Richard van der Hoff
48fec67536
Features
...
--------
- Add support for the SNI extension to federation TLS connections. Thanks to @vojeroen! ([\#3439](https://github.com/matrix-org/synapse/issues/3439 ))
- Add /_media/r0/config ([\#3184](https://github.com/matrix-org/synapse/issues/3184 ))
- speed up /members API and add `at` and `membership` params as per MSC1227 ([\#3568](https://github.com/matrix-org/synapse/issues/3568 ))
- implement `summary` block in /sync response as per MSC688 ([\#3574](https://github.com/matrix-org/synapse/issues/3574 ))
- Add lazy-loading support to /messages as per MSC1227 ([\#3589](https://github.com/matrix-org/synapse/issues/3589 ))
- Add ability to limit number of monthly active users on the server ([\#3633](https://github.com/matrix-org/synapse/issues/3633 ))
- Support more federation endpoints on workers ([\#3653](https://github.com/matrix-org/synapse/issues/3653 ))
- Basic support for room versioning ([\#3654](https://github.com/matrix-org/synapse/issues/3654 ))
- Ability to disable client/server Synapse via conf toggle ([\#3655](https://github.com/matrix-org/synapse/issues/3655 ))
- Ability to whitelist specific threepids against monthly active user limiting ([\#3662](https://github.com/matrix-org/synapse/issues/3662 ))
- Add some metrics for the appservice and federation event sending loops ([\#3664](https://github.com/matrix-org/synapse/issues/3664 ))
- Where server is disabled, block ability for locked out users to read new messages ([\#3670](https://github.com/matrix-org/synapse/issues/3670 ))
- set admin uri via config, to be used in error messages where the user should contact the administrator ([\#3687](https://github.com/matrix-org/synapse/issues/3687 ))
- Synapse's presence functionality can now be disabled with the "use_presence" configuration option. ([\#3694](https://github.com/matrix-org/synapse/issues/3694 ))
- For resource limit blocked users, prevent writing into rooms ([\#3708](https://github.com/matrix-org/synapse/issues/3708 ))
Bugfixes
--------
- Fix occasional glitches in the synapse_event_persisted_position metric ([\#3658](https://github.com/matrix-org/synapse/issues/3658 ))
- Fix bug on deleting 3pid when using identity servers that don't support unbind API ([\#3661](https://github.com/matrix-org/synapse/issues/3661 ))
- Make the tests pass on Twisted < 18.7.0 ([\#3676](https://github.com/matrix-org/synapse/issues/3676 ))
- Don’t ship recaptcha_ajax.js, use it directly from Google ([\#3677](https://github.com/matrix-org/synapse/issues/3677 ))
- Fixes test_reap_monthly_active_users so it passes under postgres ([\#3681](https://github.com/matrix-org/synapse/issues/3681 ))
- Fix mau blocking calulation bug on login ([\#3689](https://github.com/matrix-org/synapse/issues/3689 ))
- Fix missing yield in synapse.storage.monthly_active_users.initialise_reserved_users ([\#3692](https://github.com/matrix-org/synapse/issues/3692 ))
- Improve HTTP request logging to include all requests ([\#3700](https://github.com/matrix-org/synapse/issues/3700 ))
- Avoid timing out requests while we are streaming back the response ([\#3701](https://github.com/matrix-org/synapse/issues/3701 ))
- Support more federation endpoints on workers ([\#3705](https://github.com/matrix-org/synapse/issues/3705 ), [\#3713](https://github.com/matrix-org/synapse/issues/3713 ))
- Fix "Starting db txn 'get_all_updated_receipts' from sentinel context" warning ([\#3710](https://github.com/matrix-org/synapse/issues/3710 ))
- Fix bug where `state_cache` cache factor ignored environment variables ([\#3719](https://github.com/matrix-org/synapse/issues/3719 ))
- Fix bug in v0.33.3rc1 which caused infinite loops and OOMs ([\#3723](https://github.com/matrix-org/synapse/issues/3723 ))
- Fix bug introduced in v0.33.3rc1 which made the ToS give a 500 error ([\#3732](https://github.com/matrix-org/synapse/issues/3732 ))
Deprecations and Removals
-------------------------
- The Shared-Secret registration method of the legacy v1/register REST endpoint has been removed. For a replacement, please see [the admin/register API documentation](https://github.com/matrix-org/synapse/blob/master/docs/admin_api/register_api.rst ). ([\#3703](https://github.com/matrix-org/synapse/issues/3703 ))
Internal Changes
----------------
- The test suite now can run under PostgreSQL. ([\#3423](https://github.com/matrix-org/synapse/issues/3423 ))
- Refactor HTTP replication endpoints to reduce code duplication ([\#3632](https://github.com/matrix-org/synapse/issues/3632 ))
- Tests now correctly execute on Python 3. ([\#3647](https://github.com/matrix-org/synapse/issues/3647 ))
- Sytests can now be run inside a Docker container. ([\#3660](https://github.com/matrix-org/synapse/issues/3660 ))
- Port over enough to Python 3 to allow the sytests to start. ([\#3668](https://github.com/matrix-org/synapse/issues/3668 ))
- Update docker base image from alpine 3.7 to 3.8. ([\#3669](https://github.com/matrix-org/synapse/issues/3669 ))
- Rename synapse.util.async to synapse.util.async_helpers to mitigate async becoming a keyword on Python 3.7. ([\#3678](https://github.com/matrix-org/synapse/issues/3678 ))
- Synapse's tests are now formatted with the black autoformatter. ([\#3679](https://github.com/matrix-org/synapse/issues/3679 ))
- Implemented a new testing base class to reduce test boilerplate. ([\#3684](https://github.com/matrix-org/synapse/issues/3684 ))
- Rename MAU prometheus metrics ([\#3690](https://github.com/matrix-org/synapse/issues/3690 ))
- add new error type ResourceLimit ([\#3707](https://github.com/matrix-org/synapse/issues/3707 ))
- Logcontexts for replication command handlers ([\#3709](https://github.com/matrix-org/synapse/issues/3709 ))
- Update admin register API documentation to reference a real user ID. ([\#3712](https://github.com/matrix-org/synapse/issues/3712 ))
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
iQEcBAABAgAGBQJbfWKDAAoJEIofk9V1tejVHOMH/RmP+axF2K+Socnnb5Fa6RTo
pDJXVwsvNQxcwzZIJXh2TRilVr88jx7A1PHUde0TmkHRmis/bDzZxa59QRA4rFEy
fLYMx9Tyt0RzAJrEd0p7jbIkJxxPm5PbMWfCgVfzg44q0kepRW6ZVl1+kFHDrsCo
ocE03WdLe2sRHSudaVtBLOVMA1lDE22v40/7xiu9IbUOnF+WZ3cjXYX+SNr5eb6g
7YGbXOk4/E8XoUnpzH5oEnNtiDK5piRrr7o1282kecY20ZMNzKgseBHU6nYMv5ZO
TUpCPwbukmzeGHITupY1RkzpmBbEIJTCnmLP8fiUw+RWznYyFC/aRFMiWTzle3A=
=ngmn
-----END PGP SIGNATURE-----
Merge tag 'v0.33.3'
Features
--------
- Add support for the SNI extension to federation TLS connections. Thanks to @vojeroen! ([\#3439](https://github.com/matrix-org/synapse/issues/3439 ))
- Add /_media/r0/config ([\#3184](https://github.com/matrix-org/synapse/issues/3184 ))
- speed up /members API and add `at` and `membership` params as per MSC1227 ([\#3568](https://github.com/matrix-org/synapse/issues/3568 ))
- implement `summary` block in /sync response as per MSC688 ([\#3574](https://github.com/matrix-org/synapse/issues/3574 ))
- Add lazy-loading support to /messages as per MSC1227 ([\#3589](https://github.com/matrix-org/synapse/issues/3589 ))
- Add ability to limit number of monthly active users on the server ([\#3633](https://github.com/matrix-org/synapse/issues/3633 ))
- Support more federation endpoints on workers ([\#3653](https://github.com/matrix-org/synapse/issues/3653 ))
- Basic support for room versioning ([\#3654](https://github.com/matrix-org/synapse/issues/3654 ))
- Ability to disable client/server Synapse via conf toggle ([\#3655](https://github.com/matrix-org/synapse/issues/3655 ))
- Ability to whitelist specific threepids against monthly active user limiting ([\#3662](https://github.com/matrix-org/synapse/issues/3662 ))
- Add some metrics for the appservice and federation event sending loops ([\#3664](https://github.com/matrix-org/synapse/issues/3664 ))
- Where server is disabled, block ability for locked out users to read new messages ([\#3670](https://github.com/matrix-org/synapse/issues/3670 ))
- set admin uri via config, to be used in error messages where the user should contact the administrator ([\#3687](https://github.com/matrix-org/synapse/issues/3687 ))
- Synapse's presence functionality can now be disabled with the "use_presence" configuration option. ([\#3694](https://github.com/matrix-org/synapse/issues/3694 ))
- For resource limit blocked users, prevent writing into rooms ([\#3708](https://github.com/matrix-org/synapse/issues/3708 ))
Bugfixes
--------
- Fix occasional glitches in the synapse_event_persisted_position metric ([\#3658](https://github.com/matrix-org/synapse/issues/3658 ))
- Fix bug on deleting 3pid when using identity servers that don't support unbind API ([\#3661](https://github.com/matrix-org/synapse/issues/3661 ))
- Make the tests pass on Twisted < 18.7.0 ([\#3676](https://github.com/matrix-org/synapse/issues/3676 ))
- Don’t ship recaptcha_ajax.js, use it directly from Google ([\#3677](https://github.com/matrix-org/synapse/issues/3677 ))
- Fixes test_reap_monthly_active_users so it passes under postgres ([\#3681](https://github.com/matrix-org/synapse/issues/3681 ))
- Fix mau blocking calulation bug on login ([\#3689](https://github.com/matrix-org/synapse/issues/3689 ))
- Fix missing yield in synapse.storage.monthly_active_users.initialise_reserved_users ([\#3692](https://github.com/matrix-org/synapse/issues/3692 ))
- Improve HTTP request logging to include all requests ([\#3700](https://github.com/matrix-org/synapse/issues/3700 ))
- Avoid timing out requests while we are streaming back the response ([\#3701](https://github.com/matrix-org/synapse/issues/3701 ))
- Support more federation endpoints on workers ([\#3705](https://github.com/matrix-org/synapse/issues/3705 ), [\#3713](https://github.com/matrix-org/synapse/issues/3713 ))
- Fix "Starting db txn 'get_all_updated_receipts' from sentinel context" warning ([\#3710](https://github.com/matrix-org/synapse/issues/3710 ))
- Fix bug where `state_cache` cache factor ignored environment variables ([\#3719](https://github.com/matrix-org/synapse/issues/3719 ))
- Fix bug in v0.33.3rc1 which caused infinite loops and OOMs ([\#3723](https://github.com/matrix-org/synapse/issues/3723 ))
- Fix bug introduced in v0.33.3rc1 which made the ToS give a 500 error ([\#3732](https://github.com/matrix-org/synapse/issues/3732 ))
Deprecations and Removals
-------------------------
- The Shared-Secret registration method of the legacy v1/register REST endpoint has been removed. For a replacement, please see [the admin/register API documentation](https://github.com/matrix-org/synapse/blob/master/docs/admin_api/register_api.rst ). ([\#3703](https://github.com/matrix-org/synapse/issues/3703 ))
Internal Changes
----------------
- The test suite now can run under PostgreSQL. ([\#3423](https://github.com/matrix-org/synapse/issues/3423 ))
- Refactor HTTP replication endpoints to reduce code duplication ([\#3632](https://github.com/matrix-org/synapse/issues/3632 ))
- Tests now correctly execute on Python 3. ([\#3647](https://github.com/matrix-org/synapse/issues/3647 ))
- Sytests can now be run inside a Docker container. ([\#3660](https://github.com/matrix-org/synapse/issues/3660 ))
- Port over enough to Python 3 to allow the sytests to start. ([\#3668](https://github.com/matrix-org/synapse/issues/3668 ))
- Update docker base image from alpine 3.7 to 3.8. ([\#3669](https://github.com/matrix-org/synapse/issues/3669 ))
- Rename synapse.util.async to synapse.util.async_helpers to mitigate async becoming a keyword on Python 3.7. ([\#3678](https://github.com/matrix-org/synapse/issues/3678 ))
- Synapse's tests are now formatted with the black autoformatter. ([\#3679](https://github.com/matrix-org/synapse/issues/3679 ))
- Implemented a new testing base class to reduce test boilerplate. ([\#3684](https://github.com/matrix-org/synapse/issues/3684 ))
- Rename MAU prometheus metrics ([\#3690](https://github.com/matrix-org/synapse/issues/3690 ))
- add new error type ResourceLimit ([\#3707](https://github.com/matrix-org/synapse/issues/3707 ))
- Logcontexts for replication command handlers ([\#3709](https://github.com/matrix-org/synapse/issues/3709 ))
- Update admin register API documentation to reference a real user ID. ([\#3712](https://github.com/matrix-org/synapse/issues/3712 ))
2018-08-22 14:28:55 +01:00
Richard van der Hoff
4e5a4549b6
bump version to 0.33.3
2018-08-22 14:07:10 +01:00
Richard van der Hoff
f7bf181a90
fix another consent encoding fail
2018-08-21 23:14:25 +01:00
Richard van der Hoff
f7baff6f7b
Fix 500 error from /consent form
...
Fixes #3731
2018-08-21 22:47:07 +01:00
Amber Brown
3b5b64ac99
changelog
2018-08-21 03:48:55 +10:00
Richard van der Hoff
be6527325a
Fix exceptions when a connection is closed before we read the headers
...
This fixes bugs introduced in #3700 , by making sure that we behave sanely
when an incoming connection is closed before the headers are read.
2018-08-20 18:21:10 +01:00
Richard van der Hoff
55e6bdf287
Robustness fix for logcontext filter
...
Make the logcontext filter not explode if it somehow ends up with a logcontext
of None, since that infinite-loops the whole logging system.
2018-08-20 18:20:07 +01:00
Amber Brown
9a2f960736
version
2018-08-21 00:00:19 +10:00
Amber Brown
324525f40c
Port over enough to get some sytests running on Python 3 ( #3668 )
2018-08-20 23:54:49 +10:00
Erik Johnston
f2a48d87df
Use get_cache_factor_for function for state_cache
...
This allows the cache factor for `state_cache` to be individually
specified in the enviroment
2018-08-20 13:01:46 +01:00
Erik Johnston
2aa7cc6a46
Merge pull request #3713 from matrix-org/erikj/fixup_fed_logging
...
Fix logging bug in EDU handling over replication
2018-08-20 10:51:45 +01:00
Richard van der Hoff
3cef867cc1
Merge pull request #3709 from matrix-org/rav/logcontext_for_replication_commands
...
Logcontexts for replication command handlers
2018-08-17 16:22:07 +01:00
Richard van der Hoff
c144252a8c
Merge pull request #3710 from matrix-org/rav/logcontext_for_pusher_updates
...
Fix logcontexts for running pushers
2018-08-17 16:21:49 +01:00
Amber Brown
c334ca67bb
Integrate presence from hotfixes ( #3694 )
2018-08-18 01:08:45 +10:00
Amber Brown
04f5d2db62
Remove v1/register's broken shared secret functionality ( #3703 )
2018-08-18 00:55:01 +10:00
Richard van der Hoff
63260397c6
Merge pull request #3701 from matrix-org/rav/use_producer_for_responses
...
Use a producer to stream back responses
2018-08-17 14:58:45 +01:00
Richard van der Hoff
3f8709ffe4
Merge pull request #3700 from matrix-org/rav/wait_for_producers
...
Refactor request logging code
2018-08-17 14:57:45 +01:00
Neil Johnson
9fd161c6fb
Merge branch 'neilj/limit_exceeded_error' of github.com:matrix-org/synapse into neilj/limit_exceeded_error
2018-08-17 13:58:40 +01:00
Neil Johnson
0195dfbf52
server limits config docs
2018-08-17 13:58:25 +01:00
Neil Johnson
69c49d3fa3
Merge branch 'develop' into neilj/limit_exceeded_error
2018-08-17 12:44:26 +00:00
Erik Johnston
3b2dcfff78
Fix logging bug in EDU handling over replication
2018-08-17 11:11:06 +01:00
Neil Johnson
521d369e7a
remove errant yield
2018-08-17 10:12:11 +01:00
Richard van der Hoff
0e8d78f6aa
Logcontexts for replication command handlers
...
Run the handlers for replication commands as background processes. This should
improve the visibility in our metrics, and reduce the number of "running db
transaction from sentinel context" warnings.
Ideally it means converting the things that fire off deferreds into the night
into things that actually return a Deferred when they are done. I've made a bit
of a stab at this, but it will probably be leaky.
2018-08-17 00:43:43 +01:00
Richard van der Hoff
66f7dc8c87
Fix logcontexts for running pushers
...
First of all, avoid resetting the logcontext before running the pushers, to fix
the "Starting db txn 'get_all_updated_receipts' from sentinel context" warning.
Instead, give them their own "background process" logcontexts.
2018-08-17 00:32:39 +01:00
Neil Johnson
bcfeb44afe
call reap on start up and fix under reaping bug
2018-08-16 22:55:32 +01:00
Neil Johnson
372bf073c1
block event creation and room creation on hitting resource limits
2018-08-16 21:25:16 +01:00
Neil Johnson
13ad9930c8
add new error type ResourceLimit
2018-08-16 18:02:02 +01:00
Erik Johnston
7d5b1a60a3
Fix inbound federation on reader worker
...
Inbound federation requires calculating push, which in turn relies on
having access to account data.
2018-08-16 15:32:20 +01:00
Will Hunt
c151b32b1d
Add GET media/v1/config ( #3184 )
2018-08-16 14:23:38 +01:00
Matthew Hodgson
762a758fea
lazyload aware /messages ( #3589 )
2018-08-16 14:22:47 +01:00
Matthew Hodgson
3f543dc021
initial cut at a room summary API ( #3574 )
2018-08-16 09:46:50 +01:00
Neil Johnson
81d727efa9
Merge pull request #3689 from matrix-org/neilj/fix_off_by_1+maus
...
Fix Mau off by one errors
2018-08-15 16:19:41 +00:00
Matthew Hodgson
2f78f432c4
speed up /members and add at= and membership params ( #3568 )
2018-08-15 16:35:22 +01:00
Neil Johnson
86a00e05e1
Merge branch 'develop' of github.com:matrix-org/synapse into neilj/fix_off_by_1+maus
2018-08-15 16:27:08 +01:00
Richard van der Hoff
afcd655ab6
Use a producer to stream back responses
...
The problem with dumping all of the json response into the Request object at
once is that doing so starts the timeout for the next request to be received:
so if it takes longer than 60s to stream back the response to the client, the
client never gets it.
The correct solution is to use a Producer; then the timeout is only started
once all of the content is sent over the TCP connection.
2018-08-15 15:04:16 +01:00
Erik Johnston
dc56c47dc0
Merge pull request #3653 from matrix-org/erikj/split_federation
...
Move more federation APIs to workers
2018-08-15 14:59:02 +01:00
Neil Johnson
4601129c44
Merge pull request #3687 from matrix-org/neilj/admin_email
...
support admin_email config and pass through into blocking errors,
2018-08-15 13:52:25 +00:00
Erik Johnston
ef184caf30
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/split_federation
2018-08-15 14:25:46 +01:00
Erik Johnston
488ffe6fdb
Use federation handler function rather than duplicate
...
This involves renaming _persist_events to be a public function.
2018-08-15 14:17:18 +01:00
Erik Johnston
773db62a22
Rename slave TransactionStore to SlaveTransactionStore
2018-08-15 14:17:06 +01:00
Richard van der Hoff
eaaa2248ff
Refactor request logging code
...
This commit moves a bunch of the logic for deciding when to log the receipt and
completion of HTTP requests into SynapseRequest, rather than in the request
handling wrappers.
Advantages of this are:
* we get logs for *all* requests (including OPTIONS and HEADs), rather than
just those that end up hitting handlers we've remembered to decorate
correctly.
* when a request handler wires up a Producer (as the media stuff does
currently, and as other things will do soon), we log at the point that all
of the traffic has been sent to the client.
2018-08-15 13:47:52 +01:00
Neil Johnson
87a824bad1
clean up AuthError
2018-08-15 11:58:03 +01:00
Neil Johnson
c4eb97518f
Merge branch 'neilj/update_limits_error_codes' of github.com:matrix-org/synapse into neilj/admin_email
2018-08-15 11:53:01 +01:00
Neil Johnson
55afba0fc5
update admin email to uri
2018-08-15 11:41:18 +01:00
Neil Johnson
1c5e690a6b
Merge pull request #3690 from matrix-org/neilj/change_prometheus_mau_metric_name
...
combine mau metrics into one group
2018-08-15 09:53:59 +00:00
Erik Johnston
fef2e65d12
Merge pull request #3667 from matrix-org/erikj/fixup_unbind
...
Don't fail requests to unbind 3pids for non supporting ID servers
2018-08-15 10:32:12 +01:00
Neil Johnson
b8429c7c81
update error codes for resource limiting
2018-08-15 10:19:48 +01:00
Neil Johnson
ab035bdeac
replace admin_email with admin_uri for greater flexibility
2018-08-15 10:16:41 +01:00
Neil Johnson
19b433e3f4
Merge branch 'develop' of github.com:matrix-org/synapse into neilj/admin_email
2018-08-14 17:44:46 +01:00
Neil Johnson
b586b8b986
Merge branch 'develop' of github.com:matrix-org/synapse into neilj/fix_off_by_1+maus
2018-08-14 17:43:22 +01:00