Commit Graph

2778 Commits

Author SHA1 Message Date
Richard van der Hoff
a902d13180
Batch up outgoing read-receipts to reduce federation traffic. (#4890)
Rate-limit outgoing read-receipts as per #4730.
2019-03-20 16:02:25 +00:00
Erik Johnston
cc197a61a1 Disable publishing to room list when its disabled 2019-03-20 14:30:36 +00:00
Erik Johnston
2c90422146 Pull out config option 2019-03-20 14:25:58 +00:00
Erik Johnston
7529038e66 Return before we log 2019-03-20 14:25:28 +00:00
Erik Johnston
213c98c00a Add option to disable search room lists
This disables both local and remote room list searching.
2019-03-19 17:10:52 +00:00
Neil Johnson
88f0675967 fix test_auto_create_auto_join_where_no_consent (#4886) 2019-03-19 11:38:59 +00:00
Amber Brown
282c97327f
Migrate the user directory initial population to a background task (#4864) 2019-03-19 04:50:24 +11:00
Brendan Abolivier
651ad8bc96 Add ratelimiting on failed login attempts (#4865) 2019-03-18 12:57:20 +00:00
Brendan Abolivier
899e523d6d
Add ratelimiting on login (#4821)
Add two ratelimiters on login (per-IP address and per-userID).
2019-03-15 17:46:16 +00:00
Richard van der Hoff
2dee441bdb
Merge pull request #4852 from matrix-org/rav/move_rr_sending_to_worker
Move client receipt processing to federation sender worker.
2019-03-15 12:30:30 +00:00
Erik Johnston
9073cfc8bd
Merge pull request #4846 from matrix-org/hawkowl/userdir-search
Improve searching in the userdir
2019-03-14 13:54:15 +00:00
Richard van der Hoff
fdcad8eabd Move client receipt processing to federation sender worker.
This is mostly a prerequisite for #4730, but also fits with the general theme
of "move everything off the master that we possibly can".
2019-03-13 17:21:19 +00:00
Richard van der Hoff
eed7271b3b declare a ReadReceipt class
I'm going to use this in queues and things, so it'll be useful to give it more
of a structure.
2019-03-13 17:20:55 +00:00
Andrew Morgan
d42c81d724
Transfer local user's push rules on room upgrade (#4838)
Transfer push rules (notifications) on room upgrade
2019-03-12 14:42:53 +00:00
Amber Brown
c0332d095f fixup 2019-03-13 01:30:54 +11:00
Amber Brown
10480c4348 fixup 2019-03-12 21:47:14 +11:00
Amber Brown
5ba8ceab4c fixes 2019-03-12 00:35:31 +11:00
Amber Brown
4abc988c6a
initial 2019-03-11 21:11:36 +11:00
Erik Johnston
8c4896668f Merge branch 'develop' of github.com:matrix-org/synapse into erikj/soft_fail_impl 2019-03-08 11:44:20 +00:00
Erik Johnston
0ff8163eae Factor out soft fail checks 2019-03-08 11:26:33 +00:00
Erik Johnston
469b66c102
Merge pull request #4829 from matrix-org/erikj/device_list_seen_updates
When re-syncing device lists reset the state
2019-03-08 09:54:54 +00:00
Erik Johnston
d42b41544a When re-syncing device lists reset the state
We keep track of what stream IDs we've seen so that we know what updates
we've handled or missed. If we re-sync we don't know if the updates
we've seen are included in the re-sync (there may be a race), so we
should reset the seen updates.
2019-03-07 16:04:24 +00:00
Erik Johnston
c633fc02d7 Add some debug logging for device list handling 2019-03-07 15:58:03 +00:00
Amber Brown
f6135d06cf
Rewrite userdir to be faster (#4537) 2019-03-07 01:22:53 -08:00
Erik Johnston
0a6e716600
Merge pull request #4815 from matrix-org/erikj/docstrings
Add docstrings from matrix-org-hotfixes
2019-03-06 18:59:28 +00:00
Erik Johnston
8b7790e68f Port #4422 debug logging from hotfixes 2019-03-06 17:29:15 +00:00
Erik Johnston
6d13bdec91 Add docstrings from matrix-org-hotfixes 2019-03-06 17:24:26 +00:00
Erik Johnston
a9de04be72 Implement soft fail 2019-03-06 16:22:16 +00:00
Brendan Abolivier
6f3cde8b25 Make registration ratelimiter separate from the main events one 2019-03-06 11:02:42 +00:00
Brendan Abolivier
f4195f4118 Revert "Split ratelimiters in two (one for events, one for registration)"
This reverts commit d7dbad3526.
2019-03-06 10:55:22 +00:00
Brendan Abolivier
d7dbad3526 Split ratelimiters in two (one for events, one for registration) 2019-03-05 18:41:27 +00:00
Brendan Abolivier
a4c3a361b7
Add rate-limiting on registration (#4735)
* Rate-limiting for registration

* Add unit test for registration rate limiting

* Add config parameters for rate limiting on auth endpoints

* Doc

* Fix doc of rate limiting function

Co-Authored-By: babolivier <contact@brendanabolivier.com>

* Incorporate review

* Fix config parsing

* Fix linting errors

* Set default config for auth rate limiting

* Fix tests

* Add changelog

* Advance reactor instead of mocked clock

* Move parameters to registration specific config and give them more sensible default values

* Remove unused config options

* Don't mock the rate limiter un MAU tests

* Rename _register_with_store into register_with_store

* Make CI happy

* Remove unused import

* Update sample config

* Fix ratelimiting test for py2

* Add non-guest test
2019-03-05 14:25:33 +00:00
Erik Johnston
b050a10871
Merge pull request #4699 from matrix-org/erikj/stop_fed_not_in_room
Stop backpaginating when events not visible
2019-03-05 09:32:33 +00:00
Erik Johnston
aa06d26ae0 clarify comments 2019-03-05 09:16:35 +00:00
Erik Johnston
c3c542bb4a
Merge pull request #4796 from matrix-org/erikj/factor_out_e2e_keys
Allow /keys/{changes,query} API to run on worker
2019-03-05 09:06:25 +00:00
Erik Johnston
bfa7d46a10 Allow /keys/{changes,query} API to run on worker 2019-03-04 18:30:01 +00:00
Erik Johnston
157e5a8f27 Split DeviceHandler into master and worker 2019-03-04 18:29:26 +00:00
Richard van der Hoff
b29693a30b Clean up read-receipt handling.
Remove a call to run_as_background_process: there is no need to run this as a
background process, because build_and_send_edu does not block.

We may as well inline the whole of _push_remotes.
2019-03-04 18:16:43 +00:00
Erik Johnston
d1523aed6b Only check history visibility when filtering
When filtering events to send to server we check more than just history
visibility. However when deciding whether to backfill or not we only
care about the history visibility.
2019-03-04 14:43:42 +00:00
Richard van der Hoff
856c83f5f8
Avoid rebuilding Edu objects in worker mode (#4770)
In worker mode, on the federation sender, when we receive an edu for sending
over the replication socket, it is parsed into an Edu object. There is no point
extracting the contents of it so that we can then immediately build another Edu.
2019-03-04 12:57:44 +00:00
Erik Johnston
8b63fe4c26 s/get_forward_events/get_successor_events/ 2019-03-04 11:56:03 +00:00
Erik Johnston
fbc047f2a5 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/stop_fed_not_in_room 2019-03-04 11:54:58 +00:00
Richard van der Hoff
3064952939
Fix incorrect log about not persisting duplicate state event. (#4776)
We were logging this when it was not true.
2019-03-01 16:47:12 +00:00
Erik Johnston
b183fef9ac Update comments 2019-02-27 13:06:10 +00:00
Andrew Morgan
802884d4ee Merge branch 'develop' of github.com:matrix-org/synapse into anoa/public_rooms_federate_develop 2019-02-26 14:23:40 +00:00
Andrew Morgan
6fcb25202f Put function def back to the way it was 2019-02-26 14:17:38 +00:00
Andrew Morgan
7a4632af9c Prevent showing non-fed rooms in fed /publicRooms 2019-02-26 13:37:24 +00:00
Andrew Morgan
c74624a633
Revert "Prevent showing non-fed rooms in fed /publicRooms" 2019-02-26 13:20:38 +00:00
Andrew Morgan
a1a6473293
Merge pull request #4736 from matrix-org/anoa/public_rooms_federate
Config option to prevent showing non-fed rooms in fed /publicRooms
2019-02-26 13:07:15 +00:00
Andrew Morgan
c4414768af Cleaner chunk logic 2019-02-26 12:22:34 +00:00
Andrew Morgan
a712aa3a9c Correct indent 2019-02-26 12:13:55 +00:00
Andrew Morgan
16565e67db Correct docstring types and chunk logic 2019-02-26 12:12:48 +00:00
Andrew Morgan
40c2271680 Clean up room chunk logic 2019-02-26 12:04:34 +00:00
Andrew Morgan
6728bf3940 Make not showing non-federated rooms the default 2019-02-26 11:52:52 +00:00
Andrew Morgan
6946c20111 Result may be None 2019-02-26 11:27:19 +00:00
Andrew Morgan
71669a0fba Address rich comments 2019-02-26 11:25:00 +00:00
Matthew Hodgson
70ea2f4e1d switch from google.com to recaptcha.net for reCAPTCHA (#4731)
* add trivial clarification about jemalloc

* switch from google.com to recaptcha.net

because https://developers.google.com/recaptcha/docs/faq#can-i-use-recaptcha-globally
2019-02-25 11:15:36 -08:00
Andrew Morgan
1330aa4a8f elif not else if 2019-02-25 17:28:19 +00:00
Erik Johnston
65f3fbfbf7
Merge pull request #4737 from matrix-org/erikj/failure_log_tb
Log tracebacks correctly
2019-02-25 17:26:30 +00:00
Erik Johnston
c7b333c545 Log tracebacks correctly 2019-02-25 16:56:41 +00:00
Erik Johnston
9c598dddcb Fix typo 2019-02-25 16:32:02 +00:00
Andrew Morgan
8aaf7ffc44 syntax derp 2019-02-25 15:27:17 +00:00
Andrew Morgan
84c0a20dfe Simplify call to generate_room_entry 2019-02-25 15:23:27 +00:00
Erik Johnston
4b9e5076c4 Merge branch 'develop' of github.com:matrix-org/synapse into anoa/public_rooms_federate 2019-02-25 15:08:18 +00:00
Andrew Morgan
07493607a8 Docs and arg name clarification 2019-02-25 15:04:46 +00:00
Andrew Morgan
bd398b874e Don't restrict non-fed rooms over client APIs 2019-02-25 15:04:46 +00:00
Andrew Morgan
e4b078a600 Config option to prevent showing non-fed rooms in fed /publicRooms 2019-02-25 15:04:46 +00:00
Erik Johnston
d730c2c22b More comments 2019-02-25 14:45:02 +00:00
Erik Johnston
890cb048fd Assert rather than clobber the values 2019-02-25 14:42:11 +00:00
Erik Johnston
9342cc6ab1 Add comments and paranoia 2019-02-25 10:02:12 +00:00
Erik Johnston
7b288826b7 Fix backfill storing incorrect state for events 2019-02-22 11:33:51 +00:00
Richard van der Hoff
6d65659b62
Run push_receipts_to_remotes as background job (#4707)
I suspect the CPU usage metrics for this are going to /dev/null at the moment.
2019-02-21 17:50:30 +00:00
Erik Johnston
b9d6756b14
Merge pull request #4263 from rkfg/develop
Prevent crash on pagination.
2019-02-21 17:42:15 +00:00
Erik Johnston
71b625d808 Stop backpaginating when events not visible 2019-02-20 18:14:12 +00:00
Erik Johnston
dbdc565dfd Fix registration on workers (#4682)
* Move RegistrationHandler init to HomeServer

* Move post registration actions to RegistrationHandler

* Add post regisration replication endpoint

* Newsfile
2019-02-20 18:47:31 +11:00
Brendan Abolivier
a288bdf0b1
Merge pull request #4652 from matrix-org/babolivier/acme-delegated
Support .well-known delegation when issuing certificates through ACME
2019-02-19 11:15:38 +00:00
Brendan Abolivier
a8626901cd Fetch ACME domain into an instance member 2019-02-19 10:54:33 +00:00
Andrew Morgan
968a30a75c
Merge pull request #4642 from matrix-org/anoa/bans_room_upgrade
Transfer bans on room upgrade
2019-02-19 10:16:42 +00:00
Andrew Morgan
34ac75ce2c lint 2019-02-18 18:23:37 +00:00
Andrew Morgan
f8b9ca53ce Move member event processing and changelog fix 2019-02-18 17:28:52 +00:00
Erik Johnston
fc2c245a1f
Merge pull request #4666 from matrix-org/erikj/register_login_split
Split out registration to worker
2019-02-18 17:18:06 +00:00
Erik Johnston
459d3d5046
Merge pull request #4668 from matrix-org/erikj/catch_exceptions
Correctly handle HttpResponseException when handling device updates
2019-02-18 16:55:43 +00:00
Erik Johnston
af691e415c Move register_device into handler 2019-02-18 16:49:38 +00:00
Erik Johnston
c30f73c86a
Merge pull request #4667 from matrix-org/erikj/fix_revoke_guest_access_workers
Fix kicking guest users in worker mode
2019-02-18 15:55:48 +00:00
Brendan Abolivier
45bb55c6de Use a configuration parameter to give the domain to generate a certificate for 2019-02-18 15:46:23 +00:00
Erik Johnston
94960cef03 pep8 2019-02-18 15:24:13 +00:00
Erik Johnston
e07cc31cb8 Correctly handle HttpResponseException 2019-02-18 14:55:09 +00:00
Brendan Abolivier
68a53f825f Merge branch 'develop' into babolivier/acme-delegated 2019-02-18 14:52:23 +00:00
Erik Johnston
32e54b472a Fix kicking guest users in worker mode
When guest_access changes from allowed to forbidden all local guest
users should be kicked from the room. This did not happen when
revocation was received from federation on a worker.

Presumably broken in #4141
2019-02-18 14:16:07 +00:00
Andrew Morgan
915421065b Membership events are done later 2019-02-18 14:02:09 +00:00
Erik Johnston
d1b060b492
Merge pull request #4643 from matrix-org/erikj/catch_exceptions
Correctly handle RequestSendFailed exceptions
2019-02-18 13:53:24 +00:00
Andrew Morgan
9caab0c364 Transfer bans on room upgrade 2019-02-18 13:52:51 +00:00
Erik Johnston
eb2b8523ae Split out registration to worker
This allows registration to be handled by a worker, though the actual
write to the database still happens on master.

Note: due to the in-memory session map all registration requests must be
handled by the same worker.
2019-02-18 12:12:57 +00:00
Andrew Morgan
6d02a13d81
Typo in info log
Co-Authored-By: babolivier <contact@brendanabolivier.com>
2019-02-18 11:36:34 +00:00
Brendan Abolivier
f86b695cbd
Various cosmetics to make TravisCI happy 2019-02-15 12:29:34 +00:00
Brendan Abolivier
af8a2f679b
Remove unused import 2019-02-15 12:27:43 +00:00
Brendan Abolivier
1895d14e12
Support .well-known delegation when issuing certificates through ACME 2019-02-15 12:05:08 +00:00
Erik Johnston
f61b2068e6 Only fetch aliases when publishing rooms 2019-02-14 18:08:09 +00:00
Erik Johnston
eaf4d11af9 Add configurable room list publishing rules
This allows specifying who and what is allowed to be published onto the
public room list
2019-02-14 16:02:23 +00:00
Erik Johnston
7fc1196a36 Correctly handle RequestSendFailed exceptions
This mainly reduces the number of exceptions we log.
2019-02-14 14:01:04 +00:00