Richard van der Hoff
d688a51736
Merge pull request #4954 from matrix-org/rav/refactor_parse_row
...
Refactors to replication stream row update/parsing
2019-03-28 18:31:17 +00:00
Neil Johnson
c7296bcb98
remove log line for password ( #4965 )
...
Remove log line for password.
2019-03-28 17:38:01 +00:00
Andrew Morgan
7a91b9d81c
Allow password providers to bind emails ( #4947 )
...
This PR allows password provider modules to bind email addresses when a user is registering and is motivated by matrix-org/matrix-synapse-ldap3#58
2019-03-28 15:48:07 +00:00
Erik Johnston
248014379e
Merge pull request #4942 from matrix-org/erikj/fix_presence
...
Use event streams to calculate presence
2019-03-28 14:38:31 +00:00
Erik Johnston
4e5f0f7ca0
Use an assert
2019-03-28 14:05:05 +00:00
Erik Johnston
40e56997bc
Review comments
2019-03-28 13:48:41 +00:00
Richard van der Hoff
d035d62f6b
Merge remote-tracking branch 'origin/develop' into rav/refactor_parse_row
2019-03-28 13:45:14 +00:00
Richard van der Hoff
4eeb2c2f07
Merge pull request #4953 from matrix-org/rav/refactor_replication_streams
...
Split up replication.tcp.streams into smaller files
2019-03-28 13:43:25 +00:00
Amber Brown
2e060774ad
Run black
on some storage modules that the stats branch touches ( #4959 )
2019-03-29 00:37:16 +11:00
Richard van der Hoff
4b91c313a9
Combine the CurrentStateDeltaStream into the EventStream
2019-03-27 22:07:05 +00:00
Richard van der Hoff
1f6d6f918a
Make EventStream rows have a type
...
... as a precursor to combining it with the CurrentStateDelta stream.
2019-03-27 22:07:05 +00:00
Richard van der Hoff
015b3622eb
Skip building a ROW_TYPE when building updates
...
We're about to turn it straight into a JSON object anyway so building a
ROW_TYPE is a bit pointless, and reduces flexibility in the update_function.
2019-03-27 21:58:03 +00:00
Richard van der Hoff
f570916a3e
Add parse_row method to replication stream class
...
This will allow individual stream classes to override how a row is parsed.
2019-03-27 21:32:33 +00:00
Richard van der Hoff
71dcb275f1
move FederationStream out to its own file
2019-03-27 21:13:14 +00:00
Richard van der Hoff
aa1e017864
move EventsStream out to its own file
2019-03-27 21:13:14 +00:00
Richard van der Hoff
a5798de067
Move replication.tcp.streams into a package
2019-03-27 21:13:14 +00:00
Richard van der Hoff
acaa18f7dd
Fix/improve some docstrings in the replication code. ( #4949 )
2019-03-27 21:12:36 +00:00
Erik Johnston
197fae1639
Use event streams to calculate presence
...
Primarily this fixes a bug in the handling of remote users joining a
room where the server sent out the presence for all local users in the
room to all servers in the room.
We also change to using the state delta stream, rather than the
distributor, as it will make it easier to split processing out of the
master process (as well as being more flexible).
Finally, when sending presence states to newly joined servers we filter
out old presence states to reduce the number sent. Initially we filter
out states that are offline and have a last active more than a week ago,
though this can be changed down the line.
Fixes #3962
2019-03-27 13:41:36 +00:00
Neil Johnson
4aa914369b
bump version
2019-03-27 10:23:03 +00:00
Andrew Morgan
bbd244c7b2
Support 3PID login in password providers ( #4931 )
...
Adds a new method, check_3pid_auth, which gives password providers
the chance to allow authentication with third-party identifiers such
as email or msisdn.
2019-03-26 17:48:30 +00:00
Amber Brown
903f04c21f
Use the state event amount for userdir import batching, not room count ( #4944 )
2019-03-27 02:49:28 +11:00
Andrew Morgan
4a125be138
Make federation endpoints more tolerant of trailing slashes v2 ( #4935 )
...
Redo of https://github.com/matrix-org/synapse/pull/4840
2019-03-26 11:35:29 +00:00
Richard van der Hoff
8cbbedaa2b
Fix ClientReplicationStreamProtocol.__str__ ( #4929 )
...
`__str__` depended on `self.addr`, which was absent from
ClientReplicationStreamProtocol, so attempting to call str on such an object
would raise an exception.
We can calculate the peer addr from the transport, so there is no need for addr
anyway.
2019-03-25 16:41:51 +00:00
Richard van der Hoff
9bde730ef8
Fix bug where read-receipts lost their timestamps ( #4927 )
...
Make sure that they are sent correctly over the replication stream.
Fixes : #4898
2019-03-25 16:38:05 +00:00
Amber Brown
ac396a0d32
Refactor out state delta handling into its own class ( #4917 )
2019-03-25 20:37:08 +11:00
Erik Johnston
5fee9d8067
Merge pull request #4869 from matrix-org/erikj/yaml_load
...
Fix yaml warnings by using safe_load
2019-03-22 11:58:13 +00:00
Erik Johnston
3677548a82
Use yaml safe_load
2019-03-22 10:20:17 +00:00
Andrew Morgan
7bef97dfb7
Remove trailing slashes from outbound federation requests and retry on 400 ( #4840 )
...
As per #3622 , we remove trailing slashes from outbound federation requests. However, to ensure that we remain backwards compatible with previous versions of Synapse, if we receive a HTTP 400 with `M_UNRECOGNIZED`, then we are likely talking to an older version of Synapse in which case we retry with a trailing slash appended to the request path.
2019-03-21 15:07:28 +00:00
Andrew Morgan
b41c2eaadc
Clean up backoff_on_404 and metehod calls
2019-03-21 14:32:47 +00:00
Erik Johnston
01e6b405be
Merge pull request #4908 from matrix-org/erikj/block_peek_on_blocked_rooms
...
Deny peeking into rooms that have been blocked
2019-03-21 14:07:17 +00:00
Erik Johnston
3959858eaa
Merge pull request #4904 from matrix-org/erikj/fix_shutdown
...
Fixup shutdown room API
2019-03-21 11:24:42 +00:00
Erik Johnston
536a266520
Deny peeking into rooms that have been blocked
2019-03-21 11:20:13 +00:00
Erik Johnston
3ecec5ede2
Fix upsert
2019-03-21 10:21:15 +00:00
Erik Johnston
09f991a63d
Merge pull request #4896 from matrix-org/erikj/disable_room_directory
...
Add option to disable search room lists
2019-03-21 10:16:54 +00:00
Erik Johnston
cd62981a6a
Revert spurious delete
2019-03-20 17:51:27 +00:00
Erik Johnston
8d8834d3e7
comment block_room
2019-03-20 17:49:56 +00:00
Erik Johnston
aa959a6c07
Use flags
2019-03-20 17:40:29 +00:00
Erik Johnston
7d47cc1305
Move requester check into assert_accepted_privacy_policy
2019-03-20 17:08:36 +00:00
Erik Johnston
72a14860ab
Gracefully handle failing to kick user
2019-03-20 16:54:21 +00:00
Erik Johnston
6b28890543
Log new room ID
2019-03-20 16:52:28 +00:00
Erik Johnston
74c46d81fa
Only require consent for events with an associated request
...
There are a number of instances where a server or admin may puppet a
user to join/leave rooms, which we don't want to fail if the user has
not consented to the privacy policy. We fix this by adding a check to
test if the requester has an associated access_token, which is used as a
proxy to answer the question of whether the action is being done on
behalf of a real request from the user.
2019-03-20 16:50:23 +00:00
Erik Johnston
67d618e111
Allow blocking a room multiple times
2019-03-20 16:50:05 +00:00
Erik Johnston
263f2c9ce1
Merge pull request #4895 from matrix-org/erikj/disable_user_search
...
Add option to disable searching in the user dir
2019-03-20 16:47:15 +00:00
Amber Brown
4d53017432
Batching in the user directory import ( #4900 )
2019-03-21 03:06:36 +11:00
Richard van der Hoff
cdb8036161
Add a config option for torture-testing worker replication. ( #4902 )
...
Setting this to 50 or so makes a bunch of sytests fail in worker mode.
2019-03-20 16:04:35 +00:00
Richard van der Hoff
a902d13180
Batch up outgoing read-receipts to reduce federation traffic. ( #4890 )
...
Rate-limit outgoing read-receipts as per #4730 .
2019-03-20 16:02:25 +00:00
Erik Johnston
cd8c5b91ad
Fix up sample config
2019-03-20 14:35:41 +00:00
Richard van der Hoff
ab20f85c59
Update synapse/config/user_directory.py
...
Co-Authored-By: erikjohnston <erikj@jki.re>
2019-03-20 14:33:11 +00:00
Erik Johnston
cc197a61a1
Disable publishing to room list when its disabled
2019-03-20 14:30:36 +00:00
Erik Johnston
2c90422146
Pull out config option
2019-03-20 14:25:58 +00:00
Erik Johnston
7529038e66
Return before we log
2019-03-20 14:25:28 +00:00
Erik Johnston
926f29ea6d
Fix up config comments
2019-03-20 14:24:53 +00:00
Andrew Morgan
2150151abe
kwargs doesn't like commas on calling funcs either. TIL
2019-03-20 14:13:32 +00:00
Andrew Morgan
bb52a2e653
lint
2019-03-20 14:08:57 +00:00
Andrew Morgan
cd36a1283b
New test, fix issues
2019-03-20 14:00:39 +00:00
Andrew Morgan
c69df5d5d3
Fix comments. v0.99.2 -> v0.99.3
2019-03-20 11:27:18 +00:00
Andrew Morgan
551ea11559
Just return if not doing any trailing slash shennanigans
2019-03-20 11:07:36 +00:00
Erik Johnston
213c98c00a
Add option to disable search room lists
...
This disables both local and remote room list searching.
2019-03-19 17:10:52 +00:00
Erik Johnston
320667a479
Add option to disable searching in the user dir
...
We still populate it, as it can still be accessed via the admin API.
2019-03-19 16:40:19 +00:00
Richard van der Hoff
13bc1e0746
Use a regular HomeServerConfig object for unit tests
...
Rather than using a Mock for the homeserver config, use a genuine
HomeServerConfig object. This makes for a more realistic test, and means that
we don't have to keep remembering to add things to the mock config every time
we add a new config setting.
2019-03-19 11:44:43 +00:00
Richard van der Hoff
45c4e19c74
Merge remote-tracking branch 'origin/develop' into HEAD
2019-03-19 11:43:46 +00:00
Neil Johnson
88f0675967
fix test_auto_create_auto_join_where_no_consent ( #4886 )
2019-03-19 11:38:59 +00:00
Richard van der Hoff
0dbfae03f9
Enforce hs_disabled_message correctly
...
Fixes a bug where hs_disabled_message was not enforced for 3pid-based requests
if there was no server_notices_mxid configured.
2019-03-19 11:30:54 +00:00
Erik Johnston
e9eeca1314
Fix user directory background update ( #4887 )
2019-03-19 22:13:53 +11:00
Richard van der Hoff
fd463b4f5d
Comment out most options in the generated config. ( #4863 )
...
Make it so that most options in the config are optional, and commented out in
the generated config.
The reasons this is a good thing are as follows:
* If we decide that we should change the default for an option, we can do so,
and only those admins that have deliberately chosen to override that option
will be stuck on the old setting.
* It moves us towards a point where we can get rid of the super-surprising
feature of synapse where the default settings for the config come from the
generated yaml.
* It makes setting up a test config for unit testing an order of magnitude
easier (see forthcoming PR).
* It makes the generated config more consistent, and hopefully easier for users
to understand.
2019-03-19 10:06:40 +00:00
Amber Brown
282c97327f
Migrate the user directory initial population to a background task ( #4864 )
2019-03-19 04:50:24 +11:00
Andrew Morgan
a8ad39eec7
lint
2019-03-18 17:47:39 +00:00
Andrew Morgan
621e7f37f1
Better exception handling
2019-03-18 17:45:54 +00:00
Brendan Abolivier
651ad8bc96
Add ratelimiting on failed login attempts ( #4865 )
2019-03-18 12:57:20 +00:00
Brendan Abolivier
899e523d6d
Add ratelimiting on login ( #4821 )
...
Add two ratelimiters on login (per-IP address and per-userID).
2019-03-15 17:46:16 +00:00
Richard van der Hoff
3b7ceb2c69
Merge pull request #4855 from matrix-org/rav/refactor_transaction_queue
...
Split TransactionQueue up
2019-03-15 12:32:11 +00:00
Richard van der Hoff
2dee441bdb
Merge pull request #4852 from matrix-org/rav/move_rr_sending_to_worker
...
Move client receipt processing to federation sender worker.
2019-03-15 12:30:30 +00:00
Erik Johnston
b0fa3f6ff3
Merge pull request #4853 from matrix-org/erikj/worker_docker_ci
...
Allow passing --daemonize to workers
2019-03-15 10:35:38 +00:00
Erik Johnston
271cb1998b
Revert "Make federation endpoints more tolerant of trailing slashes for some endpoints ( #4793 )"
...
This reverts commit 290552fd83
.
2019-03-14 14:30:54 +00:00
Erik Johnston
9073cfc8bd
Merge pull request #4846 from matrix-org/hawkowl/userdir-search
...
Improve searching in the userdir
2019-03-14 13:54:15 +00:00
Erik Johnston
9ad448c1e5
Correctly handle all command line options
2019-03-14 13:32:14 +00:00
Andrew Morgan
ecea5af491
Correct var name
2019-03-13 21:21:03 +00:00
Andrew Morgan
b2df0e8e2c
receiving a 400 caused an exception. handle it
2019-03-13 21:08:10 +00:00
Andrew Morgan
9a2e22fd41
is this what purgatory feels like
2019-03-13 20:29:38 +00:00
Andrew Morgan
45524f2f5e
i should have given up x2
2019-03-13 20:17:39 +00:00
Andrew Morgan
8d16ffaf7a
i should have given up
2019-03-13 20:03:10 +00:00
Richard van der Hoff
02e23b36bc
Rename and move the classes
2019-03-13 20:02:56 +00:00
Andrew Morgan
26f8e2d099
there comes a time when you should give up. but you dont
2019-03-13 19:49:08 +00:00
Andrew Morgan
5ca857ad84
as above
2019-03-13 19:35:23 +00:00
Andrew Morgan
7c0295f13c
no kwargs today
2019-03-13 19:27:10 +00:00
Andrew Morgan
66cdb840a6
Or perhaps I was the one who was drunk
2019-03-13 19:18:25 +00:00
Andrew Morgan
bec313818c
go home python, you're drunk
2019-03-13 19:10:56 +00:00
Andrew Morgan
c991e7aec7
Syntax checker is bork
2019-03-13 19:08:08 +00:00
Andrew Morgan
c2d848b80d
Destructure again
2019-03-13 19:04:43 +00:00
Andrew Morgan
ee8ba397e8
Are you happy now
2019-03-13 18:48:31 +00:00
Andrew Morgan
9dd0e34679
Syntax test
2019-03-13 18:45:17 +00:00
Andrew Morgan
220607a618
Remove testing code
2019-03-13 18:43:40 +00:00
Andrew Morgan
660b77f362
Add missing docstring detail
2019-03-13 18:38:16 +00:00
Andrew Morgan
5526b054aa
Fix syntax issues
2019-03-13 18:35:21 +00:00
Andrew Morgan
09626bfd39
Switch to wrapper function around _send_request
2019-03-13 18:26:06 +00:00
Richard van der Hoff
5d89a526f1
Factor per-destination stuff out of TransactionQueue
...
This is easier than having to have a million fields keyed on destination.
2019-03-13 17:48:29 +00:00
Erik Johnston
72bfaf746d
Allow passing --daemonize to workers
2019-03-13 17:33:54 +00:00
Richard van der Hoff
fdcad8eabd
Move client receipt processing to federation sender worker.
...
This is mostly a prerequisite for #4730 , but also fits with the general theme
of "move everything off the master that we possibly can".
2019-03-13 17:21:19 +00:00
Richard van der Hoff
eed7271b3b
declare a ReadReceipt class
...
I'm going to use this in queues and things, so it'll be useful to give it more
of a structure.
2019-03-13 17:20:55 +00:00
Andrew Morgan
7998ca3a66
Document using a certificate with a full chain ( #4849 )
2019-03-13 15:26:29 +00:00
Andrew Morgan
7d053cfe10
Retry on 400:M_UNRECOGNIZED
2019-03-13 12:10:33 +00:00
Andrew Morgan
7e75d9644b
Fix paranthesis indent
2019-03-13 11:15:23 +00:00
Andrew Morgan
cf301e37d8
Add workaround note
2019-03-13 11:14:43 +00:00
Andrew Morgan
d42c81d724
Transfer local user's push rules on room upgrade ( #4838 )
...
Transfer push rules (notifications) on room upgrade
2019-03-12 14:42:53 +00:00
Amber Brown
c0332d095f
fixup
2019-03-13 01:30:54 +11:00
Amber Brown
797b6a63fc
fixup
2019-03-13 01:17:51 +11:00
Andrew Morgan
0ea8582f8b
Cleaner way of implementing trailing slashes
2019-03-12 14:11:11 +00:00
Amber Brown
81d9d1bee6
fixup
2019-03-12 22:28:48 +11:00
Amber Brown
6f5890b2fa
fixup
2019-03-12 22:27:56 +11:00
Amber Brown
10480c4348
fixup
2019-03-12 21:47:14 +11:00
Amber Brown
8b618041ef
fixup
2019-03-12 18:06:28 +11:00
Amber Brown
c980c7e31f
use the old method
2019-03-12 17:51:14 +11:00
Aaron Raimist
8ea1b41a0e
Clarify what registration_shared_secret allows for ( #2885 ) ( #4844 )
...
* Clarify what registration_shared_secret allows for (#2885 )
Signed-off-by: Aaron Raimist <aaron@raim.ist>
* Add changelog
Signed-off-by: Aaron Raimist <aaron@raim.ist>
2019-03-11 18:21:52 +00:00
Andrew Morgan
f18dca26da
Merge branch 'develop' into anoa/trailing_slashes_client
2019-03-11 17:44:29 +00:00
Andrew Morgan
290552fd83
Make federation endpoints more tolerant of trailing slashes for some endpoints ( #4793 )
...
Server side of a solution towards #3622 .
2019-03-11 17:44:03 +00:00
Andrew Morgan
a8a028dfce
Merge branch 'develop' into anoa/trailing_slashes_client
2019-03-11 17:07:31 +00:00
Erik Johnston
78c563b77c
Correctly log expected errors when fetching server keys
2019-03-11 14:11:10 +00:00
Amber Brown
78a6b950b3
fix
2019-03-12 00:50:28 +11:00
Amber Brown
1b77bd69fb
pep8
2019-03-12 00:39:12 +11:00
Amber Brown
5ba8ceab4c
fixes
2019-03-12 00:35:31 +11:00
Amber Brown
8da22e2b53
master startup
2019-03-11 21:13:35 +11:00
Amber Brown
30a8deeb68
Merge remote-tracking branch 'origin/develop' into hawkowl/userdir-search
2019-03-11 21:12:30 +11:00
Amber Brown
4abc988c6a
initial
2019-03-11 21:11:36 +11:00
Matthew Hodgson
2326e00bc4
fix incorrect encoding of filenames with spaces in ( #2090 )
...
fixes https://github.com/vector-im/riot-web/issues/3155
2019-03-11 09:53:45 +00:00
Andrew Morgan
a5dd335cd8
lint
2019-03-08 18:25:59 +00:00
Andrew Morgan
64ff11019e
Retry certain federation requests on 404
2019-03-08 18:22:47 +00:00
Andrew Morgan
525dd02bbe
Remove trailing slashes from outbound federation requests
2019-03-08 16:55:52 +00:00
Erik Johnston
fe6c12e6cd
Add comment to schema
2019-03-08 16:38:23 +00:00
Erik Johnston
5536ddba75
Make prev_state
field optional
...
The `prev_state` field on events is not specced and so synapse shouldn't
explode if an event is missing the field.
Fixes #4787
2019-03-08 15:05:32 +00:00
Erik Johnston
8c4896668f
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/soft_fail_impl
2019-03-08 11:44:20 +00:00
Erik Johnston
0ff8163eae
Factor out soft fail checks
2019-03-08 11:26:33 +00:00
Erik Johnston
469b66c102
Merge pull request #4829 from matrix-org/erikj/device_list_seen_updates
...
When re-syncing device lists reset the state
2019-03-08 09:54:54 +00:00
Erik Johnston
d42b41544a
When re-syncing device lists reset the state
...
We keep track of what stream IDs we've seen so that we know what updates
we've handled or missed. If we re-sync we don't know if the updates
we've seen are included in the re-sync (there may be a race), so we
should reset the seen updates.
2019-03-07 16:04:24 +00:00
Erik Johnston
c633fc02d7
Add some debug logging for device list handling
2019-03-07 15:58:03 +00:00
Amber Brown
f6135d06cf
Rewrite userdir to be faster ( #4537 )
2019-03-07 01:22:53 -08:00
Travis Ralston
06cbf79fe3
Merge pull request #4779 from matrix-org/travis/icons
...
Use static locations for Riot icons
2019-03-06 15:11:41 -07:00
Erik Johnston
1f85c2c0ce
Merge pull request #4818 from matrix-org/erikj/prefill_client_ips
...
Prefill client IPs cache on workers
2019-03-06 19:17:39 +00:00
Erik Johnston
0a6e716600
Merge pull request #4815 from matrix-org/erikj/docstrings
...
Add docstrings from matrix-org-hotfixes
2019-03-06 18:59:28 +00:00
Erik Johnston
c665b637de
Merge pull request #4816 from matrix-org/erikj/4422_debug
...
Port #4422 debug logging from hotfixes
2019-03-06 18:59:12 +00:00
Erik Johnston
face0c5b3c
Prefill client IPs cache on workers
2019-03-06 17:39:32 +00:00
Erik Johnston
b879870b2d
Send message after room has been shutdown
...
Currently the explanation message is sent to the abuse room before any
users are forced joined, which means it tends to get lost in the backlog
of joins.
So instead we send the message *after* we've forced joined everyone.
2019-03-06 17:35:11 +00:00
Erik Johnston
8b7790e68f
Port #4422 debug logging from hotfixes
2019-03-06 17:29:15 +00:00
Erik Johnston
6d13bdec91
Add docstrings from matrix-org-hotfixes
2019-03-06 17:24:26 +00:00
Erik Johnston
a9de04be72
Implement soft fail
2019-03-06 16:22:16 +00:00
Andrew Morgan
7b8a157b79
Merge pull request #4792 from matrix-org/anoa/replication_tokens
...
Support batch updates in the worker sender
2019-03-06 15:48:29 +00:00
Brendan Abolivier
6f3cde8b25
Make registration ratelimiter separate from the main events one
2019-03-06 11:02:42 +00:00
Brendan Abolivier
f4195f4118
Revert "Split ratelimiters in two (one for events, one for registration)"
...
This reverts commit d7dbad3526
.
2019-03-06 10:55:22 +00:00
Brendan Abolivier
d7dbad3526
Split ratelimiters in two (one for events, one for registration)
2019-03-05 18:41:27 +00:00
Brendan Abolivier
067ce795c0
Move settings from registration to ratelimiting in config file
2019-03-05 18:03:14 +00:00
Erik Johnston
16c8b4ecbd
Merge pull request #4772 from jbweston/jbweston/server-version-api
...
Add 'server_version' endpoint to admin API
2019-03-05 16:31:00 +00:00
Brendan Abolivier
a4c3a361b7
Add rate-limiting on registration ( #4735 )
...
* Rate-limiting for registration
* Add unit test for registration rate limiting
* Add config parameters for rate limiting on auth endpoints
* Doc
* Fix doc of rate limiting function
Co-Authored-By: babolivier <contact@brendanabolivier.com>
* Incorporate review
* Fix config parsing
* Fix linting errors
* Set default config for auth rate limiting
* Fix tests
* Add changelog
* Advance reactor instead of mocked clock
* Move parameters to registration specific config and give them more sensible default values
* Remove unused config options
* Don't mock the rate limiter un MAU tests
* Rename _register_with_store into register_with_store
* Make CI happy
* Remove unused import
* Update sample config
* Fix ratelimiting test for py2
* Add non-guest test
2019-03-05 14:25:33 +00:00
Andrew Morgan
b9f6163092
Simplify token replication logic
2019-03-05 13:58:30 +00:00
Andrew Morgan
ae90531036
Merge pull request #4794 from matrix-org/anoa/erroneous_dollahs
...
Remove unnecessary dollar signs
2019-03-05 09:46:59 +00:00
Erik Johnston
b050a10871
Merge pull request #4699 from matrix-org/erikj/stop_fed_not_in_room
...
Stop backpaginating when events not visible
2019-03-05 09:32:33 +00:00
Erik Johnston
9e8bca5667
Merge pull request #4799 from matrix-org/rav/clean_up_replication_code
...
Clean ups in replication notifier
2019-03-05 09:19:48 +00:00
Erik Johnston
aa06d26ae0
clarify comments
2019-03-05 09:16:35 +00:00
Erik Johnston
c3c542bb4a
Merge pull request #4796 from matrix-org/erikj/factor_out_e2e_keys
...
Allow /keys/{changes,query} API to run on worker
2019-03-05 09:06:25 +00:00
Richard van der Hoff
c7325776a7
Remove redundant PreserveLoggingContext
...
Both (!) things that register as replication listeners do the right thing wrt
logcontexts, so this is redundant.
2019-03-04 18:31:18 +00:00
Erik Johnston
bfa7d46a10
Allow /keys/{changes,query} API to run on worker
2019-03-04 18:30:01 +00:00
Erik Johnston
157e5a8f27
Split DeviceHandler into master and worker
2019-03-04 18:29:26 +00:00
Richard van der Hoff
daa10e3e66
Remove unused wait_for_replication
method
...
I guess this was used once? It's not now, anyway.
2019-03-04 18:27:32 +00:00
Richard van der Hoff
2db49ea476
Add some debug about processing read receipts.
...
I'm hoping to establish which rooms are having lots of RRs sent for them, and
how old the events are when they are sent.
2019-03-04 18:19:40 +00:00
Richard van der Hoff
b29693a30b
Clean up read-receipt handling.
...
Remove a call to run_as_background_process: there is no need to run this as a
background process, because build_and_send_edu does not block.
We may as well inline the whole of _push_remotes.
2019-03-04 18:16:43 +00:00
Erik Johnston
a84b8d56c2
Fixup slave stores
2019-03-04 18:04:57 +00:00
Richard van der Hoff
8e28bc5eee
Include a default configuration file in the 'docs' directory. ( #4791 )
2019-03-04 17:14:58 +00:00
Erik Johnston
0d2d046709
Fix missing null guard
2019-03-04 16:04:04 +00:00
Andrew Morgan
336de1d45b
Remove unnecessary dollar signs
...
A dollar sign is already appended to the end of each PATH, so there's
no need to add one in the PATH declaration as well.
2019-03-04 15:25:12 +00:00
Andrew Morgan
fe7bd23a85
Clean up logic and add comments
2019-03-04 15:08:15 +00:00
Erik Johnston
d1523aed6b
Only check history visibility when filtering
...
When filtering events to send to server we check more than just history
visibility. However when deciding whether to backfill or not we only
care about the history visibility.
2019-03-04 14:43:42 +00:00
Andrew Morgan
9f7cdf3da1
Clearer branching, fix missing list clear
2019-03-04 14:36:52 +00:00
Andrew Morgan
5f0c449dd5
Prevent replication wedging
2019-03-04 14:03:18 +00:00
Richard van der Hoff
856c83f5f8
Avoid rebuilding Edu objects in worker mode ( #4770 )
...
In worker mode, on the federation sender, when we receive an edu for sending
over the replication socket, it is parsed into an Edu object. There is no point
extracting the contents of it so that we can then immediately build another Edu.
2019-03-04 12:57:44 +00:00
Erik Johnston
8b63fe4c26
s/get_forward_events/get_successor_events/
2019-03-04 11:56:03 +00:00
Erik Johnston
fbc047f2a5
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/stop_fed_not_in_room
2019-03-04 11:54:58 +00:00
Travis Ralston
4dfbae18fe
Use static locations for Riot icons
...
See https://github.com/vector-im/riot-web/issues/9009
2019-03-01 15:06:21 -07:00
Richard van der Hoff
3064952939
Fix incorrect log about not persisting duplicate state event. ( #4776 )
...
We were logging this when it was not true.
2019-03-01 16:47:12 +00:00
Richard van der Hoff
1beebe916f
Merge branch 'master' into develop
2019-03-01 10:58:39 +00:00
Richard van der Hoff
9ac72d9543
0.99.2
2019-03-01 10:55:44 +00:00
Joseph Weston
1e8388b311
Add 'server_version' endpoint to admin API
...
This is required because the 'Server' HTTP header is not always
passed through proxies.
2019-03-01 09:56:58 +01:00
Andrew Morgan
ac61b45a75
Minor docstring fixes for MatrixFederationAgent ( #4765 )
2019-02-28 16:24:01 +00:00
Amber Brown
b131cc77df
Make 'event_id' a required parameter in federated state requests ( #4741 )
...
* make 'event_id' a required parameter in federated state requests
As per the spec: https://matrix.org/docs/spec/server_server/r0.1.1.html#id40
Signed-off-by: Joseph Weston <joseph@weston.cloud>
* add changelog entry for bugfix
Signed-off-by: Joseph Weston <joseph@weston.cloud>
* Update server.py
2019-02-27 14:35:47 -08:00
Richard van der Hoff
68f47d6744
Fix parsing of Content-Disposition headers ( #4763 )
...
* Fix parsing of Content-Disposition headers
TIL: filenames in content-dispostion headers can contain semicolons, and aren't
%-encoded.
* fix python2 incompatibility
* Fix docstrings
2019-02-27 14:29:10 -08:00
Amber Brown
f2a753ea38
Move from TravisCI to BuildKite ( #4752 )
2019-02-27 13:03:14 -08:00
Erik Johnston
76550c58d2
Merge pull request #4759 from matrix-org/erikj/3pid_client_reader
...
Move /account/3pid to client_reader
2019-02-27 16:11:21 +00:00
Erik Johnston
8267034a63
Merge pull request #4758 from matrix-org/erikj/use_presence_replication
...
When presence is disabled don't send over replication
2019-02-27 15:46:26 +00:00
Erik Johnston
54f9ce11a7
Move /account/3pid to client_reader
2019-02-27 14:26:08 +00:00
Erik Johnston
1e315017d3
When presence is enabled don't send over replication
2019-02-27 13:53:46 +00:00
Erik Johnston
4cff9376f7
Move server key queries to federation reader
2019-02-27 13:43:53 +00:00
Erik Johnston
b183fef9ac
Update comments
2019-02-27 13:06:10 +00:00
Erik Johnston
7590e9fa28
Merge pull request #4749 from matrix-org/erikj/replication_connection_backoff
...
Fix tightloop over connecting to replication server
2019-02-27 11:00:59 +00:00
Richard van der Hoff
44a4d65586
0.99.2rc1
2019-02-27 10:48:34 +00:00
Erik Johnston
6bb1c028f1
Limit cache invalidation replication line length ( #4748 )
2019-02-27 10:28:37 +00:00
Erik Johnston
6870fc496f
Move connecting logic into ClientReplicationStreamProtocol
2019-02-27 10:23:51 +00:00
Richard van der Hoff
f191be822b
Add database version to phonehome stats. ( #4753 )
2019-02-27 10:21:49 +00:00
Erik Johnston
25814921f1
Increase the max delay between retry attempts
...
Otherwise if you have many workers they can easily take out master with
their connection attempts
2019-02-26 15:12:33 +00:00
Erik Johnston
313987187e
Fix tightloop over connecting to replication server
...
If the client failed to process incoming commands during the initial set
up of the replication connection it would immediately disconnect and
reconnect, resulting in a tightloop.
This can happen, for example, when subscribing to a stream that has a
row that is too long in the backlog.
The fix here is to not consider the connection successfully set up until
the client has succesfully subscribed and caught up with the streams.
This ensures that the retry logic timers aren't reset until then,
meaning that if an error does happen during start up the client will
continue backing off before retrying again.
2019-02-26 15:05:41 +00:00
Andrew Morgan
802884d4ee
Merge branch 'develop' of github.com:matrix-org/synapse into anoa/public_rooms_federate_develop
2019-02-26 14:23:40 +00:00
Andrew Morgan
6fcb25202f
Put function def back to the way it was
2019-02-26 14:17:38 +00:00
Andrew Morgan
7a4632af9c
Prevent showing non-fed rooms in fed /publicRooms
2019-02-26 13:37:24 +00:00
Andrew Morgan
c74624a633
Revert "Prevent showing non-fed rooms in fed /publicRooms"
2019-02-26 13:20:38 +00:00
Andrew Morgan
a1a6473293
Merge pull request #4736 from matrix-org/anoa/public_rooms_federate
...
Config option to prevent showing non-fed rooms in fed /publicRooms
2019-02-26 13:07:15 +00:00
Andrew Morgan
c4414768af
Cleaner chunk logic
2019-02-26 12:22:34 +00:00
Andrew Morgan
a712aa3a9c
Correct indent
2019-02-26 12:13:55 +00:00
Andrew Morgan
16565e67db
Correct docstring types and chunk logic
2019-02-26 12:12:48 +00:00
Andrew Morgan
40c2271680
Clean up room chunk logic
2019-02-26 12:04:34 +00:00
Andrew Morgan
6728bf3940
Make not showing non-federated rooms the default
2019-02-26 11:52:52 +00:00
Andrew Morgan
6946c20111
Result may be None
2019-02-26 11:27:19 +00:00
Andrew Morgan
71669a0fba
Address rich comments
2019-02-26 11:25:00 +00:00
Erik Johnston
899a119c2b
Don't log stack trace when client has gone away during media download ( #4738 )
...
* Don't log stack trace when client has gone away during media download
* Newsfile
* Fixup newsfile
2019-02-25 11:17:22 -08:00
Richard van der Hoff
641c409e4e
Fix ACME config for python 2. ( #4717 )
...
Fixes #4675 .
2019-02-25 11:16:33 -08:00
Matthew Hodgson
70ea2f4e1d
switch from google.com to recaptcha.net for reCAPTCHA ( #4731 )
...
* add trivial clarification about jemalloc
* switch from google.com to recaptcha.net
because https://developers.google.com/recaptcha/docs/faq#can-i-use-recaptcha-globally
2019-02-25 11:15:36 -08:00
Andrew Morgan
96c408273e
Fix group's call to generate_room_entry
2019-02-25 18:00:17 +00:00
Andrew Morgan
1330aa4a8f
elif not else if
2019-02-25 17:28:19 +00:00
Erik Johnston
65f3fbfbf7
Merge pull request #4737 from matrix-org/erikj/failure_log_tb
...
Log tracebacks correctly
2019-02-25 17:26:30 +00:00
Erik Johnston
c7b333c545
Log tracebacks correctly
2019-02-25 16:56:41 +00:00
Erik Johnston
69efe6fb16
Merge pull request #4718 from matrix-org/erikj/fix_backfill_state_shred
...
Fix backfill storing incorrect state for events
2019-02-25 16:48:52 +00:00
Erik Johnston
9c598dddcb
Fix typo
2019-02-25 16:32:02 +00:00
Erik Johnston
b1a90da82e
Merge pull request #4721 from matrix-org/erikj/msc_1866
...
MSC 1866 - Use M_UNSUPPORTED_ROOM_VERSION for invite API
2019-02-25 15:53:10 +00:00
Erik Johnston
16c7afa94c
Merge pull request #4722 from matrix-org/erikj/correctly_handle_keyring_exceptions
...
Handle errors when fetching remote server keys
2019-02-25 15:53:02 +00:00
Andrew Morgan
8aaf7ffc44
syntax derp
2019-02-25 15:27:17 +00:00
Andrew Morgan
84c0a20dfe
Simplify call to generate_room_entry
2019-02-25 15:23:27 +00:00
Erik Johnston
4b9e5076c4
Merge branch 'develop' of github.com:matrix-org/synapse into anoa/public_rooms_federate
2019-02-25 15:08:18 +00:00
Andrew Morgan
07493607a8
Docs and arg name clarification
2019-02-25 15:04:46 +00:00
Andrew Morgan
bd398b874e
Don't restrict non-fed rooms over client APIs
2019-02-25 15:04:46 +00:00
Andrew Morgan
e4b078a600
Config option to prevent showing non-fed rooms in fed /publicRooms
2019-02-25 15:04:46 +00:00
Erik Johnston
d730c2c22b
More comments
2019-02-25 14:45:02 +00:00
Erik Johnston
890cb048fd
Assert rather than clobber the values
2019-02-25 14:42:11 +00:00
Erik Johnston
65d1003d01
raise_from already raises
2019-02-25 14:34:03 +00:00
Erik Johnston
9342cc6ab1
Add comments and paranoia
2019-02-25 10:02:12 +00:00
Erik Johnston
47a7e3928d
Correctly proxy exception in frontend_proxy worker
2019-02-23 15:17:57 +00:00
Erik Johnston
41285ffe5b
Handle errors when fetching remote server keys
2019-02-23 15:09:39 +00:00
Erik Johnston
59e0112209
MSC 1866 - Use M_UNSUPPORTED_ROOM_VERSION for invite API
2019-02-23 14:50:43 +00:00
Erik Johnston
d14e94bae4
Merge pull request #4716 from matrix-org/erikj/pusher_logging
...
Fix up pusher logging a bit
2019-02-22 15:29:16 +00:00
Erik Johnston
b82c9cf462
Add missing return
2019-02-22 15:27:40 +00:00
Erik Johnston
f2891d2487
Correctly handle PusherConfigException
2019-02-22 15:18:19 +00:00
Erik Johnston
a164134a53
Drop logging level of creating a pusher
2019-02-22 14:48:06 +00:00
Erik Johnston
1d9df51ff1
Correctly handle null data in HttpPusher
2019-02-22 14:47:48 +00:00
Erik Johnston
80467bbac3
Fix state cache invalidation on workers
2019-02-22 14:38:14 +00:00
Erik Johnston
7b288826b7
Fix backfill storing incorrect state for events
2019-02-22 11:33:51 +00:00
Richard van der Hoff
e07384c4e1
Add prometheus metrics for number of badge update pushes. ( #4709 )
...
We're counting the number of push notifications, but not the number of badges;
I'd like to see if they are significant.
2019-02-22 10:57:15 +00:00
Richard van der Hoff
0abb094f1a
bail out early in on_new_receipts if no pushers ( #4706 )
2019-02-21 17:51:21 +00:00
Richard van der Hoff
6d65659b62
Run push_receipts_to_remotes as background job ( #4707 )
...
I suspect the CPU usage metrics for this are going to /dev/null at the moment.
2019-02-21 17:50:30 +00:00
Erik Johnston
b9d6756b14
Merge pull request #4263 from rkfg/develop
...
Prevent crash on pagination.
2019-02-21 17:42:15 +00:00
Erik Johnston
71b625d808
Stop backpaginating when events not visible
2019-02-20 18:14:12 +00:00
Erik Johnston
82fca11fc1
Merge pull request #4694 from matrix-org/erikj/fix_sentry_config_format
...
Fixup generated metrics config
2019-02-20 14:13:38 +00:00
Richard van der Hoff
82ca6d1f9f
Add metrics for number of outgoing EDUs, by type ( #4695 )
2019-02-20 14:13:14 +00:00
Erik Johnston
3d672fec51
Fixup generated metrics config
2019-02-20 13:39:37 +00:00
Amber Brown
a06614bd2a
UPSERT many functionality ( #4644 )
2019-02-20 23:03:30 +11:00
Richard van der Hoff
b2200a8690
Logging improvements for the pusher ( #4691 )
2019-02-20 11:36:50 +00:00
Richard van der Hoff
c88bc53903
Fix TaskStopped exceptions when outbound requests time out ( #4690 )
2019-02-20 11:35:52 +00:00