Matrix-Mirrors/synapse-product
1
0
Fork 0
mirror of https://git.anonymousland.org/anonymousland/synapse-product.git synced 2024-10-01 08:25:44 -04:00
Code Issues Packages Projects Releases Wiki Activity
13,972 Commits 4 Branches 609 Tags 60 MiB
b50641e357
Commit Graph

9886 Commits

Author SHA1 Message Date
Amber Brown
7efd1d87c2 Run black on the rest of the storage module (#4996) 2019-04-03 10:07:29 +01:00
Erik Johnston
3039d61baf
Merge pull request #4991 from matrix-org/erikj/stagger_push_startup 
Make starting pushers faster during start up
 2019-04-02 18:23:32 +01:00
Andrew Morgan
66e78700a2
Transfer related groups on room upgrade (#4990) 
Transfers the m.room.related_groups state event on room upgrade.
 2019-04-02 17:15:24 +01:00
Richard van der Hoff
8530090b16
Add config.signing_key_path. (#4974) 
As requested by @andrewshadura
 2019-04-02 16:59:27 +01:00
Erik Johnston
5bec8d660d Make starting pushers faster during start up 
We start all pushers on start up and immediately start a background
process to fetch push to send. This makes start up incredibly painful
when dealing with many pushers.

Instead, let's do a quick fast DB check to see if there *may* be push to
send and only start the background processes for those pushers. We also
stagger starting up and doing those checks so that we don't try and
handle all pushers at once.
 2019-04-02 16:59:13 +01:00
Richard van der Hoff
297bf2547e
Fix sync bug when accepting invites (#4956) 
Hopefully this time we really will fix #4422.

We need to make sure that the cache on
`get_rooms_for_user_with_stream_ordering` is invalidated *before* the
SyncHandler is notified for the new events, and we can now do so reliably via
the `events` stream.
 2019-04-02 12:42:39 +01:00
Erik Johnston
4ef5d17b96 Correctly handle id_server param 2019-04-02 11:20:09 +01:00
Erik Johnston
24232514bf Remove threepid binding if id server returns 400/404/501 2019-04-02 11:20:09 +01:00
Erik Johnston
c75e2017f1 Fixup docstrings 2019-04-02 11:20:06 +01:00
Erik Johnston
862d6e5ba5 Add unbind API to /r0 as it is now stabalised 2019-04-01 15:25:19 +01:00
Erik Johnston
3715c124b3 Grandfather in existing user threepids 
We assume, as we did before, that users bound their threepid to one of
the trusted identity servers. So we simply fill the new table with all
threepids in `user_threepids` joined with the trusted identity servers.
 2019-04-01 15:25:19 +01:00
Erik Johnston
057715aaa2 Allowing specifying IS to use in unbind API. 
By default the homeserver will use the identity server used during the
binding of the 3PID to unbind the 3PID. However, we need to allow
clients to explicitly ask the homeserver to unbind via a particular
identity server, for the case where the 3PID was bound out of band from
the homeserver.

Implements MSC915.
 2019-04-01 15:25:18 +01:00
Erik Johnston
9fbbc3d9e5 For unbind poke IS used during binding of 3PID 
This changes the behaviour from using the server specified trusted
identity server to using the IS that used during the binding of the
3PID, if known.

This is the behaviour specified by MSC1915.
 2019-04-01 15:23:30 +01:00
Erik Johnston
1666c0696a Track IS used to bind 3PIDs 
This will then be used to know which IS to default to when unbinding the
threepid.
 2019-04-01 15:23:01 +01:00
Neil Johnson
62988f73fd Merge branch 'master' into develop 2019-04-01 14:08:53 +01:00
Neil Johnson
35442efb75 0.99.3 2019-04-01 12:49:03 +00:00
Richard van der Hoff
54a87a7b08
Collect room-version variations into one place (#4969) 
Collect all the things that make room-versions different to one another into
one place, so that it's easier to define new room versions.
 2019-04-01 10:24:38 +01:00
Richard van der Hoff
902cdc63b6
Merge pull request #4955 from matrix-org/rav/merge_state_into_events 
Combine the CurrentStateDeltaStream into the EventStream
 2019-03-28 18:32:13 +00:00
Richard van der Hoff
d688a51736
Merge pull request #4954 from matrix-org/rav/refactor_parse_row 
Refactors to replication stream row update/parsing
 2019-03-28 18:31:17 +00:00
Neil Johnson
c7296bcb98
remove log line for password (#4965) 
Remove log line for password.
 2019-03-28 17:38:01 +00:00
Andrew Morgan
7a91b9d81c
Allow password providers to bind emails (#4947) 
This PR allows password provider modules to bind email addresses when a user is registering and is motivated by matrix-org/matrix-synapse-ldap3#58
 2019-03-28 15:48:07 +00:00
Erik Johnston
248014379e
Merge pull request #4942 from matrix-org/erikj/fix_presence 
Use event streams to calculate presence
 2019-03-28 14:38:31 +00:00
Erik Johnston
4e5f0f7ca0 Use an assert 2019-03-28 14:05:05 +00:00
Erik Johnston
40e56997bc Review comments 2019-03-28 13:48:41 +00:00
Richard van der Hoff
d035d62f6b Merge remote-tracking branch 'origin/develop' into rav/refactor_parse_row 2019-03-28 13:45:14 +00:00
Richard van der Hoff
4eeb2c2f07
Merge pull request #4953 from matrix-org/rav/refactor_replication_streams 
Split up replication.tcp.streams into smaller files
 2019-03-28 13:43:25 +00:00
Amber Brown
2e060774ad
Run black on some storage modules that the stats branch touches (#4959) 2019-03-29 00:37:16 +11:00
Richard van der Hoff
4b91c313a9 Combine the CurrentStateDeltaStream into the EventStream 2019-03-27 22:07:05 +00:00
Richard van der Hoff
1f6d6f918a Make EventStream rows have a type 
... as a precursor to combining it with the CurrentStateDelta stream.
 2019-03-27 22:07:05 +00:00
Richard van der Hoff
015b3622eb Skip building a ROW_TYPE when building updates 
We're about to turn it straight into a JSON object anyway so building a
ROW_TYPE is a bit pointless, and reduces flexibility in the update_function.
 2019-03-27 21:58:03 +00:00
Richard van der Hoff
f570916a3e Add parse_row method to replication stream class 
This will allow individual stream classes to override how a row is parsed.
 2019-03-27 21:32:33 +00:00
Richard van der Hoff
71dcb275f1 move FederationStream out to its own file 2019-03-27 21:13:14 +00:00
Richard van der Hoff
aa1e017864 move EventsStream out to its own file 2019-03-27 21:13:14 +00:00
Richard van der Hoff
a5798de067 Move replication.tcp.streams into a package 2019-03-27 21:13:14 +00:00
Richard van der Hoff
acaa18f7dd
Fix/improve some docstrings in the replication code. (#4949) 2019-03-27 21:12:36 +00:00
Erik Johnston
197fae1639 Use event streams to calculate presence 
Primarily this fixes a bug in the handling of remote users joining a
room where the server sent out the presence for all local users in the
room to all servers in the room.

We also change to using the state delta stream, rather than the
distributor, as it will make it easier to split processing out of the
master process (as well as being more flexible).

Finally, when sending presence states to newly joined servers we filter
out old presence states to reduce the number sent. Initially we filter
out states that are offline and have a last active more than a week ago,
though this can be changed down the line.

Fixes #3962
 2019-03-27 13:41:36 +00:00
Neil Johnson
4aa914369b bump version 2019-03-27 10:23:03 +00:00
Andrew Morgan
bbd244c7b2
Support 3PID login in password providers (#4931) 
Adds a new method, check_3pid_auth, which gives password providers
the chance to allow authentication with third-party identifiers such
as email or msisdn.
 2019-03-26 17:48:30 +00:00
Amber Brown
903f04c21f
Use the state event amount for userdir import batching, not room count (#4944) 2019-03-27 02:49:28 +11:00
Andrew Morgan
4a125be138
Make federation endpoints more tolerant of trailing slashes v2 (#4935) 
Redo of https://github.com/matrix-org/synapse/pull/4840
 2019-03-26 11:35:29 +00:00
Richard van der Hoff
8cbbedaa2b
Fix ClientReplicationStreamProtocol.__str__ (#4929) 
`__str__` depended on `self.addr`, which was absent from
ClientReplicationStreamProtocol, so attempting to call str on such an object
would raise an exception.

We can calculate the peer addr from the transport, so there is no need for addr
anyway.
 2019-03-25 16:41:51 +00:00
Richard van der Hoff
9bde730ef8
Fix bug where read-receipts lost their timestamps (#4927) 
Make sure that they are sent correctly over the replication stream.

Fixes: #4898
 2019-03-25 16:38:05 +00:00
Amber Brown
ac396a0d32
Refactor out state delta handling into its own class (#4917) 2019-03-25 20:37:08 +11:00
Erik Johnston
5fee9d8067
Merge pull request #4869 from matrix-org/erikj/yaml_load 
Fix yaml warnings by using safe_load
 2019-03-22 11:58:13 +00:00
Erik Johnston
3677548a82 Use yaml safe_load 2019-03-22 10:20:17 +00:00
Andrew Morgan
7bef97dfb7
Remove trailing slashes from outbound federation requests and retry on 400 (#4840) 
As per #3622, we remove trailing slashes from outbound federation requests. However, to ensure that we remain backwards compatible with previous versions of Synapse, if we receive a HTTP 400 with `M_UNRECOGNIZED`, then we are likely talking to an older version of Synapse in which case we retry with a trailing slash appended to the request path.
 2019-03-21 15:07:28 +00:00
Andrew Morgan
b41c2eaadc Clean up backoff_on_404 and metehod calls 2019-03-21 14:32:47 +00:00
Erik Johnston
01e6b405be
Merge pull request #4908 from matrix-org/erikj/block_peek_on_blocked_rooms 
Deny peeking into rooms that have been blocked
 2019-03-21 14:07:17 +00:00
Erik Johnston
3959858eaa
Merge pull request #4904 from matrix-org/erikj/fix_shutdown 
Fixup shutdown room API
 2019-03-21 11:24:42 +00:00
Erik Johnston
536a266520 Deny peeking into rooms that have been blocked 2019-03-21 11:20:13 +00:00
Erik Johnston
3ecec5ede2 Fix upsert 2019-03-21 10:21:15 +00:00
Erik Johnston
09f991a63d
Merge pull request #4896 from matrix-org/erikj/disable_room_directory 
Add option to disable search room lists
 2019-03-21 10:16:54 +00:00
Erik Johnston
cd62981a6a Revert spurious delete 2019-03-20 17:51:27 +00:00
Erik Johnston
8d8834d3e7 comment block_room 2019-03-20 17:49:56 +00:00
Erik Johnston
aa959a6c07 Use flags 2019-03-20 17:40:29 +00:00
Erik Johnston
7d47cc1305 Move requester check into assert_accepted_privacy_policy 2019-03-20 17:08:36 +00:00
Erik Johnston
72a14860ab Gracefully handle failing to kick user 2019-03-20 16:54:21 +00:00
Erik Johnston
6b28890543 Log new room ID 2019-03-20 16:52:28 +00:00
Erik Johnston
74c46d81fa Only require consent for events with an associated request 
There are a number of instances where a server or admin may puppet a
user to join/leave rooms, which we don't want to fail if the user has
not consented to the privacy policy. We fix this by adding a check to
test if the requester has an associated access_token, which is used as a
proxy to answer the question of whether the action is being done on
behalf of a real request from the user.
 2019-03-20 16:50:23 +00:00
Erik Johnston
67d618e111 Allow blocking a room multiple times 2019-03-20 16:50:05 +00:00
Erik Johnston
263f2c9ce1
Merge pull request #4895 from matrix-org/erikj/disable_user_search 
Add option to disable searching in the user dir
 2019-03-20 16:47:15 +00:00
Amber Brown
4d53017432
Batching in the user directory import (#4900) 2019-03-21 03:06:36 +11:00
Richard van der Hoff
cdb8036161
Add a config option for torture-testing worker replication. (#4902) 
Setting this to 50 or so makes a bunch of sytests fail in worker mode.
 2019-03-20 16:04:35 +00:00
Richard van der Hoff
a902d13180
Batch up outgoing read-receipts to reduce federation traffic. (#4890) 
Rate-limit outgoing read-receipts as per #4730.
 2019-03-20 16:02:25 +00:00
Erik Johnston
cd8c5b91ad Fix up sample config 2019-03-20 14:35:41 +00:00
Richard van der Hoff
ab20f85c59
Update synapse/config/user_directory.py 
Co-Authored-By: erikjohnston <erikj@jki.re>
 2019-03-20 14:33:11 +00:00
Erik Johnston
cc197a61a1 Disable publishing to room list when its disabled 2019-03-20 14:30:36 +00:00
Erik Johnston
2c90422146 Pull out config option 2019-03-20 14:25:58 +00:00
Erik Johnston
7529038e66 Return before we log 2019-03-20 14:25:28 +00:00
Erik Johnston
926f29ea6d Fix up config comments 2019-03-20 14:24:53 +00:00
Andrew Morgan
2150151abe kwargs doesn't like commas on calling funcs either. TIL 2019-03-20 14:13:32 +00:00
Andrew Morgan
bb52a2e653 lint 2019-03-20 14:08:57 +00:00
Andrew Morgan
cd36a1283b New test, fix issues 2019-03-20 14:00:39 +00:00
Andrew Morgan
c69df5d5d3 Fix comments. v0.99.2 -> v0.99.3 2019-03-20 11:27:18 +00:00
Andrew Morgan
551ea11559 Just return if not doing any trailing slash shennanigans 2019-03-20 11:07:36 +00:00
Erik Johnston
213c98c00a Add option to disable search room lists 
This disables both local and remote room list searching.
 2019-03-19 17:10:52 +00:00
Erik Johnston
320667a479 Add option to disable searching in the user dir 
We still populate it, as it can still be accessed via the admin API.
 2019-03-19 16:40:19 +00:00
Richard van der Hoff
13bc1e0746 Use a regular HomeServerConfig object for unit tests 
Rather than using a Mock for the homeserver config, use a genuine
HomeServerConfig object. This makes for a more realistic test, and means that
we don't have to keep remembering to add things to the mock config every time
we add a new config setting.
 2019-03-19 11:44:43 +00:00
Richard van der Hoff
45c4e19c74 Merge remote-tracking branch 'origin/develop' into HEAD 2019-03-19 11:43:46 +00:00
Neil Johnson
88f0675967 fix test_auto_create_auto_join_where_no_consent (#4886) 2019-03-19 11:38:59 +00:00
Richard van der Hoff
0dbfae03f9 Enforce hs_disabled_message correctly 
Fixes a bug where hs_disabled_message was not enforced for 3pid-based requests
if there was no server_notices_mxid configured.
 2019-03-19 11:30:54 +00:00
Erik Johnston
e9eeca1314 Fix user directory background update (#4887) 2019-03-19 22:13:53 +11:00
Richard van der Hoff
fd463b4f5d
Comment out most options in the generated config. (#4863) 
Make it so that most options in the config are optional, and commented out in
the generated config.

The reasons this is a good thing are as follows:

* If we decide that we should change the default for an option, we can do so,
  and only those admins that have deliberately chosen to override that option
  will be stuck on the old setting.

* It moves us towards a point where we can get rid of the super-surprising
  feature of synapse where the default settings for the config come from the
  generated yaml.

* It makes setting up a test config for unit testing an order of magnitude
  easier (see forthcoming PR).

* It makes the generated config more consistent, and hopefully easier for users
  to understand.
 2019-03-19 10:06:40 +00:00
Amber Brown
282c97327f
Migrate the user directory initial population to a background task (#4864) 2019-03-19 04:50:24 +11:00
Andrew Morgan
a8ad39eec7 lint 2019-03-18 17:47:39 +00:00
Andrew Morgan
621e7f37f1 Better exception handling 2019-03-18 17:45:54 +00:00
Brendan Abolivier
651ad8bc96 Add ratelimiting on failed login attempts (#4865) 2019-03-18 12:57:20 +00:00
Brendan Abolivier
899e523d6d
Add ratelimiting on login (#4821) 
Add two ratelimiters on login (per-IP address and per-userID).
 2019-03-15 17:46:16 +00:00
Richard van der Hoff
3b7ceb2c69
Merge pull request #4855 from matrix-org/rav/refactor_transaction_queue 
Split TransactionQueue up
 2019-03-15 12:32:11 +00:00
Richard van der Hoff
2dee441bdb
Merge pull request #4852 from matrix-org/rav/move_rr_sending_to_worker 
Move client receipt processing to federation sender worker.
 2019-03-15 12:30:30 +00:00
Erik Johnston
b0fa3f6ff3
Merge pull request #4853 from matrix-org/erikj/worker_docker_ci 
Allow passing --daemonize to workers
 2019-03-15 10:35:38 +00:00
Erik Johnston
271cb1998b Revert "Make federation endpoints more tolerant of trailing slashes for some endpoints (#4793)" 
This reverts commit 290552fd83.
 2019-03-14 14:30:54 +00:00
Erik Johnston
9073cfc8bd
Merge pull request #4846 from matrix-org/hawkowl/userdir-search 
Improve searching in the userdir
 2019-03-14 13:54:15 +00:00
Erik Johnston
9ad448c1e5 Correctly handle all command line options 2019-03-14 13:32:14 +00:00
Andrew Morgan
ecea5af491 Correct var name 2019-03-13 21:21:03 +00:00
Andrew Morgan
b2df0e8e2c receiving a 400 caused an exception. handle it 2019-03-13 21:08:10 +00:00
Andrew Morgan
9a2e22fd41 is this what purgatory feels like 2019-03-13 20:29:38 +00:00
Andrew Morgan
45524f2f5e i should have given up x2 2019-03-13 20:17:39 +00:00
Andrew Morgan
8d16ffaf7a i should have given up 2019-03-13 20:03:10 +00:00
Richard van der Hoff
02e23b36bc Rename and move the classes 2019-03-13 20:02:56 +00:00
Andrew Morgan
26f8e2d099 there comes a time when you should give up. but you dont 2019-03-13 19:49:08 +00:00
Andrew Morgan
5ca857ad84 as above 2019-03-13 19:35:23 +00:00
Andrew Morgan
7c0295f13c no kwargs today 2019-03-13 19:27:10 +00:00
Andrew Morgan
66cdb840a6 Or perhaps I was the one who was drunk 2019-03-13 19:18:25 +00:00
Andrew Morgan
bec313818c go home python, you're drunk 2019-03-13 19:10:56 +00:00
Andrew Morgan
c991e7aec7 Syntax checker is bork 2019-03-13 19:08:08 +00:00
Andrew Morgan
c2d848b80d Destructure again 2019-03-13 19:04:43 +00:00
Andrew Morgan
ee8ba397e8 Are you happy now 2019-03-13 18:48:31 +00:00
Andrew Morgan
9dd0e34679 Syntax test 2019-03-13 18:45:17 +00:00
Andrew Morgan
220607a618 Remove testing code 2019-03-13 18:43:40 +00:00
Andrew Morgan
660b77f362 Add missing docstring detail 2019-03-13 18:38:16 +00:00
Andrew Morgan
5526b054aa Fix syntax issues 2019-03-13 18:35:21 +00:00
Andrew Morgan
09626bfd39 Switch to wrapper function around _send_request 2019-03-13 18:26:06 +00:00
Richard van der Hoff
5d89a526f1 Factor per-destination stuff out of TransactionQueue 
This is easier than having to have a million fields keyed on destination.
 2019-03-13 17:48:29 +00:00
Erik Johnston
72bfaf746d Allow passing --daemonize to workers 2019-03-13 17:33:54 +00:00
Richard van der Hoff
fdcad8eabd Move client receipt processing to federation sender worker. 
This is mostly a prerequisite for #4730, but also fits with the general theme
of "move everything off the master that we possibly can".
 2019-03-13 17:21:19 +00:00
Richard van der Hoff
eed7271b3b declare a ReadReceipt class 
I'm going to use this in queues and things, so it'll be useful to give it more
of a structure.
 2019-03-13 17:20:55 +00:00
Andrew Morgan
7998ca3a66
Document using a certificate with a full chain (#4849) 2019-03-13 15:26:29 +00:00
Andrew Morgan
7d053cfe10 Retry on 400:M_UNRECOGNIZED 2019-03-13 12:10:33 +00:00
Andrew Morgan
7e75d9644b Fix paranthesis indent 2019-03-13 11:15:23 +00:00
Andrew Morgan
cf301e37d8 Add workaround note 2019-03-13 11:14:43 +00:00
Andrew Morgan
d42c81d724
Transfer local user's push rules on room upgrade (#4838) 
Transfer push rules (notifications) on room upgrade
 2019-03-12 14:42:53 +00:00
Amber Brown
c0332d095f fixup 2019-03-13 01:30:54 +11:00
Amber Brown
797b6a63fc fixup 2019-03-13 01:17:51 +11:00
Andrew Morgan
0ea8582f8b Cleaner way of implementing trailing slashes 2019-03-12 14:11:11 +00:00
Amber Brown
81d9d1bee6 fixup 2019-03-12 22:28:48 +11:00
Amber Brown
6f5890b2fa fixup 2019-03-12 22:27:56 +11:00
Amber Brown
10480c4348 fixup 2019-03-12 21:47:14 +11:00
Amber Brown
8b618041ef fixup 2019-03-12 18:06:28 +11:00
Amber Brown
c980c7e31f use the old method 2019-03-12 17:51:14 +11:00
Aaron Raimist
8ea1b41a0e Clarify what registration_shared_secret allows for (#2885) (#4844) 
* Clarify what registration_shared_secret allows for (#2885)

Signed-off-by: Aaron Raimist <aaron@raim.ist>

* Add changelog

Signed-off-by: Aaron Raimist <aaron@raim.ist>
 2019-03-11 18:21:52 +00:00
Andrew Morgan
f18dca26da Merge branch 'develop' into anoa/trailing_slashes_client 2019-03-11 17:44:29 +00:00
Andrew Morgan
290552fd83
Make federation endpoints more tolerant of trailing slashes for some endpoints (#4793) 
Server side of a solution towards #3622.
 2019-03-11 17:44:03 +00:00
Andrew Morgan
a8a028dfce Merge branch 'develop' into anoa/trailing_slashes_client 2019-03-11 17:07:31 +00:00
Erik Johnston
78c563b77c Correctly log expected errors when fetching server keys 2019-03-11 14:11:10 +00:00
Amber Brown
78a6b950b3 fix 2019-03-12 00:50:28 +11:00
Amber Brown
1b77bd69fb pep8 2019-03-12 00:39:12 +11:00
Amber Brown
5ba8ceab4c fixes 2019-03-12 00:35:31 +11:00
Amber Brown
8da22e2b53 master startup 2019-03-11 21:13:35 +11:00
Amber Brown
30a8deeb68 Merge remote-tracking branch 'origin/develop' into hawkowl/userdir-search 2019-03-11 21:12:30 +11:00
Amber Brown
4abc988c6a
initial 2019-03-11 21:11:36 +11:00
Matthew Hodgson
2326e00bc4 fix incorrect encoding of filenames with spaces in (#2090) 
fixes https://github.com/vector-im/riot-web/issues/3155
 2019-03-11 09:53:45 +00:00
Andrew Morgan
a5dd335cd8 lint 2019-03-08 18:25:59 +00:00
Andrew Morgan
64ff11019e Retry certain federation requests on 404 2019-03-08 18:22:47 +00:00
Andrew Morgan
525dd02bbe Remove trailing slashes from outbound federation requests 2019-03-08 16:55:52 +00:00
Erik Johnston
fe6c12e6cd Add comment to schema 2019-03-08 16:38:23 +00:00
Erik Johnston
5536ddba75 Make prev_state field optional 
The `prev_state` field on events is not specced and so synapse shouldn't
explode if an event is missing the field.

Fixes #4787
 2019-03-08 15:05:32 +00:00
Erik Johnston
8c4896668f Merge branch 'develop' of github.com:matrix-org/synapse into erikj/soft_fail_impl 2019-03-08 11:44:20 +00:00
Erik Johnston
0ff8163eae Factor out soft fail checks 2019-03-08 11:26:33 +00:00
Erik Johnston
469b66c102
Merge pull request #4829 from matrix-org/erikj/device_list_seen_updates 
When re-syncing device lists reset the state
 2019-03-08 09:54:54 +00:00