Erik Johnston
5be34fc3e3
Actually check for None rather falsey
2019-05-15 17:30:23 +01:00
Erik Johnston
e6459c26b4
Actually implement idempotency
2019-05-15 17:28:33 +01:00
Richard van der Hoff
1757e2d7c3
Merge branch 'master' into develop
2019-05-15 14:09:30 +01:00
Richard van der Hoff
4a926f528e
0.99.4
2019-05-15 13:58:45 +01:00
Erik Johnston
b50641e357
Add simple pagination API
2019-05-15 13:36:51 +01:00
Erik Johnston
efe3c7977a
Add simple send_relation API and track in DB
2019-05-15 13:36:51 +01:00
Erik Johnston
54d77107c1
Make generating SQL bounds for pagination generic
...
This will allow us to reuse the same structure when we paginate e.g.
relations
2019-05-15 11:30:05 +01:00
Erik Johnston
0aba6c8251
Merge pull request #5183 from matrix-org/erikj/async_serialize_event
...
Allow client event serialization to be async
2019-05-15 10:36:30 +01:00
Erik Johnston
d94544051b
Merge pull request #5184 from matrix-org/erikj/expose_get_events_as_array
...
Expose DataStore._get_events as get_events_as_list
2019-05-15 10:17:38 +01:00
Erik Johnston
8ed2f182f7
Update docstring with correct return type
...
Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2019-05-15 09:52:52 +01:00
Erik Johnston
52ddc6c0ed
Update docstring with correct type
...
Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2019-05-15 09:52:15 +01:00
David Baker
6ca88c4693
Only check 3pids not in use when registering
...
We checked that 3pids were not already in use before we checked if
we were going to return the account previously registered in the
same UI auth session, in which case the 3pids will definitely
be in use.
https://github.com/vector-im/riot-web/issues/9586
2019-05-14 19:04:59 +01:00
Richard van der Hoff
daa2fb6317
comment about user_joined_room
2019-05-14 18:53:09 +01:00
Erik Johnston
4fb44fb5b9
Expose DataStore._get_events as get_events_as_list
...
This is in preparation for reaction work which requires it.
2019-05-14 13:37:44 +01:00
Erik Johnston
a80e6b53f9
Newsfile
2019-05-14 13:12:23 +01:00
Erik Johnston
b54b03f9e1
Allow client event serialization to be async
2019-05-14 11:58:01 +01:00
Amber Brown
df2ebd75d3
Migrate all tests to use the dict-based config format instead of hanging items off HomeserverConfig ( #5171 )
2019-05-13 15:01:14 -05:00
Andrew Morgan
5a4b328f52
Add ability to blacklist ip ranges for federation traffic ( #5043 )
2019-05-13 19:05:06 +01:00
Brendan Abolivier
2e1129b5f7
0.99.4rc1
2019-05-13 16:11:21 +01:00
David Baker
516a5fb64b
Merge remote-tracking branch 'origin/develop' into dbkr/add_dummy_flow_to_recaptcha_only
2019-05-13 15:54:25 +01:00
David Baker
9e99143c47
Merge remote-tracking branch 'origin/develop' into dbkr/add_dummy_flow_to_recaptcha_only
2019-05-13 15:37:03 +01:00
Brendan Abolivier
bb93757b32
Fix CI after new release of isort
2019-05-13 15:19:44 +01:00
Andrew Morgan
2f48c4e1ae
URL preview blacklisting fixes ( #5155 )
...
Prevents a SynapseError being raised inside of a IResolutionReceiver and instead opts to just return 0 results. This thus means that we have to lump a failed lookup and a blacklisted lookup together with the same error message, but the substitute should be generic enough to cover both cases.
2019-05-10 10:32:44 -07:00
David Baker
04299132af
Re-order flows so that email auth is done last
...
It's more natural for the user if the bit that takes them away
from the registration flow comes last. Adding the dummy stage allows
us to do the stages in this order without the ambiguity.
2019-05-10 13:58:03 +01:00
David Baker
9c61dce3c8
Comment
2019-05-10 11:14:55 +01:00
David Baker
8714ff6d51
Add a DUMMY stage to captcha-only registration flow
...
This allows the client to complete the email last which is more
natual for the user. Without this stage, if the client would
complete the recaptcha (and terms, if enabled) stages and then the
registration request would complete because you've now completed a
flow, even if you were intending to complete the flow that's the
same except has email auth at the end.
Adding a dummy auth stage to the recaptcha-only flow means it's
always unambiguous which flow the client was trying to complete.
Longer term we should think about changing the protocol so the
client explicitly says which flow it's trying to complete.
vector-im/riot-web#9586
2019-05-10 11:09:53 +01:00
David Baker
c2bb7476c9
Revert 085ae346ac
...
Accidentally went straight to develop
2019-05-10 11:08:01 +01:00
David Baker
085ae346ac
Add a DUMMY stage to captcha-only registration flow
...
This allows the client to complete the email last which is more
natual for the user. Without this stage, if the client would
complete the recaptcha (and terms, if enabled) stages and then the
registration request would complete because you've now completed a
flow, even if you were intending to complete the flow that's the
same except has email auth at the end.
Adding a dummy auth stage to the recaptcha-only flow means it's
always unambiguous which flow the client was trying to complete.
Longer term we should think about changing the protocol so the
client explicitly says which flow it's trying to complete.
https://github.com/vector-im/riot-web/issues/9586
2019-05-10 10:52:24 +01:00
Richard van der Hoff
130f932cbc
Run black
on per_destination_queue
...
... mostly to fix pep8 fails
2019-05-09 16:27:02 +01:00
Quentin Dufour
11ea16777f
Limit the number of EDUs in transactions to 100 as expected by receiver ( #5138 )
...
Fixes #3951 .
2019-05-09 11:01:41 +01:00
Matthew Hodgson
c0e0740bef
add options to require an access_token to GET /profile and /publicRooms on CS API ( #5083 )
...
This commit adds two config options:
* `restrict_public_rooms_to_local_users`
Requires auth to fetch the public rooms directory through the CS API and disables fetching it through the federation API.
* `require_auth_for_profile_requests`
When set to `true`, requires that requests to `/profile` over the CS API are authenticated, and only returns the user's profile if the requester shares a room with the profile's owner, as per MSC1301.
MSC1301 also specifies a behaviour for federation (only returning the profile if the server asking for it shares a room with the profile's owner), but that's currently really non-trivial to do in a not too expensive way. Next step is writing down a MSC that allows a HS to specify which user sent the profile query. In this implementation, Synapse won't send a profile query over federation if it doesn't believe it already shares a room with the profile's owner, though.
Groups have been intentionally omitted from this commit.
2019-05-08 18:26:56 +01:00
Erik Johnston
c8c069db92
Merge pull request #5037 from matrix-org/erikj/limit_inflight_dns
...
Limit in flight DNS requests
2019-05-08 17:11:03 +01:00
Brendan Abolivier
1473058b5e
Do checks on aliases for incoming m.room.aliases events ( #5128 )
...
Follow-up to #5124
Also added a bunch of checks to make sure everything (both the stuff added on #5124 and this PR) works as intended.
2019-05-08 17:01:30 +01:00
Erik Johnston
de655e669a
Merge pull request #5104 from matrix-org/erikj/ratelimit_3pid_invite
...
Ratelimit 3pid invites
2019-05-07 10:12:49 +01:00
Richard van der Hoff
59e2d2694d
Remove the requirement to authenticate for /admin/server_version. ( #5122 )
...
This endpoint isn't much use for its intended purpose if you first need to get
yourself an admin's auth token.
I've restricted it to the `/_synapse/admin` path to make it a bit easier to
lock down for those concerned about exposing this information. I don't imagine
anyone is using it in anger currently.
2019-05-07 09:29:30 +01:00
Richard van der Hoff
836d3adcce
Merge branch 'master' into develop
2019-05-03 19:25:01 +01:00
Richard van der Hoff
fa21455e08
0.99.3.2
2019-05-03 18:56:24 +01:00
Richard van der Hoff
e3281d7d26
pin urllib3 to <1.25
2019-05-03 18:33:10 +01:00
Richard van der Hoff
863ec09622
0.99.3.1
2019-05-03 16:03:24 +01:00
Richard van der Hoff
a845abbf3a
Merge pull request #5134 from matrix-org/rav/url_preview_blacklist
...
Blacklist 0.0.0.0 and :: by default for URL previews
2019-05-03 15:59:20 +01:00
Richard van der Hoff
1565ebec2c
more config comment updates
2019-05-03 15:50:59 +01:00
Richard van der Hoff
1acfb9e9f0
Merge pull request #5133 from matrix-org/rav/systemrandom
...
Use SystemRandom for token generation.
2019-05-03 15:39:30 +01:00
Richard van der Hoff
1a7104fde3
Blacklist 0.0.0.0 and :: by default for URL previews
2019-05-03 15:35:49 +01:00
Richard van der Hoff
247dc1bd0b
Use SystemRandom for token generation
2019-05-03 13:02:55 +01:00
Erik Johnston
176f31c2e3
Rate limit early
2019-05-02 15:23:08 +01:00
Richard van der Hoff
12f9d51e82
Add admin api for sending server_notices ( #5121 )
2019-05-02 11:59:16 +01:00
Brendan Abolivier
c193b39134
Merge pull request #5124 from matrix-org/babolivier/aliases
...
Add some limitations to alias creation
2019-05-02 11:22:40 +01:00
Brendan Abolivier
84196cb231
Add some limitations to alias creation
2019-05-02 11:05:11 +01:00
Richard van der Hoff
0836cbb9f5
Factor out an "assert_requester_is_admin" function ( #5120 )
...
Rather than copying-and-pasting the same four lines hundreds of times
2019-05-02 10:45:52 +01:00
Richard van der Hoff
f203c98794
fix examples
2019-05-01 17:49:56 +01:00
Richard van der Hoff
40e576e29c
Move admin api impl to its own package
...
It doesn't really belong under rest/client/v1 any more.
2019-05-01 15:44:30 +01:00
Richard van der Hoff
8e9ca83537
Move admin API to a new prefix
2019-05-01 15:44:30 +01:00
Richard van der Hoff
579b637b6c
Move admin API away from ClientV1RestServlet
2019-05-01 15:16:04 +01:00
Travis Ralston
8c5b1e30d4
Add a default .m.rule.tombstone push rule ( #4867 )
...
* Add a default .m.rule.tombstone push rule
In support of MSC1930: https://github.com/matrix-org/matrix-doc/pull/1930
* changelog
* Appease the changelog linter
2019-04-29 15:40:31 -06:00
Richard van der Hoff
b31cc1c613
Merge pull request #5100 from matrix-org/rav/verification_hackery
...
Improve logging when event-signature checking fails
2019-04-29 13:19:32 +01:00
Erik Johnston
d6118c5be6
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/ratelimit_3pid_invite
2019-04-26 18:14:23 +01:00
Erik Johnston
28a81ed62f
Ratelimit 3pid invites
...
We do ratelimit sending the 3PID invite events, but that happens after
spamming the identity server.
2019-04-26 18:06:25 +01:00
Richard van der Hoff
bd0d45ca69
Fix infinite loop in presence handler
...
Fixes #5102
2019-04-26 11:14:49 +01:00
Richard van der Hoff
837d7f85a9
more logging improvements
2019-04-25 22:17:59 +01:00
Richard van der Hoff
fd8fb32bdd
remove extraneous exception logging
2019-04-25 22:02:03 +01:00
Richard van der Hoff
7ca638c761
Clarify logging when PDU signature checking fails
2019-04-25 20:55:12 +01:00
Andrew Morgan
6824ddd93d
Config option for verifying federation certificates (MSC 1711) ( #4967 )
2019-04-25 14:22:49 +01:00
Michael Kaye
788163e204
Remove log error for .well-known/matrix/client ( #4972 )
2019-04-24 17:44:06 +01:00
Katie Wolfe
7e07dc429f
Lint
...
I probably should've just run autopep8 in the first place...
Signed-off-by: Katie Wolfe <katie@dnaf.moe>
2019-04-24 12:43:18 -04:00
Amber Brown
6b2b9a58c4
Prevent "producer not unregistered" message ( #5009 )
2019-04-24 17:37:32 +01:00
Katie Wolfe
b3e5db402d
Clean up code
...
Signed-off-by: Katie Wolfe <katie@dnaf.moe>
2019-04-24 12:26:55 -04:00
Katie Wolfe
5d3ed79944
Show heroes if room name or canonical alias are empty
...
Fixes #4194
Signed-off-by: Katie Wolfe <katie@dnaf.moe>
2019-04-24 12:26:53 -04:00
Katie Wolfe
7f025eb425
Show heroes if room name or canonical alias are empty
...
Fixes #4194
Signed-off-by: Katie Wolfe <katie@dnaf.moe>
2019-04-24 12:26:29 -04:00
Brendan Abolivier
f8826d31cd
Don't crash on lack of expiry templates
2019-04-18 14:50:05 +01:00
Erik Johnston
ca90336a69
Merge branch 'develop' of github.com:matrix-org/synapse into babolivier/account_expiration
2019-04-17 19:44:40 +01:00
Brendan Abolivier
eaf41a943b
Add management endpoints for account validity
2019-04-17 19:34:45 +01:00
Brendan Abolivier
91934025b9
Merge pull request #5047 from matrix-org/babolivier/account_expiration
...
Send out emails with links to extend an account's validity period
2019-04-17 14:57:39 +01:00
Brendan Abolivier
20f0617e87
Send out emails with links to extend an account's validity period
2019-04-17 14:42:20 +01:00
Brendan Abolivier
49ff74da9b
Merge pull request #5071 from matrix-org/babolivier/3pid-check
...
Make sure we're not registering the same 3pid twice
2019-04-17 14:37:42 +01:00
Brendan Abolivier
600ec04739
Make sure we're not registering the same 3pid twice
2019-04-17 14:23:01 +01:00
Erik Johnston
fd2fcb817c
Merge pull request #5070 from matrix-org/erikj/postpath
...
Remove usage of request.postpath
2019-04-17 09:03:26 +01:00
Erik Johnston
a1eb4c6d2f
Merge pull request #5065 from matrix-org/erikj/fix_versions
...
VersionRestServlet doesn't take a param
2019-04-16 17:52:36 +01:00
Erik Johnston
ad010f6306
Remove usage of request.postpath
...
This is an undocumented variable in twisted, and relies on the servlet
being mounted in the right way.
This also breaks getting push rules on workers.
2019-04-16 17:42:50 +01:00
Silke Hofstra
a137f4eac0
Add systemd-python to optional dependencies ( #4339 )
...
Using systemd-python allows for logging to the systemd journal,
as is documented in: `synapse/contrib/systemd/log_config.yaml`.
Signed-off-by: Silke Hofstra <silke@slxh.eu>
2019-04-16 20:41:17 +10:00
Erik Johnston
38642614cf
VersionRestServlet doesn't take a param
2019-04-15 19:39:47 +01:00
Erik Johnston
6e27a8620f
Merge pull request #5063 from matrix-org/erikj/move_endpoints
...
Move some rest endpoints to client reader
2019-04-15 18:55:01 +01:00
Erik Johnston
ec638a1602
Only handle GET requests for /push_rules
2019-04-15 18:51:48 +01:00
Erik Johnston
d5adf297e6
Move some rest endpoints to client reader
2019-04-15 17:21:03 +01:00
Andrew Morgan
caa76e6021
Remove periods from copyright headers ( #5046 )
2019-04-11 17:08:13 +01:00
Erik Johnston
e446921def
Merge pull request #5033 from matrix-org/erikj/fix_schema_delta
...
Fix schema upgrade when dropping tables
2019-04-10 10:22:35 +01:00
Richard van der Hoff
329688c161
Fix disappearing exceptions in manhole. ( #5035 )
...
Avoid sending syntax errors from the manhole to sentry.
2019-04-10 07:23:48 +01:00
Erik Johnston
a0fc256d65
Limit in flight DNS requests
...
This is to work around a bug in twisted where a large number of
concurrent DNS requests cause it to tight loop forever.
c.f. https://twistedmatrix.com/trac/ticket/9620#ticket
2019-04-09 17:23:42 +01:00
Brendan Abolivier
bfc8fdf1fc
Merge pull request #5027 from matrix-org/babolivier/account_expiration
...
Add time-based account expiration
2019-04-09 17:02:41 +01:00
Brendan Abolivier
747aa9f8ca
Add account expiration feature
2019-04-09 16:46:04 +01:00
Erik Johnston
50d2a3059d
Fix schema upgrade when dropping tables
...
We need to drop tables in the correct order due to foreign table
constraints (on `application_services`), otherwise the DROP TABLE
command will fail.
Introduced in #4992 .
2019-04-09 14:39:18 +01:00
Richard van der Hoff
644b86677f
Merge pull request #5030 from matrix-org/rav/rewrite_g_s_v_k
...
Rewrite Datastore.get_server_verify_keys
2019-04-09 11:30:23 +01:00
Richard van der Hoff
4abf5aa81a
Bump psycopg requirement ( #5032 )
2019-04-09 11:29:50 +01:00
Richard van der Hoff
18b69be00f
Rewrite Datastore.get_server_verify_keys
...
Rewrite this so that it doesn't hammer the database.
2019-04-09 00:00:10 +01:00
Richard van der Hoff
f50efcb65d
Replace SlavedKeyStore with a shim
...
since we're pulling everything out of KeyStore anyway, we may as well simplify
it.
2019-04-08 23:59:07 +01:00
Richard van der Hoff
f88a9e6323
Remove redundant merged_keys dict
...
There's no point in collecting a merged dict of keys: it is sufficient to
consider just the new keys which have been fetched by the most recent
key_fetch_fns.
2019-04-08 22:36:18 +01:00
Richard van der Hoff
3352baac4b
Remove unused server_tls_certificates functions ( #5028 )
...
These have been unused since #4120 , and with the demise of perspectives, it is
unlikely that they will ever be used again.
2019-04-08 21:50:18 +01:00
Neil Johnson
b25e387c0d
add context to phonehome stats ( #5020 )
...
add context to phonehome stats
2019-04-08 15:47:39 +01:00
Richard van der Hoff
67d7b44784
Merge pull request #5024 from matrix-org/rav/record_correct_server_in_serverkeys
...
Fix from_server buglet in get_keys_from_perspectives
2019-04-08 15:40:37 +01:00
Neil Johnson
2d951686a7
drop tables listed in #1830 ( #4992 )
...
Tables dropped:
* application_services,
* application_services_regex,
* transaction_id_to_pdu,
* stats_reporting
* current_state_resets
* event_content_hashes
* event_destinations
* event_edge_hashes
* event_signatures
* feedback
* room_hosts
* state_forward_extremities
2019-04-08 15:37:26 +01:00
Richard van der Hoff
7d2a0c848e
Fix from_server buglet in get_keys_from_perspectives
...
make sure we store the name of the server the keys came from, rather than the
origin server, after doing a fetch-from-perspectives.
2019-04-08 12:51:16 +01:00
Richard van der Hoff
7fc1e17f4c
Merge pull request #5001 from matrix-org/rav/keyring_cleanups
...
Cleanups in the Keyring
2019-04-08 12:47:09 +01:00
Richard van der Hoff
6ae9361510
Hoist server_name check out of process_v2_response
...
It's easier to check it in the caller than to complicate the interface with an
extra param.
2019-04-04 19:12:54 +01:00
Richard van der Hoff
ef27d434d1
Clean up Keyring.process_v2_response
...
Make this just return the key dict, rather than a single-entry dict mapping the
server name to the key dict. It's easy for the caller to get the server name
from from the response object anyway.
2019-04-04 19:12:54 +01:00
Richard van der Hoff
b43d9a920b
Fix docstring on get_server_keys_json
2019-04-04 18:54:03 +01:00
Brendan Abolivier
8e85493b0c
Add config option to block users from looking up 3PIDs ( #5010 )
2019-04-04 17:25:47 +01:00
Amber Brown
a33a5abc4c
Clean up the database pagination code ( #5007 )
...
* rewrite & simplify
* changelog
* cleanup potential sql injection
2019-04-05 00:21:16 +11:00
Erik Johnston
616e6a10bd
Merge pull request #5002 from matrix-org/erikj/delete_group
...
Add delete group admin API
2019-04-04 14:15:41 +01:00
Andrew Morgan
db265f0642
Prevent kicking users who aren't in the room ( #4999 )
...
Prevent kick events from succeeding if the user is not currently in the room.
2019-04-04 13:05:51 +01:00
Marcel Krüger
9f5d206c4a
Avoid redundant URL encoding ( #4555 )
...
* Do not double encode fallback redirect URL
Signed-off-by: Marcel Fabian Krüger <zauguin@gmail.com>
2019-04-04 12:05:56 +01:00
Erik Johnston
c192bf8970
Add admin API for group deletion
2019-04-03 16:29:52 +01:00
Erik Johnston
4a2e13631d
Add functions to delete a group
2019-04-03 16:29:52 +01:00
Andrew Morgan
4a4d5c4fd6
Fix grammar and document get_current_users_in_room ( #4998 )
2019-04-03 14:32:20 +01:00
Neil Johnson
e8419554ff
Remove presence lists ( #4989 )
...
Remove presence list support as per MSC 1819
2019-04-03 11:11:15 +01:00
Erik Johnston
8f549c1177
Merge pull request #4982 from matrix-org/erikj/msc1915
...
Implement MSC1915 - 3PID unbind APIs
2019-04-03 11:07:09 +01:00
Amber Brown
7efd1d87c2
Run black on the rest of the storage module ( #4996 )
2019-04-03 10:07:29 +01:00
Erik Johnston
3039d61baf
Merge pull request #4991 from matrix-org/erikj/stagger_push_startup
...
Make starting pushers faster during start up
2019-04-02 18:23:32 +01:00
Andrew Morgan
66e78700a2
Transfer related groups on room upgrade ( #4990 )
...
Transfers the m.room.related_groups state event on room upgrade.
2019-04-02 17:15:24 +01:00
Richard van der Hoff
8530090b16
Add config.signing_key_path. ( #4974 )
...
As requested by @andrewshadura
2019-04-02 16:59:27 +01:00
Erik Johnston
5bec8d660d
Make starting pushers faster during start up
...
We start all pushers on start up and immediately start a background
process to fetch push to send. This makes start up incredibly painful
when dealing with many pushers.
Instead, let's do a quick fast DB check to see if there *may* be push to
send and only start the background processes for those pushers. We also
stagger starting up and doing those checks so that we don't try and
handle all pushers at once.
2019-04-02 16:59:13 +01:00
Richard van der Hoff
297bf2547e
Fix sync bug when accepting invites ( #4956 )
...
Hopefully this time we really will fix #4422 .
We need to make sure that the cache on
`get_rooms_for_user_with_stream_ordering` is invalidated *before* the
SyncHandler is notified for the new events, and we can now do so reliably via
the `events` stream.
2019-04-02 12:42:39 +01:00
Erik Johnston
4ef5d17b96
Correctly handle id_server param
2019-04-02 11:20:09 +01:00
Erik Johnston
24232514bf
Remove threepid binding if id server returns 400/404/501
2019-04-02 11:20:09 +01:00
Erik Johnston
c75e2017f1
Fixup docstrings
2019-04-02 11:20:06 +01:00
Erik Johnston
862d6e5ba5
Add unbind API to /r0 as it is now stabalised
2019-04-01 15:25:19 +01:00
Erik Johnston
3715c124b3
Grandfather in existing user threepids
...
We assume, as we did before, that users bound their threepid to one of
the trusted identity servers. So we simply fill the new table with all
threepids in `user_threepids` joined with the trusted identity servers.
2019-04-01 15:25:19 +01:00
Erik Johnston
057715aaa2
Allowing specifying IS to use in unbind API.
...
By default the homeserver will use the identity server used during the
binding of the 3PID to unbind the 3PID. However, we need to allow
clients to explicitly ask the homeserver to unbind via a particular
identity server, for the case where the 3PID was bound out of band from
the homeserver.
Implements MSC915.
2019-04-01 15:25:18 +01:00
Erik Johnston
9fbbc3d9e5
For unbind poke IS used during binding of 3PID
...
This changes the behaviour from using the server specified trusted
identity server to using the IS that used during the binding of the
3PID, if known.
This is the behaviour specified by MSC1915.
2019-04-01 15:23:30 +01:00
Erik Johnston
1666c0696a
Track IS used to bind 3PIDs
...
This will then be used to know which IS to default to when unbinding the
threepid.
2019-04-01 15:23:01 +01:00
Neil Johnson
62988f73fd
Merge branch 'master' into develop
2019-04-01 14:08:53 +01:00
Neil Johnson
35442efb75
0.99.3
2019-04-01 12:49:03 +00:00
Richard van der Hoff
54a87a7b08
Collect room-version variations into one place ( #4969 )
...
Collect all the things that make room-versions different to one another into
one place, so that it's easier to define new room versions.
2019-04-01 10:24:38 +01:00
Richard van der Hoff
902cdc63b6
Merge pull request #4955 from matrix-org/rav/merge_state_into_events
...
Combine the CurrentStateDeltaStream into the EventStream
2019-03-28 18:32:13 +00:00
Richard van der Hoff
d688a51736
Merge pull request #4954 from matrix-org/rav/refactor_parse_row
...
Refactors to replication stream row update/parsing
2019-03-28 18:31:17 +00:00
Neil Johnson
c7296bcb98
remove log line for password ( #4965 )
...
Remove log line for password.
2019-03-28 17:38:01 +00:00
Andrew Morgan
7a91b9d81c
Allow password providers to bind emails ( #4947 )
...
This PR allows password provider modules to bind email addresses when a user is registering and is motivated by matrix-org/matrix-synapse-ldap3#58
2019-03-28 15:48:07 +00:00
Erik Johnston
248014379e
Merge pull request #4942 from matrix-org/erikj/fix_presence
...
Use event streams to calculate presence
2019-03-28 14:38:31 +00:00
Erik Johnston
4e5f0f7ca0
Use an assert
2019-03-28 14:05:05 +00:00
Erik Johnston
40e56997bc
Review comments
2019-03-28 13:48:41 +00:00
Richard van der Hoff
d035d62f6b
Merge remote-tracking branch 'origin/develop' into rav/refactor_parse_row
2019-03-28 13:45:14 +00:00
Richard van der Hoff
4eeb2c2f07
Merge pull request #4953 from matrix-org/rav/refactor_replication_streams
...
Split up replication.tcp.streams into smaller files
2019-03-28 13:43:25 +00:00
Amber Brown
2e060774ad
Run black
on some storage modules that the stats branch touches ( #4959 )
2019-03-29 00:37:16 +11:00
Richard van der Hoff
4b91c313a9
Combine the CurrentStateDeltaStream into the EventStream
2019-03-27 22:07:05 +00:00
Richard van der Hoff
1f6d6f918a
Make EventStream rows have a type
...
... as a precursor to combining it with the CurrentStateDelta stream.
2019-03-27 22:07:05 +00:00
Richard van der Hoff
015b3622eb
Skip building a ROW_TYPE when building updates
...
We're about to turn it straight into a JSON object anyway so building a
ROW_TYPE is a bit pointless, and reduces flexibility in the update_function.
2019-03-27 21:58:03 +00:00
Richard van der Hoff
f570916a3e
Add parse_row method to replication stream class
...
This will allow individual stream classes to override how a row is parsed.
2019-03-27 21:32:33 +00:00
Richard van der Hoff
71dcb275f1
move FederationStream out to its own file
2019-03-27 21:13:14 +00:00
Richard van der Hoff
aa1e017864
move EventsStream out to its own file
2019-03-27 21:13:14 +00:00
Richard van der Hoff
a5798de067
Move replication.tcp.streams into a package
2019-03-27 21:13:14 +00:00
Richard van der Hoff
acaa18f7dd
Fix/improve some docstrings in the replication code. ( #4949 )
2019-03-27 21:12:36 +00:00
Erik Johnston
197fae1639
Use event streams to calculate presence
...
Primarily this fixes a bug in the handling of remote users joining a
room where the server sent out the presence for all local users in the
room to all servers in the room.
We also change to using the state delta stream, rather than the
distributor, as it will make it easier to split processing out of the
master process (as well as being more flexible).
Finally, when sending presence states to newly joined servers we filter
out old presence states to reduce the number sent. Initially we filter
out states that are offline and have a last active more than a week ago,
though this can be changed down the line.
Fixes #3962
2019-03-27 13:41:36 +00:00
Neil Johnson
4aa914369b
bump version
2019-03-27 10:23:03 +00:00
Andrew Morgan
bbd244c7b2
Support 3PID login in password providers ( #4931 )
...
Adds a new method, check_3pid_auth, which gives password providers
the chance to allow authentication with third-party identifiers such
as email or msisdn.
2019-03-26 17:48:30 +00:00
Amber Brown
903f04c21f
Use the state event amount for userdir import batching, not room count ( #4944 )
2019-03-27 02:49:28 +11:00
Andrew Morgan
4a125be138
Make federation endpoints more tolerant of trailing slashes v2 ( #4935 )
...
Redo of https://github.com/matrix-org/synapse/pull/4840
2019-03-26 11:35:29 +00:00
Richard van der Hoff
8cbbedaa2b
Fix ClientReplicationStreamProtocol.__str__ ( #4929 )
...
`__str__` depended on `self.addr`, which was absent from
ClientReplicationStreamProtocol, so attempting to call str on such an object
would raise an exception.
We can calculate the peer addr from the transport, so there is no need for addr
anyway.
2019-03-25 16:41:51 +00:00
Richard van der Hoff
9bde730ef8
Fix bug where read-receipts lost their timestamps ( #4927 )
...
Make sure that they are sent correctly over the replication stream.
Fixes : #4898
2019-03-25 16:38:05 +00:00
Amber Brown
ac396a0d32
Refactor out state delta handling into its own class ( #4917 )
2019-03-25 20:37:08 +11:00
Erik Johnston
5fee9d8067
Merge pull request #4869 from matrix-org/erikj/yaml_load
...
Fix yaml warnings by using safe_load
2019-03-22 11:58:13 +00:00
Erik Johnston
3677548a82
Use yaml safe_load
2019-03-22 10:20:17 +00:00
Andrew Morgan
7bef97dfb7
Remove trailing slashes from outbound federation requests and retry on 400 ( #4840 )
...
As per #3622 , we remove trailing slashes from outbound federation requests. However, to ensure that we remain backwards compatible with previous versions of Synapse, if we receive a HTTP 400 with `M_UNRECOGNIZED`, then we are likely talking to an older version of Synapse in which case we retry with a trailing slash appended to the request path.
2019-03-21 15:07:28 +00:00
Andrew Morgan
b41c2eaadc
Clean up backoff_on_404 and metehod calls
2019-03-21 14:32:47 +00:00
Erik Johnston
01e6b405be
Merge pull request #4908 from matrix-org/erikj/block_peek_on_blocked_rooms
...
Deny peeking into rooms that have been blocked
2019-03-21 14:07:17 +00:00
Erik Johnston
3959858eaa
Merge pull request #4904 from matrix-org/erikj/fix_shutdown
...
Fixup shutdown room API
2019-03-21 11:24:42 +00:00
Erik Johnston
536a266520
Deny peeking into rooms that have been blocked
2019-03-21 11:20:13 +00:00
Erik Johnston
3ecec5ede2
Fix upsert
2019-03-21 10:21:15 +00:00
Erik Johnston
09f991a63d
Merge pull request #4896 from matrix-org/erikj/disable_room_directory
...
Add option to disable search room lists
2019-03-21 10:16:54 +00:00
Erik Johnston
cd62981a6a
Revert spurious delete
2019-03-20 17:51:27 +00:00
Erik Johnston
8d8834d3e7
comment block_room
2019-03-20 17:49:56 +00:00
Erik Johnston
aa959a6c07
Use flags
2019-03-20 17:40:29 +00:00
Erik Johnston
7d47cc1305
Move requester check into assert_accepted_privacy_policy
2019-03-20 17:08:36 +00:00
Erik Johnston
72a14860ab
Gracefully handle failing to kick user
2019-03-20 16:54:21 +00:00
Erik Johnston
6b28890543
Log new room ID
2019-03-20 16:52:28 +00:00
Erik Johnston
74c46d81fa
Only require consent for events with an associated request
...
There are a number of instances where a server or admin may puppet a
user to join/leave rooms, which we don't want to fail if the user has
not consented to the privacy policy. We fix this by adding a check to
test if the requester has an associated access_token, which is used as a
proxy to answer the question of whether the action is being done on
behalf of a real request from the user.
2019-03-20 16:50:23 +00:00
Erik Johnston
67d618e111
Allow blocking a room multiple times
2019-03-20 16:50:05 +00:00
Erik Johnston
263f2c9ce1
Merge pull request #4895 from matrix-org/erikj/disable_user_search
...
Add option to disable searching in the user dir
2019-03-20 16:47:15 +00:00
Amber Brown
4d53017432
Batching in the user directory import ( #4900 )
2019-03-21 03:06:36 +11:00
Richard van der Hoff
cdb8036161
Add a config option for torture-testing worker replication. ( #4902 )
...
Setting this to 50 or so makes a bunch of sytests fail in worker mode.
2019-03-20 16:04:35 +00:00
Richard van der Hoff
a902d13180
Batch up outgoing read-receipts to reduce federation traffic. ( #4890 )
...
Rate-limit outgoing read-receipts as per #4730 .
2019-03-20 16:02:25 +00:00
Erik Johnston
cd8c5b91ad
Fix up sample config
2019-03-20 14:35:41 +00:00
Richard van der Hoff
ab20f85c59
Update synapse/config/user_directory.py
...
Co-Authored-By: erikjohnston <erikj@jki.re>
2019-03-20 14:33:11 +00:00
Erik Johnston
cc197a61a1
Disable publishing to room list when its disabled
2019-03-20 14:30:36 +00:00
Erik Johnston
2c90422146
Pull out config option
2019-03-20 14:25:58 +00:00
Erik Johnston
7529038e66
Return before we log
2019-03-20 14:25:28 +00:00
Erik Johnston
926f29ea6d
Fix up config comments
2019-03-20 14:24:53 +00:00
Andrew Morgan
2150151abe
kwargs doesn't like commas on calling funcs either. TIL
2019-03-20 14:13:32 +00:00
Andrew Morgan
bb52a2e653
lint
2019-03-20 14:08:57 +00:00
Andrew Morgan
cd36a1283b
New test, fix issues
2019-03-20 14:00:39 +00:00
Andrew Morgan
c69df5d5d3
Fix comments. v0.99.2 -> v0.99.3
2019-03-20 11:27:18 +00:00
Andrew Morgan
551ea11559
Just return if not doing any trailing slash shennanigans
2019-03-20 11:07:36 +00:00
Erik Johnston
213c98c00a
Add option to disable search room lists
...
This disables both local and remote room list searching.
2019-03-19 17:10:52 +00:00
Erik Johnston
320667a479
Add option to disable searching in the user dir
...
We still populate it, as it can still be accessed via the admin API.
2019-03-19 16:40:19 +00:00
Richard van der Hoff
13bc1e0746
Use a regular HomeServerConfig object for unit tests
...
Rather than using a Mock for the homeserver config, use a genuine
HomeServerConfig object. This makes for a more realistic test, and means that
we don't have to keep remembering to add things to the mock config every time
we add a new config setting.
2019-03-19 11:44:43 +00:00
Richard van der Hoff
45c4e19c74
Merge remote-tracking branch 'origin/develop' into HEAD
2019-03-19 11:43:46 +00:00
Neil Johnson
88f0675967
fix test_auto_create_auto_join_where_no_consent ( #4886 )
2019-03-19 11:38:59 +00:00
Richard van der Hoff
0dbfae03f9
Enforce hs_disabled_message correctly
...
Fixes a bug where hs_disabled_message was not enforced for 3pid-based requests
if there was no server_notices_mxid configured.
2019-03-19 11:30:54 +00:00
Erik Johnston
e9eeca1314
Fix user directory background update ( #4887 )
2019-03-19 22:13:53 +11:00
Richard van der Hoff
fd463b4f5d
Comment out most options in the generated config. ( #4863 )
...
Make it so that most options in the config are optional, and commented out in
the generated config.
The reasons this is a good thing are as follows:
* If we decide that we should change the default for an option, we can do so,
and only those admins that have deliberately chosen to override that option
will be stuck on the old setting.
* It moves us towards a point where we can get rid of the super-surprising
feature of synapse where the default settings for the config come from the
generated yaml.
* It makes setting up a test config for unit testing an order of magnitude
easier (see forthcoming PR).
* It makes the generated config more consistent, and hopefully easier for users
to understand.
2019-03-19 10:06:40 +00:00
Amber Brown
282c97327f
Migrate the user directory initial population to a background task ( #4864 )
2019-03-19 04:50:24 +11:00
Andrew Morgan
a8ad39eec7
lint
2019-03-18 17:47:39 +00:00
Andrew Morgan
621e7f37f1
Better exception handling
2019-03-18 17:45:54 +00:00
Brendan Abolivier
651ad8bc96
Add ratelimiting on failed login attempts ( #4865 )
2019-03-18 12:57:20 +00:00
Brendan Abolivier
899e523d6d
Add ratelimiting on login ( #4821 )
...
Add two ratelimiters on login (per-IP address and per-userID).
2019-03-15 17:46:16 +00:00
Richard van der Hoff
3b7ceb2c69
Merge pull request #4855 from matrix-org/rav/refactor_transaction_queue
...
Split TransactionQueue up
2019-03-15 12:32:11 +00:00
Richard van der Hoff
2dee441bdb
Merge pull request #4852 from matrix-org/rav/move_rr_sending_to_worker
...
Move client receipt processing to federation sender worker.
2019-03-15 12:30:30 +00:00
Erik Johnston
b0fa3f6ff3
Merge pull request #4853 from matrix-org/erikj/worker_docker_ci
...
Allow passing --daemonize to workers
2019-03-15 10:35:38 +00:00
Erik Johnston
271cb1998b
Revert "Make federation endpoints more tolerant of trailing slashes for some endpoints ( #4793 )"
...
This reverts commit 290552fd83
.
2019-03-14 14:30:54 +00:00
Erik Johnston
9073cfc8bd
Merge pull request #4846 from matrix-org/hawkowl/userdir-search
...
Improve searching in the userdir
2019-03-14 13:54:15 +00:00
Erik Johnston
9ad448c1e5
Correctly handle all command line options
2019-03-14 13:32:14 +00:00
Andrew Morgan
ecea5af491
Correct var name
2019-03-13 21:21:03 +00:00
Andrew Morgan
b2df0e8e2c
receiving a 400 caused an exception. handle it
2019-03-13 21:08:10 +00:00
Andrew Morgan
9a2e22fd41
is this what purgatory feels like
2019-03-13 20:29:38 +00:00
Andrew Morgan
45524f2f5e
i should have given up x2
2019-03-13 20:17:39 +00:00
Andrew Morgan
8d16ffaf7a
i should have given up
2019-03-13 20:03:10 +00:00
Richard van der Hoff
02e23b36bc
Rename and move the classes
2019-03-13 20:02:56 +00:00
Andrew Morgan
26f8e2d099
there comes a time when you should give up. but you dont
2019-03-13 19:49:08 +00:00
Andrew Morgan
5ca857ad84
as above
2019-03-13 19:35:23 +00:00
Andrew Morgan
7c0295f13c
no kwargs today
2019-03-13 19:27:10 +00:00
Andrew Morgan
66cdb840a6
Or perhaps I was the one who was drunk
2019-03-13 19:18:25 +00:00
Andrew Morgan
bec313818c
go home python, you're drunk
2019-03-13 19:10:56 +00:00
Andrew Morgan
c991e7aec7
Syntax checker is bork
2019-03-13 19:08:08 +00:00
Andrew Morgan
c2d848b80d
Destructure again
2019-03-13 19:04:43 +00:00
Andrew Morgan
ee8ba397e8
Are you happy now
2019-03-13 18:48:31 +00:00
Andrew Morgan
9dd0e34679
Syntax test
2019-03-13 18:45:17 +00:00
Andrew Morgan
220607a618
Remove testing code
2019-03-13 18:43:40 +00:00
Andrew Morgan
660b77f362
Add missing docstring detail
2019-03-13 18:38:16 +00:00
Andrew Morgan
5526b054aa
Fix syntax issues
2019-03-13 18:35:21 +00:00
Andrew Morgan
09626bfd39
Switch to wrapper function around _send_request
2019-03-13 18:26:06 +00:00
Richard van der Hoff
5d89a526f1
Factor per-destination stuff out of TransactionQueue
...
This is easier than having to have a million fields keyed on destination.
2019-03-13 17:48:29 +00:00
Erik Johnston
72bfaf746d
Allow passing --daemonize to workers
2019-03-13 17:33:54 +00:00
Richard van der Hoff
fdcad8eabd
Move client receipt processing to federation sender worker.
...
This is mostly a prerequisite for #4730 , but also fits with the general theme
of "move everything off the master that we possibly can".
2019-03-13 17:21:19 +00:00
Richard van der Hoff
eed7271b3b
declare a ReadReceipt class
...
I'm going to use this in queues and things, so it'll be useful to give it more
of a structure.
2019-03-13 17:20:55 +00:00
Andrew Morgan
7998ca3a66
Document using a certificate with a full chain ( #4849 )
2019-03-13 15:26:29 +00:00
Andrew Morgan
7d053cfe10
Retry on 400:M_UNRECOGNIZED
2019-03-13 12:10:33 +00:00
Andrew Morgan
7e75d9644b
Fix paranthesis indent
2019-03-13 11:15:23 +00:00
Andrew Morgan
cf301e37d8
Add workaround note
2019-03-13 11:14:43 +00:00
Andrew Morgan
d42c81d724
Transfer local user's push rules on room upgrade ( #4838 )
...
Transfer push rules (notifications) on room upgrade
2019-03-12 14:42:53 +00:00
Amber Brown
c0332d095f
fixup
2019-03-13 01:30:54 +11:00
Amber Brown
797b6a63fc
fixup
2019-03-13 01:17:51 +11:00
Andrew Morgan
0ea8582f8b
Cleaner way of implementing trailing slashes
2019-03-12 14:11:11 +00:00
Amber Brown
81d9d1bee6
fixup
2019-03-12 22:28:48 +11:00
Amber Brown
6f5890b2fa
fixup
2019-03-12 22:27:56 +11:00
Amber Brown
10480c4348
fixup
2019-03-12 21:47:14 +11:00
Amber Brown
8b618041ef
fixup
2019-03-12 18:06:28 +11:00
Amber Brown
c980c7e31f
use the old method
2019-03-12 17:51:14 +11:00
Aaron Raimist
8ea1b41a0e
Clarify what registration_shared_secret allows for ( #2885 ) ( #4844 )
...
* Clarify what registration_shared_secret allows for (#2885 )
Signed-off-by: Aaron Raimist <aaron@raim.ist>
* Add changelog
Signed-off-by: Aaron Raimist <aaron@raim.ist>
2019-03-11 18:21:52 +00:00
Andrew Morgan
f18dca26da
Merge branch 'develop' into anoa/trailing_slashes_client
2019-03-11 17:44:29 +00:00
Andrew Morgan
290552fd83
Make federation endpoints more tolerant of trailing slashes for some endpoints ( #4793 )
...
Server side of a solution towards #3622 .
2019-03-11 17:44:03 +00:00
Andrew Morgan
a8a028dfce
Merge branch 'develop' into anoa/trailing_slashes_client
2019-03-11 17:07:31 +00:00
Erik Johnston
78c563b77c
Correctly log expected errors when fetching server keys
2019-03-11 14:11:10 +00:00
Amber Brown
78a6b950b3
fix
2019-03-12 00:50:28 +11:00