Richard van der Hoff
63d6ad1064
Stronger typing in the federation handler ( #6480 )
...
replace the event_info dict with an attrs thing
2019-12-05 15:02:35 +00:00
Richard van der Hoff
e1f4c83f41
Sanity-check the rooms of auth events before pulling them in. ( #6472 )
2019-12-05 14:14:45 +00:00
Richard van der Hoff
e203874caa
get rid of (most of) have_events from _update_auth_events_and_context_for_auth ( #6468 )
...
have_events was a map from event_id to rejection reason (or None) for events
which are in our local database. It was used as filter on the list of
event_ids being passed into get_events_as_list. However, since
get_events_as_list will ignore any event_ids that are unknown or rejected, we
can equivalently just leave it to get_events_as_list to do the filtering.
That means that we don't have to keep `have_events` up-to-date, and can use
`have_seen_events` instead of `get_seen_events_with_rejection` in the one place
we do need it.
2019-12-04 17:27:32 +00:00
Erik Johnston
b9449012db
Merge pull request #6441 from syamgk/fix-parameter-mismatch
...
Fix issue #6406 parameter mismatch
2019-12-04 15:42:33 +00:00
Brendan Abolivier
c530f9af4d
Merge pull request #6329 from matrix-org/babolivier/context_filters
...
Filter state, events_before and events_after in /context requests
2019-12-04 15:24:16 +00:00
Brendan Abolivier
9dc84b7989
Merge branch 'develop' into babolivier/context_filters
2019-12-04 14:23:44 +00:00
Brendan Abolivier
08a436ecb2
Incorporate review
2019-12-04 14:18:46 +00:00
Syam G Krishnan
ce1c975ebc
Issue #6406 Fix parameter mismatch
...
Signed-off-by: Syam G Krishnan <syamgk01@gmail.com>
2019-12-04 13:03:46 +05:30
Brendan Abolivier
54dd5dc12b
Add ephemeral messages support (MSC2228) ( #6409 )
...
Implement part [MSC2228](https://github.com/matrix-org/matrix-doc/pull/2228 ). The parts that differ are:
* the feature is hidden behind a configuration flag (`enable_ephemeral_messages`)
* self-destruction doesn't happen for state events
* only implement support for the `m.self_destruct_after` field (not the `m.self_destruct` one)
* doesn't send synthetic redactions to clients because for this specific case we consider the clients to be able to destroy an event themselves, instead we just censor it (by pruning its JSON) in the database
2019-12-03 19:19:45 +00:00
Andrew Morgan
72078e4be5
Transfer power level state events on room upgrade ( #6237 )
2019-12-02 15:11:32 +00:00
Erik Johnston
1c3a61529f
Merge pull request #6434 from matrix-org/erikj/msc2367_membership_reasons
...
Implement MSC 2367 - Membership Reasons
2019-11-29 13:30:36 +00:00
Brendan Abolivier
708cef88cf
Discard retention policies when retrieving state
...
Purge jobs don't delete the latest event in a room in order to keep the forward extremity and not break the room. On the other hand, get_state_events, when given an at_token argument calls filter_events_for_client to know if the user can see the event that matches that (sync) token. That function uses the retention policies of the events it's given to filter out those that are too old from a client's view.
Some clients, such as Riot, when loading a room, request the list of members for the latest sync token it knows about, and get confused to the point of refusing to send any message if the server tells it that it can't get that information. This can happen very easily with the message retention feature turned on and a room with low activity so that the last event sent becomes too old according to the room's retention policy.
An easy and clean fix for that issue is to discard the room's retention policies when retrieving state.
2019-11-28 19:26:26 +00:00
Erik Johnston
2173785f0d
Propagate reason in remotely rejected invites
2019-11-28 11:31:56 +00:00
Andrew Morgan
a9c44d4008
Remove local threepids on account deactivation ( #6426 )
2019-11-28 10:40:42 +00:00
Hubert Chathi
0d27aba900
add etag and count to key backup endpoints ( #5858 )
2019-11-27 16:14:44 -05:00
Brendan Abolivier
d31f69afa0
Merge pull request #6358 from matrix-org/babolivier/message_retention
...
Implement message retention policies (MSC1763)
2019-11-27 15:04:38 +00:00
Richard van der Hoff
651d930f16
Merge pull request #6343 from matrix-org/rav/event_auth/4
...
Refactor _update_auth_events_and_context_for_auth
2019-11-26 18:15:03 +00:00
Brendan Abolivier
9e937c28ee
Merge branch 'develop' into babolivier/message_retention
2019-11-26 17:53:57 +00:00
Erik Johnston
f9f1c8acbb
Merge pull request #6332 from matrix-org/erikj/query_devices_fix
...
Fix caching devices for remote servers in worker.
2019-11-26 12:56:05 +00:00
Richard van der Hoff
4d394d6415
remove confusing fixme
2019-11-26 12:32:37 +00:00
Andrew Morgan
3916e1b97a
Clean up newline quote marks around the codebase ( #6362 )
2019-11-21 12:00:14 +00:00
Brendan Abolivier
83446a18fb
Merge pull request #6335 from matrix-org/erikj/rc_login_cleanups
...
Only do `rc_login` ratelimiting on succesful login.
2019-11-20 09:52:38 +00:00
Brendan Abolivier
a6fc6754f8
Fix 3PID invite exchange
2019-11-19 14:07:39 +00:00
Brendan Abolivier
7c24d0f443
Lint
2019-11-19 13:22:37 +00:00
Erik Johnston
c7376cdfe3
Apply suggestions from code review
...
Co-Authored-By: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
Co-Authored-By: Brendan Abolivier <babolivier@matrix.org>
2019-11-18 17:10:16 +00:00
Richard van der Hoff
870c00e278
Merge remote-tracking branch 'origin/develop' into rav/event_auth/4
2019-11-18 12:05:36 +00:00
Andrew Morgan
bc29a19731
Replace instance variations of homeserver with correct case/spacing
2019-11-12 13:08:12 +00:00
Richard van der Hoff
f41027f746
Use get_events_as_list rather than lots of calls to get_event
...
It's more efficient and clearer.
2019-11-08 12:21:28 +00:00
Richard van der Hoff
f8407975e7
Update some docstrings and comments
2019-11-08 12:18:20 +00:00
Richard van der Hoff
772d414975
Simplify _update_auth_events_and_context_for_auth
...
move event_key calculation into _update_context_for_auth_events, since it's
only used there.
2019-11-08 11:40:11 +00:00
Erik Johnston
f713c01e2b
Merge pull request #6295 from matrix-org/erikj/split_purge_history
...
Split purge API into events vs state and add PurgeEventsStorage
2019-11-08 10:19:15 +00:00
Andrew Morgan
e914cf12f6
Merge pull request #6235 from matrix-org/anoa/room_upgrade_groups
2019-11-07 11:12:22 +00:00
Brendan Abolivier
eda14737cf
Also filter state events
2019-11-06 18:14:03 +00:00
Andrew Morgan
55bc8d531e
raise exception after multiple failures
2019-11-06 16:52:54 +00:00
Andrew Morgan
1fe3cc2c9c
Address review comments
2019-11-06 14:54:24 +00:00
Andrew Morgan
4059d61e26
Don't forget to ratelimit calls outside of RegistrationHandler
2019-11-06 12:01:54 +00:00
Andrew Morgan
b33c4f7a82
Numeric ID checker now checks @0, don't ratelimit on checking
2019-11-06 11:55:00 +00:00
Erik Johnston
f697b4b4a2
Add failed auth ratelimiting to UIA
2019-11-06 11:08:58 +00:00
Erik Johnston
541f1b92d9
Only do rc_login ratelimiting on succesful login.
...
We were doing this in a number of places which meant that some login
code paths incremented the counter multiple times.
It was also applying ratelimiting to UIA endpoints, which was probably
not intentional.
In particular, some custom auth modules were calling
`check_user_exists`, which incremented the counters, meaning that people
would fail to login sometimes.
2019-11-06 11:08:58 +00:00
Richard van der Hoff
807ec3bd99
Fix bug which caused rejected events to be stored with the wrong room state ( #6320 )
...
Fixes a bug where rejected events were persisted with the wrong state group.
Also fixes an occasional internal-server-error when receiving events over
federation which are rejected and (possibly because they are
backwards-extremities) have no prev_group.
Fixes #6289 .
2019-11-06 10:01:39 +00:00
Richard van der Hoff
0e3ab8afdc
Add some checks that we aren't using state from rejected events ( #6330 )
...
* Raise an exception if accessing state for rejected events
Add some sanity checks on accessing state_group etc for
rejected events.
* Skip calculating push actions for rejected events
It didn't actually cause any bugs, because rejected events get filtered out at
various later points, but there's not point in trying to calculate the push
actions for a rejected event.
2019-11-05 22:13:37 +00:00
Erik Johnston
c16e192e2f
Fix caching devices for remote servers in worker.
...
When the `/keys/query` API is hit on client_reader worker Synapse may
decide that it needs to resync some remote deivces. Usually this happens
on master, and then gets cached. However, that fails on workers and so
it falls back to fetching devices from remotes directly, which may in
turn fail if the remote is down.
2019-11-05 15:49:43 +00:00
Brendan Abolivier
cb2cbe4d26
Only filter if a filter was provided
2019-11-05 15:28:36 +00:00
Brendan Abolivier
f141af4c79
Update copyright
2019-11-05 14:52:38 +00:00
Brendan Abolivier
1dffa78701
Filter events_before and events_after in /context requests
...
While the current version of the spec doesn't say much about how this endpoint uses filters (see https://github.com/matrix-org/matrix-doc/issues/2338 ), the current implementation is that some fields of an EventFilter apply (the ones that are used when running the SQL query) and others don't (the ones that are used by the filter itself) because we don't call event_filter.filter(...). This seems counter-intuitive and probably not what we want so this commit fixes it.
2019-11-05 12:21:59 +00:00
Andrew Morgan
c2203bea57
Re-add docstring, with caveats detailed
2019-11-04 18:17:11 +00:00
Andrew Morgan
0287d033ee
Transfer upgraded rooms on groups
2019-11-04 18:08:50 +00:00
Brendan Abolivier
09957ce0e4
Implement per-room message retention policies
2019-11-04 17:09:22 +00:00
Erik Johnston
6a0092d371
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/split_purge_history
2019-11-04 13:29:35 +00:00
Richard van der Hoff
c6516adbe0
Factor out an _AsyncEventContextImpl ( #6298 )
...
The intention here is to make it clearer which fields we can expect to be
populated when: notably, that the _event_type etc aren't used for the
synchronous impl of EventContext.
2019-11-01 16:19:09 +00:00
Richard van der Hoff
1cb84c6486
Support for routing outbound HTTP requests via a proxy ( #6239 )
...
The `http_proxy` and `HTTPS_PROXY` env vars can be set to a `host[:port]` value which should point to a proxy.
The address of the proxy should be excluded from IP blacklists such as the `url_preview_ip_range_blacklist`.
The proxy will then be used for
* push
* url previews
* phone-home stats
* recaptcha validation
* CAS auth validation
It will *not* be used for:
* Application Services
* Identity servers
* Outbound federation
* In worker configurations, connections from workers to masters
Fixes #4198 .
2019-11-01 14:07:44 +00:00
Andrew Morgan
fe1f2b4520
Remove last usages of deprecated logging.warn method ( #6314 )
2019-11-01 12:03:44 +00:00
Andrew Morgan
ace947e8da
Depublish a room from the public rooms list when it is upgraded ( #6232 )
2019-11-01 10:28:09 +00:00
Hubert Chathi
6f4bc6d01d
Merge branch 'develop' into cross-signing_federation
2019-10-31 22:38:21 -04:00
Amber Brown
020add5099
Update black to 19.10b0 ( #6304 )
...
* update version of black and also fix the mypy config being overridden
2019-11-01 02:43:24 +11:00
Erik Johnston
cd581338cf
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/split_purge_history
2019-10-31 15:19:26 +00:00
Erik Johnston
dfe0cd71b6
Merge pull request #6294 from matrix-org/erikj/add_state_storage
...
Add StateGroupStorage interface
2019-10-31 16:17:53 +01:00
Andrew Morgan
54fef094b3
Remove usage of deprecated logger.warn method from codebase ( #6271 )
...
Replace every instance of `logger.warn` with `logger.warning` as the former is deprecated.
2019-10-31 10:23:24 +00:00
Erik Johnston
7c8c97e635
Split purge API into events vs state
2019-10-30 15:23:37 +00:00
Erik Johnston
69f0054ce6
Port to use state storage
2019-10-30 14:46:54 +00:00
Hubert Chathi
7d7eac61be
Merge branch 'develop' into cross-signing_federation
2019-10-30 10:17:10 -04:00
Hubert Chathi
bc32f102cd
black
2019-10-30 10:07:36 -04:00
Hubert Chathi
d78b1e339d
apply changes as a result of PR review
2019-10-30 10:01:53 -04:00
Erik Johnston
ec6de1cc7d
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/split_out_persistence_store
2019-10-30 13:37:04 +00:00
Andrew Morgan
a2276d4d3c
Fix log line that was printing undefined value ( #6278 )
2019-10-30 11:28:48 +00:00
Erik Johnston
2c35ffead2
Port receipt and read markers to async/wait
2019-10-29 15:08:22 +00:00
Richard van der Hoff
172f264ed3
Improve signature checking on some federation APIs ( #6262 )
...
Make sure that we check that events sent over /send_join, /send_leave, and
/invite, are correctly signed and come from the expected servers.
2019-10-28 12:43:23 +00:00
Erik Johnston
f666d6f5d7
Remove repeated calls to config.stats_enabled.
...
Turns out that fetching variables from the config object is expensive,
so doing it once at startup avoids unnecessary work.
2019-10-25 10:28:36 +01:00
Hubert Chathi
ff05c9b760
don't error if federation query doesn't have cross-signing keys
2019-10-24 21:46:11 -04:00
Erik Johnston
3ca4c7c516
Use new EventPersistenceStore
2019-10-23 16:15:03 +01:00
Hubert Chathi
480eac30eb
black
2019-10-22 22:37:16 -04:00
Hubert Chathi
404e8c8532
vendor-prefix the EDU name until MSC1756 is merged into the spec
2019-10-22 22:33:23 -04:00
Hubert Chathi
1fabf82d50
update to work with newer code, and fix formatting
2019-10-22 21:44:58 -04:00
Hubert Chathi
41ad35b523
add missing param
2019-10-22 19:06:29 -04:00
Hubert Chathi
cfdb84422d
make black happy
2019-10-22 19:06:06 -04:00
Hubert Chathi
8d3542a64e
implement federation parts of cross-signing
2019-10-22 19:04:35 -04:00
Hubert Chathi
2761731634
Merge pull request #5726 from matrix-org/uhoreg/e2e_cross-signing2-part2
...
Cross-signing [3/4] -- uploading signatures edition
2019-10-22 17:05:06 +01:00
Richard van der Hoff
93eaeec75a
Remove Auth.check method ( #6217 )
...
This method was somewhat redundant, and confusing.
2019-10-18 19:43:36 +02:00
Hubert Chathi
36adfaedab
Merge branch 'develop' into cross-signing_sig_upload
2019-10-18 18:34:42 +01:00
Hubert Chathi
125eb45e19
fix doc strings
2019-10-18 16:56:16 +01:00
Hubert Chathi
0a9d22808c
Merge branch 'uhoreg/e2e_cross-signing_merged' into develop
2019-10-18 15:57:40 +01:00
Richard van der Hoff
2b22faded7
Merge remote-tracking branch 'origin/develop' into rav/event_auth/1
2019-10-18 12:35:13 +01:00
Hubert Chathi
cfc28325a6
Merge branch 'develop' into uhoreg/e2e_cross-signing_merged
2019-10-18 11:39:40 +01:00
Richard van der Hoff
80003dfcd5
Merge remote-tracking branch 'origin/develop' into rav/event_auth/1
2019-10-18 10:11:40 +01:00
Erik Johnston
5859a5c569
Fix presence timeouts when synchrotron restarts. ( #6212 )
...
* Fix presence timeouts when synchrotron restarts.
Handling timeouts would fail if there was an external process that had
timed out, e.g. a synchrotron restarting. This was due to a couple of
variable name typoes.
Fixes #3715 .
2019-10-18 06:42:26 +01:00
Richard van der Hoff
1ba359a11f
rip out some unreachable code
...
The only possible rejection reason is AUTH_ERROR, so all of this is unreachable.
2019-10-17 21:45:59 +01:00
Hubert Chathi
06fc66c81e
Merge pull request #6193 from matrix-org/uhoreg/interpret_device_key_in_storage
...
make storage layer in charge of interpreting the device key data
2019-10-11 15:19:06 -04:00
Hubert Chathi
691dd67fcd
Merge pull request #6189 from matrix-org/uhoreg/e2e_backup_optional_version
...
make version optional in body of e2e backup version update
2019-10-11 10:11:59 -04:00
Hubert Chathi
7a0dce9259
make sure we actually return something
2019-10-10 20:31:30 -04:00
Hubert Chathi
4908fb3b30
make storage layer in charge of interpreting the device key data
2019-10-10 19:15:30 -04:00
werner291
b5b03b7079
Add domain validation when creating room with list of invitees ( #6121 )
2019-10-10 13:05:48 +01:00
Richard van der Hoff
a139420a3c
Fix races in room stats (and other) updates. ( #6187 )
...
Hopefully this will fix the occasional failures we were seeing in the room directory.
The problem was that events are not necessarily persisted (and `current_state_delta_stream` updated) in the same order as their stream_id. So for instance current_state_delta 9 might be persisted *before* current_state_delta 8. Then, when the room stats saw stream_id 9, it assumed it had done everything up to 9, and never came back to do stream_id 8.
We can solve this easily by only processing up to the stream_id where we know all events have been persisted.
2019-10-10 11:29:01 +01:00
Andrew Morgan
da815c1f69
Move tag/push rules room upgrade checking ealier ( #6155 )
...
It turns out that _local_membership_update doesn't run when you join a new, remote room. It only runs if you're joining a room that your server already knows about. This would explain #4703 and #5295 and why the transfer would work in testing and some rooms, but not others. This would especially hit single-user homeservers.
The check has been moved to right after the room has been joined, and works much more reliably. (Though it may still be a bit awkward of a place).
2019-10-10 10:06:45 +01:00
Hubert Chathi
4535a07f4a
make version optional in body of e2e backup version update
...
to agree with latest version of the MSC
2019-10-09 17:54:03 -04:00
Andrew Morgan
ea7d938bca
Remove unused public room list timeout param ( #6179 )
...
* Remove unused public room list timeout param
* Add changelog
2019-10-08 13:51:25 +01:00
Erik Johnston
2cb7466993
Merge pull request #6161 from matrix-org/erikj/dont_regen_user_id_on_failure
...
Don't regenerate numeric user ID if registration fails.
2019-10-07 16:56:23 +01:00
Richard van der Hoff
276ae5c63e
add some logging to the rooms stats updates, to try to track down a flaky test ( #6167 )
2019-10-07 14:41:39 +01:00
Brendan Abolivier
ae0b78cb1f
Merge pull request #6147 from matrix-org/babolivier/3pid-invite-revoked
...
Don't 500 when trying to exchange a revoked 3PID invite
2019-10-04 12:09:05 +01:00
Brendan Abolivier
21d51ab598
Typo
2019-10-04 11:21:24 +01:00
Brendan Abolivier
4676732ca0
Lint
2019-10-04 11:18:28 +01:00
Brendan Abolivier
81d51ce48b
Incorporate review
2019-10-04 11:16:19 +01:00
Erik Johnston
ab8a64772b
Remove unused variable
2019-10-03 17:42:32 +01:00
Erik Johnston
693156aaf4
Don't regenerate numeric user ID if registration fails.
...
This causes huge amounts of DB IO if registrations start to fail e.g.
because the DB is struggling with IO.
2019-10-03 17:33:54 +01:00
Brendan Abolivier
6527fa18c1
Add test case
2019-10-03 11:24:36 +01:00
Erik Johnston
03cf4385e0
Fix public room list pagination.
...
We incorrectly used `room_id` as to bound the result set, even though we
order by `joined_members, room_id`, leading to incorrect results after
pagination.
2019-10-02 15:11:17 +01:00
Erik Johnston
a5166e4d5f
Land improved room list based on room stats ( #6019 )
...
Use room_stats and room_state for room directory search
2019-10-02 14:08:35 +01:00
Brendan Abolivier
972c9f65d7
Lint
2019-10-02 12:17:46 +01:00
Andrew Morgan
2a1470cd05
Fix yields and copy instead of move push rules on room upgrade ( #6144 )
...
Copy push rules during a room upgrade from the old room to the new room, instead of deleting them from the old room.
For instance, we've defined upgrading of a room multiple times to be possible, and push rules won't be transferred on the second upgrade if they're deleted during the first.
Also fix some missing yields that probably broke things quite a bit.
2019-10-02 12:04:22 +01:00
Brendan Abolivier
5705ecaec6
Don't 500 code when trying to exchange a revoked 3PID invite
...
While this is not documented in the spec (but should be), Riot (and other clients) revoke 3PID invites by sending a m.room.third_party_invite event with an empty ({}) content to the room's state.
When the invited 3PID gets associated with a MXID, the identity server (which doesn't know about revocations) sends down to the MXID's homeserver all of the undelivered invites it has for this 3PID. The homeserver then tries to talk to the inviting homeserver in order to exchange these invite for m.room.member events.
When one of the invite is revoked, the inviting homeserver responds with a 500 error because it tries to extract a 'display_name' property from the content, which is empty. This might cause the invited server to consider that the server is down and not try to exchange other, valid invites (or at least delay it).
This fix handles the case of revoked invites by avoiding trying to fetch a 'display_name' from the original invite's content, and letting the m.room.member event fail the auth rules (because, since the original invite's content is empty, it doesn't have public keys), which results in sending a 403 with the correct error message to the invited server.
2019-10-02 11:19:43 +01:00
Brendan Abolivier
3e42d47a5a
Incorporate review
2019-09-27 16:15:01 +01:00
Brendan Abolivier
af92110c46
Update synapse/handlers/deactivate_account.py
...
Co-Authored-By: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2019-09-27 16:12:15 +01:00
Brendan Abolivier
bbe2a0f339
Update synapse/handlers/deactivate_account.py
...
Co-Authored-By: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2019-09-27 16:10:36 +01:00
Brendan Abolivier
873fe7883c
Lint
2019-09-27 15:21:03 +01:00
Brendan Abolivier
72a2708ac6
Fixup and add some logging
2019-09-27 15:13:39 +01:00
Brendan Abolivier
5257a2fb1c
Reject pending invites on deactivation
2019-09-27 14:49:53 +01:00
Andrew Morgan
8c27bc8b60
Move lookup-related functions from RoomMemberHandler to IdentityHandler ( #5978 )
...
Just to have all the methods that make calls to identity services in one place.
2019-09-27 10:36:20 +01:00
Neil Johnson
034db2ba21
Fix dummy event insertion consent bug ( #6053 )
...
Fixes #5905
2019-09-26 11:47:53 +01:00
Neil Johnson
77dc7093a7
Threepid validity checks on msisdns should not be dependent on 'threepid_behaviour_email'. ( #6104 )
...
Fixes #6103
2019-09-25 12:29:35 +01:00
Richard van der Hoff
990928abde
Stop advertising unsupported flows for registration ( #6107 )
...
If email or msisdn verification aren't supported, let's stop advertising them
for registration.
Fixes #6100 .
2019-09-25 12:10:26 +01:00
Richard van der Hoff
2cd98812ba
Refactor the user-interactive auth handling ( #6105 )
...
Pull the checkers out to their own classes, rather than having them lost in a
massive 1000-line class which does everything.
This is also preparation for some more intelligent advertising of flows, as per #6100
2019-09-25 11:33:03 +01:00
Hubert Chathi
c3635c9459
make isort happy
2019-09-24 16:21:03 -04:00
Hubert Chathi
f4b6d43ec3
add some comments
2019-09-24 16:19:54 -04:00
Hubert Chathi
39864f45ec
drop some logger lines to debug
2019-09-24 15:26:45 -04:00
Hubert Chathi
26113fb7de
make changes based on PR feedback
2019-09-24 14:12:20 -04:00
Richard van der Hoff
9b7c4f4078
Merge remote-tracking branch 'origin/develop' into rav/saml_mapping_work
2019-09-24 17:03:50 +01:00
J. Ryan Stinnett
40fb00f5b7
Add sid to next_link for email validation ( #6097 )
2019-09-24 14:39:50 +01:00
Richard van der Hoff
ed8b92f0d2
Merge remote-tracking branch 'origin/develop' into rav/saml_mapping_work
2019-09-24 12:57:32 +01:00
Andrew Morgan
50776261e1
Add submit_url response parameter to msisdn /requestToken ( #6079 )
...
Second part of solving #6076
Fixes #6076
We return a submit_url parameter on calls to POST */msisdn/requestToken so that clients know where to submit token information to.
2019-09-23 21:21:03 +01:00
Andrew Morgan
e08ea43463
Use the federation blacklist for requests to untrusted Identity Servers ( #6000 )
...
Uses a SimpleHttpClient instance equipped with the federation_ip_range_blacklist list for requests to identity servers provided by user input. Does not use a blacklist when contacting identity servers specified by account_threepid_delegates. The homeserver trusts the latter and we don't want to prevent homeserver admins from specifying delegates that are on internal IP addresses.
Fixes #5935
2019-09-23 20:23:20 +01:00
Andrew Morgan
2c99c63453
Add POST submit_token endpoint for MSISDN ( #6078 )
...
First part of solving #6076
2019-09-23 17:49:00 +01:00
Andrew Morgan
30af161af2
Implement MSC2290 ( #6043 )
...
Implements MSC2290. This PR adds two new endpoints, /unstable/account/3pid/add and /unstable/account/3pid/bind. Depending on the progress of that MSC the unstable prefix may go away.
This PR also removes the blacklist on some 3PID tests which occurs in #6042 , as the corresponding Sytest PR changes them to use the new endpoints.
Finally, it also modifies the account deactivation code such that it doesn't just try to deactivate 3PIDs that were bound to the user's account, but any 3PIDs that were bound through the homeserver on that user's account.
2019-09-23 16:50:27 +01:00
Andrew Morgan
885a4726b7
Return timeout error to user for identity server calls ( #6073 )
2019-09-23 14:37:23 +01:00
Andrew Morgan
df3401a71d
Allow HS to send emails when adding an email to the HS ( #6042 )
2019-09-20 15:21:30 +01:00
Richard van der Hoff
366dc7d2d4
Merge commit '33757bad1' into rav/saml_mapping_work
2019-09-20 11:21:40 +01:00
Richard van der Hoff
33757bad19
More better logging
2019-09-20 11:20:02 +01:00
Richard van der Hoff
b65327ff66
Merge branch 'develop' into rav/saml_mapping_work
2019-09-19 18:13:31 +01:00
Richard van der Hoff
7423fade92
better logging
2019-09-19 17:16:50 +01:00
Andrew Morgan
6670bd4072
v2 3PID Invites (part of MSC2140) ( #5979 )
...
3PID invites require making a request to an identity server to check that the invited 3PID has an Matrix ID linked, and if so, what it is.
These requests are being made on behalf of a user. The user will supply an identity server and an access token for that identity server. The homeserver will then forward this request with the access token (using an `Authorization` header) and, if the given identity server doesn't support v2 endpoints, will fall back to v1 (which doesn't require any access tokens).
Requires: ~~#5976~~
2019-09-17 18:05:13 +01:00
Erik Johnston
70c52821ce
Fix race condition in room stats. ( #6029 )
...
Broke in #5971
Basically the bug is that if get_current_state_deltas returns no new updates and we then take the max pos, its possible that we miss an update that happens in between the two calls. (e.g. get_current_state_deltas looks up to stream pos 5, then an event persists and so getting the max stream pos returns 6, meaning that next time we check for things with a stream pos bigger than 6)
2019-09-17 12:41:23 +01:00
Richard van der Hoff
a8ac40445c
Record mappings from saml users in an external table
...
We want to assign unique mxids to saml users based on an incrementing
suffix. For that to work, we need to record the allocated mxid in a separate
table.
2019-09-13 16:01:46 +01:00
Andrew Morgan
9fc71dc5ee
Use the v2 Identity Service API for lookups (MSC2134 + MSC2140) ( #5976 )
...
This is a redo of https://github.com/matrix-org/synapse/pull/5897 but with `id_access_token` accepted.
Implements [MSC2134](https://github.com/matrix-org/matrix-doc/pull/2134 ) plus Identity Service v2 authentication ala [MSC2140](https://github.com/matrix-org/matrix-doc/pull/2140 ).
Identity lookup-related functions were also moved from `RoomMemberHandler` to `IdentityHandler`.
2019-09-11 16:02:42 +01:00
Erik Johnston
cbcbfe64a2
Merge pull request #6015 from matrix-org/erikj/ratelimit_admin_redaction
...
Allow use of different ratelimits for admin redactions.
2019-09-11 15:39:38 +01:00
Andrew Morgan
9c555f37e3
Add note about extra arg to send_membership_event, remove arg in remote_reject_invite ( #6009 )
...
Some small fixes to `room_member.py` found while doing other PRs.
1. Add requester to the base `_remote_reject_invite` method.
2. `send_membership_event`'s docstring was out of date and took in a `remote_room_hosts` arg that was not used and no calling function provided.
2019-09-11 14:23:24 +01:00
Erik Johnston
57dd41a45b
Fix comments
...
Co-Authored-By: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2019-09-11 13:54:50 +01:00
Andrew Morgan
3505ffcda7
Fix existing v2 identity server calls (MSC2140) ( #6013 )
...
Two things I missed while implementing [MSC2140](https://github.com/matrix-org/matrix-doc/pull/2140/files#diff-c03a26de5ac40fb532de19cb7fc2aaf7R80 ).
1. Access tokens should be provided to the identity server as `access_token`, not `id_access_token`, even though the homeserver may accept the tokens as `id_access_token`.
2. Access tokens must be sent to the identity server in a query parameter, the JSON body is not allowed.
We now send the access token as part of an `Authorization: ...` header, which fixes both things.
The breaking code was added in https://github.com/matrix-org/synapse/pull/5892
Sytest PR: https://github.com/matrix-org/sytest/pull/697
2019-09-11 11:59:45 +01:00
Erik Johnston
c64c3bb4c5
Fix how we check for self redaction
2019-09-11 11:18:04 +01:00
Erik Johnston
54ce81c86d
Allow use of different ratelimits for admin redactions.
...
This is useful to allow room admins to quickly deal with a large number
of abusive messages.
2019-09-11 10:46:38 +01:00
Andrew Morgan
cd17a2085e
Remove origin parameter from add_display_name_to_third_party_invite and add params to docstring ( #6010 )
...
Another small fixup noticed during work on a larger PR. The `origin` field of `add_display_name_to_third_party_invite` is not used and likely was just carried over from the `on_PUT` method of `FederationThirdPartyInviteExchangeServlet` which, like all other servlets, provides an `origin` argument.
Since it's not used anywhere in the handler function though, we should remove it from the function arguments.
2019-09-11 10:37:17 +01:00
Erik Johnston
5e9b05d7da
Merge pull request #6011 from matrix-org/anoa/fix_3pid_validation
...
Use account_threepid_delegate for 3pid validation
2019-09-10 18:15:07 +01:00
Andrew Morgan
b5833a2abf
Add changelog
2019-09-10 17:56:10 +01:00
Andrew Morgan
60d3c57bd0
Use account_threepid_delegate for 3pid validation
2019-09-10 17:56:10 +01:00
Jason Robinson
be618e0551
Only count real users when checking for auto-creation of auto-join room
...
Previously if the first registered user was a "support" or "bot" user,
when the first real user registers, the auto-join rooms were not
created.
Fix to exclude non-real (ie users with a special user type) users
when counting how many users there are to determine whether we should
auto-create a room.
Signed-off-by: Jason Robinson <jasonr@matrix.org>
2019-09-09 14:48:08 +03:00
Hubert Chathi
d3f2fbcfe5
add function docs
2019-09-07 14:13:18 -04:00
Hubert Chathi
8e86f5b65c
Merge branch 'develop' into uhoreg/e2e_cross-signing_merged
2019-09-07 13:20:34 -04:00
Hubert Chathi
0d61d1d735
Merge branch 'develop' into cross-signing_sig_upload
2019-09-07 13:14:45 -04:00
Hubert Chathi
415d0a00e0
run black
2019-09-06 16:46:45 -04:00
Hubert Chathi
561cbba057
split out signature processing into separate functions
2019-09-06 16:44:24 -04:00
Andrew Morgan
78801e7f9e
Ensure a sid parameter is passed to bind_threepid ( #5995 )
...
`sid` is required to be part of `three_pid_creds`. We were 500'ing if it wasn't provided instead of returning `M_MISSING_PARAM`.
2019-09-06 15:36:50 +01:00
Erik Johnston
142c9325c2
Correctly handle non-bool m.federate flag
2019-09-06 14:21:06 +01:00
Andrew Morgan
0c0b82b6d1
Allow Synapse to send registration emails + choose Synapse or an external server to handle 3pid validation ( #5987 )
...
This is a combination of a few different PRs, finally all being merged into `develop`:
* #5875
* #5876
* #5868 (This one added the `/versions` flag but the flag itself was actually [backed out](891afb57cb (diff-e591d42d30690ffb79f63bb726200891)#5969 . What's left is just giving /versions access to the config file, which could be useful in the future)
* #5835
* #5969
* #5940
Clients should not actually use the new registration functionality until https://github.com/matrix-org/synapse/pull/5972 is merged.
UPGRADE.rst, changelog entries and config file changes should all be reviewed closely before this PR is merged.
2019-09-06 11:35:28 +01:00
Hubert Chathi
369462da74
avoid modifying input parameter
2019-09-05 17:03:31 -04:00
Erik Johnston
1a6ae33309
Merge pull request #5984 from matrix-org/joriks/opentracing_link_send_to_edu_contexts
...
Link the send loop with the edus contexts
2019-09-05 15:22:24 +01:00
Jorik Schellekens
1d65292e94
Link the send loop with the edus contexts
...
The contexts were being filtered too early so the send loop wasn't
being linked to them unless the destination
was whitelisted.
2019-09-05 14:42:37 +01:00
Andrew Morgan
a0d294c306
Switch to using v2 Identity Service APIs other than lookup (MSC 2140) ( #5892 )
2019-09-05 14:31:22 +01:00
Andrew Morgan
90d17a3d28
Add POST /_matrix/client/r0/account/3pid/unbind (MSC2140) ( #5980 )
...
Implements `POST /_matrix/client/r0/account/3pid/unbind` from [MSC2140](https://github.com/matrix-org/matrix-doc/blob/dbkr/tos_2/proposals/2140-terms-of-service-2.md#post-_matrixclientr0account3pidunbind ).
2019-09-05 14:00:30 +01:00
Hubert Chathi
c8dc740a94
update with newer coding style
2019-09-04 22:30:45 -04:00
Hubert Chathi
7d6c70fc7a
make black happy
2019-09-04 22:04:12 -04:00
Hubert Chathi
ac4746ac4b
allow uploading signatures of master key signed by devices
2019-09-04 22:03:41 -04:00
Hubert Chathi
4bb4544784
implement device signature uploading/fetching
2019-09-04 20:02:56 -04:00
Hubert Chathi
faf72a4c40
Merge branch 'develop' into cross-signing_keys
2019-09-04 19:12:29 -04:00
Andrew Morgan
b736c6cd3a
Remove bind_email and bind_msisdn ( #5964 )
...
Removes the `bind_email` and `bind_msisdn` parameters from the `/register` C/S API endpoint as per [MSC2140: Terms of Service for ISes and IMs](https://github.com/matrix-org/matrix-doc/pull/2140/files#diff-c03a26de5ac40fb532de19cb7fc2aaf7R107 ).
2019-09-04 18:24:23 +01:00
Erik Johnston
6e834e94fc
Fix and refactor room and user stats ( #5971 )
...
Previously the stats were not being correctly populated.
2019-09-04 13:04:27 +01:00
Andrew Morgan
ea128a3e8e
code cleanups
2019-09-03 21:05:06 +01:00
Andrew Morgan
6b6086b8bf
Fix docstring
2019-09-03 20:00:09 +01:00
Andrew Morgan
a98b8583c6
Remove unnecessary variable declaration
2019-09-03 19:58:51 +01:00
Matthew Hodgson
8401bcd206
fix typo
2019-09-03 12:44:14 +01:00
Andrew Morgan
2a44782666
Remove double return statements ( #5962 )
...
Remove all the "double return" statements which were a result of us removing all the instances of
```
defer.returnValue(...)
return
```
statements when we switched to python3 fully.
2019-09-03 11:42:45 +01:00
Jorik Schellekens
a90d16dabc
Opentrace device lists ( #5853 )
...
Trace device list changes.
2019-09-03 10:21:30 +01:00
Andrew Morgan
36f34e6f3d
Remove unused methods from c/s api v1 in register.py ( #5963 )
...
These methods were part of the v1 C/S API. Remove them as they are no longer used by any code paths.
2019-09-02 18:29:21 +01:00
Andrew Morgan
4548d1f87e
Remove unnecessary parentheses around return statements ( #5931 )
...
Python will return a tuple whether there are parentheses around the returned values or not.
I'm just sick of my editor complaining about this all over the place :)
2019-08-30 16:28:26 +01:00
Andrew Morgan
3057095a5d
Revert "Use the v2 lookup API for 3PID invites ( #5897 )" ( #5937 )
...
This reverts commit 71fc04069a#5892 was required for it to work correctly.
2019-08-30 12:00:20 +01:00
Hubert Chathi
e3d3fbf63f
Merge branch 'uhoreg/e2e_cross-signing_merged' into cross-signing_keys
2019-08-28 17:36:46 -07:00
Hubert Chathi
96bda56370
black
2019-08-28 17:18:40 -07:00
Hubert Chathi
3b0b22cb05
use stream ID generator instead of timestamp
2019-08-28 17:17:21 -07:00
Will Hunt
c8fa620d7a
Merge pull request #5902 from matrix-org/hs/exempt-support-users-from-consent
...
Exempt support users from consent
2019-08-28 16:31:40 +01:00
Andrew Morgan
71fc04069a
Use the v2 lookup API for 3PID invites ( #5897 )
...
Fixes https://github.com/matrix-org/synapse/issues/5861
Adds support for the v2 lookup API as defined in [MSC2134](https://github.com/matrix-org/matrix-doc/pull/2134 ). Currently this is only used for 3PID invites.
Sytest PR: https://github.com/matrix-org/sytest/pull/679
2019-08-28 14:59:26 +02:00
Amber Brown
7dc398586c
Implement a structured logging output system. ( #5680 )
2019-08-28 21:18:53 +10:00
reivilibre
a3f0635686
Merge pull request #5914 from matrix-org/rei/admin_getadmin
...
Add GET method to admin API /users/@user:dom/admin
2019-08-28 09:44:22 +01:00
reivilibre
7ccc251415
Merge pull request #5859 from matrix-org/rei/msc2197
...
MSC2197 Search Filters over Federation
2019-08-28 09:00:21 +01:00
Olivier Wilkinson (reivilibre)
c88a119259
Add GET method to admin API /users/@user:dom/admin
...
Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
2019-08-27 13:12:27 +01:00
Richard van der Hoff
ccb15a5bbe
Merge pull request #5906 from matrix-org/neilj/increase_display_name_limit
...
Increase profile display name limit
2019-08-27 11:52:59 +01:00
reivilibre
1a7e6eb633
Add Admin API capability to set adminship of a user ( #5878 )
...
Admin API: Set adminship of a user
2019-08-27 10:14:00 +01:00
Neil Johnson
27d3fc421a
Increase max display name limit
2019-08-24 22:33:43 +01:00
Will Hunt
c998f25006
Apply suggestions from code review
...
Co-Authored-By: Erik Johnston <erik@matrix.org>
2019-08-23 10:28:54 +01:00
Half-Shot
9ba32f6573
Exempt bot users
2019-08-23 09:56:31 +01:00
Half-Shot
886eceba3e
Return user_type in get_user_by_id
2019-08-23 09:14:52 +01:00
Jorik Schellekens
8767b63a82
Propagate opentracing contexts through EDUs ( #5852 )
...
Propagate opentracing contexts through EDUs
Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2019-08-22 18:21:10 +01:00
Brendan Abolivier
1c5b8c6222
Revert "Add "require_consent" parameter for registration"
...
This reverts commit 3320aaab3a
2019-08-22 14:47:34 +01:00
Half-Shot
3320aaab3a
Add "require_consent" parameter for registration
2019-08-22 14:21:54 +01:00
Jorik Schellekens
9a6f2be572
Opentrace e2e keys ( #5855 )
...
Add opentracing tags and logs for e2e keys
2019-08-22 11:28:12 +01:00
Richard van der Hoff
119aa31b10
Servlet to purge old rooms ( #5845 )
2019-08-22 10:42:59 +01:00
Hubert Chathi
814f253f1b
make isort happy
2019-08-21 13:22:15 -07:00
Hubert Chathi
7c3abc6572
apply PR review suggestions
2019-08-21 13:19:35 -07:00
Erik Johnston
d514dac0b2
Merge pull request #5860 from matrix-org/erikj/update_5704_comments
...
Remove logging for #5407 and update comments
2019-08-19 10:20:59 +01:00
Erik Johnston
748aa38378
Remove logging for #5407 and update comments
2019-08-15 12:02:18 +01:00
Olivier Wilkinson (reivilibre)
6fadb560fc
Support MSC2197 outbound with unstable prefix
...
Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
2019-08-15 10:59:37 +01:00
Michael Telatynski
baee288fb4
Don't create broken room when power_level_content_override.users does not contain creator_id. ( #5633 )
2019-08-15 09:45:57 +01:00
Erik Johnston
6881f21f3e
Handle TimelineBatch being limited and empty.
...
This hopefully addresses #5407 by gracefully handling an empty but
limited TimelineBatch. We also add some logging to figure out how this
is happening.
2019-08-06 12:59:00 +01:00
Hubert Chathi
fac1cdc562
make changes from PR review
2019-08-01 21:51:19 -04:00
Hubert Chathi
336c546d6a
Merge branch 'cross-signing_hidden' into cross-signing_keys
2019-08-01 16:31:40 -04:00
Brendan Abolivier
8ed9e63432
Account validity: allow defining HTML templates to serve the us… ( #5807 )
...
Account validity: allow defining HTML templates to serve the user on account renewal attempt
2019-08-01 16:09:25 +02:00
Erik Johnston
58af30a6c7
Merge pull request #5802 from matrix-org/erikj/deny_redacting_different_room
...
Deny redaction of events in a different room.
2019-08-01 13:14:46 +01:00
Erik Johnston
0f632f3a57
Merge pull request #5790 from matrix-org/erikj/groups_request_errors
...
Handle RequestSendFailed exception correctly in more places.
2019-08-01 13:14:08 +01:00
Brendan Abolivier
a4a9ded4d0
Allow defining HTML templates to serve the user on account renewal
2019-08-01 11:59:27 +02:00
Erik Johnston
cf89266b98
Deny redaction of events in a different room.
...
We already correctly filter out such redactions, but we should also deny
them over the CS API.
2019-07-31 16:12:27 +01:00
Andrew Morgan
58a755cdc3
Remove duplicate return statement
2019-07-31 13:24:51 +01:00
Erik Johnston
b4d5ff0af7
Don't log as exception when failing durig backfill
2019-07-30 13:19:22 +01:00
Erik Johnston
1ec7d656dd
Unwrap error
2019-07-30 13:09:02 +01:00
Erik Johnston
458e51df7a
Fix error handling when fetching remote device keys
2019-07-30 13:07:02 +01:00
Richard van der Hoff
8c97f6414c
Remove non-functional 'expire_access_token' setting ( #5782 )
...
The `expire_access_token` didn't do what it sounded like it should do. What it
actually did was make Synapse enforce the 'time' caveat on macaroons used as
access tokens, but since our access token macaroons never contained such a
caveat, it was always a no-op.
(The code to add 'time' caveats was removed back in v0.18.5, in #1656 )
2019-07-30 08:25:02 +01:00
Amber Brown
865077f1d1
Room Complexity Client Implementation ( #5783 )
2019-07-30 02:47:27 +10:00
Erik Johnston
aecae8f397
Correctly handle errors doing requests to group servers
2019-07-29 17:21:57 +01:00
Amber Brown
97a8b4caf7
Move some timeout checking logs to DEBUG #5785
2019-07-30 02:02:18 +10:00
Jorik Schellekens
85b0bd8fe0
Update the device list cache when keys/query is called ( #5693 )
2019-07-29 16:34:44 +01:00
Richard van der Hoff
1a93daf353
Merge pull request #5744 from matrix-org/erikj/log_leave_origin_mismatch
...
Log when we receive a /make_* request from a different origin
2019-07-26 12:38:37 +01:00
Richard van der Hoff
d1020653fc
Log when we receive a /make_* request from a different origin
2019-07-26 10:08:22 +01:00
Hubert Chathi
c659b9f94f
allow uploading keys for cross-signing
2019-07-25 11:08:24 -04:00
Erik Johnston
2276936bac
Merge pull request #5743 from matrix-org/erikj/log_origin_receipts_mismatch
...
Log when we receive receipt from a different origin
2019-07-24 13:27:57 +01:00
Richard van der Hoff
f30a71a67b
Stop trying to fetch events with event_id=None. ( #5753 )
...
`None` is not a valid event id, so queuing up a database fetch for it seems
like a silly thing to do.
I considered making `get_event` return `None` if `event_id is None`, but then
its interaction with `allow_none` seemed uninituitive, and strong typing ftw.
2019-07-24 13:16:18 +01:00
Amber Brown
4806651744
Replace returnValue with return ( #5736 )
2019-07-23 23:00:55 +10:00
Erik Johnston
3db1377b26
Log when we receive receipt from a different origin
2019-07-23 13:31:03 +01:00
Richard van der Hoff
9c70a02a9c
Ignore redactions of m.room.create events ( #5701 )
2019-07-17 19:08:02 +01:00
Andrew Morgan
18c516698e
Return a different error from Invalid Password when a user is deactivated ( #5674 )
...
Return `This account has been deactivated` instead of `Invalid password` when a user is deactivated.
2019-07-15 11:45:29 +01:00
Erik Johnston
d86321300a
Merge pull request #5589 from matrix-org/erikj/admin_exfiltrate_data
...
Add basic function to get all data for a user out of synapse
2019-07-15 10:04:02 +01:00
Richard van der Hoff
5f158ec039
Implement access token expiry ( #5660 )
...
Record how long an access token is valid for, and raise a soft-logout once it
expires.
2019-07-12 17:26:02 +01:00
Richard van der Hoff
1890cfcf82
Inline issue_access_token ( #5659 )
...
this is only used in one place, so it's clearer if we inline it and reduce the
API surface.
Also, fixes a buglet where we would create an access token even if we were
about to block the user (we would never return the AT, so the user could never
use it, but it was still created and added to the db.)
2019-07-11 04:10:07 +10:00
Brendan Abolivier
8ab3444fdf
Merge pull request #5658 from matrix-org/babolivier/is-json
...
Send 3PID bind requests as JSON data
2019-07-10 17:01:26 +01:00
Richard van der Hoff
953dbb7980
Remove access-token support from RegistrationStore.register ( #5642 )
...
The 'token' param is no longer used anywhere except the tests, so let's kill
that off too.
2019-07-10 16:26:49 +01:00
Brendan Abolivier
b2a2e96ea6
Typo
2019-07-10 15:56:21 +01:00
Brendan Abolivier
f77e997619
Send 3PID bind requests as JSON data
2019-07-10 15:46:42 +01:00
Brendan Abolivier
65434da75d
Merge pull request #5638 from matrix-org/babolivier/invite-json
...
Use JSON when querying the IS's /store-invite endpoint
2019-07-09 18:48:38 +01:00
Hubert Chathi
7b3bc755a3
remove unused and unnecessary check for FederationDeniedError ( #5645 )
...
FederationDeniedError is a subclass of SynapseError, which is a subclass of
CodeMessageException, so if e is a FederationDeniedError, then this check for
FederationDeniedError will never be reached since it will be caught by the
check for CodeMessageException above. The check for CodeMessageException does
almost the same thing as this check (since FederationDeniedError initialises
with code=403 and msg="Federation denied with %s."), so may as well just keep
allowing it to handle this case.
2019-07-09 18:37:39 +01:00
Brendan Abolivier
af67c7c1de
Merge pull request #5644 from matrix-org/babolivier/profile-allow-self
...
Allow newly-registered users to lookup their own profiles
2019-07-09 10:25:40 +01:00
Richard van der Hoff
824707383b
Remove access-token support from RegistrationHandler.register ( #5641 )
...
Nothing uses this now, so we can remove the dead code, and clean up the
API.
Since we're changing the shape of the return value anyway, we take the
opportunity to give the method a better name.
2019-07-08 19:01:08 +01:00
Brendan Abolivier
7556851665
Allow newly-registered users to lookup their own profiles
...
When a user creates an account and the 'require_auth_for_profile_requests' config flag is set, and a client that performed the registration wants to lookup the newly-created profile, the request will be denied because the user doesn't share a room with themselves yet.
2019-07-08 17:31:00 +01:00
Richard van der Hoff
b70e080b59
Better logging for auto-join. ( #5643 )
...
It was pretty unclear what was going on, so I've added a couple of log lines.
2019-07-08 17:14:51 +01:00
Brendan Abolivier
57eacee4f4
Merge branch 'develop' into babolivier/invite-json
2019-07-08 15:49:23 +01:00
Richard van der Hoff
1af2fcd492
Move get_or_create_user to test code ( #5628 )
...
This is only used in tests, so...
2019-07-08 23:52:26 +10:00
Brendan Abolivier
f05c7d62bc
Lint
2019-07-08 14:29:27 +01:00
Brendan Abolivier
1a807dfe68
Use application/json when querying the IS's /store-invite endpoint
2019-07-08 14:19:39 +01:00
Richard van der Hoff
80cc82a445
Remove support for invite_3pid_guest. ( #5625 )
...
This has never been documented, and I'm not sure it's ever been used outside
sytest.
It's quite a lot of poorly-maintained code, so I'd like to get rid of it.
For now I haven't removed the database table; I suggest we leave that for a
future clearout.
2019-07-05 16:47:58 +01:00
Erik Johnston
b4f5416dd9
pep8
2019-07-05 14:41:29 +01:00
Erik Johnston
eadb13d2e9
Remove FileExfiltrationWriter
2019-07-05 14:15:00 +01:00
Erik Johnston
7f0d8e4288
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/admin_exfiltrate_data
2019-07-05 14:08:21 +01:00
Erik Johnston
9ccea16d45
Assume key existence. Update docstrings
2019-07-05 14:07:56 +01:00
Erik Johnston
c061d4f237
Fixup from review comments.
2019-07-04 11:41:06 +01:00
Amber Brown
463b072b12
Move logging utilities out of the side drawer of util/ and into logging/ ( #5606 )
2019-07-04 00:07:04 +10:00
Erik Johnston
d0b849c86d
Apply comment fixups from code review
...
Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2019-07-03 15:03:38 +01:00
Richard van der Hoff
91753cae59
Fix a number of "Starting txn from sentinel context" warnings ( #5605 )
...
Fixes #5602 , #5603
2019-07-03 09:31:27 +01:00
Erik Johnston
8ee69f299c
Add basic function to get all data for a user out of synapse
2019-07-02 12:09:04 +01:00
Richard van der Hoff
6eecb6e500
Complete the SAML2 implementation ( #5422 )
...
* SAML2 Improvements and redirect stuff
Signed-off-by: Alexander Trost <galexrt@googlemail.com>
* Code cleanups and simplifications.
Also: share the saml client between redirect and response handlers.
* changelog
* Revert redundant changes to static js
* Move all the saml stuff out to a centralised handler
* Add support for tracking SAML2 sessions.
This allows us to correctly handle `allow_unsolicited: False`.
* update sample config
* cleanups
* update sample config
* rename BaseSSORedirectServlet for consistency
* Address review comments
2019-07-02 11:18:11 +01:00
Richard van der Hoff
b4fd86a9b4
Merge branch 'develop' into rav/saml2_client
2019-07-01 14:21:03 +01:00
Richard van der Hoff
3bcb13edd0
Address review comments
2019-07-01 12:13:22 +01:00
Erik Johnston
04196a4dae
Merge pull request #5507 from matrix-org/erikj/presence_sync_tighloop
...
Fix sync tightloop bug.
2019-07-01 11:43:10 +01:00
Erik Johnston
915280f1ed
Fixup comment
2019-07-01 10:22:42 +01:00
Brendan Abolivier
c7ff297dde
Merge pull request #5576 from matrix-org/babolivier/3pid-invite-ratelimit
...
Don't update the ratelimiter before sending a 3PID invite
2019-06-28 17:43:48 +01:00
Brendan Abolivier
15d9fc31bd
Only ratelimit when sending the email
...
If we do the opposite, an event can arrive after or while sending the email and the 3PID invite event will get ratelimited.
2019-06-28 16:04:05 +01:00
Brendan Abolivier
01d0f8e701
Don't update the ratelimiter before sending a 3PID invite
...
This would cause emails being sent, but Synapse responding with a 429 when creating the event. The client would then retry, and with bad timing the same scenario would happen again. Some testing I did ended up sending me 10 emails for one single invite because of this.
2019-06-28 15:22:16 +01:00
Daniel Hoffend
9646a593ac
Added possibilty to disable local password authentication ( #5092 )
...
Signed-off-by: Daniel Hoffend <dh@dotlan.net>
2019-06-27 18:37:29 +01:00
Erik Johnston
729f5a4fb6
Review comments
2019-06-27 16:06:23 +01:00
Richard van der Hoff
28db0ae537
cleanups
2019-06-27 00:37:41 +01:00
Richard van der Hoff
36f4953dec
Add support for tracking SAML2 sessions.
...
This allows us to correctly handle `allow_unsolicited: False`.
2019-06-26 23:50:55 +01:00
Richard van der Hoff
3705322103
Move all the saml stuff out to a centralised handler
2019-06-26 22:52:02 +01:00
Erik Johnston
8624db3194
Refactor and comment sync device list code
2019-06-26 19:39:49 +01:00
Erik Johnston
806a06daf2
Rename get_users_whose_devices_changed
2019-06-26 19:39:19 +01:00
Erik Johnston
a2f6d31a63
Refactor get_user_ids_changed to pull less from DB
...
When a client asks for users whose devices have changed since a token we
used to pull *all* users from the database since the token, which could
easily be thousands of rows for old tokens.
This PR changes this to only check for changes for users the client is
actually interested in.
Fixes #5553
2019-06-26 12:03:44 +01:00
Andrew Morgan
ef8c62758c
Prevent multiple upgrades on the same room at once ( #5051 )
...
Closes #4583
Does slightly less than #5045 , which prevented a room from being upgraded multiple times, one after another. This PR still allows that, but just prevents two from happening at the same time.
Mostly just to mitigate the fact that servers are slow and it can take a moment for the room upgrade to actually complete. We don't want people sending another request to upgrade the room when really they just thought the first didn't go through.
2019-06-25 14:19:21 +01:00
Erik Johnston
25433f212d
Merge pull request #5531 from matrix-org/erikj/workers_pagination_token
...
Fix /messages on workers when no from param specified.
2019-06-24 15:30:10 +01:00
Richard van der Hoff
edea4bb5be
Allow configuration of the path used for ACME account keys.
...
Because sticking it in the same place as the config isn't necessarily the right
thing to do.
2019-06-24 13:51:22 +01:00
Richard van der Hoff
21bf4318b5
Factor acme bits out to a separate file ( #5521 )
...
This makes some of the conditional-import hoop-jumping easier.
2019-06-24 11:33:56 +01:00
Erik Johnston
dddf20e8e1
Fix /messages on workers when no from param specified.
...
If no `from` param is specified we calculate and use the "current
token" that inlcuded typing, presence, etc. These are unused during
pagination and are not available on workers, so we simply don't
calculate them.
2019-06-24 10:06:51 +01:00
Erik Johnston
8181e290a9
Fix sync tightloop bug.
...
If, for some reason, presence updates take a while to persist then it
can trigger clients to tightloop calling `/sync` due to the presence
handler returning updates but not advancing the stream token.
Fixes #5503 .
2019-06-21 11:10:27 +01:00
Amber Brown
32e7c9e7f2
Run Black. ( #5482 )
2019-06-20 19:32:02 +10:00
Erik Johnston
7dcf984075
Merge pull request #5042 from matrix-org/erikj/fix_get_missing_events_error
...
Handle the case of `get_missing_events` failing
2019-06-19 13:20:09 +01:00
Erik Johnston
e0be8d7016
Merge pull request #5480 from matrix-org/erikj/extremities_dummy_events
...
Add experimental option to reduce extremities.
2019-06-19 13:19:18 +01:00
Erik Johnston
554609288b
Run as background process and fix comments
2019-06-19 11:33:03 +01:00
David Baker
f2d2ae03da
Add some logging to 3pid invite sig verification ( #5015 )
...
I had to add quite a lot of logging to diagnose a problem with 3pid
invites - we only logged the one failure which isn't all that
informative.
NB. I'm not convinced the logic of this loop is right: I think it
should just accept a single valid signature from a trusted source
rather than fail if *any* signature is invalid. Also it should
probably not skip the rest of middle loop if a check fails? However,
I'm deliberately not changing the logic here.
2019-06-18 22:51:24 +01:00
Erik Johnston
2b20d0fb59
Fix logline
2019-06-18 16:12:53 +01:00
Erik Johnston
19b80fe68a
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/fix_get_missing_events_error
2019-06-18 16:11:43 +01:00
Erik Johnston
b42f90470f
Add experimental option to reduce extremities.
...
Adds new config option `cleanup_extremities_with_dummy_events` which
periodically sends dummy events to rooms with more than 10 extremities.
THIS IS REALLY EXPERIMENTAL.
2019-06-18 15:02:18 +01:00
Brendan Abolivier
112cf5a73a
Add third party rules hook for 3PID invites
2019-06-17 17:39:09 +01:00
Erik Johnston
187d2837a9
Add third party rules hook into create room
2019-06-17 16:41:19 +01:00
Erik Johnston
6840ebeef8
Merge pull request #5385 from matrix-org/erikj/reduce_http_exceptions
...
Handle HttpResponseException when using federation client.
2019-06-17 13:54:47 +01:00
Erik Johnston
a9dab970b8
Merge pull request #5464 from matrix-org/erikj/3pid_remote_invite_state
...
Fix 3PID invite room state over federation.
2019-06-17 10:18:28 +01:00
Brendan Abolivier
f12e1f029c
Merge pull request #5440 from matrix-org/babolivier/third_party_event_rules
...
Allow server admins to define implementations of extra rules for allowing or denying incoming events
2019-06-14 19:37:59 +01:00
Brendan Abolivier
f874b16b2e
Add plugin APIs for implementations of custom event rules.
2019-06-14 18:16:03 +01:00
Erik Johnston
304a1376c2
Fix 3PID invite room state over federation.
...
Fixes that when a user exchanges a 3PID invite for a proper invite over
federation it does not include the `invite_room_state` key.
This was due to synapse incorrectly sending out two invite requests.
2019-06-14 16:19:11 +01:00
Brendan Abolivier
6d56a694f4
Don't send renewal emails to deactivated users
2019-06-14 15:05:56 +01:00
Brendan Abolivier
d0530382ee
Track deactivated accounts in the database ( #5378 )
2019-06-14 13:18:24 +01:00
Erik Johnston
49e01e5710
Fix defaults on checking threepids
2019-06-10 23:09:31 +01:00
Erik Johnston
a46ef1e3a4
Handle HttpResponseException when using federation client.
...
Otherwise we just log exceptions everywhere.
2019-06-07 10:29:35 +01:00
Andrew Morgan
3719680ee4
Add ability to perform password reset via email without trusting the identity server ( #5377 )
...
Sends password reset emails from the homeserver instead of proxying to the identity server. This is now the default behaviour for security reasons. If you wish to continue proxying password reset requests to the identity server you must now enable the email.trust_identity_server_for_password_resets option.
This PR is a culmination of 3 smaller PRs which have each been separately reviewed:
* #5308
* #5345
* #5368
2019-06-06 17:34:07 +01:00
Richard van der Hoff
71063a69b8
Fix missing logcontext for PresenceHandler.on_shutdown. ( #5369 )
...
Fixes some warnings, and a scary-looking stacktrace when sytest kills the
process.
2019-06-06 14:45:17 +01:00
Brendan Abolivier
8f06344e11
Merge pull request #5089 from dnaf/m-heroes-empty-room-name
...
Make /sync return heroes if room name or canonical alias are empty
2019-06-06 11:18:13 +01:00
Brendan Abolivier
64fa928792
Simplify condition
2019-06-06 10:34:12 +01:00
Brendan Abolivier
b9c43c8463
Merge pull request #5355 from matrix-org/babolivier/heroes_left_members
...
Include left members in room summaries' heroes
2019-06-06 10:05:27 +01:00
Erik Johnston
75538813fc
Fix background updates to handle redactions/rejections ( #5352 )
...
* Fix background updates to handle redactions/rejections
In background updates based on current state delta stream we need to
handle that we may not have all the events (or at least that
`get_events` may raise an exception).
2019-06-06 00:45:46 +10:00
Brendan Abolivier
0a2f522644
Simplify condition
2019-06-05 14:02:29 +01:00
Brendan Abolivier
40596aec0e
Merge branch 'develop' into m-heroes-empty-room-name
2019-06-05 13:38:01 +01:00
Brendan Abolivier
dbbaf25dd3
Do user_id != me checks before deciding whether we should pick heroes from the joined members or the parted ones
2019-06-05 11:50:27 +01:00
Erik Johnston
bc3d6b918b
Add logging when request fails and clarify we ignore errors.
2019-06-05 11:37:51 +01:00
Erik Johnston
220a733d73
Fix handling of failures when calling /event_auth.
...
When processing an incoming event over federation, we may try and
resolve any unexpected differences in auth events. This is a
non-essential process and so should not stop the processing of the event
if it fails (e.g. due to the remote disappearing or not implementing the
necessary endpoints).
Fixes #3330
2019-06-03 09:56:45 +01:00
Richard van der Hoff
d828d1dc57
Merge pull request #5309 from matrix-org/rav/limit_displayname_length
...
Limit displaynames and avatar URLs
2019-06-01 11:34:50 +01:00
Richard van der Hoff
d16c6375fe
Limit displaynames and avatar URLs
...
These end up in join events everywhere, so let's limit them.
Fixes #5079
2019-06-01 10:44:36 +01:00
Richard van der Hoff
5726378ece
Fix "db txn 'update_presence' from sentinel context" log messages ( #5275 )
...
Fixes #4414 .
2019-05-28 21:20:11 +01:00
Erik Johnston
d16f5574b6
Merge pull request #5220 from matrix-org/erikj/dont_bundle_live_events
...
Don't bundle aggregations with events in /sync or /events or state queries
2019-05-24 10:36:31 +01:00
Erik Johnston
4cb577c23f
Don't bundle aggs for /state and /members etc APIs
2019-05-24 09:52:33 +01:00
Andrew Morgan
6368150a74
Add config option for setting homeserver's default room version ( #5223 )
...
Replaces DEFAULT_ROOM_VERSION constant with a method that first checks the config, then returns a hardcoded value if the option is not present.
That hardcoded value is now located in the server.py config file.
2019-05-23 15:00:20 +01:00
Richard van der Hoff
85d1e03b9d
Simplifications and comments in do_auth ( #5227 )
...
I was staring at this function trying to figure out wtf it was actually
doing. This is (hopefully) a non-functional refactor which makes it a bit
clearer.
2019-05-23 11:17:42 +01:00
Amber Brown
4a30e4acb4
Room Statistics ( #4338 )
2019-05-21 11:36:50 -05:00
Erik Johnston
9526aa96a6
Merge pull request #5212 from matrix-org/erikj/deny_multiple_reactions
...
Block attempts to annotate the same event twice
2019-05-21 17:08:14 +01:00
Erik Johnston
44b8ba484e
Fix words
2019-05-21 16:51:45 +01:00
Richard van der Hoff
c4aef549ad
Exclude soft-failed events from fwd-extremity candidates. ( #5146 )
...
When considering the candidates to be forward-extremities, we must exclude soft
failures.
Hopefully fixes #5090 .
2019-05-21 16:10:54 +01:00
Erik Johnston
de7672b78f
Don't bundle events in /sync or /events
...
As we'll send down the annotations too anyway, so this just ends up
confusing clients.
2019-05-21 13:54:09 +01:00
Erik Johnston
c7ec06e8a6
Block attempts to annotate the same event twice
2019-05-20 17:39:05 +01:00
ReidAnderson
3787133c9e
Limit UserIds to a length that fits in a state key ( #5198 )
2019-05-20 11:20:08 +01:00
bytepoets-blo
291e1eea5e
fix mapping of return values for get_or_register_3pid_guest ( #5177 )
...
* fix mapping of return values for get_or_register_3pid_guest
2019-05-17 17:27:14 +01:00
Brendan Abolivier
cd32375846
Add option to disable per-room profiles
2019-05-16 14:34:28 +01:00
Matthew Hodgson
4a6d5de98c
Make /sync attempt to return device updates for both joined and invited users ( #3484 )
2019-05-16 13:23:43 +01:00
Amber Brown
f1e5b41388
Make all the rate limiting options more consistent ( #5181 )
2019-05-15 12:06:04 -05:00
Erik Johnston
b54b03f9e1
Allow client event serialization to be async
2019-05-14 11:58:01 +01:00
Matthew Hodgson
c0e0740bef
add options to require an access_token to GET /profile and /publicRooms on CS API ( #5083 )
...
This commit adds two config options:
* `restrict_public_rooms_to_local_users`
Requires auth to fetch the public rooms directory through the CS API and disables fetching it through the federation API.
* `require_auth_for_profile_requests`
When set to `true`, requires that requests to `/profile` over the CS API are authenticated, and only returns the user's profile if the requester shares a room with the profile's owner, as per MSC1301.
MSC1301 also specifies a behaviour for federation (only returning the profile if the server asking for it shares a room with the profile's owner), but that's currently really non-trivial to do in a not too expensive way. Next step is writing down a MSC that allows a HS to specify which user sent the profile query. In this implementation, Synapse won't send a profile query over federation if it doesn't believe it already shares a room with the profile's owner, though.
Groups have been intentionally omitted from this commit.
2019-05-08 18:26:56 +01:00
Brendan Abolivier
1473058b5e
Do checks on aliases for incoming m.room.aliases events ( #5128 )
...
Follow-up to #5124
Also added a bunch of checks to make sure everything (both the stuff added on #5124 and this PR) works as intended.
2019-05-08 17:01:30 +01:00
Erik Johnston
de655e669a
Merge pull request #5104 from matrix-org/erikj/ratelimit_3pid_invite
...
Ratelimit 3pid invites
2019-05-07 10:12:49 +01:00
Erik Johnston
176f31c2e3
Rate limit early
2019-05-02 15:23:08 +01:00
Brendan Abolivier
84196cb231
Add some limitations to alias creation
2019-05-02 11:05:11 +01:00
Erik Johnston
d6118c5be6
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/ratelimit_3pid_invite
2019-04-26 18:14:23 +01:00
Erik Johnston
28a81ed62f
Ratelimit 3pid invites
...
We do ratelimit sending the 3PID invite events, but that happens after
spamming the identity server.
2019-04-26 18:06:25 +01:00
Richard van der Hoff
bd0d45ca69
Fix infinite loop in presence handler
...
Fixes #5102
2019-04-26 11:14:49 +01:00
Katie Wolfe
7e07dc429f
Lint
...
I probably should've just run autopep8 in the first place...
Signed-off-by: Katie Wolfe <katie@dnaf.moe>
2019-04-24 12:43:18 -04:00
Katie Wolfe
b3e5db402d
Clean up code
...
Signed-off-by: Katie Wolfe <katie@dnaf.moe>
2019-04-24 12:26:55 -04:00
Katie Wolfe
5d3ed79944
Show heroes if room name or canonical alias are empty
...
Fixes #4194
Signed-off-by: Katie Wolfe <katie@dnaf.moe>
2019-04-24 12:26:53 -04:00
Katie Wolfe
7f025eb425
Show heroes if room name or canonical alias are empty
...
Fixes #4194
Signed-off-by: Katie Wolfe <katie@dnaf.moe>
2019-04-24 12:26:29 -04:00
Erik Johnston
ca90336a69
Merge branch 'develop' of github.com:matrix-org/synapse into babolivier/account_expiration
2019-04-17 19:44:40 +01:00
Brendan Abolivier
eaf41a943b
Add management endpoints for account validity
2019-04-17 19:34:45 +01:00
Brendan Abolivier
91934025b9
Merge pull request #5047 from matrix-org/babolivier/account_expiration
...
Send out emails with links to extend an account's validity period
2019-04-17 14:57:39 +01:00
Brendan Abolivier
20f0617e87
Send out emails with links to extend an account's validity period
2019-04-17 14:42:20 +01:00
Erik Johnston
c132c8e505
Handle the case of get_missing_events failing
...
Currently if a call to `/get_missing_events` fails we log an exception
and stop processing the top level event we received over federation.
Instead let's try and handle it sensibly given it is a somewhat expected
failure mode.
2019-04-10 10:39:54 +01:00
Brendan Abolivier
8e85493b0c
Add config option to block users from looking up 3PIDs ( #5010 )
2019-04-04 17:25:47 +01:00
Andrew Morgan
db265f0642
Prevent kicking users who aren't in the room ( #4999 )
...
Prevent kick events from succeeding if the user is not currently in the room.
2019-04-04 13:05:51 +01:00
Andrew Morgan
4a4d5c4fd6
Fix grammar and document get_current_users_in_room ( #4998 )
2019-04-03 14:32:20 +01:00
Neil Johnson
e8419554ff
Remove presence lists ( #4989 )
...
Remove presence list support as per MSC 1819
2019-04-03 11:11:15 +01:00
Erik Johnston
8f549c1177
Merge pull request #4982 from matrix-org/erikj/msc1915
...
Implement MSC1915 - 3PID unbind APIs
2019-04-03 11:07:09 +01:00
Andrew Morgan
66e78700a2
Transfer related groups on room upgrade ( #4990 )
...
Transfers the m.room.related_groups state event on room upgrade.
2019-04-02 17:15:24 +01:00
Erik Johnston
4ef5d17b96
Correctly handle id_server param
2019-04-02 11:20:09 +01:00
Erik Johnston
24232514bf
Remove threepid binding if id server returns 400/404/501
2019-04-02 11:20:09 +01:00
Erik Johnston
c75e2017f1
Fixup docstrings
2019-04-02 11:20:06 +01:00
Erik Johnston
057715aaa2
Allowing specifying IS to use in unbind API.
...
By default the homeserver will use the identity server used during the
binding of the 3PID to unbind the 3PID. However, we need to allow
clients to explicitly ask the homeserver to unbind via a particular
identity server, for the case where the 3PID was bound out of band from
the homeserver.
Implements MSC915.
2019-04-01 15:25:18 +01:00
Erik Johnston
9fbbc3d9e5
For unbind poke IS used during binding of 3PID
...
This changes the behaviour from using the server specified trusted
identity server to using the IS that used during the binding of the
3PID, if known.
This is the behaviour specified by MSC1915.
2019-04-01 15:23:30 +01:00
Erik Johnston
1666c0696a
Track IS used to bind 3PIDs
...
This will then be used to know which IS to default to when unbinding the
threepid.
2019-04-01 15:23:01 +01:00
Richard van der Hoff
54a87a7b08
Collect room-version variations into one place ( #4969 )
...
Collect all the things that make room-versions different to one another into
one place, so that it's easier to define new room versions.
2019-04-01 10:24:38 +01:00
Andrew Morgan
7a91b9d81c
Allow password providers to bind emails ( #4947 )
...
This PR allows password provider modules to bind email addresses when a user is registering and is motivated by matrix-org/matrix-synapse-ldap3#58
2019-03-28 15:48:07 +00:00
Erik Johnston
4e5f0f7ca0
Use an assert
2019-03-28 14:05:05 +00:00
Erik Johnston
40e56997bc
Review comments
2019-03-28 13:48:41 +00:00
Erik Johnston
197fae1639
Use event streams to calculate presence
...
Primarily this fixes a bug in the handling of remote users joining a
room where the server sent out the presence for all local users in the
room to all servers in the room.
We also change to using the state delta stream, rather than the
distributor, as it will make it easier to split processing out of the
master process (as well as being more flexible).
Finally, when sending presence states to newly joined servers we filter
out old presence states to reduce the number sent. Initially we filter
out states that are offline and have a last active more than a week ago,
though this can be changed down the line.
Fixes #3962
2019-03-27 13:41:36 +00:00
Andrew Morgan
bbd244c7b2
Support 3PID login in password providers ( #4931 )
...
Adds a new method, check_3pid_auth, which gives password providers
the chance to allow authentication with third-party identifiers such
as email or msisdn.
2019-03-26 17:48:30 +00:00
Amber Brown
ac396a0d32
Refactor out state delta handling into its own class ( #4917 )
2019-03-25 20:37:08 +11:00
Erik Johnston
01e6b405be
Merge pull request #4908 from matrix-org/erikj/block_peek_on_blocked_rooms
...
Deny peeking into rooms that have been blocked
2019-03-21 14:07:17 +00:00
Erik Johnston
3959858eaa
Merge pull request #4904 from matrix-org/erikj/fix_shutdown
...
Fixup shutdown room API
2019-03-21 11:24:42 +00:00
Erik Johnston
536a266520
Deny peeking into rooms that have been blocked
2019-03-21 11:20:13 +00:00
Erik Johnston
09f991a63d
Merge pull request #4896 from matrix-org/erikj/disable_room_directory
...
Add option to disable search room lists
2019-03-21 10:16:54 +00:00
Erik Johnston
cd62981a6a
Revert spurious delete
2019-03-20 17:51:27 +00:00
Erik Johnston
aa959a6c07
Use flags
2019-03-20 17:40:29 +00:00
Erik Johnston
7d47cc1305
Move requester check into assert_accepted_privacy_policy
2019-03-20 17:08:36 +00:00
Erik Johnston
74c46d81fa
Only require consent for events with an associated request
...
There are a number of instances where a server or admin may puppet a
user to join/leave rooms, which we don't want to fail if the user has
not consented to the privacy policy. We fix this by adding a check to
test if the requester has an associated access_token, which is used as a
proxy to answer the question of whether the action is being done on
behalf of a real request from the user.
2019-03-20 16:50:23 +00:00
Richard van der Hoff
a902d13180
Batch up outgoing read-receipts to reduce federation traffic. ( #4890 )
...
Rate-limit outgoing read-receipts as per #4730 .
2019-03-20 16:02:25 +00:00
Erik Johnston
cc197a61a1
Disable publishing to room list when its disabled
2019-03-20 14:30:36 +00:00
Erik Johnston
2c90422146
Pull out config option
2019-03-20 14:25:58 +00:00
Erik Johnston
7529038e66
Return before we log
2019-03-20 14:25:28 +00:00
Erik Johnston
213c98c00a
Add option to disable search room lists
...
This disables both local and remote room list searching.
2019-03-19 17:10:52 +00:00
Neil Johnson
88f0675967
fix test_auto_create_auto_join_where_no_consent ( #4886 )
2019-03-19 11:38:59 +00:00
Amber Brown
282c97327f
Migrate the user directory initial population to a background task ( #4864 )
2019-03-19 04:50:24 +11:00
Brendan Abolivier
651ad8bc96
Add ratelimiting on failed login attempts ( #4865 )
2019-03-18 12:57:20 +00:00
Brendan Abolivier
899e523d6d
Add ratelimiting on login ( #4821 )
...
Add two ratelimiters on login (per-IP address and per-userID).
2019-03-15 17:46:16 +00:00
Richard van der Hoff
2dee441bdb
Merge pull request #4852 from matrix-org/rav/move_rr_sending_to_worker
...
Move client receipt processing to federation sender worker.
2019-03-15 12:30:30 +00:00
Erik Johnston
9073cfc8bd
Merge pull request #4846 from matrix-org/hawkowl/userdir-search
...
Improve searching in the userdir
2019-03-14 13:54:15 +00:00
Richard van der Hoff
fdcad8eabd
Move client receipt processing to federation sender worker.
...
This is mostly a prerequisite for #4730 , but also fits with the general theme
of "move everything off the master that we possibly can".
2019-03-13 17:21:19 +00:00
Richard van der Hoff
eed7271b3b
declare a ReadReceipt class
...
I'm going to use this in queues and things, so it'll be useful to give it more
of a structure.
2019-03-13 17:20:55 +00:00
Andrew Morgan
d42c81d724
Transfer local user's push rules on room upgrade ( #4838 )
...
Transfer push rules (notifications) on room upgrade
2019-03-12 14:42:53 +00:00
Amber Brown
c0332d095f
fixup
2019-03-13 01:30:54 +11:00
Amber Brown
10480c4348
fixup
2019-03-12 21:47:14 +11:00
Amber Brown
5ba8ceab4c
fixes
2019-03-12 00:35:31 +11:00
Amber Brown
4abc988c6a
initial
2019-03-11 21:11:36 +11:00
Erik Johnston
8c4896668f
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/soft_fail_impl
2019-03-08 11:44:20 +00:00
Erik Johnston
0ff8163eae
Factor out soft fail checks
2019-03-08 11:26:33 +00:00
Erik Johnston
469b66c102
Merge pull request #4829 from matrix-org/erikj/device_list_seen_updates
...
When re-syncing device lists reset the state
2019-03-08 09:54:54 +00:00
Erik Johnston
d42b41544a
When re-syncing device lists reset the state
...
We keep track of what stream IDs we've seen so that we know what updates
we've handled or missed. If we re-sync we don't know if the updates
we've seen are included in the re-sync (there may be a race), so we
should reset the seen updates.
2019-03-07 16:04:24 +00:00
Erik Johnston
c633fc02d7
Add some debug logging for device list handling
2019-03-07 15:58:03 +00:00
Amber Brown
f6135d06cf
Rewrite userdir to be faster ( #4537 )
2019-03-07 01:22:53 -08:00
Erik Johnston
0a6e716600
Merge pull request #4815 from matrix-org/erikj/docstrings
...
Add docstrings from matrix-org-hotfixes
2019-03-06 18:59:28 +00:00
Erik Johnston
8b7790e68f
Port #4422 debug logging from hotfixes
2019-03-06 17:29:15 +00:00
Erik Johnston
6d13bdec91
Add docstrings from matrix-org-hotfixes
2019-03-06 17:24:26 +00:00
Erik Johnston
a9de04be72
Implement soft fail
2019-03-06 16:22:16 +00:00
Brendan Abolivier
6f3cde8b25
Make registration ratelimiter separate from the main events one
2019-03-06 11:02:42 +00:00
Brendan Abolivier
f4195f4118
Revert "Split ratelimiters in two (one for events, one for registration)"
...
This reverts commit d7dbad3526
2019-03-06 10:55:22 +00:00
Brendan Abolivier
d7dbad3526
Split ratelimiters in two (one for events, one for registration)
2019-03-05 18:41:27 +00:00
Brendan Abolivier
a4c3a361b7
Add rate-limiting on registration ( #4735 )
...
* Rate-limiting for registration
* Add unit test for registration rate limiting
* Add config parameters for rate limiting on auth endpoints
* Doc
* Fix doc of rate limiting function
Co-Authored-By: babolivier <contact@brendanabolivier.com>
* Incorporate review
* Fix config parsing
* Fix linting errors
* Set default config for auth rate limiting
* Fix tests
* Add changelog
* Advance reactor instead of mocked clock
* Move parameters to registration specific config and give them more sensible default values
* Remove unused config options
* Don't mock the rate limiter un MAU tests
* Rename _register_with_store into register_with_store
* Make CI happy
* Remove unused import
* Update sample config
* Fix ratelimiting test for py2
* Add non-guest test
2019-03-05 14:25:33 +00:00
Erik Johnston
b050a10871
Merge pull request #4699 from matrix-org/erikj/stop_fed_not_in_room
...
Stop backpaginating when events not visible
2019-03-05 09:32:33 +00:00
Erik Johnston
aa06d26ae0
clarify comments
2019-03-05 09:16:35 +00:00
Erik Johnston
c3c542bb4a
Merge pull request #4796 from matrix-org/erikj/factor_out_e2e_keys
...
Allow /keys/{changes,query} API to run on worker
2019-03-05 09:06:25 +00:00
Erik Johnston
bfa7d46a10
Allow /keys/{changes,query} API to run on worker
2019-03-04 18:30:01 +00:00
Erik Johnston
157e5a8f27
Split DeviceHandler into master and worker
2019-03-04 18:29:26 +00:00
Richard van der Hoff
b29693a30b
Clean up read-receipt handling.
...
Remove a call to run_as_background_process: there is no need to run this as a
background process, because build_and_send_edu does not block.
We may as well inline the whole of _push_remotes.
2019-03-04 18:16:43 +00:00
Erik Johnston
d1523aed6b
Only check history visibility when filtering
...
When filtering events to send to server we check more than just history
visibility. However when deciding whether to backfill or not we only
care about the history visibility.
2019-03-04 14:43:42 +00:00
Richard van der Hoff
856c83f5f8
Avoid rebuilding Edu objects in worker mode ( #4770 )
...
In worker mode, on the federation sender, when we receive an edu for sending
over the replication socket, it is parsed into an Edu object. There is no point
extracting the contents of it so that we can then immediately build another Edu.
2019-03-04 12:57:44 +00:00
Erik Johnston
8b63fe4c26
s/get_forward_events/get_successor_events/
2019-03-04 11:56:03 +00:00
Erik Johnston
fbc047f2a5
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/stop_fed_not_in_room
2019-03-04 11:54:58 +00:00
Richard van der Hoff
3064952939
Fix incorrect log about not persisting duplicate state event. ( #4776 )
...
We were logging this when it was not true.
2019-03-01 16:47:12 +00:00
Erik Johnston
b183fef9ac
Update comments
2019-02-27 13:06:10 +00:00
Andrew Morgan
802884d4ee
Merge branch 'develop' of github.com:matrix-org/synapse into anoa/public_rooms_federate_develop
2019-02-26 14:23:40 +00:00
Andrew Morgan
6fcb25202f
Put function def back to the way it was
2019-02-26 14:17:38 +00:00
Andrew Morgan
7a4632af9c
Prevent showing non-fed rooms in fed /publicRooms
2019-02-26 13:37:24 +00:00
Andrew Morgan
c74624a633
Revert "Prevent showing non-fed rooms in fed /publicRooms"
2019-02-26 13:20:38 +00:00
Andrew Morgan
a1a6473293
Merge pull request #4736 from matrix-org/anoa/public_rooms_federate
...
Config option to prevent showing non-fed rooms in fed /publicRooms
2019-02-26 13:07:15 +00:00
Andrew Morgan
c4414768af
Cleaner chunk logic
2019-02-26 12:22:34 +00:00
Andrew Morgan
a712aa3a9c
Correct indent
2019-02-26 12:13:55 +00:00
Andrew Morgan
16565e67db
Correct docstring types and chunk logic
2019-02-26 12:12:48 +00:00
Andrew Morgan
40c2271680
Clean up room chunk logic
2019-02-26 12:04:34 +00:00
Andrew Morgan
6728bf3940
Make not showing non-federated rooms the default
2019-02-26 11:52:52 +00:00
Andrew Morgan
6946c20111
Result may be None
2019-02-26 11:27:19 +00:00
Andrew Morgan
71669a0fba
Address rich comments
2019-02-26 11:25:00 +00:00
Matthew Hodgson
70ea2f4e1d
switch from google.com to recaptcha.net for reCAPTCHA ( #4731 )
...
* add trivial clarification about jemalloc
* switch from google.com to recaptcha.net
because https://developers.google.com/recaptcha/docs/faq#can-i-use-recaptcha-globally
2019-02-25 11:15:36 -08:00
Andrew Morgan
1330aa4a8f
elif not else if
2019-02-25 17:28:19 +00:00
Erik Johnston
65f3fbfbf7
Merge pull request #4737 from matrix-org/erikj/failure_log_tb
...
Log tracebacks correctly
2019-02-25 17:26:30 +00:00
Erik Johnston
c7b333c545
Log tracebacks correctly
2019-02-25 16:56:41 +00:00
Erik Johnston
9c598dddcb
Fix typo
2019-02-25 16:32:02 +00:00
Andrew Morgan
8aaf7ffc44
syntax derp
2019-02-25 15:27:17 +00:00
Andrew Morgan
84c0a20dfe
Simplify call to generate_room_entry
2019-02-25 15:23:27 +00:00
Erik Johnston
4b9e5076c4
Merge branch 'develop' of github.com:matrix-org/synapse into anoa/public_rooms_federate
2019-02-25 15:08:18 +00:00
Andrew Morgan
07493607a8
Docs and arg name clarification
2019-02-25 15:04:46 +00:00
Andrew Morgan
bd398b874e
Don't restrict non-fed rooms over client APIs
2019-02-25 15:04:46 +00:00
Andrew Morgan
e4b078a600
Config option to prevent showing non-fed rooms in fed /publicRooms
2019-02-25 15:04:46 +00:00
Erik Johnston
d730c2c22b
More comments
2019-02-25 14:45:02 +00:00
Erik Johnston
890cb048fd
Assert rather than clobber the values
2019-02-25 14:42:11 +00:00
Erik Johnston
9342cc6ab1
Add comments and paranoia
2019-02-25 10:02:12 +00:00
Erik Johnston
7b288826b7
Fix backfill storing incorrect state for events
2019-02-22 11:33:51 +00:00
Richard van der Hoff
6d65659b62
Run push_receipts_to_remotes as background job ( #4707 )
...
I suspect the CPU usage metrics for this are going to /dev/null at the moment.
2019-02-21 17:50:30 +00:00
Erik Johnston
b9d6756b14
Merge pull request #4263 from rkfg/develop
...
Prevent crash on pagination.
2019-02-21 17:42:15 +00:00
Erik Johnston
71b625d808
Stop backpaginating when events not visible
2019-02-20 18:14:12 +00:00
Erik Johnston
dbdc565dfd
Fix registration on workers ( #4682 )
...
* Move RegistrationHandler init to HomeServer
* Move post registration actions to RegistrationHandler
* Add post regisration replication endpoint
* Newsfile
2019-02-20 18:47:31 +11:00
Brendan Abolivier
a288bdf0b1
Merge pull request #4652 from matrix-org/babolivier/acme-delegated
...
Support .well-known delegation when issuing certificates through ACME
2019-02-19 11:15:38 +00:00
Brendan Abolivier
a8626901cd
Fetch ACME domain into an instance member
2019-02-19 10:54:33 +00:00
Andrew Morgan
968a30a75c
Merge pull request #4642 from matrix-org/anoa/bans_room_upgrade
...
Transfer bans on room upgrade
2019-02-19 10:16:42 +00:00
Andrew Morgan
34ac75ce2c
lint
2019-02-18 18:23:37 +00:00
Andrew Morgan
f8b9ca53ce
Move member event processing and changelog fix
2019-02-18 17:28:52 +00:00
Erik Johnston
fc2c245a1f
Merge pull request #4666 from matrix-org/erikj/register_login_split
...
Split out registration to worker
2019-02-18 17:18:06 +00:00
Erik Johnston
459d3d5046
Merge pull request #4668 from matrix-org/erikj/catch_exceptions
...
Correctly handle HttpResponseException when handling device updates
2019-02-18 16:55:43 +00:00
Erik Johnston
af691e415c
Move register_device into handler
2019-02-18 16:49:38 +00:00
Erik Johnston
c30f73c86a
Merge pull request #4667 from matrix-org/erikj/fix_revoke_guest_access_workers
...
Fix kicking guest users in worker mode
2019-02-18 15:55:48 +00:00
Brendan Abolivier
45bb55c6de
Use a configuration parameter to give the domain to generate a certificate for
2019-02-18 15:46:23 +00:00
Erik Johnston
94960cef03
pep8
2019-02-18 15:24:13 +00:00
Erik Johnston
e07cc31cb8
Correctly handle HttpResponseException
2019-02-18 14:55:09 +00:00
Brendan Abolivier
68a53f825f
Merge branch 'develop' into babolivier/acme-delegated
2019-02-18 14:52:23 +00:00
Erik Johnston
32e54b472a
Fix kicking guest users in worker mode
...
When guest_access changes from allowed to forbidden all local guest
users should be kicked from the room. This did not happen when
revocation was received from federation on a worker.
Presumably broken in #4141
2019-02-18 14:16:07 +00:00
Andrew Morgan
915421065b
Membership events are done later
2019-02-18 14:02:09 +00:00
Erik Johnston
d1b060b492
Merge pull request #4643 from matrix-org/erikj/catch_exceptions
...
Correctly handle RequestSendFailed exceptions
2019-02-18 13:53:24 +00:00
Andrew Morgan
9caab0c364
Transfer bans on room upgrade
2019-02-18 13:52:51 +00:00
Erik Johnston
eb2b8523ae
Split out registration to worker
...
This allows registration to be handled by a worker, though the actual
write to the database still happens on master.
Note: due to the in-memory session map all registration requests must be
handled by the same worker.
2019-02-18 12:12:57 +00:00
Andrew Morgan
6d02a13d81
Typo in info log
...
Co-Authored-By: babolivier <contact@brendanabolivier.com>
2019-02-18 11:36:34 +00:00
Brendan Abolivier
f86b695cbd
Various cosmetics to make TravisCI happy
2019-02-15 12:29:34 +00:00
Brendan Abolivier
af8a2f679b
Remove unused import
2019-02-15 12:27:43 +00:00
Brendan Abolivier
1895d14e12
Support .well-known delegation when issuing certificates through ACME
2019-02-15 12:05:08 +00:00
Erik Johnston
f61b2068e6
Only fetch aliases when publishing rooms
2019-02-14 18:08:09 +00:00
Erik Johnston
eaf4d11af9
Add configurable room list publishing rules
...
This allows specifying who and what is allowed to be published onto the
public room list
2019-02-14 16:02:23 +00:00
Erik Johnston
7fc1196a36
Correctly handle RequestSendFailed exceptions
...
This mainly reduces the number of exceptions we log.
2019-02-14 14:01:04 +00:00
Amber Brown
bb4fd8f927
Run black on user directory code ( #4635 )
2019-02-13 23:05:32 +11:00
Erik Johnston
ba3f27b69a
Merge pull request #4608 from matrix-org/anoa/acls_room_upgrade
...
Transfer Server ACLs on room upgrade
2019-02-12 13:20:06 +00:00
Andrew Morgan
a126f86eec
Transfer Server ACLs on room upgrade
2019-02-11 11:30:37 +00:00
Hubert Chathi
afae8442b5
make sure version is in body and wrap in linearizer queue
...
also add tests
2019-02-08 01:32:45 -05:00
Hubert Chathi
d9e424bf64
re-try to make isort happy
2019-02-06 22:18:41 -05:00
Hubert Chathi
9ff620a518
fix import to make isort happy
2019-02-06 21:32:52 -05:00
Hubert Chathi
8248637173
add new endpoint to update backup versions
2019-02-06 17:57:10 -05:00
Hubert Chathi
664c81e8b7
return proper error codes for some 404s
2019-02-06 17:47:22 -05:00
Andrew Morgan
c45fd0dda0
Merge pull request #4530 from matrix-org/anoa/room_upgrade_federatable
...
Copy over non-federatable trait on room upgrade
2019-01-31 20:28:37 +00:00
Richard van der Hoff
07dfe148de
Add some debug for membership syncing issues ( #4538 )
...
I can't figure out what's going on with #4422 and #4436 ; perhaps this will help.
2019-01-31 18:30:40 +00:00
Andrew Morgan
3ed3cb4339
New function for getting room's create event
2019-01-31 18:21:39 +00:00
Andrew Morgan
bbb97a35fd
Merge branch 'develop' of github.com:matrix-org/synapse into anoa/room_upgrade_federatable
2019-01-31 18:11:00 +00:00
Andrew Morgan
fb50934b8f
lint
2019-01-31 11:34:45 +00:00
Andrew Morgan
d621c5562e
Copy over non-federatable trait on room upgrade
2019-01-30 16:33:51 +00:00
Erik Johnston
ed8c5e4cda
Fix remote invite rejections not comming down sync
...
This was broken in PR #4405 , commit 886e5ac
2019-01-30 15:46:27 +00:00
Richard van der Hoff
7615a8ced1
ACME config cleanups ( #4525 )
...
* Handle listening for ACME requests on IPv6 addresses
the weird url-but-not-actually-a-url-string doesn't handle IPv6 addresses
without extra quoting. Building a string which you are about to parse again
seems like a weird choice. Let's just use listenTCP, which is consistent with
what we do elsewhere.
* Clean up the default ACME config
make it look a bit more consistent with everything else, and tweak the defaults
to listen on port 80.
* newsfile
2019-01-30 14:17:55 +00:00
Amber Brown
3f189c902e
Fix flake8 ( #4519 )
2019-01-30 10:53:17 +00:00
Erik Johnston
67b82f1336
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/redactions_eiah
2019-01-29 22:58:38 +00:00
Erik Johnston
a696c48133
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/redactions_eiah
2019-01-29 22:00:33 +00:00
Erik Johnston
55d9024835
Use snder and not event ID domain to check if ours
...
The transaction queue only sends out events that we generate. This was
done by checking domain of event ID, but that can no longer be used.
Instead, we may as well use the sender field.
2019-01-29 16:54:23 +00:00
Erik Johnston
b1fffca345
Remove event ID usage when checking if new room
...
The event ID is changing, so we can no longer get the domain from it. On
the other hand, the check is unnecessary.
2019-01-29 16:15:02 +00:00
Erik Johnston
b8d75ef53e
Merge pull request #4481 from matrix-org/erikj/event_builder
...
Refactor event building into EventBuilder
2019-01-29 14:07:23 +00:00
Erik Johnston
5891a6edc8
Correctly set context.app_service
2019-01-29 12:09:10 +00:00
Erik Johnston
7709d2bd16
Implement rechecking of redactions
2019-01-29 11:56:20 +00:00
Erik Johnston
6598992b01
Fixup calls to comput_event_signature
...
We currently pass FrozenEvent instead of `dict` to
`compute_event_signature`, which works by accident due to `dict(event)`
producing the correct result.
This fixes PR #4493 commit 855a151
2019-01-29 11:41:58 +00:00
Erik Johnston
be47cfa9c9
Refactor event building into EventBuilder
...
This is so that everything is done in one place, making it easier to
change the event format based on room version
2019-01-29 11:13:00 +00:00
Erik Johnston
073f6c2e5e
Merge pull request #4494 from matrix-org/erikj/fixup_event_validator
...
Split up event validation between event and builder
2019-01-29 10:55:07 +00:00
Erik Johnston
f1a04462eb
Merge pull request #4482 from matrix-org/erikj/event_auth_room_version
...
Pass through room version to event auth
2019-01-28 20:09:38 +00:00
Erik Johnston
d758d5310e
Correctly use default room version if none is set
2019-01-28 17:26:39 +00:00
Erik Johnston
b872c7b1b4
Split up event validation between event and builder
...
The validator was being run on the EventBuilder objects, and so the
validator only checked a subset of fields. With the upcoming
EventBuilder refactor even fewer fields will be there to validate.
To get around this we split the validation into those that can be run
against an EventBuilder and those run against a fully fledged event.
2019-01-28 17:00:14 +00:00
Andrew Morgan
f0e96ab66a
Change return syntax in doc string
2019-01-28 14:09:45 +00:00
Andrew Morgan
4026d555fa
Merge branch 'develop' of github.com:matrix-org/synapse into anoa/dm_room_upgrade
2019-01-28 14:08:24 +00:00
Andrew Morgan
1ce463963d
Reuse predecessor method
2019-01-28 14:08:18 +00:00
Erik Johnston
ae2a957dba
Pass through room version to event auth
2019-01-25 18:31:41 +00:00
Erik Johnston
b6dce9b9fd
Merge pull request #4470 from matrix-org/erikj/require_format_version
...
Require event format version to parse or create events
2019-01-25 15:59:36 +00:00
Andrew Morgan
9244a3089e
Fixes
2019-01-25 11:48:38 +00:00
Andrew Morgan
da0d2219d2
Clean up direct_rooms access
2019-01-25 11:37:12 +00:00
Andrew Morgan
8265995498
Use python magic
2019-01-25 11:26:06 +00:00
Andrew Morgan
c4cdafa81f
Destructure account data tuple before use
2019-01-25 11:24:28 +00:00
Andrew Morgan
516456b763
Remove unnecessary null check
2019-01-25 11:22:14 +00:00
Andrew Morgan
6f3fda79ce
Move room_tag declaration to be closer to its use
2019-01-25 11:21:25 +00:00
Andrew Morgan
821b65aeb5
Merge branch 'develop' of github.com:matrix-org/synapse into anoa/dm_room_upgrade
2019-01-25 11:09:53 +00:00
Andrew Morgan
0862d35b8e
Move tag and direct state copying into separate function
2019-01-25 11:09:34 +00:00
Andrew Morgan
b1b6dba2d2
Merge pull request #4415 from matrix-org/anoa/full_search_upgraded_rooms
...
Ability to search entire room history after upgrading room
2019-01-25 10:49:58 +00:00
Erik Johnston
a50cf929c1
Require event format version to parse or create events
2019-01-25 10:32:19 +00:00
Erik Johnston
62514bb81b
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/msc_1813
2019-01-25 10:07:08 +00:00
Erik Johnston
edc1e21dbe
Merge pull request #4448 from matrix-org/erikj/get_pdu_versions
...
Add room_version param to get_pdu
2019-01-24 18:47:15 +00:00
Erik Johnston
80bcca659e
Merge pull request #4405 from matrix-org/erikj/fixup_rejecting_invites
...
Store rejected remote invite events as outliers
2019-01-24 18:33:27 +00:00
Erik Johnston
d148c43050
Review comments
2019-01-24 18:31:23 +00:00
Erik Johnston
9139b87be4
Remove unecessary setting of outlier bit
2019-01-24 18:04:02 +00:00
Erik Johnston
b8082a5445
Use term 'out of band membership' instead
2019-01-24 17:33:19 +00:00
Richard van der Hoff
03c85335d1
Apply suggestions from code review
...
Co-Authored-By: anoadragon453 <1342360+anoadragon453@users.noreply.github.com>
2019-01-24 17:22:09 +00:00
Andrew Morgan
075ff3ede9
Change default timeout value from 0 to None
2019-01-24 15:10:22 +00:00
Andrew Morgan
5b1dc94083
Use self.clock instead of datetime
2019-01-24 14:59:50 +00:00
Andrew Morgan
a2d85144e5
isort
2019-01-24 14:22:26 +00:00
Andrew Morgan
5541645e80
lint
2019-01-24 12:45:32 +00:00
Andrew Morgan
068aa1d228
Time out filtered room dir queries after 60s
2019-01-24 12:44:27 +00:00
Erik Johnston
7c288c2250
Clarify the invite flows
2019-01-23 20:07:47 +00:00
Erik Johnston
07f62da55a
Remove unnecessary '_sign_event'
2019-01-23 20:04:57 +00:00
Erik Johnston
886e5acc76
Store rejected remote invite events as outliers
...
Currently they're stored as non-outliers even though the server isn't in
the room, which can be problematic in places where the code assumes it
has the state for all non outlier events.
In particular, there is an edge case where persisting the leave event
triggers a state resolution, which requires looking up the room version
from state. Since the server doesn't have the state, this causes an
exception to be thrown.
2019-01-23 20:04:57 +00:00
Erik Johnston
6a41d2a187
Add room_version param to get_pdu
...
When we add new event format we'll need to know the event format or room
version when parsing events.
2019-01-23 17:19:58 +00:00
Erik Johnston
67cd4dad81
Implement MSC 1813 - Add room version to make APIs
...
We also implement `make_membership_event` converting the returned
room version to an event format version.
2019-01-23 16:51:46 +00:00
Andrew Morgan
82a92ba535
Add metric for user dir current event stream position
2019-01-23 15:34:47 +00:00
Amber Brown
6129e52f43
Support ACME for certificate provisioning ( #4384 )
2019-01-23 19:39:06 +11:00
Andrew Morgan
766a172b99
lint
2019-01-22 13:51:40 +00:00
Andrew Morgan
c4875d8c76
Prevent duplicate room IDs in m.direct
2019-01-22 12:13:46 +00:00
Andrew Morgan
c9bfb058d8
Fix a bug with single-room search searching all rooms
...
* Create a new method for getting predecessor rooms
* Remove formatting change
2019-01-22 12:00:41 +00:00
Andrew Morgan
8086a5c05e
Fix comments
2019-01-22 11:16:23 +00:00
Andrew Morgan
48951f437f
Join logic covers both room creator and arbitrary users
2019-01-22 11:00:04 +00:00
Andrew Morgan
8c85f0833d
tags, m.direct copying over correctly
2019-01-22 11:00:04 +00:00
Andrew Morgan
25d64a846a
Fix typos
2019-01-22 11:00:04 +00:00
Andrew Morgan
ea8903fcc9
Migrating dm and room tags work for migrator
2019-01-22 11:00:04 +00:00
Andrew Morgan
887ca93a1b
Prevent crash on user who doesn't have any direct rooms
2019-01-22 11:00:04 +00:00
Andrew Morgan
4ff6d22245
Preserve DM status of a room on upgrade
...
Signed-off-by: Andrew Morgan <andrew@amorgan.xyz>
2019-01-22 11:00:04 +00:00
Andrew Morgan
75942af1db
Fix typo
2019-01-22 11:00:04 +00:00
Andrew Morgan
702c4b750c
Migrate encryption state on room upgrade ( #4411 )
...
* Migrate encryption state on room upgrade
Signed-off-by: Andrew Morgan <andrew@amorgan.xyz>
* Add changelog file
2019-01-21 20:42:58 +11:00
Andrew Morgan
df3a661e4a
Search for messages across predecessor rooms
...
Signed-off-by: Andrew Morgan <andrew@amorgan.xyz>
2019-01-18 11:19:20 +00:00
Richard van der Hoff
9feb5d0b71
sign_request -> build_auth_headers ( #4408 )
...
Just got very confused about the fact that the headers are only an output, not
an input.
2019-01-17 12:40:09 +00:00
Richard van der Hoff
05e1296649
don't store more remote device lists if they have more than 1K devices ( #4397 )
2019-01-16 23:14:11 +00:00
Matthew Hodgson
27128145e6
fix NPE in /messages by checking if all events were filtered out ( #4330 )
2019-01-02 23:37:39 +00:00
Nathan Pennie
81b513416e
Fixed line length
2018-12-19 14:12:33 -05:00
Nathan Pennie
668e6625b0
Fix the variable names used for account_data
2018-12-19 06:53:02 -05:00
Neil Johnson
d2f7c4e6b1
create support user ( #4141 )
...
Allow for the creation of a support user.
A support user can access the server, join rooms, interact with other users, but does not appear in the user directory nor does it contribute to monthly active user limits.
2018-12-14 18:20:59 +00:00
Richard van der Hoff
30da50a5b8
Initialise user displayname from SAML2 data ( #4272 )
...
When we register a new user from SAML2 data, initialise their displayname
correctly.
2018-12-07 14:44:46 +01:00
Richard van der Hoff
c588b9b9e4
Factor SSO success handling out of CAS login ( #4264 )
...
This is mostly factoring out the post-CAS-login code to somewhere we can reuse
it for other SSO flows, but it also fixes the userid mapping while we're at it.
2018-12-07 13:10:07 +01:00
rkfg
ae19a7db8c
Prevent crash on pagination.
2018-12-06 13:32:05 +03:00
Richard van der Hoff
b0c24a66ec
Rip out half-implemented m.login.saml2 support ( #4265 )
...
* Rip out half-implemented m.login.saml2 support
This was implemented in an odd way that left most of the work to the client, in
a way that I really didn't understand. It's going to be a pain to maintain, so
let's start by ripping it out.
* drop undocumented dependency on dateutil
It turns out we were relying on dateutil being pulled in transitively by
pysaml2. There's no need for that bloat.
2018-12-06 19:44:38 +11:00
Travis Ralston
158ffb92f1
Add an option to disable search for homeservers which may not be interested in it ( #4230 )
...
This is useful for homeservers not intended for users, such as bot-only homeservers or ones that only process IoT data.
2018-12-04 12:01:02 +01:00
Neil Johnson
7039ece8fb
Neilj/fix autojoin ( #4223 )
...
* Fix auto join failures for servers that require user consent
* Fix auto join failures for servers that require user consent
2018-11-28 22:24:57 +11:00
Travis Ralston
ab4526a153
Remove duplicate slashes in generated consent URLs
2018-11-15 20:41:53 -07:00
Erik Johnston
dc59ad5334
Remove hack to support rejoining rooms
2018-11-09 14:58:09 +00:00
Erik Johnston
30dd27afff
Simplify to always drop events if server isn't in the room
2018-11-09 11:36:45 +00:00
Travis Ralston
0f5e51f726
Add config variables for enabling terms auth and the policy name ( #4142 )
...
So people can still collect consent the old way if they want to.
2018-11-06 10:32:34 +00:00
Hubert Chathi
f1087106cf
handle empty backups according to latest spec proposal ( #4123 )
...
fixes #4056
2018-11-05 17:59:29 -05:00
Erik Johnston
bc80b3f454
Add helpers for getting prev and auth events ( #4139 )
...
* Add helpers for getting prev and auth events
This is in preparation for allowing the event format to change between
room versions.
2018-11-06 00:35:15 +11:00
Amber Brown
cb7a6b2379
Fix typing being reset causing infinite syncs ( #4127 )
2018-11-03 00:19:23 +11:00
Travis Ralston
a8d41c6aff
Include a version query string arg for the consent route
2018-10-31 13:19:28 -06:00
Travis Ralston
d1e7b9c44c
Merge branch 'develop' into travis/login-terms
2018-10-31 13:15:14 -06:00
Richard van der Hoff
1729ba1650
Merge pull request #4101 from matrix-org/rav/aliases_for_upgrades
...
Attempt to move room aliases on room upgrades
2018-10-31 17:52:18 +00:00
Richard van der Hoff
0f8591a5a8
Avoid else clause on exception for clarity
2018-10-31 15:43:57 +00:00
Richard van der Hoff
94c7fadc98
Attempt to move room aliases on room upgrades
2018-10-31 15:43:57 +00:00
Richard van der Hoff
9b827c40ca
Log some bits about event creation ( #4121 )
...
I found these helpful in debugging my room upgrade tests.
2018-10-31 15:42:23 +00:00
Amber Brown
086e1a8f3e
Fix search 500ing ( #4122 )
2018-10-31 06:26:46 +11:00
Amber Brown
3bade14ec0
Fix search 500ing ( #4122 )
2018-10-31 04:33:41 +11:00
Amber Brown
4cd1c9f2ff
Delete the disused & unspecced identicon functionality ( #4106 )
2018-10-29 23:57:24 +11:00
Richard van der Hoff
7fbfea062e
Merge pull request #4100 from matrix-org/rav/room_upgrade_avatar
...
Remember to copy the avatar on room upgrades
2018-10-29 12:49:21 +00:00
Richard van der Hoff
db24d7f15e
Better handling of odd PLs during room upgrades
...
Fixes handling of rooms where we have permission to send the tombstone, but not
other state. We need to (a) fail more gracefully when we can't send the PLs in
the old room, and (b) not set the PLs in the new room until we are done with
the other stuff.
2018-10-27 00:54:26 +01:00
Richard van der Hoff
5caf79b312
Remember to copy the avatar on room upgrades
2018-10-26 23:56:40 +01:00
Richard van der Hoff
54bbe71867
optimise state copying
2018-10-26 22:51:34 +01:00
Richard van der Hoff
193cadc988
Address review comments
...
Improve comments, get old room state from the context we already have
2018-10-26 17:10:30 +01:00
Richard van der Hoff
e6babc27d5
restrict PLs in old room
2018-10-25 19:18:25 +01:00
Richard van der Hoff
3a263bf3ae
copy state
2018-10-25 19:18:25 +01:00
Richard van der Hoff
1b9f253e20
preserve PLs
2018-10-25 19:10:24 +01:00
Richard van der Hoff
4cda300058
preserve room visibility
2018-10-25 19:10:24 +01:00
Richard van der Hoff
0f7d1c9906
Basic initial support for room upgrades
...
Currently just creates a new, empty, room, and sends a tombstone in the old
room.
2018-10-25 19:10:24 +01:00
Richard van der Hoff
e1948175ee
Allow power_level_content_override=None for _send_events_for_new_room
2018-10-25 19:10:24 +01:00
Richard van der Hoff
7f7b2cd3de
Make room_member_handler a member of RoomCreationHandler
...
... to save passing it into `_send_events_for_new_room`
2018-10-25 19:10:18 +01:00
Richard van der Hoff
871c4abfec
Factor _generate_room_id out of create_room
...
we're going to need this for room upgrades.
2018-10-25 18:23:09 +01:00
Erik Johnston
cb53ce9d64
Refactor state group lookup to reduce DB hits ( #4011 )
...
Currently when fetching state groups from the data store we make two
hits two the database: once for members and once for non-members (unless
request is filtered to one or the other). This adds needless load to the
datbase, so this PR refactors the lookup to make only a single database
hit.
2018-10-25 17:49:55 +01:00
Erik Johnston
b94a43d5b5
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/alias_disallow_list
2018-10-25 15:25:31 +01:00
Neil Johnson
f7f487e14c
Merge branch 'develop' of github.com:matrix-org/synapse into matthew/autocreate_autojoin
2018-10-25 14:40:06 +01:00
Travis Ralston
54def42c19
Merge branch 'develop' into travis/login-terms
2018-10-24 13:22:59 -06:00
Neil Johnson
9532caf6ef
remove trailing whiter space
2018-10-24 16:08:25 +01:00
Richard van der Hoff
94a49e0636
fix tuple
...
Co-Authored-By: neilisfragile <neil@matrix.org>
2018-10-24 14:39:23 +01:00
Erik Johnston
3904cbf307
Merge pull request #4040 from matrix-org/erikj/states_res_v2_rebase
...
Add v2 state resolution algorithm
2018-10-24 11:12:12 +01:00
Richard van der Hoff
e0b9d5f0af
Merge pull request #4075 from matrix-org/rav/fix_pusher_logcontexts
...
Clean up the way logcontexts and threads work in the pushers
2018-10-24 09:53:57 +01:00
Erik Johnston
dacbeb2e03
Comment
2018-10-24 09:47:49 +01:00
Erik Johnston
810715f79a
Rename resolve_events_with_factory
2018-10-24 09:44:22 +01:00
Erik Johnston
cb23aa4c42
Merge pull request #4063 from matrix-org/erikj/cleaup_alias_creation
...
Clean up room alias creation
2018-10-24 09:40:27 +01:00
Neil Johnson
a67d8ace9b
remove errant exception and style
2018-10-23 17:44:39 +01:00
Travis Ralston
43c3f0b02f
Merge pull request #3969 from turt2live/travis/fix-federated-group-requests
...
Handle HttpResponseException more safely for federated groups
2018-10-23 10:41:04 -06:00
Travis Ralston
3e704822be
Comments help
2018-10-23 10:25:31 -06:00
Richard van der Hoff
b3f6dddad2
Give some more things logcontexts ( #4077 )
2018-10-23 14:29:17 +01:00
Richard van der Hoff
5c445114d3
Correctly account for cpu usage by background threads ( #4074 )
...
Wrap calls to deferToThread() in a thing which uses a child logcontext to
attribute CPU usage to the right request.
While we're in the area, remove the logcontext_tracer stuff, which is never
used, and afaik doesn't work.
Fixes #4064
2018-10-23 13:12:32 +01:00
Richard van der Hoff
e7a16c6210
Remove redundant run_as_background_process() from pusherpool
...
`on_new_notifications` and `on_new_receipts` in `HttpPusher` and `EmailPusher`
now always return synchronously, so we can remove the `defer.gatherResults` on
their results, and the `run_as_background_process` wrappers can be removed too
because the PusherPool methods will now complete quickly enough.
2018-10-22 16:12:11 +01:00
Erik Johnston
084046456e
Add config option to control alias creation
2018-10-19 10:22:45 +01:00
Erik Johnston
74e7617083
Clean up room alias creation
2018-10-19 10:11:56 +01:00
Travis Ralston
49a044aa5f
Merge branch 'develop' into travis/login-terms
2018-10-18 09:57:58 -06:00
Will Hunt
d6a7797dd1
Fix roomlist since tokens on Python 3 ( #4046 )
...
Thanks @Half-Shot !!!
2018-10-17 23:04:55 +11:00
Neil Johnson
c6584f4b5f
clean up config error logic and imports
2018-10-17 11:36:41 +01:00
Richard van der Hoff
fc0f13dd03
Fix incorrect truncation in get_missing_events
...
It's quite important that get_missing_events returns the *latest* events in the
room; however we were pulling event ids out of the database until we got *at
least* 10, and then taking the *earliest* of the results.
We also shouldn't really be relying on depth, and should be checking the
room_id.
2018-10-16 21:10:04 +01:00
Erik Johnston
15133477ee
Fix up use of resolve_events_with_factory
2018-10-16 16:28:42 +01:00
Travis Ralston
a8ed93a4b5
pep8
2018-10-15 16:10:29 -06:00
Travis Ralston
442734ff9e
Ensure the terms params are actually provided
2018-10-15 14:56:13 -06:00
Travis Ralston
762a0982aa
Python is hard
2018-10-15 14:46:09 -06:00
Travis Ralston
f293d124b6
Merge branch 'develop' into travis/login-terms
2018-10-15 14:44:32 -06:00
David Baker
03c11032c3
Merge pull request #4019 from matrix-org/dbkr/e2e_backups
...
E2E backups
2018-10-15 10:19:43 +01:00
Neil Johnson
1ccafb0c5e
no need to join room if creator
2018-10-13 21:14:21 +01:00
Travis Ralston
dd99db846d
Update login terms structure for the proposed language support
2018-10-12 18:03:27 -06:00
Travis Ralston
7ede650956
Merge branch 'develop' into travis/login-terms
2018-10-12 16:24:07 -06:00
Travis Ralston
164f8e4843
isort
2018-10-12 15:11:59 -06:00
Travis Ralston
7bb651de6a
More sane handling of group errors and pep8
2018-10-12 14:53:30 -06:00
Travis Ralston
e3586f7c06
Merge branch 'develop' into travis/fix-federated-group-requests
2018-10-12 14:49:58 -06:00
Neil Johnson
a2bfb778c8
improve auto room join logic, comments and tests
2018-10-12 18:17:36 +01:00
David Baker
8c0ff0287a
Linting soothes the savage PEP8 monster
2018-10-12 13:47:43 +01:00
David Baker
83e72bb2f0
PR feedback pt. 1
2018-10-12 11:26:18 +01:00
Erik Johnston
7e561b5c1a
Add description to counter metric
2018-10-10 11:41:15 +01:00
Erik Johnston
3cbe8331e6
Track number of non-empty sync responses instead
2018-10-10 11:23:17 +01:00
Erik Johnston
395276b405
Append _total to metric and fix up spelling
2018-10-10 09:24:39 +01:00
Erik Johnston
bdc27d6716
Add metric to count lazy member sync requests
2018-10-09 14:17:52 +01:00
David Baker
dc045ef202
Merge remote-tracking branch 'origin/develop' into dbkr/e2e_backups
2018-10-09 10:05:02 +01:00
David Baker
f4a4dbcad1
Apparently this blank line is Very Important
2018-10-09 09:47:04 +01:00
David Baker
0c905ee015
be python3 compatible
2018-10-09 09:39:13 +01:00
Erik Johnston
6982320572
Remove unnecessary extra function call layer
2018-10-08 14:06:19 +01:00
Erik Johnston
8a1817f0d2
Use errback pattern and catch async failures
2018-10-08 13:29:47 +01:00
Neil Johnson
ed82043efb
Merge branch 'develop' into matthew/autocreate_autojoin
2018-10-04 17:26:59 +01:00
Neil Johnson
2dadc092b8
move logic into register, fix room alias localpart bug, tests
2018-10-04 17:00:27 +01:00
Travis Ralston
537d0b7b36
Use a flag rather than a new route for the public policy
...
This also means that the template now has optional parameters, which will need to be documented somehow.
2018-10-03 17:50:11 -06:00
Travis Ralston
149c4f1765
Supply params for terms auth stage
...
As per https://github.com/matrix-org/matrix-doc/pull/1692
2018-10-03 15:57:42 -06:00
Travis Ralston
fd99787162
Incorporate Dave's work for GDPR login flows
...
As per https://github.com/vector-im/riot-web/issues/7168#issuecomment-419996117
2018-10-03 15:57:42 -06:00
Erik Johnston
495a9d06bb
Fix exception handling in fetching remote profiles
2018-10-03 11:34:30 +01:00
Richard van der Hoff
8c41b0ca66
Merge pull request #3989 from matrix-org/rav/better_stacktraces
...
Avoid reraise, to improve stacktraces
2018-10-02 10:47:38 +01:00
Erik Johnston
bc29946809
Merge pull request #3986 from matrix-org/erikj/fix_sync_with_redacted_state
...
Fix lazy loaded sync with rejected state events
2018-10-02 11:38:35 +02:00
Richard van der Hoff
8174c6725b
Avoid reraise, to improve stacktraces
2018-10-01 18:50:34 +01:00
Richard van der Hoff
b5b93f45d5
Merge pull request #3968 from matrix-org/rav/fix_federation_errors
...
Fix exceptions when handling incoming transactions
2018-10-01 15:54:24 +01:00
Erik Johnston
82f922b4af
Fix lazy loaded sync with rejected state events
...
In particular, we assume that the name and canonical alias events in
the state have not been rejected. In practice this may not be the case
(though we should probably think about fixing that) so lets ensure that
we gracefully handle that case, rather than 404'ing the sync request
like we do now.
2018-10-01 14:19:40 +01:00
Matthew Hodgson
23b6a0537f
emit room aliases event
2018-09-29 02:19:37 +01:00
Matthew Hodgson
5b68f29f48
fix thinkos
2018-09-29 02:14:40 +01:00
Matthew Hodgson
8f646f2d04
fix UTs
2018-09-28 15:37:28 +01:00
Richard van der Hoff
3deaad2fb4
Merge pull request #3964 from matrix-org/rav/remove_localhost_checks
...
remove spurious federation checks on localhost
2018-09-28 13:35:47 +01:00
Richard van der Hoff
965154d60a
Fix complete fail to do the right thing
2018-09-28 12:45:54 +01:00
Matthew Hodgson
07340cdaca
untested stab at autocreating autojoin rooms
2018-09-28 01:42:53 +01:00
Richard van der Hoff
f094f715cf
Merge remote-tracking branch 'origin/develop' into rav/fix_federation_errors
2018-09-27 15:18:21 +01:00
Richard van der Hoff
36c62a67c4
Merge pull request #3794 from matrix-org/erikj/faster_typing
...
Improve performance of getting typing updates for replication
2018-09-27 15:14:26 +01:00
Richard van der Hoff
e1e3e77bfd
Merge pull request #3967 from matrix-org/rav/federation_handler_cleanups
...
Clarifications in FederationHandler
2018-09-27 15:06:59 +01:00
Amber Brown
a512e637ac
Merge pull request #3970 from schnuffle/develop-py3
...
Replaced all occurences of e.message with str(e)
2018-09-27 23:39:45 +10:00
Schnuffle
dc5db01ff2
Replaced all occurences of e.message with str(e)
...
Signed-off-by: Schnuffle <schnuffle@github.com>
2018-09-27 13:38:50 +02:00
Richard van der Hoff
333bee27f5
Include event when resolving state for missing prevs
...
If we have a forward extremity for a room as `E`, and you receive `A`, `B`,
s.t. `A -> B -> E`, and `B` also points to an unknown event `X`, then we need
to do state res between `X` and `E`.
When that happens, we need to make sure we include `X` in the state that goes
into the state res alg.
Fixes #3934 .
2018-09-27 11:37:39 +01:00
Richard van der Hoff
bd61c82bdf
Include state from remote servers in pdu handling
...
If we've fetched state events from remote servers in order to resolve the state
for a new event, we need to actually pass those events into
resolve_events_with_factory (so that it can do the state res) and then persist
the ones we need - otherwise other bits of the codebase get confused about why
we have state groups pointing to non-existent events.
2018-09-27 11:37:39 +01:00
Richard van der Hoff
a215b698c4
Fix "unhashable type: 'list'" exception in federation handling
...
get_state_groups returns a map from state_group_id to a list of FrozenEvents,
so was very much the wrong thing to be putting as one of the entries in the
list passed to resolve_events_with_factory (which expects maps from
(event_type, state_key) to event id).
We actually want get_state_groups_ids().values() rather than
get_state_groups().
This fixes the main problem in #3923 , but there are other problems with this
bit of code which get discovered once you do so.
2018-09-27 11:37:39 +01:00
Richard van der Hoff
28223841e0
more comments
2018-09-27 11:31:51 +01:00
Richard van der Hoff
e3c159863d
Clarifications in FederationHandler
...
* add some comments on things that look a bit bogus
* rename this `state` variable to avoid confusion with the `state` used
elsewhere in this function. (There was no actual conflict, but it was
a confusing bit of spaghetti.)
2018-09-27 11:31:51 +01:00
Richard van der Hoff
92abd3d6d6
Merge pull request #3966 from matrix-org/rav/rx_txn_logging_2
...
Logging improvements
2018-09-27 11:27:46 +01:00
Richard van der Hoff
4a15a3e4d5
Include eventid in log lines when processing incoming federation transactions ( #3959 )
...
when processing incoming transactions, it can be hard to see what's going on,
because we process a bunch of stuff in parallel, and because we may end up
recursively working our way through a chain of three or four events.
This commit creates a way to use logcontexts to add the relevant event ids to
the log lines.
2018-09-27 11:25:34 +01:00
Travis Ralston
82fa31799c
Remove debugging statement
2018-09-26 14:01:02 -06:00
Travis Ralston
b4c3bc1734
Handle HttpResponseException more safely for federated groups
2018-09-26 13:48:04 -06:00
Richard van der Hoff
e70b4ce069
Logging improvements
...
Some logging tweaks to help with debugging incoming federation transactions
2018-09-26 17:36:14 +01:00
Amber Brown
c2185f14d7
Merge pull request #3924 from matrix-org/rav/clean_up_on_receive_pdu
...
Comments and interface cleanup for on_receive_pdu
2018-09-26 21:41:26 +10:00
Matthew Hodgson
787d22ed6c
Only lazy load self-members on initial sync
...
Given we have disabled lazy loading for incr syncs in #3840 , we can make self-LL more efficient by only doing it on initial sync. Also adds a bounds check for if/when we change our mind, so that we don't try to include LL members on sync responses with no timeline.
2018-09-25 00:49:26 +01:00
Matthew Hodgson
a2ddaa90f2
Always LL ourselves if we're in a room to simplify clients ( #3916 )
...
Should fix https://github.com/vector-im/riot-web/issues/7209
2018-09-20 21:21:54 +01:00
Richard van der Hoff
703de4ec13
Comments and interface cleanup for on_receive_pdu
...
Add some informative comments about what's going on here.
Also, `sent_to_us_directly` and `get_missing` were doing the same thing (apart
from in `_handle_queued_pdus`, which looks like a bug), so let's get rid of
`get_missing` and use `sent_to_us_directly` consistently.
2018-09-20 13:06:55 +01:00
Richard van der Hoff
642199570c
Improve the logging when handling a federation transaction ( #3904 )
...
Let's try to rationalise the logging that happens when we are processing an
incoming transaction, to make it easier to figure out what is going wrong when
they take ages. In particular:
- make everything start with a [room_id event_id] prefix
- make sure we log a warning when catching exceptions rather than just turning
them into other, more cryptic, exceptions.
2018-09-19 17:28:18 +01:00
Erik Johnston
ce846bb620
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/faster_typing
2018-09-19 15:08:36 +01:00
Amber Brown
f773ecbd61
Merge pull request #3903 from matrix-org/rav/increase_get_missing_events_timeout
...
Bump timeout on get_missing_events request
2018-09-19 17:57:48 +10:00
Richard van der Hoff
a219ce8726
Use directory server for room joins ( #3899 )
...
When we do a join, always try the server we used for the alias lookup first.
Fixes #2418
2018-09-18 18:27:37 +01:00
Richard van der Hoff
550007cb0e
Bump timeout on get_missing_events request
2018-09-18 15:02:51 +01:00
Richard van der Hoff
286d6930b7
Merge pull request #3879 from matrix-org/matthew/fix-autojoin
...
don't ratelimit autojoins
2018-09-18 13:07:01 +01:00
Richard van der Hoff
f75b9961c6
Reinstate missing null check
2018-09-17 16:52:02 +01:00
Richard van der Hoff
85a43f4167
Return a 404 when deleting unknown room alias
...
As per https://github.com/matrix-org/matrix-doc/issues/1675
Fixes https://github.com/matrix-org/synapse/issues/2782
2018-09-17 13:19:00 +01:00
Matthew Hodgson
d42d79e3c3
don't ratelimit autojoins
2018-09-15 22:27:41 +01:00
Matthew Hodgson
024be6cf18
don't filter membership events based on history visibility ( #3874 )
...
don't filter membership events based on history visibility
as we will already have filtered the messages in the timeline, and state events
are always visible.
and because @erikjohnston said so.
2018-09-14 18:12:52 +01:00
Matthew Hodgson
2ac1abbc7e
show heroes if a room has a 'deleted' name/canonical_alias ( #3851 )
2018-09-12 17:11:05 +01:00
Matthew Hodgson
0403cf0783
argh pep8
2018-09-12 16:54:28 +01:00
Matthew Hodgson
0e200e366d
correctly log gappy sync metrics
2018-09-12 16:47:20 +01:00
Matthew Hodgson
11bfc2af1c
fix logline
2018-09-12 16:45:42 +01:00
Matthew Hodgson
b041115415
Speed up lazy loading ( #3827 )
...
* speed up room summaries by pulling their data from room_memberships rather than room state
* disable LL for incr syncs, and log incr sync stats (#3840 )
2018-09-12 00:50:39 +01:00
Erik Johnston
f60c9e2a01
Don't send empty tags list down sync
2018-09-06 17:01:41 +01:00
Erik Johnston
7baf66ef5d
Send existing room tags down sync on join
...
When a user joined a room any existing tags were not sent down the sync
stream. Ordinarily this isn't a problem because the user needs to be in
the room to have set tags in it, however synapse will sometimes add tags
for a user to a room, e.g. for server notices, which need to come down
sync.
2018-09-06 16:46:51 +01:00
Hubert Chathi
3801b8aa03
try to make flake8 and isort happy
2018-09-06 11:35:19 -04:00
Amber Brown
ee7c8bd2b5
Merge pull request #3795 from matrix-org/erikj/faster_sync_state
...
User iter* during sync state calculations
2018-09-07 00:24:43 +10:00
Amber Brown
2608ebc04c
Port handlers/ to Python 3 ( #3803 )
2018-09-07 00:22:23 +10:00
Erik Johnston
7419764351
User iter* during sync state calculations
2018-09-05 16:19:50 +01:00
Erik Johnston
5f02017aea
Improve performance of getting typing updates for replication
...
Fetching the list of all new typing notifications involved iterating
over all rooms and comparing their serial. Lets move to using a stream
change cache, like we do for other streams.
2018-09-05 10:20:40 +01:00
Erik Johnston
567363e497
Merge pull request #3737 from matrix-org/erikj/remove_redundant_state_func
...
Remove unnecessary resolve_events_with_state_map
2018-09-03 16:19:41 +01:00
Neil Johnson
ea068d6f3c
fix bug where preserved threepid user comes to sign up and server is mau blocked
2018-08-31 10:49:14 +01:00
Matthew Hodgson
b1580f50fe
don't return non-LL-member state in incremental sync state blocks ( #3760 )
...
don't return non-LL-member state in incremental sync state blocks
2018-08-28 23:25:58 +01:00
Hubert Chathi
16a31c6fce
update to newer Synapse APIs
2018-08-24 22:51:25 -04:00
Hubert Chathi
83caead95a
Merge branch 'develop' into e2e_backups
2018-08-24 11:44:26 -04:00
Erik Johnston
5c261107c9
Remove unnecessary resolve_events_with_state_map
...
We only ever used the synchronous resolve_events_with_state_map in one
place, which is trivial to replace with the async version.
2018-08-22 15:41:15 +01:00
Erik Johnston
764030cf63
Merge pull request #3659 from matrix-org/erikj/split_profiles
...
Allow profile updates to happen on workers
2018-08-22 11:35:55 +01:00
Erik Johnston
8432e2ebd7
Rename WorkerProfileHandler to BaseProfileHandler
2018-08-22 10:13:40 +01:00
Erik Johnston
a81f140880
Add assert to ensure handler is only run on master
2018-08-22 10:11:21 +01:00
Erik Johnston
47b25ba5f3
Remove redundant vars
2018-08-22 10:09:05 +01:00
Erik Johnston
4d664278af
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/refactor_state_handler
2018-08-20 14:49:43 +01:00
Erik Johnston
8dee601054
Remove redundant room_version checks
2018-08-20 14:48:53 +01:00
Erik Johnston
e21c368b8b
Revert spurious change
2018-08-20 13:54:51 +01:00
Richard van der Hoff
c144252a8c
Merge pull request #3710 from matrix-org/rav/logcontext_for_pusher_updates
...
Fix logcontexts for running pushers
2018-08-17 16:21:49 +01:00
Amber Brown
c334ca67bb
Integrate presence from hotfixes ( #3694 )
2018-08-18 01:08:45 +10:00
Erik Johnston
ab822a2d1f
Add some fixmes
2018-08-17 15:31:50 +01:00
Erik Johnston
91cdb6de08
Call UserDirectoryHandler methods directly
...
Turns out that the user directory handling is fairly racey as a bunch
of stuff assumes that the processing happens on master, which it doesn't
when there is a synapse.app.user_dir worker. So lets just call the
function directly until we actually get round to fixing it, since it
doesn't make the situation any worse.
2018-08-17 15:26:13 +01:00
Erik Johnston
782689bd40
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/split_profiles
2018-08-17 14:15:48 +01:00
Erik Johnston
ca87ad1def
Split ProfileHandler into master and worker
2018-08-17 14:15:14 +01:00
Richard van der Hoff
66f7dc8c87
Fix logcontexts for running pushers
...
First of all, avoid resetting the logcontext before running the pushers, to fix
the "Starting db txn 'get_all_updated_receipts' from sentinel context" warning.
Instead, give them their own "background process" logcontexts.
2018-08-17 00:32:39 +01:00
Neil Johnson
372bf073c1
block event creation and room creation on hitting resource limits
2018-08-16 21:25:16 +01:00
Matthew Hodgson
762a758fea
lazyload aware /messages ( #3589 )
2018-08-16 14:22:47 +01:00
Matthew Hodgson
3f543dc021
initial cut at a room summary API ( #3574 )
2018-08-16 09:46:50 +01:00
Neil Johnson
81d727efa9
Merge pull request #3689 from matrix-org/neilj/fix_off_by_1+maus
...
Fix Mau off by one errors
2018-08-15 16:19:41 +00:00
Matthew Hodgson
2f78f432c4
speed up /members and add at= and membership params ( #3568 )
2018-08-15 16:35:22 +01:00
Neil Johnson
86a00e05e1
Merge branch 'develop' of github.com:matrix-org/synapse into neilj/fix_off_by_1+maus
2018-08-15 16:27:08 +01:00
Erik Johnston
dc56c47dc0
Merge pull request #3653 from matrix-org/erikj/split_federation
...
Move more federation APIs to workers
2018-08-15 14:59:02 +01:00
Neil Johnson
4601129c44
Merge pull request #3687 from matrix-org/neilj/admin_email
...
support admin_email config and pass through into blocking errors,
2018-08-15 13:52:25 +00:00
Erik Johnston
ef184caf30
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/split_federation
2018-08-15 14:25:46 +01:00
Erik Johnston
488ffe6fdb
Use federation handler function rather than duplicate
...
This involves renaming _persist_events to be a public function.
2018-08-15 14:17:18 +01:00
Erik Johnston
fef2e65d12
Merge pull request #3667 from matrix-org/erikj/fixup_unbind
...
Don't fail requests to unbind 3pids for non supporting ID servers
2018-08-15 10:32:12 +01:00
Neil Johnson
19b433e3f4
Merge branch 'develop' of github.com:matrix-org/synapse into neilj/admin_email
2018-08-14 17:44:46 +01:00
Neil Johnson
414d54b61a
Merge pull request #3670 from matrix-org/neilj/mau_sync_block
...
Block ability to read via sync if mau limit exceeded
2018-08-14 15:21:31 +00:00
Neil Johnson
8f9a7eb58d
support admin_email config and pass through into blocking errors, return AuthError in all cases
2018-08-14 15:11:54 +01:00
Neil Johnson
c74c71128d
remove blank line
2018-08-14 15:06:24 +01:00
Neil Johnson
ed4bc3d2fc
fix off by 1s on mau
2018-08-14 15:04:48 +01:00
Neil Johnson
bd92c8eaa7
Merge branch 'neilj/admin_email' of github.com:matrix-org/synapse into neilj/fix_off_by_1+maus
2018-08-14 14:56:45 +01:00
Neil Johnson
99ebaed8e6
Update register.py
...
remove comments
2018-08-14 14:55:55 +01:00
Neil Johnson
9b5bf3d858
Merge branch 'neilj/admin_email' of github.com:matrix-org/synapse into neilj/fix_off_by_1+maus
2018-08-14 14:51:38 +01:00
Neil Johnson
e25d87d97b
Merge branch 'neilj/mau_sync_block' of github.com:matrix-org/synapse into neilj/fix_off_by_1+maus
2018-08-14 14:32:18 +01:00
Neil Johnson
f4b49152e2
support admin_email config and pass through into blocking errors, return AuthError in all cases
2018-08-13 21:09:47 +01:00
Neil Johnson
ce7de9ae6b
Revert "support admin_email config and pass through into blocking errors, return AuthError in all cases"
...
This reverts commit 0d43f991a1
2018-08-13 18:06:18 +01:00
Neil Johnson
0d43f991a1
support admin_email config and pass through into blocking errors, return AuthError in all cases
2018-08-13 18:00:23 +01:00
Amber Brown
99dd975dae
Run tests under PostgreSQL ( #3423 )
2018-08-13 16:47:46 +10:00
Matthew Hodgson
72788cf9c1
support DELETE /version with no args
2018-08-12 19:14:31 -04:00
Matthew Hodgson
fe87890b18
implement remaining tests and make them work
2018-08-12 19:14:31 -04:00
Matthew Hodgson
5e42c45c96
switch get_current_version_info back to being get_version_info
2018-08-12 19:14:31 -04:00
Matthew Hodgson
982edca380
fix flakes
2018-08-12 19:14:31 -04:00
Matthew Hodgson
234611f347
fix typos
2018-08-12 19:14:31 -04:00
Matthew Hodgson
14b3da63a3
add a tonne of docstring; make upload_room_keys properly assert version
2018-08-12 19:14:31 -04:00
Matthew Hodgson
9f500cb39e
more docstring for the e2e_room_keys rest
2018-08-12 19:14:31 -04:00
Matthew Hodgson
ca0b052307
fix factoring out of _should_replace_room_key
2018-08-12 19:14:31 -04:00
Matthew Hodgson
cac0253799
rename room_key_version table correctly, and fix opt args
2018-08-12 19:14:31 -04:00
Matthew Hodgson
0abb205b47
blindly incorporate PR review - needs testing & fixing
2018-08-12 19:14:31 -04:00
Matthew Hodgson
69e51c7ba4
make /room_keys/version work
2018-08-12 19:14:31 -04:00
Matthew Hodgson
8ae64b270f
implement /room_keys/version too (untested)
2018-08-12 19:14:31 -04:00
Matthew Hodgson
6b8c07abc2
make it work and fix pep8
2018-08-12 19:13:09 -04:00
Matthew Hodgson
0bc4627a73
interim WIP checkin; doesn't build yet
2018-08-12 18:23:10 -04:00
Matthew Hodgson
53ace904b2
total WIP skeleton for /room_keys API
2018-08-12 18:23:10 -04:00
Amber Brown
b37c472419
Rename async to async_helpers because async is a keyword on Python 3.7 ( #3678 )
2018-08-10 23:50:21 +10:00
Neil Johnson
09cf130898
only block on sync where user is not part of the mau cohort
2018-08-09 17:39:12 +01:00
Erik Johnston
ce6db0e547
Choose state algorithm based on room version
2018-08-09 14:58:47 +01:00
Neil Johnson
69ce057ea6
block sync if auth checks fail
2018-08-09 12:26:27 +01:00
Neil Johnson
a5ef110749
Merge branch 'develop' of github.com:matrix-org/synapse into neilj/mau_sync_block
2018-08-09 11:40:37 +01:00
Erik Johnston
b179537f2a
Move clean_room_for_join to master
2018-08-09 10:37:38 +01:00
Erik Johnston
484a0ebdfc
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/split_profiles
2018-08-09 10:16:29 +01:00
Erik Johnston
5785b93711
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/split_federation
2018-08-09 10:16:16 +01:00
Erik Johnston
bf7598f582
Log when we 3pid/unbind request fails
2018-08-09 10:09:56 +01:00
Erik Johnston
2bdafaf3c1
Merge pull request #3632 from matrix-org/erikj/refactor_repl_servlet
...
Add helper base class for generating new replication endpoints
2018-08-09 10:06:23 +01:00
Richard van der Hoff
bb89c84614
Merge pull request #3664 from matrix-org/rav/federation_metrics
...
more metrics for the federation and appservice senders
2018-08-08 23:16:58 +01:00
Neil Johnson
e92fb00f32
sync auth blocking
2018-08-08 17:54:49 +01:00
Richard van der Hoff
8521ae13e3
Merge pull request #3654 from matrix-org/rav/room_versions
...
Support for room versioning
2018-08-08 17:10:53 +01:00
Neil Johnson
990fe9fc23
Merge pull request #3633 from matrix-org/neilj/mau_tracker
...
API for monthly_active_users table
2018-08-08 12:44:15 +00:00
Erik Johnston
360ba89c50
Don't fail requests to unbind 3pids for non supporting ID servers
...
Older identity servers may not support the unbind 3pid request, so we
shouldn't fail the requests if we received one of 400/404/501. The
request still fails if we receive e.g. 500 responses, allowing clients
to retry requests on transient identity server errors that otherwise do
support the API.
Fixes #3661
2018-08-08 12:06:18 +01:00
Richard van der Hoff
53bca4690b
more metrics for the federation and appservice senders
2018-08-07 19:09:48 +01:00
Richard van der Hoff
3523f5432a
Don't expose default_room_version as config opt
2018-08-07 12:51:57 +01:00
Erik Johnston
495cb100d1
Allow profile changes to happen on workers
2018-08-07 10:50:26 +01:00
Erik Johnston
a3f5bf79a0
Add EDU/query handling over replication
2018-08-06 15:23:31 +01:00
Erik Johnston
e26dbd82ef
Add replication APIs for persisting federation events
2018-08-06 15:02:28 +01:00
Richard van der Hoff
f900d50824
include known room versions in outgoing make_joins
2018-08-06 13:45:37 +01:00
Neil Johnson
e40a510fbf
py3 fix
2018-08-03 23:19:13 +01:00
Neil Johnson
886be75ad1
bug fixes
2018-08-03 22:29:03 +01:00
Will Hunt
16d9701892
Return M_NOT_FOUND when a profile could not be found. ( #3596 )
2018-08-03 19:08:05 +01:00
Richard van der Hoff
0ca459ea33
Basic support for room versioning
...
This is the first tranche of support for room versioning. It includes:
* setting the default room version in the config file
* new room_version param on the createRoom API
* storing the version of newly-created rooms in the m.room.create event
* fishing the version of existing rooms out of the m.room.create event
2018-08-03 16:08:32 +01:00
Neil Johnson
897c51d274
Merge branch 'develop' of github.com:matrix-org/synapse into neilj/mau_tracker
2018-08-03 13:40:47 +01:00
Erik Johnston
cb298ff623
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/refactor_repl_servlet
2018-08-03 09:25:15 +01:00
Richard van der Hoff
1fa98495d0
Merge pull request #3639 from matrix-org/rav/refactor_error_handling
...
Clean up handling of errors from outbound requests
2018-08-02 17:38:24 +01:00
Neil Johnson
74b1d46ad9
do mau checks based on monthly_active_users table
2018-08-02 16:57:35 +01:00
Richard van der Hoff
704c3e6239
Merge branch 'master' into develop
2018-08-02 15:43:30 +01:00
Richard van der Hoff
0bf5ec0db7
Check room visibility for /event/ requests
...
Make sure that the user has permission to view the requeseted event for
/event/{eventId} and /room/{roomId}/event/{eventId} requests.
Also check that the event is in the given room for
/room/{roomId}/event/{eventId}, for sanity.
2018-08-02 15:03:27 +01:00
Richard van der Hoff
14fa9d4d92
Avoid extra db lookups
...
Since we're about to look up the events themselves anyway, we can skip the
extra db queries here.
2018-08-02 13:55:51 +01:00
Richard van der Hoff
0a65450d04
Validation for events/rooms in fed requests
...
When we get a federation request which refers to an event id, make sure that
said event is in the room the caller claims it is in.
(patch supplied by @turt2live)
2018-08-02 13:48:40 +01:00
Erik Johnston
40c1c59cf4
Merge pull request #3621 from matrix-org/erikj/split_fed_store
...
Split out DB writes in federation handler
2018-08-02 10:41:42 +01:00
Neil Johnson
085435e13a
Merge pull request #3630 from matrix-org/neilj/mau_sign_in_log_in_limits
...
Initial impl of capping MAU
2018-08-01 15:58:45 +00:00
Richard van der Hoff
01e93f48ed
Kill off MatrixCodeMessageException
...
This code brings the SimpleHttpClient into line with the
MatrixFederationHttpClient by having it raise HttpResponseExceptions when a
request fails (rather than trying to parse for matrix errors and maybe raising
MatrixCodeMessageException).
Then, whenever we were checking for MatrixCodeMessageException and turning them
into SynapseErrors, we now need to check for HttpResponseExceptions and call
to_synapse_error.
2018-08-01 16:02:46 +01:00
Amber Brown
da7785147d
Python 3: Convert some unicode/bytes uses ( #3569 )
2018-08-02 00:54:06 +10:00
Neil Johnson
303f1c851f
Merge branch 'develop' of github.com:matrix-org/synapse into neilj/mau_sign_in_log_in_limits
2018-08-01 13:42:50 +01:00
Erik Johnston
a6d7b74915
update docs
2018-08-01 13:39:14 +01:00
Erik Johnston
4b256b9271
_persist_auth_tree no longer returns anything
2018-08-01 13:39:07 +01:00
Neil Johnson
0aba3d361a
count_monthly_users() async
2018-08-01 11:47:58 +01:00
Neil Johnson
7931393495
make count_monthly_users async synapse/handlers/auth.py
2018-08-01 10:21:56 +01:00
Richard van der Hoff
bdbdceeafa
rename replication_layer to federation_client
...
I have HAD ENOUGH of trying to remember wtf a replication layer is in terms of
classes.
2018-07-31 15:44:05 +01:00
Erik Johnston
443da003bc
Use new helper base class for membership requests
2018-07-31 14:32:23 +01:00
Erik Johnston
729b672823
Use new helper base class for ReplicationSendEventRestServlet
2018-07-31 14:32:23 +01:00
Neil Johnson
df2235e7fa
coding style
2018-07-31 13:16:20 +01:00
Neil Johnson
251e6c1210
limit register and sign in on number of monthly users
2018-07-30 15:55:57 +01:00
Erik Johnston
143f1a2532
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/split_fed_store
2018-07-30 09:56:18 +01:00
Matthew Hodgson
e9b2d047f6
make /context lazyload & filter aware ( #3567 )
...
make /context lazyload & filter aware.
2018-07-27 15:12:50 +01:00
Richard van der Hoff
b0b5566f36
Merge pull request #3391 from t3chguy/t3chguy/default_inviter_display_name_3pid
...
if inviter_display_name == ""||None then default to inviter MXID
2018-07-27 10:08:39 +01:00
Matthew Hodgson
a75231b507
Deduplicate redundant lazy-loaded members ( #3331 )
...
* attempt at deduplicating lazy-loaded members
as per the proposal; we can deduplicate redundant lazy-loaded members
which are sent in the same sync sequence. we do this heuristically
rather than requiring the client to somehow tell us which members it
has chosen to cache, by instead caching the last N members sent to
a client, and not sending them again. For now we hardcode N to 100.
Each cache for a given (user,device) tuple is in turn cached for up to
X minutes (to avoid the caches building up). For now we hardcode X to 30.
* add include_redundant_members filter option & make it work
* remove stale todo
* add tests for _get_some_state_from_cache
* incorporate review
2018-07-26 22:51:30 +01:00
Richard van der Hoff
03751a6420
Fix some looping_call calls which were broken in #3604
...
It turns out that looping_call does check the deferred returned by its
callback, and (at least in the case of client_ips), we were relying on this,
and I broke it in #3604 .
Update run_as_background_process to return the deferred, and make sure we
return it to clock.looping_call.
2018-07-26 11:48:08 +01:00
Matthew Hodgson
1bcd0490c2
Merge pull request #2970 from matrix-org/matthew/filter_members
...
Implement the lazy_load_members room state filter parameter
2018-07-26 00:03:01 +01:00
Travis Ralston
d8e65ed7e1
Fix a minor documentation typo in on_make_leave
2018-07-25 15:44:41 -06:00
Matthew Hodgson
2565804030
Merge branch 'develop' into matthew/filter_members
2018-07-25 17:27:49 +01:00
Erik Johnston
78a691d005
Split out DB writes in federation handler
...
This will allow us to easily add an internal replication API to proxy
these reqeusts to master, so that we can move federation APIs to
workers.
2018-07-25 16:22:56 +01:00
Richard van der Hoff
55acd6856c
Fix updating of cached remote profiles
...
_update_remote_profile_cache was missing its `defer.inlineCallbacks`, so when
it was called, would just return a generator object, without actually running
any of the method body.
2018-07-25 10:34:48 +01:00
Michael Telatynski
87951d3891
Merge branch 'develop' of github.com:matrix-org/synapse into t3chguy/default_inviter_display_name_3pid
2018-07-24 17:17:46 +01:00
Erik Johnston
8b8c4f34a3
Replace usage of get_current_toke with StreamToken.START
...
This allows us to handle /context/ requests on the client_reader worker
without having to pull in all the various stream handlers (e.g.
precence, typing, pushers etc). The only thing the token gets used for
is pagination, and that ignores everything but the room portion of the
token.
2018-07-24 16:49:17 +01:00
Richard van der Hoff
a321f78991
Merge pull request #3586 from matrix-org/rav/optimise_resolve_state_groups
...
Fixes and optimisations for resolve_state_groups
2018-07-24 14:11:45 +01:00
Matthew Hodgson
1a01a5b964
clarify comment on p_ids
2018-07-24 14:03:15 +01:00
Matthew Hodgson
eb1d911ab7
rather than adding ll_ids, remove them from p_ids
2018-07-24 13:40:49 +01:00
Matthew Hodgson
d19fba3655
Merge branch 'develop' into matthew/filter_members
2018-07-24 12:39:54 +01:00
Matthew Hodgson
cd241d6bda
incorporate more review
2018-07-24 12:39:40 +01:00
Richard van der Hoff
a678145010
Merge branch 'develop' into rav/logcontext_fixes
2018-07-24 10:43:30 +01:00
Erik Johnston
536bc63a4e
Merge branch 'develop' into erikj/client_apis_move
2018-07-24 09:57:05 +01:00
Richard van der Hoff
cf2d15c6a9
another couple of logcontext leaks
2018-07-24 00:57:48 +01:00
Richard van der Hoff
8dff6e0322
Logcontext fixes
...
Fix some random logcontext leaks.
2018-07-24 00:37:17 +01:00
Richard van der Hoff
c1f80effbe
Handle delta_ids being None in _update_context_for_auth_events
...
it's easier to create the new state group as a delta from the existing one.
(There's an outside chance this will help with
https://github.com/matrix-org/synapse/issues/3364 )
2018-07-23 22:06:50 +01:00
Matthew Hodgson
adfe29ec0b
Merge branch 'develop' into matthew/filter_members
2018-07-23 19:21:37 +01:00
Matthew Hodgson
254fb430d1
incorporate review
2018-07-23 19:21:20 +01:00
Erik Johnston
0faa3223cd
Fix missing attributes on workers.
...
This was missed during the transition from attribute to getter for
getting state from context.
2018-07-23 16:28:00 +01:00
Erik Johnston
0b0b24cb82
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/client_apis_move
2018-07-23 13:21:15 +01:00
Erik Johnston
027bc01a1b
Add support for updating state
2018-07-23 13:17:25 +01:00
Erik Johnston
e42510ba63
Use new getters
2018-07-23 13:17:22 +01:00
Amber Brown
3132b89f12
Make the rest of the .iterwhatever go away ( #3562 )
2018-07-21 15:47:18 +10:00
Erik Johnston
5c88bb722f
Move PaginationHandler to its own file
2018-07-20 15:32:23 +01:00
Erik Johnston
0ecf68aedc
Move check_in_room_or_world_readable to Auth
2018-07-20 15:30:59 +01:00
Richard van der Hoff
7c712f95bb
Combine Limiter and Linearizer
...
Linearizer was effectively a Limiter with max_count=1, so rather than
maintaining two sets of code, let's combine them.
2018-07-20 13:11:43 +01:00
Richard van der Hoff
8462c26485
Improvements to the Limiter
...
* give them names, to improve logging
* use a deque rather than a list for efficiency
2018-07-20 12:50:27 +01:00
Matthew Hodgson
bcaec2915a
incorporate review
2018-07-19 19:03:50 +01:00
Matthew Hodgson
924eb34d94
add a filtered_types param to limit filtering to specific types
2018-07-19 18:32:02 +01:00
Matthew Hodgson
be3adfc331
merge develop pydoc for _get_state_for_groups
2018-07-19 11:26:04 +01:00
Erik Johnston
bacdf0cbf9
Move RoomContextHandler out of Handlers
...
This is in preparation for moving GET /context/ to a worker
2018-07-18 15:33:03 +01:00
Erik Johnston
8cb8df55e9
Split MessageHandler into read only and writers
...
This will let us call the read only parts from workers, and so be able
to move some APIs off of master, e.g. the `/state` API.
2018-07-18 15:33:03 +01:00
Richard van der Hoff
dab00faa83
Merge pull request #3367 from matrix-org/rav/drop_re_signing_hacks
...
Remove event re-signing hacks
2018-07-18 12:46:27 +01:00
Richard van der Hoff
9c04b4abf9
Merge pull request #3541 from matrix-org/rav/optimize_filter_events_for_server
...
Refactor and optimze filter_events_for_server
2018-07-17 14:01:39 +01:00
Richard van der Hoff
ea69d35651
Move filter_events_for_server out of FederationHandler
...
for easier unit testing.
2018-07-16 13:06:24 +01:00
Matthew Hodgson
ba22b6a456
typo
2018-07-13 12:03:39 +01:00
Matthew Hodgson
ea752bdd99
s/becuase/because/g
2018-07-10 17:58:18 +01:00
Amber Brown
49af402019
run isort
2018-07-09 16:09:20 +10:00
Richard van der Hoff
a4ab491371
Merge branch 'develop' into rav/drop_re_signing_hacks
2018-07-04 07:13:38 +01:00
Erik Johnston
e3b4043800
Merge pull request #3456 from matrix-org/hawkowl/federation-prevevent-checking
...
Check the state of prev_events a bit more thoroughly when coming over federation
2018-06-29 13:55:02 +01:00
Amber Brown
6350bf925e
Attempt to be more performant on PyPy ( #3462 )
2018-06-28 14:49:57 +01:00
Amber Brown
99800de63d
try and clean up
2018-06-27 11:40:27 +01:00
Amber Brown
f03a5d1a17
pep8
2018-06-27 11:38:14 +01:00
Amber Brown
a7ecf34b70
cleanups
2018-06-27 11:36:03 +01:00
Amber Brown
77078d6c8e
handle federation not telling us about prev_events
2018-06-27 11:27:32 +01:00
Matthew Hodgson
c7f6b420ae
Merge pull request #3448 from matrix-org/matthew/gdpr-deactivate-admin-api
...
add GDPR erase param to deactivate API
2018-06-26 10:43:14 +01:00
Matthew Hodgson
1e788db430
add GDPR erase param to deactivate API
2018-06-26 10:26:54 +01:00
Erik Johnston
d72fb9a448
Merge pull request #3442 from matrix-org/matthew/allow-unconsented-parts
...
allow non-consented users to still part rooms (to let us autopart them)
2018-06-25 20:14:18 +01:00
Erik Johnston
df48f7ef37
Actually fix it
2018-06-25 20:03:41 +01:00
Erik Johnston
a0e8a53c6d
Comment
2018-06-25 19:57:38 +01:00
Erik Johnston
7bdc5c8fa3
Fix bug with assuming wrong type
2018-06-25 19:56:02 +01:00
Erik Johnston
ea7a9c0483
Add fast path to _filter_events_for_server
...
Most rooms have a trivial history visibility like "shared" or
"world_readable", especially large rooms, so lets not bother getting the
full membership of those rooms in that case.
2018-06-25 19:49:13 +01:00
Matthew Hodgson
0269367f18
allow non-consented users to still part rooms (to let us autopart them)
2018-06-25 17:56:10 +01:00
Matthew Hodgson
784189b1f4
typos
2018-06-25 17:37:16 +01:00
Matthew Hodgson
6eb861b67f
typo
2018-06-25 17:37:16 +01:00
Erik Johnston
33fdcfa957
Merge pull request #3441 from matrix-org/erikj/redo_erasure
...
Fix user erasure and re-enable
2018-06-25 14:37:01 +01:00
Amber Brown
07cad26d65
Remove all global reactor imports & pass it around explicitly ( #3424 )
2018-06-25 14:08:28 +01:00
Erik Johnston
244484bf3c
Revert "Revert "Merge pull request #3431 from matrix-org/rav/erasure_visibility""
...
This reverts commit 1d009013b3
2018-06-25 13:42:55 +01:00
Richard van der Hoff
1d009013b3
Revert "Merge pull request #3431 from matrix-org/rav/erasure_visibility"
...
This reverts commit ce0d911156b4a5d767a9
2018-06-22 16:35:10 +01:00
Erik Johnston
28ddc6cfbe
Also log number of events for serach context
2018-06-22 15:42:11 +01:00
Erik Johnston
4b4cec3989
Add some logging to search queries
2018-06-22 15:42:11 +01:00
Erik Johnston
ce0d911156
Merge pull request #3431 from matrix-org/rav/erasure_visibility
...
Support hiding events from deleted users
2018-06-22 15:06:44 +01:00
Erik Johnston
b4a5d767a9
Merge pull request #3428 from matrix-org/erikj/persisted_pdu
...
Simplify get_persisted_pdu
2018-06-22 14:47:43 +01:00
Amber Brown
77ac14b960
Pass around the reactor explicitly ( #3385 )
2018-06-22 09:37:10 +01:00
Richard van der Hoff
02bfc581f8
Merge pull request #3399 from costacruise/master
...
Add error code to room creation error
2018-06-20 17:26:25 +01:00
Michael Wagner
19cd3120ec
Add error code to room creation error
...
This error code is mentioned in the documentation at https://matrix.org/docs/api/client-server/#!/Room32creation/createRoom
2018-06-14 14:08:40 +02:00
Amber Brown
a61738b316
Remove run_on_reactor ( #3395 )
2018-06-14 18:27:37 +10:00
Michael Telatynski
94700e55fa
if inviter_display_name == ""||None then default to inviter MXID
...
to prevent email invite from "None"
2018-06-13 10:31:01 +01:00
Richard van der Hoff
b6faef2ad7
Filter out erased messages
...
Redact any messges sent by erased users.
2018-06-12 09:53:18 +01:00
Richard van der Hoff
f1023ebf4b
mark accounts as erased when requested
2018-06-12 09:53:18 +01:00
Richard van der Hoff
9fc5b74b24
simplify get_persisted_pdu
...
it doesn't make much sense to use get_persisted_pdu on the receive path: just
get the event straight from the store.
2018-06-12 09:51:31 +01:00
David Baker
187a546bff
Merge pull request #3276 from matrix-org/dbkr/unbind
...
Remove email addresses / phone numbers from ID servers when they're removed from synapse
2018-06-11 16:02:00 +01:00
Matthew Hodgson
c96d882a02
Merge branch 'develop' into matthew/filter_members
2018-06-10 12:26:14 +03:00
Richard van der Hoff
c6b1441c52
Fix event filtering in get_missing_events handler
2018-06-08 14:15:31 +01:00
Richard van der Hoff
0834b49c6a
Fix event filtering in get_missing_events handler
2018-06-08 11:34:46 +01:00
Richard van der Hoff
8503dd0047
Remove event re-signing hacks
...
These "temporary fixes" have been here three and a half years, and I can't find
any events in the matrix.org database where the calculated signature differs
from what's in the db. It's time for them to go away.
2018-06-07 16:08:29 +01:00
David Baker
d62162bbec
doc fixes
2018-06-05 18:09:13 +01:00
David Baker
6a29e815fc
Fix comment
2018-06-04 12:01:23 +01:00
David Baker
e44150a6de
Missing yield
2018-06-04 12:01:13 +01:00
David Baker
f731e42baf
docstring
2018-06-04 12:00:51 +01:00
Matthew Hodgson
28f09fcdd5
Merge branch 'develop' into matthew/filter_members
2018-06-04 00:09:17 +03:00
Matthew Hodgson
5f6122fe10
more comments
2018-06-04 00:08:52 +03:00
Amber Brown
c936a52a9e
Consistently use six's iteritems and wrap lazy keys/values in list() if they're not meant to be lazy ( #3307 )
2018-05-31 19:03:47 +10:00
Richard van der Hoff
34b85df7f5
Update some comments and docstrings in SyncHandler
2018-05-29 22:31:18 +01:00
Richard van der Hoff
4a9cbdbc15
Exempt AS-registered users from doing gdpr
2018-05-29 19:54:32 +01:00
Matthew Hodgson
a6c8f7c875
add pydoc
2018-05-29 01:09:55 +01:00
Matthew Hodgson
7a6df013cc
merge develop
2018-05-29 00:25:22 +01:00
Amber Brown
754826a830
Merge remote-tracking branch 'origin/develop' into 3218-official-prom
2018-05-28 18:57:23 +10:00
Richard van der Hoff
41921ac01b
Merge pull request #3287 from matrix-org/rav/allow_leaving_server_notices_room
...
Let users leave the server notice room after joining
2018-05-25 11:15:45 +01:00
Richard van der Hoff
757ed27258
Let users leave the server notice room after joining
...
They still can't reject invites, but we let them leave it.
2018-05-25 11:07:21 +01:00
Adrian Tschira
17a70cf6e9
Misc. py3 fixes
...
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-05-24 20:20:33 +02:00
Amber Brown
1f69693347
Merge pull request #3244 from NotAFile/py3-six-4
...
replace some iteritems with six
2018-05-24 13:04:07 -05:00
David Baker
77a23e2e05
Merge remote-tracking branch 'origin/develop' into dbkr/unbind
2018-05-24 16:20:53 +01:00
David Baker
7a1af504d7
Remove users from user directory on deactivate
2018-05-24 15:59:58 +01:00
Erik Johnston
f72d5a44d5
Merge pull request #3261 from matrix-org/erikj/pagination_fixes
...
Fix federation backfill bugs
2018-05-24 14:52:03 +01:00
Erik Johnston
68399fc4de
Merge pull request #3267 from matrix-org/erikj/iter_filter
...
Use iter* methods for _filter_events_for_server
2018-05-24 14:44:57 +01:00
David Baker
9700d15611
pep8
2018-05-24 11:23:15 +01:00
David Baker
a21a41bad7
comment
2018-05-24 11:19:59 +01:00
David Baker
b3bff53178
Unbind 3pids when they're deleted too
2018-05-24 11:08:05 +01:00
Richard van der Hoff
a0b3946fe2
Merge branch 'release-v0.30.0' into rav/localpart_in_consent_uri
2018-05-23 16:06:03 +01:00
Richard van der Hoff
3ff6f50eac
Use the localpart in the consent uri
...
... because it's shorter.
2018-05-23 15:28:23 +01:00
Richard van der Hoff
82191b08f6
Support for putting %(consent_uri)s in messages
...
Make it possible to put the URI in the error message and the server notice that
get sent by the server
2018-05-23 15:24:31 +01:00
David Baker
2c7866d664
Hit the 3pid unbind endpoint on deactivation
2018-05-23 14:38:56 +01:00
Erik Johnston
5aaa3189d5
s/values/itervalues/
2018-05-23 10:13:05 +01:00
Erik Johnston
0a4bca4134
Use iter* methods for _filter_events_for_server
2018-05-23 10:04:23 +01:00
Amber Brown
53cc2cde1f
cleanup
2018-05-22 17:32:57 -05:00
Erik Johnston
e85b5a0ff7
Use iter* methods
2018-05-22 19:02:48 +01:00
Erik Johnston
586b66b197
Fix that states is a dict of dicts
2018-05-22 19:02:36 +01:00
Richard van der Hoff
08bfc48abf
custom error code for not leaving server notices room
2018-05-22 17:27:27 +01:00
David Baker
0a078026ea
comment typo
2018-05-22 17:14:06 +01:00
Amber Brown
8f5a688d42
cleanups, self-registration
2018-05-22 10:56:03 -05:00
Amber Brown
a8990fa2ec
Merge remote-tracking branch 'origin/develop' into 3218-official-prom
2018-05-22 10:50:26 -05:00
Erik Johnston
cb2a2ad791
get_domains_from_state returns list of tuples
2018-05-22 16:23:39 +01:00
Richard van der Hoff
a5e2941aad
Reject attempts to send event before privacy consent is given
...
Returns an M_CONSENT_NOT_GIVEN error (cf
https://github.com/matrix-org/matrix-doc/issues/1252 ) if consent is not yet
given.
2018-05-22 12:00:47 +01:00
Richard van der Hoff
8810685df9
Stub out ServerNoticesSender on the workers
...
... and have the sync endpoints call it directly rather than obsure indirection
via PresenceHandler
2018-05-22 11:54:51 +01:00
Richard van der Hoff
9ea219c514
Send users a server notice about consent
...
When a user first syncs, we will send them a server notice asking them to
consent to the privacy policy if they have not already done so.
2018-05-22 11:54:51 +01:00
Amber Brown
df9f72d9e5
replacing portions
2018-05-21 19:47:37 -05:00
Adrian Tschira
933bf2dd35
replace some iteritems with six
...
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-05-19 17:59:26 +02:00
Richard van der Hoff
6d9dc67139
Merge pull request #3232 from matrix-org/rav/server_notices_room
...
Infrastructure for a server notices room
2018-05-18 11:28:04 +01:00
Richard van der Hoff
67af392712
Merge pull request #3233 from matrix-org/rav/remove_dead_code
...
Remove unused `update_external_syncs`
2018-05-18 11:22:43 +01:00
Richard van der Hoff
26305788fe
Make sure we reject attempts to invite the notices user
2018-05-18 11:18:39 +01:00
Erik Johnston
fa30ac38cc
Merge pull request #3221 from matrix-org/erikj/purge_token
...
Make purge_history operate on tokens
2018-05-18 10:35:23 +01:00
Richard van der Hoff
88d3405332
fix missing yield for server_notices_room
2018-05-17 18:33:45 +01:00
Richard van der Hoff
d43d480d86
Remove unused update_external_syncs
...
This method isn't used anywhere. Burninate it.
2018-05-17 18:22:19 +01:00
Richard van der Hoff
fed62e21ad
Infrastructure for a server notices room
...
Server Notices use a special room which the user can't dismiss. They are
created on demand when some other bit of the code calls send_notice.
(This doesn't actually do much yet becuse we don't call send_notice anywhere)
2018-05-17 17:58:25 +01:00
Richard van der Hoff
c46367d0d7
Move RoomCreationHandler out of synapse.handlers.Handlers
...
Handlers is deprecated nowadays, so let's move this out before I add a new
dependency on it.
Also fix the docstrings on create_room.
2018-05-17 09:08:42 +01:00
Erik Johnston
5f27ed75ad
Make purge_history operate on tokens
...
As we're soon going to change how topological_ordering works
2018-05-15 16:23:50 +01:00
David Baker
8cbbfd16fb
Merge pull request #3201 from matrix-org/dbkr/leave_rooms_on_deactivate
...
Part user from rooms on account deactivate
2018-05-14 11:31:48 +01:00
David Baker
6b49628e3b
Catch failure to part user from room
2018-05-10 12:23:53 +01:00
David Baker
217bc53c98
Many docstrings
2018-05-10 12:20:40 +01:00
David Baker
4d298506dd
Oops, don't call function passed to run_in_background
2018-05-10 11:57:13 +01:00
Erik Johnston
bfe1f73855
Merge pull request #3199 from matrix-org/erikj/pagination_sync
...
Refactor sync APIs to reuse pagination API
2018-05-09 16:16:56 +01:00
Erik Johnston
d26bec8a43
Add comment to sync as to why code path is split
2018-05-09 15:56:07 +01:00
Erik Johnston
fcf55f2255
Fix returned token is no longer a tuple
2018-05-09 15:43:00 +01:00
David Baker
294e9a0c9b
Prefix internal functions
2018-05-09 15:10:37 +01:00
David Baker
52281e4c54
Indent fail
2018-05-09 15:06:16 +01:00
David Baker
7e8726b8fb
Part deactivated users in the background
...
One room at a time so we don't take out the whole server with leave
events, and restart at server restart.
2018-05-09 14:54:28 +01:00
Erik Johnston
e2accd7f1d
Refactor sync APIs to reuse pagination API
...
The sync API often returns events in a topological rather than stream
ordering, e.g. when the user joined the room or on initial sync. When
this happens we can reuse existing pagination storage functions.
2018-05-09 13:43:39 +01:00
Erik Johnston
c4af4c24ca
Refactor get_recent_events_for_room return type
...
There is no reason to return a tuple of tokens when the last token is
always the token passed as an argument. Changing it makes it consistent
with other storage APIs
2018-05-09 11:55:34 +01:00
David Baker
bf98fa0864
Part user from rooms on account deactivate
...
This implements this very crudely: this probably isn't viable
because parting a user from all their rooms could take a long time,
and if the HS gets restarted in that time the process will be
aborted.
2018-05-08 15:58:35 +01:00
Richard van der Hoff
ca7211104e
Merge branch 'release-v0.28.1' into develop
2018-05-01 18:16:57 +01:00
Richard van der Hoff
33f469ba19
Apply some limits to depth to counter abuse
...
* When creating a new event, cap its depth to 2^63 - 1
* When receiving events, reject any without a sensible depth
As per https://docs.google.com/document/d/1I3fi2S-XnpO45qrpCsowZv8P8dHcNZ4fsBsbOW7KABI
2018-05-01 17:54:19 +01:00
Richard van der Hoff
63ae5cbf34
Merge pull request #3143 from matrix-org/rav/remove_redundant_preserve_fn
...
Remove redundant call to preserve_fn
2018-04-30 10:23:59 +01:00
Richard van der Hoff
bc2017a594
Merge pull request #3153 from NotAFile/py3-httplib
...
move httplib import to six
2018-04-30 01:26:42 +01:00
Richard van der Hoff
683149c1f9
Merge pull request #3151 from NotAFile/py3-xrange-1
...
Move more xrange to six
2018-04-30 01:20:06 +01:00
Richard van der Hoff
3b0e431c82
Merge pull request #3150 from NotAFile/py3-listcomp-yield
...
Don't yield in list comprehensions
2018-04-30 01:11:41 +01:00
Richard van der Hoff
db75c86e84
Merge branch 'develop' into py3-xrange-1
2018-04-30 01:02:25 +01:00
Richard van der Hoff
af3cc50511
Remove redundant call to preserve_fn
...
submit_event_for_as doesn't return a deferred anyway, so this is pointless.
2018-04-30 00:48:36 +01:00
Adrian Tschira
d82b6ea9e6
Move more xrange to six
...
plus a bonus next()
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-04-28 13:57:00 +02:00
Adrian Tschira
94f4d7f49e
move httplib import to six
2018-04-28 13:43:34 +02:00
Adrian Tschira
cdb4647a80
Don't yield in list comprehensions
...
I've tried to grep for more of this with no success.
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-04-28 13:36:30 +02:00
Richard van der Hoff
453adf00b6
pep8; remove spurious import
2018-04-27 14:32:08 +01:00
Richard van der Hoff
fc149b4eeb
Merge remote-tracking branch 'origin/develop' into rav/use_run_in_background
2018-04-27 14:31:23 +01:00
Richard van der Hoff
2a13af23bc
Use run_in_background in preference to preserve_fn
...
While I was going through uses of preserve_fn for other PRs, I converted places
which only use the wrapped function once to use run_in_background, to avoid
creating the function object.
2018-04-27 12:55:51 +01:00
Richard van der Hoff
41d4b07a53
Merge pull request #3142 from matrix-org/rav/reraise
...
reraise exceptions more carefully
2018-04-27 12:16:19 +01:00
Richard van der Hoff
9c3da24561
Merge pull request #3138 from matrix-org/rav/catch_unhandled_exceptions
...
Improve exception handling for background processes
2018-04-27 11:47:49 +01:00
Richard van der Hoff
6493b22b42
reraise exceptions more carefully
...
We need to be careful (under python 2, at least) that when we reraise an
exception after doing some error handling, we actually reraise the original
exception rather than anything that might have been raised (and handled) during
the error handling.
2018-04-27 11:40:06 +01:00
Richard van der Hoff
605defb9e4
Add missing consumeErrors
...
In general we want defer.gatherResults to consumeErrors, rather than having
exceptions hanging around and getting logged as CRITICAL unhandled errors.
2018-04-27 11:16:28 +01:00
Richard van der Hoff
9255a6cb17
Improve exception handling for background processes
...
There were a bunch of places where we fire off a process to happen in the
background, but don't have any exception handling on it - instead relying on
the unhandled error being logged when the relevent deferred gets
garbage-collected.
This is unsatisfactory for a number of reasons:
- logging on garbage collection is best-effort and may happen some time after
the error, if at all
- it can be hard to figure out where the error actually happened.
- it is logged as a scary CRITICAL error which (a) I always forget to grep for
and (b) it's not really CRITICAL if a background process we don't care about
fails.
So this is an attempt to add exception handling to everything we fire off into
the background.
2018-04-27 11:07:40 +01:00
Richard van der Hoff
77ebef9d43
Merge pull request #3118 from matrix-org/rav/reject_prev_events
...
Reject events which have lots of prev_events
2018-04-23 17:51:38 +01:00
Richard van der Hoff
9b9c38373c
Remove spurious param
2018-04-23 12:00:06 +01:00
Richard van der Hoff
8dc4a6144b
Merge pull request #3107 from NotAFile/py3-bool-nonzero
...
add __bool__ alias to __nonzero__ methods
2018-04-20 15:43:39 +01:00
Richard van der Hoff
3de7d9fe99
accept stupid events over backfill
2018-04-20 11:41:03 +01:00
Richard van der Hoff
11a67b7c9d
Merge pull request #3093 from matrix-org/rav/response_cache_wrap
...
Refactor ResponseCache usage
2018-04-20 11:31:17 +01:00
Richard van der Hoff
bc381d5798
Merge pull request #3117 from matrix-org/rav/refactor_have_events
...
Refactor store.have_events
2018-04-20 10:26:12 +01:00
Richard van der Hoff
b1dfbc3c40
Refactor store.have_events
...
It turns out that most of the time we were calling have_events, we were only
using half of the result. Replace have_events with have_seen_events and
get_rejection_reasons, so that we can see what's going on a bit more clearly.
2018-04-20 10:25:56 +01:00
Richard van der Hoff
1f4b498b73
Add some comments
2018-04-18 00:15:36 +01:00
Richard van der Hoff
e585228860
Check events on backfill too
2018-04-18 00:06:42 +01:00
Richard van der Hoff
9b7794262f
Reject events which have too many auth_events or prev_events
...
... this should protect us from being dossed by people making silly events
(deliberately or otherwise)
2018-04-18 00:06:42 +01:00
Richard van der Hoff
639480e14a
Avoid creating events with huge numbers of prev_events
...
In most cases, we limit the number of prev_events for a given event to 10
events. This fixes a particular code path which created events with huge
numbers of prev_events.
2018-04-16 18:41:37 +01:00
Adrian Tschira
f63ff73c7f
add __bool__ alias to __nonzero__ methods
...
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-04-15 20:40:47 +02:00
Richard van der Hoff
b78395b7fe
Refactor ResponseCache usage
...
Adds a `.wrap` method to ResponseCache which wraps up the boilerplate of a
(get, set) pair, and then use it throughout the codebase.
This will be largely non-functional, but does include the following functional
changes:
* federation_server.on_context_state_request: drops use of _server_linearizer
which looked redundant and could cause incorrect cache misses by yielding
between the get and the set.
* RoomListHandler.get_remote_public_room_list(): fixes logcontext leaks
* the wrap function includes some logging. I'm hoping this won't be too noisy
on production.
2018-04-12 13:02:15 +01:00
Richard van der Hoff
d5c74b9f6c
Merge pull request #3092 from matrix-org/rav/response_cache_metrics
...
Add metrics for ResponseCache
2018-04-12 12:59:36 +01:00
Erik Johnston
19ceb4851f
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/processed_event_lag
2018-04-12 11:36:07 +01:00
Erik Johnston
f67e906e18
Set all metrics at the same time
2018-04-12 11:18:19 +01:00
Erik Johnston
971059a733
Merge pull request #3088 from matrix-org/erikj/as_parallel
...
Send events to ASes concurrently
2018-04-12 10:42:36 +01:00
Erik Johnston
4dae4a97ed
Track last processed event received_ts
2018-04-11 14:27:09 +01:00
Erik Johnston
92e34615c5
Track where event stream processing have gotten up to
2018-04-11 12:13:40 +01:00
Erik Johnston
121591568b
Send events to ASes concurrently
2018-04-11 09:56:00 +01:00
Richard van der Hoff
b3384232a0
Add metrics for ResponseCache
2018-04-10 23:14:47 +01:00
Erik Johnston
eaa2ebf20b
Merge pull request #3079 from matrix-org/erikj/limit_concurrent_sends
...
Limit concurrent event sends for a room
2018-04-10 16:43:58 +01:00
Erik Johnston
f8e8ec013b
Note why we're limiting concurrent event sends
2018-04-10 14:00:46 +01:00
Richard van der Hoff
a3599dda97
Merge pull request #2996 from krombel/allow_auto_join_rooms
...
move handling of auto_join_rooms to RegisterHandler
2018-04-10 01:11:00 +01:00
Richard van der Hoff
d2c6f4d626
Merge pull request #3080 from matrix-org/rav/fix_500_on_rejoin
...
Return a 404 rather than a 500 on rejoining empty rooms
2018-04-09 17:32:36 +01:00
Richard van der Hoff
f3ef60662f
Return a 404 rather than a 500 on rejoining empty rooms
...
Filter ourselves out of the server list before checking for an empty remote
host list, to fix 500 error
Fixes #2141
2018-04-09 12:56:22 +01:00
Erik Johnston
e5082494eb
Limit concurrent event sends for a room
2018-04-09 12:07:39 +01:00
David Baker
b370fe61c0
Implement group join API
2018-04-06 15:43:27 +01:00
Luke Barnard
e089100c62
Merge pull request #3045 from matrix-org/dbkr/group_joinable
...
Add joinability for groups
2018-04-05 15:57:49 +01:00
Richard van der Hoff
725a72ec5a
Merge pull request #3000 from NotAFile/change-except-style
...
Replace old style error catching with 'as' keyword
2018-04-04 10:45:22 +01:00
Luke Barnard
eb8d8d6f57
Use join_policy API instead of joinable
...
The API is now under
/groups/$group_id/setting/m.join_policy
and expects a JSON blob of the shape
```json
{
"m.join_policy": {
"type": "invite"
}
}
```
where "invite" could alternatively be "open".
2018-04-03 16:16:40 +01:00
Richard van der Hoff
8da39ad98f
Merge pull request #3049 from matrix-org/rav/use_staticjson
...
Use static JSONEncoders
2018-04-03 15:18:32 +01:00
Richard van der Hoff
05630758f2
Use static JSONEncoders
...
using json.dumps with custom options requires us to create a new JSONEncoder on
each call. It's more efficient to create one upfront and reuse it.
2018-03-29 23:13:33 +01:00
Richard van der Hoff
fcfe7f6ad3
Use simplejson throughout
...
Let's use simplejson rather than json, for consistency.
2018-03-29 22:45:52 +01:00
Richard van der Hoff
9a0db062af
Merge pull request #3034 from matrix-org/rav/fix_key_claim_errors
...
Fix error when claiming e2e keys from offline servers
2018-03-28 14:50:38 +01:00
David Baker
79452edeee
Add joinability for groups
...
Adds API to set the 'joinable' flag, and corresponding flag in the
table.
2018-03-28 14:03:37 +01:00
Krombel
6152e253d8
Merge branch 'develop' of into allow_auto_join_rooms
2018-03-28 14:45:28 +02:00
Richard van der Hoff
a134c572a6
Stringify exceptions for keys/{query,claim}
...
Make sure we stringify any exceptions we return from keys/query and keys/claim,
to avoid a 'not JSON serializable' error later
Fixes #3010
2018-03-27 17:15:06 +01:00
Richard van der Hoff
c2a5cf2fe3
factor out exception handling for keys/claim and keys/query
...
this stuff is badly c&p'ed
2018-03-27 17:11:23 +01:00
Erik Johnston
3e0c0660b3
Also do check inside linearizer
2018-03-27 13:01:34 +01:00
Erik Johnston
44cd6e1358
PEP8
2018-03-26 12:06:48 +01:00
Erik Johnston
a052aa42e7
Linearize calls to _generate_user_id
2018-03-26 12:02:20 +01:00
Erik Johnston
fa72803490
Merge branch 'master' of github.com:matrix-org/synapse into develop
2018-03-19 11:41:01 +00:00
Matthew Hodgson
b2f2282947
make lazy_load_members configurable in filters
2018-03-19 01:15:13 +00:00
Matthew Hodgson
478af0f720
reshuffle todo & comments
2018-03-19 01:00:12 +00:00
Matthew Hodgson
366f730bf6
only get member state IDs for incremental syncs if we're filtering
2018-03-18 21:40:43 +00:00
Matthew Hodgson
fc5397fdf5
remove debug
2018-03-16 01:44:55 +00:00
Matthew Hodgson
4f0493c850
fix tsm search again
2018-03-16 01:43:37 +00:00
Matthew Hodgson
f7dcc404f2
add state_ids for timeline entries
2018-03-16 01:37:53 +00:00
Matthew Hodgson
5b3b3aada8
simplify timeline_start_members
2018-03-16 01:17:34 +00:00
Matthew Hodgson
3bc5bd2d22
make incr syncs work
2018-03-16 00:52:04 +00:00
Erik Johnston
926ba76e23
Replace ujson with simplejson
2018-03-15 23:43:31 +00:00
NotAFile
2cc9f76bc3
replace old style error catching with 'as' keyword
...
This is both easier to read and compatible with python3 (not that that
matters)
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-03-15 16:11:17 +01:00
Krombel
91ea0202e6
move handling of auto_join_rooms to RegisterHandler
...
Currently the handling of auto_join_rooms only works when a user
registers itself via public register api. Registrations via
registration_shared_secret and ModuleApi do not work
This auto_joins the users in the registration handler which enables
the auto join feature for all 3 registration paths.
This is related to issue #2725
Signed-Off-by: Matthias Kesler <krombel@krombel.de>
2018-03-14 16:45:37 +01:00
Matthew Hodgson
056a6df546
Merge branch 'develop' into matthew/filter_members
2018-03-14 15:38:05 +00:00
Erik Johnston
57db62e554
Merge pull request #2992 from matrix-org/erikj/implement_member_workre
...
Implement RoomMemberWorkerHandler
2018-03-14 14:29:33 +00:00
Erik Johnston
62ad701326
s/join/joined/ in notify_user_membership_change
2018-03-14 14:17:43 +00:00
Erik Johnston
3f0f06cb31
Split RoomMemberWorkerHandler to separate file
2018-03-14 11:41:45 +00:00
Erik Johnston
3e839e0548
Merge pull request #2989 from matrix-org/erikj/profile_cache_master
...
Only update remote profile cache on master
2018-03-14 09:42:27 +00:00
Matthew Hodgson
4d0cfef6ee
add copyright to nudge CI
2018-03-14 00:02:20 +00:00
Matthew Hodgson
c9d72e4571
oops
2018-03-13 23:46:45 +00:00
Matthew Hodgson
ccca02846d
make it work
2018-03-13 22:31:41 +00:00
Matthew Hodgson
12350e3f9a
merge proper fix to bug 2969
2018-03-13 22:11:58 +00:00
Matthew Hodgson
14a9d2f73d
ensure we always include the members for a given timeline block
2018-03-13 22:03:42 +00:00
Erik Johnston
b27320b550
Implement RoomMemberWorkerHandler
2018-03-13 18:26:00 +00:00
Erik Johnston
350331d466
_remote_join and co take a requester
2018-03-13 17:50:39 +00:00
Erik Johnston
1a69c6d590
Merge pull request #2987 from matrix-org/erikj/split_room_member_handler
...
Split RoomMemberHandler into base and master class
2018-03-13 17:40:00 +00:00
Erik Johnston
df8ff682a7
Only update remote profile cache on master
2018-03-13 17:38:21 +00:00
Erik Johnston
6dbebef141
Add missing param to docstrings
2018-03-13 17:15:32 +00:00
Erik Johnston
16adb11cc0
Correct import order
2018-03-13 16:57:07 +00:00
Erik Johnston
82f16faa78
Move user_*_room distributor stuff to master class
...
I added yields when calling user_left_room, but they shouldn't matter on
the master process as they always return None anyway.
2018-03-13 16:38:15 +00:00
Erik Johnston
b78717b87b
Split RoomMemberHandler into base and master class
...
The intention here is to split the class into the bits that can be done
on workers and the bits that have to be done on the master.
In future there will also be a class that can be run on the worker,
which will delegate work to the master when necessary.
2018-03-13 16:37:41 +00:00
Erik Johnston
95cb401ae0
Merge pull request #2978 from matrix-org/erikj/refactor_replication_layer
...
Remove ReplicationLayer and user Client/Server directly
2018-03-13 15:45:08 +00:00
Erik Johnston
5d8476d8ff
Merge pull request #2981 from matrix-org/erikj/factor_remote_leave
...
Factor out _remote_reject_invite in RoomMember
2018-03-13 15:44:56 +00:00
Erik Johnston
56e709857c
Merge pull request #2979 from matrix-org/erikj/no_handlers
...
Don't build handlers on workers unnecessarily
2018-03-13 13:46:38 +00:00
Erik Johnston
cb9f8e527c
s/replication_client/federation_client/
2018-03-13 13:26:52 +00:00
Erik Johnston
bf8e97bd3c
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/factor_remote_leave
2018-03-13 13:17:08 +00:00
Erik Johnston
ea3442c15c
Add docstring
2018-03-13 13:16:21 +00:00
Erik Johnston
16469a4f15
Merge pull request #2980 from matrix-org/erikj/rm_priv
...
Make RoomMemberHandler functions private that can be
2018-03-13 13:11:11 +00:00
Erik Johnston
c82111a55f
Merge pull request #2982 from matrix-org/erikj/fix_extra_users
...
extra_users is actually a list of UserIDs
2018-03-13 13:11:04 +00:00
Erik Johnston
da87791975
Merge pull request #2983 from matrix-org/erikj/rename_register_3pid
...
Refactor get_or_register_3pid_guest
2018-03-13 13:10:52 +00:00
Erik Johnston
8b3573a8b2
Refactor get_or_register_3pid_guest
2018-03-13 12:08:58 +00:00
Erik Johnston
0f942f68c1
Factor out _remote_reject_invite in RoomMember
2018-03-13 11:22:45 +00:00
Erik Johnston
d0fcc48f9d
extra_users is actually a list of UserIDs
2018-03-13 11:20:06 +00:00
Erik Johnston
31becf4ac3
Make functions private that can be
2018-03-13 11:15:16 +00:00
Erik Johnston
265b993b8a
Split replication layer into two
2018-03-13 10:55:47 +00:00
Erik Johnston
c3f79c9da5
Split out edu/query registration to a separate class
2018-03-13 10:24:27 +00:00
Richard van der Hoff
e48c7aac4d
Add transactional API to history purge
...
Make the purge request return quickly, and allow scripts to poll for updates.
2018-03-12 16:22:55 +00:00
Richard van der Hoff
1708412f56
Return an error when doing two purges on a room
...
Queuing up purges doesn't sound like a good thing.
2018-03-12 16:22:54 +00:00
Matthew Hodgson
8713365265
typos
2018-03-11 20:10:25 +00:00
Matthew Hodgson
9b334b3f97
WIP experiment in lazyloading room members
2018-03-11 20:01:41 +00:00
Erik Johnston
a56d54dcb7
Fix up log message
2018-03-07 11:55:31 +00:00
Erik Johnston
02a1296ad6
Fix typo
2018-03-07 11:55:31 +00:00
Erik Johnston
8cb44da4aa
Fix race in sync when joining room
...
The race happens when the user joins a room at the same time as doing a
sync. We fetch the current token and then get the rooms the user is in.
If the join happens after the current token, but before we get the rooms
we end up sending down a partial room entry in the sync.
This is fixed by looking at the stream ordering of the membership
returned by get_rooms_for_user, and handling the case when that stream
ordering is after the current token.
2018-03-07 11:55:31 +00:00
Richard van der Hoff
8ffaacbee3
Merge pull request #2949 from krombel/use_bcrypt_checkpw
...
use bcrypt.checkpw
2018-03-06 11:56:06 +00:00
Richard van der Hoff
b2932107bb
Merge pull request #2946 from matrix-org/rav/timestamp_to_purge
...
Implement purge_history by timestamp
2018-03-06 11:20:23 +00:00
Krombel
ed9b5eced4
use bcrypt.checkpw
...
in bcrypt 3.1.0 checkpw got introduced (already 2 years ago)
This makes use of that with enhancements which might get introduced
by that
Signed-Off-by: Matthias Kesler <krombel@krombel.de>
2018-03-05 18:02:59 +01:00
Erik Johnston
d4ffe61d4f
Remove ability for AS users to call /events and /sync
...
This functionality has been deprecated for a while as well as being
broken for a while. Instead of fixing it lets just remove it entirely.
See: https://github.com/matrix-org/matrix-doc/issues/1144
2018-03-05 15:44:46 +00:00
Richard van der Hoff
f8bfcd7e0d
Provide a means to pass a timestamp to purge_history
2018-03-05 14:37:23 +00:00
Erik Johnston
d960d23830
Add missing yield during 3pid signature checks
2018-03-02 11:03:18 +00:00
Erik Johnston
9e08a93a7b
Merge pull request #2927 from matrix-org/erikj/read_marker_caches
...
Improve caching for read_marker API
2018-03-01 17:12:34 +00:00
Erik Johnston
a83c514d1f
Improve caching for read_marker API
...
We add a new storage function to get a paritcular type of room account
data. This allows us to prefill the cache when updating that acount
data.
2018-03-01 17:08:17 +00:00
Erik Johnston
529c026ac1
Move back to hs.is_mine
2018-03-01 16:49:12 +00:00
Erik Johnston
784f036306
Move RoomMemberHandler out of Handlers
2018-03-01 14:36:50 +00:00
Erik Johnston
f381d63813
Check event auth on the worker
2018-03-01 10:18:37 +00:00
Erik Johnston
6b8604239f
Correctly send ratelimit and extra_users params
2018-03-01 10:08:39 +00:00
Erik Johnston
f756f961ea
Fixup comments
2018-03-01 10:05:27 +00:00
Erik Johnston
28e973ac11
Calculate push actions on worker
2018-02-28 18:02:30 +00:00
Erik Johnston
d874d4f2d7
Delete from push_actions_staging in federation too
2018-02-20 11:37:52 +00:00
Erik Johnston
b96278d6fe
Ensure that we delete staging push actions on errors
2018-02-15 15:47:06 +00:00
Erik Johnston
c0c9327fe0
Merge pull request #2854 from matrix-org/erikj/event_create_worker
...
Create a worker for event creation
2018-02-13 18:07:10 +00:00
Richard van der Hoff
10b34dbb9a
Merge pull request #2858 from matrix-org/rav/purge_updates
...
delete_local_events for purge_room_history
2018-02-09 14:11:00 +00:00
Richard van der Hoff
74fcbf741b
delete_local_events for purge_history
...
Add a flag which makes the purger delete local events
2018-02-09 13:07:41 +00:00
Richard van der Hoff
671540dccf
rename delete_old_state -> purge_history
...
(beacause it deletes more than state)
2018-02-09 13:07:41 +00:00
Erik Johnston
50fe92cd26
Move presence handling into handle_new_client_event
...
As we want to have it run on the main synapse instance
2018-02-07 10:34:09 +00:00
Erik Johnston
24dd73028a
Add replication http endpoint for event sending
2018-02-07 10:32:32 +00:00
Erik Johnston
e3624fad5f
Remove pointless ratelimit check
...
The intention was for the check to be called as early as possible in the
request, but actually was called just before the main ratelimit check,
so was fairly pointless.
2018-02-07 10:30:25 +00:00
Erik Johnston
617199d73d
Merge pull request #2847 from matrix-org/erikj/separate_event_creation
...
Split event creation into a separate handler
2018-02-06 17:01:17 +00:00
Erik Johnston
3e1e69ccaf
Update copyright
2018-02-06 16:40:38 +00:00
Erik Johnston
770b2252ca
s/_create_new_client_event/create_new_client_event/
2018-02-06 16:40:30 +00:00
Erik Johnston
3d33eef6fc
Store state groups separately from events ( #2784 )
...
* Split state group persist into seperate storage func
* Add per database engine code for state group id gen
* Move store_state_group to StateReadStore
This allows other workers to use it, and so resolve state.
* Hook up store_state_group
* Fix tests
* Rename _store_mult_state_groups_txn
* Rename StateGroupReadStore
* Remove redundant _have_persisted_state_group_txn
* Update comments
* Comment compute_event_context
* Set start val for state_group_id_seq
... otherwise we try to recreate old state groups
* Update comments
* Don't store state for outliers
* Update comment
* Update docstring as state groups are ints
2018-02-06 14:31:24 +00:00
Erik Johnston
3fa362502c
Update places where we create events
2018-02-05 16:01:48 +00:00
Erik Johnston
5ff3d23564
Split event creation into a separate handler
2018-02-05 16:01:48 +00:00
Richard van der Hoff
0cbda53819
Rename resolve_state_groups -> resolve_state_groups_for_events
...
(to make way for a method that actually just does the state group resolution)
2018-02-01 12:40:00 +00:00
Neil Johnson
73560237d6
add white space line
2018-01-26 00:15:10 +00:00
Neil Johnson
86c4f49a31
rather than try reconstruct the results object, better to guard against the xrange step argument being 0
2018-01-26 00:12:02 +00:00
Neil Johnson
f632083576
fix return type, should be a dict
2018-01-25 23:52:17 +00:00
Neil Johnson
6c6e197b0a
fix PEP8 violation
2018-01-25 23:47:46 +00:00
Neil Johnson
d02e43b15f
remove white space
2018-01-25 23:29:46 +00:00
Neil Johnson
349c739966
synapse 500s on a call to publicRooms in the case where the number of public rooms is zero, the specific cause is due to xrange trying to use a step value of zero, but if the total room number really is zero then it makes sense to just bail and save the extra processing
2018-01-25 23:28:44 +00:00
Matthew Hodgson
ab9f844aaf
Add federation_domain_whitelist option ( #2820 )
...
Add federation_domain_whitelist
gives a way to restrict which domains your HS is allowed to federate with.
useful mainly for gracefully preventing a private but internet-connected HS from trying to federate to the wider public Matrix network
2018-01-22 19:11:18 +01:00
Matthew Hodgson
447f4f0d5f
rewrite based on PR feedback:
...
* [ ] split config options into allowed_local_3pids and registrations_require_3pid
* [ ] simplify and comment logic for picking registration flows
* [ ] fix docstring and move check_3pid_allowed into a new util module
* [ ] use check_3pid_allowed everywhere
@erikjohnston PTAL
2018-01-19 15:33:55 +00:00
Matthew Hodgson
9d332e0f79
fix up v1, and improve errors
2018-01-19 00:53:58 +00:00
Richard van der Hoff
a5213df1f7
Sanity checking for user ids
...
Check the user_id passed to a couple of APIs for validity, to avoid
"IndexError: list index out of range" exception which looks scary and results
in a 500 rather than a more useful error.
Fixes #1432 , among other things
2018-01-17 14:28:54 +00:00
Richard van der Hoff
a027c2af8d
Metrics for events processed in appservice and fed sender
...
More metrics I wished I'd had
2018-01-15 18:23:24 +00:00
Erik Johnston
825a07a974
Merge pull request #2773 from matrix-org/erikj/hash_bg
...
Do bcrypt hashing in a background thread
2018-01-10 18:11:41 +00:00
Erik Johnston
f8e1ab5fee
Do bcrypt hashing in a background thread
2018-01-10 18:01:28 +00:00
Michael Telatynski
5f07f5694c
fix order of operations derp and also use .get to default to {}
...
Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
2018-01-10 15:11:35 +00:00
Michael Telatynski
8c9d5b4873
Fix publicised groups API (singular) over federation
...
which was missing its fed client API, since there is no other API
it might as well reuse the bulk one and unwrap it
Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
2018-01-10 15:04:51 +00:00
Erik Johnston
d8a6c734fa
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/createroom_content
2017-12-07 14:24:01 +00:00
Erik Johnston
ef045dcd71
Copy dict in update_membership too
2017-12-07 14:17:15 +00:00
Matthew Hodgson
33cb7ef0b7
Merge pull request #2723 from matrix-org/matthew/search-all-local-users
...
Add all local users to the user_directory and optionally search them
2017-12-05 11:09:47 +00:00
Richard van der Hoff
da1010c83a
support custom login types for validating users
...
Wire the custom login type support from password providers into the UI-auth
user-validation flows.
2017-12-05 09:43:30 +00:00
Richard van der Hoff
d7ea8c4800
Factor out a validate_user_via_ui_auth method
...
Collect together all the places that validate a logged-in user via UI auth.
2017-12-05 09:42:30 +00:00
Richard van der Hoff
d5f9fb06b0
Refactor UI auth implementation
...
Instead of returning False when auth is incomplete, throw an exception which
can be caught with a wrapper.
2017-12-05 09:40:05 +00:00
Matthew Hodgson
c22e73293a
speed up the rate of initial spam for users
2017-12-04 18:05:28 +00:00
Matthew Hodgson
74e0cc74ce
fix pep8 and tests
2017-12-04 15:11:38 +00:00
Matthew Hodgson
1bd40ca73e
switch to a simpler 'search_all_users' button as per review feedback
2017-12-04 14:58:39 +00:00
Matthew Hodgson
f397153dfc
Merge branch 'develop' into matthew/search-all-local-users
2017-11-30 01:51:38 +00:00
Matthew Hodgson
a4bb133b68
fix thinkos galore
2017-11-30 01:17:15 +00:00
Matthew Hodgson
cd3697e8b7
kick the user_directory index when new users register
2017-11-29 18:33:34 +00:00
Matthew Hodgson
3241c7aac3
untested WIP but might actually work
2017-11-29 18:27:05 +00:00
Matthew Hodgson
47d99a20d5
Add user_directory_include_pattern config param to expand search results to additional users
...
Initial commit; this doesn't work yet - the LIKE filtering seems too aggressive.
It also needs _do_initial_spam to be aware of prepopulating the whole user_directory_search table with all users...
...and it needs a handle_user_signup() or something to be added so that new signups get incrementally added to the table too.
Committing it here as a WIP
2017-11-29 16:46:45 +00:00
Richard van der Hoff
ad7e570d07
Delete devices in various logout situations
...
Make sure that we delete devices whenever a user is logged out due to any of
the following situations:
* /logout
* /logout_all
* change password
* deactivate account (by the user or by an admin)
* invalidate access token from a dynamic module
Fixes #2672 .
2017-11-29 16:44:35 +00:00
Richard van der Hoff
ae31f8ce45
Move set_password into its own handler
...
Non-functional refactoring to move set_password. This means that we'll be able
to properly deactivate devices and access tokens without introducing a
dependency loop.
2017-11-29 16:44:35 +00:00
Richard van der Hoff
7ca5c68233
Move deactivate_account into its own handler
...
Non-functional refactoring to move deactivate_account. This means that we'll be
able to properly deactivate devices and access tokens without introducing a
dependency loop.
2017-11-29 16:44:35 +00:00
Richard van der Hoff
2c6d63922a
Remove pushers when deleting access tokens
...
Whenever an access token is invalidated, we should remove the associated
pushers.
2017-11-29 16:44:35 +00:00
Erik Johnston
dfbc45302e
PEP8
2017-11-28 15:23:26 +00:00
Erik Johnston
c4c1d170af
Fix wrong avatars when inviting multiple users when creating room
...
We reused the `content` dictionary between invite requests, which meant they could end up reusing the profile info for a previous user
2017-11-28 15:19:15 +00:00
Luke Barnard
cdf6fc15b0
Merge pull request #2686 from matrix-org/luke/as-flair
...
Add automagical AS Publicised Group(s)
2017-11-17 10:13:46 +00:00
Jurek
eded7084d2
Fix auth handler #2678
2017-11-17 10:07:27 +00:00
Matthew Hodgson
34c3d0a386
typo
2017-11-17 01:54:02 +00:00
Luke Barnard
5b48eec4a1
Make sure we check AS groups for lookup on bulk
2017-11-16 17:55:15 +00:00
Jurek
624a8bbd67
Fix auth handler #2678
2017-11-16 17:19:02 +00:00
Luke Barnard
97bd18af4e
Add automagical AS Publicised Group(s)
...
via registration file "users" namespace:
```YAML
...
namespaces:
users:
- exclusive: true
regex: '.*luke.*'
group_id: '+all_the_lukes:hsdomain'
...
```
This is part of giving App Services their own groups for matching users. With this, ghost users will be given the appeareance that they are in a group and that they have publicised the fact, but _only_ from the perspective of the `get_publicised_groups_for_user` API.
2017-11-16 16:44:55 +00:00
Richard van der Hoff
1b83c09c03
Merge pull request #2675 from matrix-org/rav/remove_broken_logcontext_funcs
...
Remove preserve_context_over_{fn, deferred}
2017-11-15 11:13:53 +00:00
Richard van der Hoff
7e6fa29cb5
Remove preserve_context_over_{fn, deferred}
...
Both of these functions ae known to leak logcontexts. Replace the remaining
calls to them and kill them off.
2017-11-14 11:22:42 +00:00
Richard van der Hoff
44a1bfd6a6
Reshuffle room list request code
...
I'm not entirely sure if this will actually help anything, but it simplifies
the code and might give further clues about why room list search requests are
blowing out the get_current_state_ids caches.
2017-11-14 10:29:58 +00:00
Richard van der Hoff
1fc66c7460
Add a load of logging to the room_list handler
...
So we can see what it gets up to.
2017-11-14 10:23:47 +00:00
Richard van der Hoff
9b803ccc98
Revert "Allow upper-case characters in mxids"
...
This reverts commit b70b646903
2017-11-09 21:57:24 +00:00
Richard van der Hoff
b70b646903
Allow upper-case characters in mxids
...
Because we're never going to be able to fix this :'(
2017-11-09 19:36:13 +00:00
Erik Johnston
e8814410ef
Have an explicit API to update room config
2017-11-08 16:13:27 +00:00
Erik Johnston
94ff2cda73
Revert "Modify group room association API to allow modification of is_public"
2017-11-08 15:43:34 +00:00
Erik Johnston
d305987b40
Merge pull request #2631 from xyzz/fix_appservice_event_backlog
...
Fix appservices being backlogged and not receiving new events due to a bug in notify_interested_services
2017-11-08 11:54:10 +00:00
Richard van der Hoff
02a9a93bde
Merge pull request #2649 from matrix-org/rav/fix_delta_on_state_res
...
Fix bug in state group storage
2017-11-08 09:22:13 +00:00
Richard van der Hoff
e148438e97
s/items/iteritems/
2017-11-08 09:21:41 +00:00
Ilya Zhuravlev
d46386d57e
Remove useless assignment in notify_interested_services
2017-11-07 22:23:22 +03:00
Matthew Hodgson
228ccf1fe3
Merge pull request #2643 from matrix-org/matthew/user_dir_typos
...
Fix various embarrassing typos around user_directory and add some doc.
2017-11-07 17:31:11 +00:00
Richard van der Hoff
780dbb378f
Update deltas when doing auth resolution
...
Fixes a bug where the persisted state groups were different to those actually
being used after auth resolution.
2017-11-07 16:43:00 +00:00
Richard van der Hoff
1ca4288135
factor out _update_context_for_auth_events
...
This is duplicated, so let's factor it out before fixing it
2017-11-07 16:43:00 +00:00
Erik Johnston
5ef5e14ecc
Merge pull request #2636 from farialima/me-master
...
Fix for #2635 : correctly update rooms avatar/display name when modified by admin
2017-11-07 13:49:27 +00:00
Matthew Hodgson
d1622e080f
s/intial/initial/
2017-11-04 19:35:14 +00:00
Francois Granade
f103b91ffa
removed unused import flagged by flake8a
2017-11-03 18:45:49 +01:00
Francois Granade
fa4f337b49
Fix for issue 2635: correctly update rooms avatar/display name when modified by admin
2017-11-03 18:25:04 +01:00
Ilya Zhuravlev
45fbe4ff67
Fix appservices being backlogged and not receiving new events due to a bug in notify_interested_services
2017-11-02 22:49:43 +03:00
Richard van der Hoff
1189be43a2
Factor _AccountHandler proxy out to ModuleApi
...
We're going to need to use this from places that aren't password auth, so let's
move it to a proper class.
2017-11-02 14:36:11 +00:00
David Baker
b19d9e2174
Merge pull request #2624 from matrix-org/rav/password_provider_notify_logout
...
Notify auth providers on logout
2017-11-02 10:55:17 +00:00
David Baker
1f080a6c97
Merge pull request #2623 from matrix-org/rav/callbacks_for_auth_providers
...
Allow password_auth_providers to return a callback
2017-11-02 10:49:03 +00:00
David Baker
04897c9dc1
Merge pull request #2622 from matrix-org/rav/db_access_for_auth_providers
...
Let auth providers get to the database
2017-11-02 10:41:25 +00:00
Richard van der Hoff
979eed4362
Fix user-interactive password auth
...
this got broken in the previous commit
2017-11-01 17:03:20 +00:00
Richard van der Hoff
bc8a5c0330
Notify auth providers on logout
...
Provide a hook by which auth providers can be notified of logouts.
2017-11-01 16:51:51 +00:00
Richard van der Hoff
4c8f94ac94
Allow password_auth_providers to return a callback
...
... so that they have a way to record access tokens.
2017-11-01 16:51:03 +00:00
Richard van der Hoff
846a94fbc9
Merge pull request #2620 from matrix-org/rav/auth_non_password
...
Let password auth providers handle arbitrary login types
2017-11-01 16:45:33 +00:00
Richard van der Hoff
3cd6b22c7b
Let password auth providers handle arbitrary login types
...
Provide a hook where password auth providers can say they know about other
login types, and get passed the relevant parameters
2017-11-01 16:43:57 +00:00
David Baker
4f0488b307
Merge remote-tracking branch 'origin/develop' into rav/refactor_accesstoken_delete
2017-11-01 16:20:19 +00:00
David Baker
e5e930aec3
Merge pull request #2615 from matrix-org/rav/break_auth_device_dep
...
Break dependency of auth_handler on device_handler
2017-11-01 16:06:31 +00:00
David Baker
fbbacb284e
Merge pull request #2613 from matrix-org/rav/kill_refresh_tokens
...
Remove the last vestiges of refresh_tokens
2017-11-01 15:57:35 +00:00
Richard van der Hoff
dd13310fb8
Move access token deletion into auth handler
...
Also move duplicated deactivation code into the auth handler.
I want to add some hooks when we deactivate an access token, so let's bring it
all in here so that there's somewhere to put it.
2017-11-01 15:46:22 +00:00
Richard van der Hoff
74c56f794c
Break dependency of auth_handler on device_handler
...
I'm going to need to make the device_handler depend on the auth_handler, so I
need to break this dependency to avoid a cycle.
It turns out that the auth_handler was only using the device_handler in one
place which was an edge case which we can more elegantly handle by throwing an
error rather than fixing it up.
2017-11-01 10:27:06 +00:00
Richard van der Hoff
356bcafc44
Remove the last vestiges of refresh_tokens
2017-10-31 20:35:58 +00:00
Richard van der Hoff
3e0aaad190
Let auth providers get to the database
...
Somewhat open to abuse, but also somewhat unavoidable :/
2017-10-31 17:22:29 +00:00
Luke Barnard
20fe347906
Modify group room association API to allow modification of is_public
...
also includes renamings to make things more consistent.
2017-10-31 17:04:28 +00:00
Richard van der Hoff
1b65ae00ac
Refactor some logic from LoginRestServlet into AuthHandler
...
I'm going to need some more flexibility in handling login types in password
auth providers, so as a first step, move some stuff from LoginRestServlet into
AuthHandler.
In particular, we pass everything other than SAML, JWT and token logins down to
the AuthHandler, which now has responsibility for checking the login type and
fishing the password out of the login dictionary, as well as qualifying the
user_id if need be. Ideally SAML, JWT and token would go that way too, but
there's no real need for it right now and I'm trying to minimise impact.
This commit *should* be non-functional.
2017-10-31 10:48:41 +00:00
Richard van der Hoff
110b373e9c
Merge pull request #2589 from matrix-org/rav/as_deactivate_account
...
Allow ASes to deactivate their own users
2017-10-27 12:29:32 +01:00
Richard van der Hoff
785bd7fd75
Allow ASes to deactivate their own users
2017-10-27 00:01:00 +01:00
Erik Johnston
566e21eac8
Update room_list.py
2017-10-26 11:39:54 +01:00
Erik Johnston
2a7e9faeec
Do logcontexts outside ResponseCache
2017-10-25 15:21:08 +01:00
Erik Johnston
33a9026cdf
Add logging and fix log contexts for publicRooms
2017-10-25 10:26:06 +01:00
Richard van der Hoff
eaaabc6c4f
replace 'except:' with 'except Exception:'
...
what could possibly go wrong
2017-10-23 15:52:32 +01:00
Richard van der Hoff
3267b81b81
Merge pull request #2561 from matrix-org/rav/id_checking
...
Updates to ID checking
2017-10-23 14:39:20 +01:00
Richard van der Hoff
29812c628b
Allow = in mxids and groupids
...
... because the spec says we should.
2017-10-20 23:42:53 +01:00
Richard van der Hoff
58fbbe0f1d
Disallow capital letters in userids
...
Factor out a common function for checking user ids and group ids, which forbids
capitals.
2017-10-20 23:37:22 +01:00
Richard van der Hoff
631d7b87b5
Remove pointless create() method
...
It just calls the constructor, so we may as well kill it rather than having
random codepaths.
2017-10-20 22:14:55 +01:00
Luke Barnard
a3ac4f6b0a
_create_rererouter for get_invited_users_in_group
2017-10-16 15:41:03 +01:00
Luke Barnard
2c5972f87f
Implement GET /groups/$groupId/invited_users
2017-10-16 15:31:11 +01:00
Luke Barnard
6079d0027a
Log a warning when no profile for invited member
...
And return empty profile
2017-10-16 14:20:45 +01:00
David Baker
b8d8ed1ba9
Merge pull request #2531 from matrix-org/dbkr/spamcheck_error_messages
...
Allow error strings from spam checker
2017-10-12 10:31:03 +01:00
Erik Johnston
b752507b48
Fix fetching remote summaries
2017-10-11 16:59:18 +01:00
Richard van der Hoff
68fd82e840
Merge pull request #2530 from matrix-org/rav/fix_receipt_logcontext
...
fix a logcontext leak in read receipt handling
2017-10-11 15:08:53 +01:00
David Baker
b78bae2d51
fix isinstance
2017-10-11 14:49:09 +01:00
David Baker
c3b7a45e84
Allow error strings from spam checker
2017-10-11 14:39:22 +01:00
Richard van der Hoff
c3e190ce67
fix a logcontext leak in read receipt handling
2017-10-11 14:37:20 +01:00
Erik Johnston
4ce4379235
Fix attestations to check correct server name
2017-10-11 14:11:43 +01:00
Erik Johnston
535cc49f27
Merge pull request #2466 from matrix-org/erikj/groups_merged
...
Initial Group Implementation
2017-10-11 13:20:07 +01:00
Erik Johnston
bc7f3eb32f
Merge pull request #2483 from jeremycline/unfreeze-ujson-dump
...
Unfreeze event before serializing with ujson
2017-10-11 13:18:52 +01:00
Erik Johnston
84e27a592d
Merge pull request #2490 from matrix-org/erikj/drop_left_room_events
...
Ignore incoming events for rooms that we have left
2017-10-10 11:58:32 +01:00
Richard van der Hoff
c8f568ddf9
Fix up deferred handling in federation.py
...
* Avoid preserve_context_over_deferred, which is broken
* set consumeErrors=True on defer.gatherResults, to avoid spurious "unhandled
failure" erros
2017-10-06 22:14:24 +01:00
David Baker
f878e6f8af
Spam checking: add the invitee to user_may_invite
2017-10-05 14:02:28 +01:00
David Baker
d8ce68b09b
spam check room publishing
2017-10-04 14:29:33 +01:00
David Baker
78d4ced829
un-double indent
2017-10-04 12:44:27 +01:00
David Baker
197c14dbcf
Add room creation checks to spam checker
...
Lets the spam checker deny attempts to create rooms and add aliases
to them.
2017-10-04 10:47:54 +01:00
