Matrix-Mirrors/synapse-product
1
0
Fork 0
mirror of https://git.anonymousland.org/anonymousland/synapse-product.git synced 2024-12-18 23:54:19 -05:00
Code Issues Packages Projects Releases Wiki Activity
9,567 Commits 4 Branches 609 Tags 60 MiB
4c8f94ac94
Commit Graph

1942 Commits

Author SHA1 Message Date
Mark Haines
3e279411fe Improve the error handling for bad invites received over federation 2017-06-30 16:20:30 +01:00
Erik Johnston
c72058bcc6 Use an ExpiringCache for storing registration sessions 
This is because pruning them was a significant performance drain on
matrix.org
 2017-06-29 14:08:37 +01:00
Erik Johnston
816605a137 Merge pull request #2307 from matrix-org/erikj/user_ip_batch 
Batch upsert user ips
 2017-06-27 15:08:32 +01:00
Erik Johnston
a0a561ae85 Fix up client ips to read from pending data 2017-06-27 14:46:12 +01:00
Erik Johnston
1bce3e6b35 Remove unused variables 2017-06-26 14:03:27 +01:00
Erik Johnston
e3cbec10c1 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/ensure_round_trip 2017-06-26 14:02:44 +01:00
Erik Johnston
44c722931b Make some more params configurable 2017-06-22 14:59:52 +01:00
Erik Johnston
2d520a9826 Typo. ARGH. 2017-06-22 14:42:39 +01:00
Erik Johnston
24d894e2e2 Fix thinko in unhandled user spam 2017-06-22 14:39:05 +01:00
Erik Johnston
1217c7da91 Don't work out users who share room with appservice users 2017-06-21 12:00:41 +01:00
Erik Johnston
e5ae386ea4 Handle all cases of sending membership events 2017-06-19 16:07:54 +01:00
Erik Johnston
b490299a3b Change to create new room and join other users 2017-06-19 14:10:13 +01:00
Erik Johnston
d7fe6b356c Add shutdown room API 2017-06-19 12:37:27 +01:00
Erik Johnston
fcf01dd88e Reject local events that don't round trip the DB 2017-06-19 11:33:40 +01:00
Erik Johnston
6aa5bc8635 Initial worker impl 2017-06-16 11:47:11 +01:00
Erik Johnston
a9d6fa8b2b Include users who share room with requester in user directory 2017-06-15 10:17:21 +01:00
Erik Johnston
4564b05483 Implement updating users who share rooms on the fly 2017-06-15 10:17:17 +01:00
Erik Johnston
72613bc379 Implement initial population of users who share rooms table 2017-06-15 09:59:04 +01:00
Erik Johnston
d9fd937e39 Fix user directory insertion due to missing room_id 2017-06-13 11:50:24 +01:00
Erik Johnston
6ae8373d40 Don't assume existance of events when updating user directory 2017-06-13 10:19:26 +01:00
Erik Johnston
a837765e8c Merge pull request #2266 from matrix-org/erikj/host_in_room 
Change is_host_joined to use current_state table
 2017-06-12 09:49:51 +01:00
Erik Johnston
1664948e41 Comment 2017-06-09 13:05:05 +01:00
Erik Johnston
0185b75381 Change is_host_joined to use current_state table 
This bypasses a bug where using the state groups to figure out if a host
is in a room sometimes errors if the servers isn't in the room. (For
example when the server rejected an invite to a remote room)
 2017-06-09 10:52:26 +01:00
Erik Johnston
7132e5cdff Mark remote invite rejections as outliers 2017-06-09 10:08:18 +01:00
Erik Johnston
f45f07ab86 Merge pull request #2258 from matrix-org/erikj/user_dir 
Don't start user_directory handling on workers
 2017-06-07 14:04:50 +01:00
Erik Johnston
a053ff3979 Merge pull request #2248 from matrix-org/erikj/state_fixup 
Faster cache for get_joined_hosts
 2017-06-07 14:01:06 +01:00
Erik Johnston
ecdd2a3658 Don't start user_directory handling on workers 2017-06-07 12:02:53 +01:00
Erik Johnston
2f34ad31ac Add some logging to user directory 2017-06-07 11:50:44 +01:00
Erik Johnston
09e4bc0501 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/state_fixup 2017-06-07 11:05:23 +01:00
Erik Johnston
1a01af079e Handle profile updates in user directory 2017-06-01 15:39:51 +01:00
Erik Johnston
21e255a8f1 Split the table in two 2017-06-01 14:50:46 +01:00
Erik Johnston
7233341eac Comments 2017-06-01 13:11:38 +01:00
Erik Johnston
8be6fd95a3 Check if host is still in room 2017-06-01 13:05:39 +01:00
Erik Johnston
9c7db2491b Fix removing users 2017-06-01 11:36:50 +01:00
Erik Johnston
0fe6f3c521 Bug fixes and logging 
- Check if room is public when a user joins before adding to user dir
- Fix typo of field name "content.join_rules" -> "content.join_rule"
 2017-06-01 11:09:49 +01:00
Erik Johnston
cc7609aa9f Comment briefly on how we keep user_directory up to date 2017-05-31 17:11:18 +01:00
Erik Johnston
b2d8d07109 Lifts things into separate function 2017-05-31 17:00:24 +01:00
Erik Johnston
f9791498ae Typos 2017-05-31 16:50:57 +01:00
Erik Johnston
f091061711 Fix tests 2017-05-31 16:34:40 +01:00
Erik Johnston
dc51af3d03 Pull max id from correct table 2017-05-31 15:13:49 +01:00
Erik Johnston
350622a107 Handle the server leaving a public room 2017-05-31 15:11:36 +01:00
Erik Johnston
63fda37e20 Add comments 2017-05-31 15:00:29 +01:00
Erik Johnston
3b5f22ca40 Add search 2017-05-31 14:00:01 +01:00
Erik Johnston
b5db4ed5f6 Update room column when room becomes unpublic 2017-05-31 13:40:28 +01:00
Erik Johnston
168524543f Add call later 2017-05-31 11:59:36 +01:00
Erik Johnston
3e123b8497 Start later 2017-05-31 11:56:27 +01:00
Erik Johnston
42137efde7 Don't go round in circles 2017-05-31 11:55:13 +01:00
Erik Johnston
eeb2f9e546 Add user_directory to database 2017-05-31 11:51:01 +01:00
Erik Johnston
23da638360 Fix typing tests 2017-05-26 10:02:04 +01:00
Erik Johnston
11f139a647 Merge pull request #2247 from matrix-org/erikj/auth_event 
Only store event_auth for state events
 2017-05-24 16:46:34 +01:00
Erik Johnston
c049472b8a Only store event_auth for state events 2017-05-24 15:23:31 +01:00
Erik Johnston
b4f59c7e27 Add count of one time keys to sync stream 2017-05-19 15:47:55 +01:00
Erik Johnston
cafe659f72 Store ActionGenerator in HomeServer 2017-05-19 10:09:56 +01:00
Erik Johnston
66d8ffabbd Faster push rule calculation via push specific cache 
We add a push rule specific cache that ensures that we can reuse
calculated push rules appropriately when a user join/leaves.
 2017-05-17 16:55:40 +01:00
Erik Johnston
6fa8148ccb Merge pull request #2228 from matrix-org/erikj/speed_up_get_hosts 
Speed up get_joined_hosts
 2017-05-16 17:40:55 +01:00
Erik Johnston
13f540ef1b Speed up get_joined_hosts 2017-05-16 16:05:22 +01:00
Erik Johnston
ec5c4499f4 Make presence use cached users/hosts in room 2017-05-16 16:01:43 +01:00
Luke Barnard
ccad2ed824 Modify condition on empty localpart 2017-05-10 17:34:30 +01:00
Luke Barnard
369195caa5 Modify register/available to be GET with query param 
- GET is now the method for register/available
- a query parameter "username" is now used

Also, empty usernames are now handled with an error message on registration or via register/available: `User ID cannot be empty`
 2017-05-10 17:23:55 +01:00
Erik Johnston
a3648f84b2 Merge pull request #2208 from matrix-org/erikj/ratelimit_overrid 
Add per user ratelimiting overrides
 2017-05-10 15:54:48 +01:00
Erik Johnston
b990b2fce5 Add per user ratelimiting overrides 2017-05-10 11:05:43 +01:00
Richard van der Hoff
aedaba018f Replace some instances of preserve_context_over_deferred 2017-05-09 19:04:56 +01:00
Richard van der Hoff
de042b3b88 Do some logging when one-time-keys get claimed 
might help us figure out if https://github.com/vector-im/riot-web/issues/3868
has happened.
 2017-05-09 19:04:56 +01:00
Richard van der Hoff
a7e9d8762d Allow clients to upload one-time-keys with new sigs 
When a client retries a key upload, don't give an error if the signature has
changed (but the key is the same).

Fixes https://github.com/vector-im/riot-android/issues/1208, hopefully.
 2017-05-09 19:04:56 +01:00
Erik Johnston
27c28eaa27 Merge pull request #2190 from matrix-org/erikj/mark_remote_as_back_more 
Always mark remotes as up if we receive a signed request from them
 2017-05-05 14:08:12 +01:00
Erik Johnston
653d90c1a5 Comment 2017-05-05 14:01:17 +01:00
Erik Johnston
7b222fc56e Remove redundant reset of destination timers 2017-05-05 11:14:09 +01:00
Erik Johnston
b843631d71 Add comment and TODO 2017-05-05 10:59:32 +01:00
Erik Johnston
f346048a6e Handle exceptions thrown in handling remote device list updates 2017-05-05 10:34:10 +01:00
David Baker
60833c8978 Merge pull request #2147 from matrix-org/dbkr/http_request_propagate_error 
Propagate errors sensibly from proxied IS requests
 2017-05-03 11:23:25 +01:00
Erik Johnston
0c27383dd7 Merge pull request #2170 from matrix-org/erikj/fed_hole_state 
Don't fetch state for missing events that we fetched
 2017-05-03 10:49:37 +01:00
Erik Johnston
ef862186dd Merge together redundant calculations/logging 2017-05-03 10:06:43 +01:00
Erik Johnston
3e5a62ecd8 Add more granular event send metrics 2017-05-02 14:23:26 +01:00
Erik Johnston
2347efc065 Fixup 2017-04-28 12:46:53 +01:00
Erik Johnston
9b147cd730 Remove unncessary call in _get_missing_events_for_pdu 2017-04-28 11:55:25 +01:00
Erik Johnston
3a9f5bf6dd Don't fetch state for missing events that we fetched 2017-04-28 11:26:46 +01:00
Erik Johnston
25a96e0c63 Merge pull request #2163 from matrix-org/erikj/fix_invite_state 
Fix invite state to always include all events
 2017-04-27 17:36:30 +01:00
Erik Johnston
87ae59f5e9 Typo 2017-04-27 15:16:21 +01:00
Erik Johnston
e42b4ebf0f Add some extra logging for edge cases of federation 2017-04-27 14:38:21 +01:00
Erik Johnston
34e682d385 Fix invite state to always include all events 2017-04-26 16:18:08 +01:00
David Baker
82ae0238f9 Revert accidental commit 2017-04-26 11:43:16 +01:00
David Baker
81804909d3 Merge remote-tracking branch 'origin/develop' into dbkr/http_request_propagate_error 2017-04-26 11:31:55 +01:00
David Baker
1a9255c12e Use CodeMessageException subclass instead 
Parse json errors from get_json client methods and throw special
errors.
 2017-04-25 19:30:55 +01:00
David Baker
2e165295b7 Merge remote-tracking branch 'origin/develop' into dbkr/http_request_propagate_error 2017-04-21 11:35:52 +01:00
David Baker
a90a0f5c8a Propagate errors sensibly from proxied IS requests 
When we're proxying Matrix endpoints, parse out Matrix error
responses and turn them into SynapseErrors so they can be
propagated sensibly upstream.
 2017-04-21 11:32:48 +01:00
Richard van der Hoff
0cdb32fc43 Remove redundant try/except clauses 
The `except SynapseError` clauses were pointless because the wrapped functions
would never throw a `SynapseError` (they either throw a `CodeMessageException`
or a `RuntimeError`).

The `except CodeMessageException` is now also pointless because the caller
treats all exceptions equally, so we may as well just throw the
`CodeMessageException`.
 2017-04-21 01:32:01 +01:00
Richard van der Hoff
838810b76a Broaden the conditions for locally_rejecting invites 
The logic for marking invites as locally rejected was all well and good, but
didn't happen when the remote server returned a 500, or wasn't reachable, or
had no DNS, or whatever.

Just expand the except clause to catch everything.

Fixes https://github.com/matrix-org/synapse/issues/761.
 2017-04-21 01:31:37 +01:00
Richard van der Hoff
736b9a4784 Remove redundant function 
inline `reject_remote_invite`, which only existed to make tracing the callflow
more difficult.
 2017-04-21 01:31:09 +01:00
Luke Barnard
3fb8784c92 m.read_marker -> m.fully_read (#2128) 
Also:
 - change the REST endpoint to have a "S" on the end (so it's now /read_markers)
 - change the content of the m.read_up_to event to have the key "event_id" instead of "marker".
 2017-04-18 17:46:15 +01:00
Luke Barnard
78f0ddbfad Merge pull request #2120 from matrix-org/luke/read-markers 
Implement Read Marker API
 2017-04-13 14:21:31 +01:00
Luke Barnard
b9557064bf Simplify is_event_after logic 2017-04-12 14:36:20 +01:00
Erik Johnston
247c736b9b Merge pull request #2115 from matrix-org/erikj/dedupe_federation_repl 
Reduce federation replication traffic
 2017-04-12 11:07:13 +01:00
Luke Barnard
69a18514e9 Only notify user, not entire room 2017-04-12 10:50:37 +01:00
Luke Barnard
122cd52ce4 Remove comment, simplify null-guard 2017-04-12 10:48:32 +01:00
Erik Johnston
c7ddb5ef7a Reuse get_interested_parties 2017-04-12 10:16:26 +01:00
Luke Barnard
77fb2b72ae Handle no previous RM 2017-04-12 09:47:29 +01:00
Luke Barnard
867822fa1e flake8 2017-04-11 17:36:04 +01:00
Luke Barnard
73880268ef Refactor event ordering check to events store 2017-04-11 17:34:09 +01:00
Luke Barnard
131485ef66 Copyright 2017-04-11 17:33:51 +01:00
Luke Barnard
0127423027 flake8 2017-04-11 17:07:07 +01:00
Erik Johnston
414522aed5 Move get_interested_parties 2017-04-11 15:33:26 +01:00
Erik Johnston
2be8a281d2 Comments 2017-04-11 15:28:24 +01:00
Erik Johnston
6308ac45b0 Move get_interested_remotes back to presence handler 2017-04-11 15:19:26 +01:00
Luke Barnard
d892079844 Finish implementing RM endpoint 
- This change causes a 405 to be sent if "m.read_marker" is set via /account_data
 - This also fixes-up the RM endpoint so that it actually Works.
 2017-04-11 15:01:39 +01:00
lukebarnard
e263c26690 Initial commit of RM server-side impl 
(See https://docs.google.com/document/d/1UWqdS-e1sdwkLDUY0wA4gZyIkRp-ekjsLZ8k6g_Zvso/edit#heading=h.lndohpg8at5u)
 2017-04-11 11:55:30 +01:00
Erik Johnston
f3cf3ff8b6 Merge branch 'master' of github.com:matrix-org/synapse into develop 2017-04-11 11:13:32 +01:00
Erik Johnston
29574fd5b3 Reduce federation presence replication traffic 
This is mainly done by moving the calculation of where to send presence
updates from the presence handler to the transaction queue, so we only
need to send the presence event (and not the destinations) across the
replication connection. Before we were duplicating by sending the full
state across once per destination.
 2017-04-10 16:48:30 +01:00
Richard van der Hoff
64765e5199 When we do an invite rejection, save the signed leave event to the db 
During a rejection of an invite received over federation, we ask a remote
server to make us a `leave` event, then sign it, then send that with
`send_leave`.

We were saving the *unsigned* version of the event (which has a different event
id to the signed version) to our db (and sending it to the clients), whereas
other servers in the room will have seen the *signed* version. We're not aware
of any actual problems that caused, except that it makes the database confusing
to look at and generally leaves the room in a weird state.
 2017-04-07 14:39:32 +01:00
Erik Johnston
98ce212093 Merge pull request #2103 from matrix-org/erikj/no-double-encode 
Don't double encode replication data
 2017-04-07 09:39:52 +01:00
Erik Johnston
877c029c16 Use iteritems 2017-04-06 15:51:22 +01:00
Erik Johnston
96b9b6c127 Don't double json encode typing replication data 2017-04-05 11:34:20 +01:00
Erik Johnston
62b89daac6 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/repl_tcp_server 2017-04-04 09:46:16 +01:00
Richard van der Hoff
773e64cc1a Merge pull request #2095 from matrix-org/rav/cull_log_preserves 
Cull spurious PreserveLoggingContexts
 2017-04-03 17:02:25 +01:00
Richard van der Hoff
30bcbf775a Accept join events from all servers 
Make sure that we accept join events from any server, rather than just the
origin server, to make the federation join dance work correctly.

(Fixes #1893).
 2017-04-03 15:58:07 +01:00
Richard van der Hoff
7eb9f34cc3 Remove spurious yield 
In `MessageHandler`, remove `yield` on call to `Notifier.on_new_room_event`:
it doesn't return anything anyway.
 2017-04-03 15:44:19 +01:00
Richard van der Hoff
0b08c48fc5 Remove more spurious PreserveLoggingContexts 
Remove `PreserveLoggingContext` around calls to `Notifier.on_new_room_event`;
there is no problem if the logcontext is set when calling it.
 2017-04-03 15:43:37 +01:00
Erik Johnston
36d2b66f90 Add a timestamp to USER_SYNC command 
This timestamp is used to indicate when the user last sync'd
 2017-03-31 15:42:22 +01:00
Erik Johnston
9d0170ac6c Fix up presence 2017-03-31 11:36:32 +01:00
Erik Johnston
63fcc42990 Remove user from process_presence when stops syncing 2017-03-30 14:26:08 +01:00
Erik Johnston
e9dd8370b0 Add functions to presence to support remote syncs 
The TCP replication protocol streams deltas of who has started or
stopped syncing. This is different from the HTTP API which periodically
sends the full list of users who are syncing. This commit adds support
for the new TCP style of sending deltas.
 2017-03-30 13:25:14 +01:00
Erik Johnston
24d35ab47b Add new storage functions for new replication 
The new replication protocol will keep all the streams separate, rather
than muxing multiple streams into one.
 2017-03-30 11:48:35 +01:00
Erik Johnston
7fc1f1e2b6 Cache hosts in room 2017-03-24 11:46:24 +00:00
Richard van der Hoff
9397edb28b Merge pull request #2050 from matrix-org/rav/federation_backoff 
push federation retry limiter down to matrixfederationclient
 2017-03-23 22:27:01 +00:00
Richard van der Hoff
5a16cb4bf0 Ignore backoff history for invites, aliases, and roomdirs 
Add a param to the federation client which lets us ignore historical backoff
data for federation queries, and set it for a handful of operations.
 2017-03-23 12:23:22 +00:00
Richard van der Hoff
4bd597d9fc push federation retry limiter down to matrixfederationclient 
rather than having to instrument everywhere we make a federation call,
make the MatrixFederationHttpClient manage the retry limiter.
 2017-03-23 09:28:46 +00:00
Richard van der Hoff
e08f81d96a Add a missing yield in device key upload 
(this would only very very rarely actually be a useful thing, so the main
problem was the logcontext leak...)
 2017-03-23 00:16:43 +00:00
Richard van der Hoff
eddce9d74a Merge pull request #2027 from matrix-org/rav/logcontext_leaks 
A few fixes to logcontext things
 2017-03-20 11:53:36 +00:00
Richard van der Hoff
f40c2db05a Stop preserve_fn leaking context into the reactor 
Fix a bug in ``logcontext.preserve_fn`` which made it leak context into the
reactor, and add a test for it.

Also, get rid of ``logcontext.reset_context_after_deferred``, which tried to do
the same thing but had its own, different, set of bugs.
 2017-03-18 00:07:43 +00:00
Richard van der Hoff
5068fb16a5 Refactoring and cleanups 
A few non-functional changes:

* A bunch of docstrings to document types
* Split `EventsStore._persist_events_txn` up a bit. Hopefully it's a bit more
  readable.
* Rephrase `EventFederationStore._update_min_depth_for_room_txn` to avoid
  mind-bending conditional.
* Rephrase rejected/outlier conditional in `_update_outliers_txn` to avoid
  mind-bending conditional.
 2017-03-17 15:06:07 +00:00
Richard van der Hoff
2abe85d50e Merge pull request #2016 from matrix-org/rav/queue_pdus_during_join 
Queue up federation PDUs while a room join is in progress
 2017-03-17 11:32:44 +00:00
Erik Johnston
da146657c9 Comments 2017-03-16 13:04:07 +00:00
Erik Johnston
a158c36a8a Comment 2017-03-16 11:57:45 +00:00
Erik Johnston
6957bfdca6 Don't recreate so many sets 2017-03-16 11:54:26 +00:00
Erik Johnston
2ccf3b241c Implement no op for room stream in sync 2017-03-16 11:06:41 +00:00
Richard van der Hoff
9ce53a3861 Queue up federation PDUs while a room join is in progress 
This just takes the existing `room_queues` logic and moves it out to
`on_receive_pdu` instead of `_process_received_pdu`, which ensures that we
don't start trying to fetch prev_events and whathaveyou until the join has
completed.
 2017-03-15 18:01:11 +00:00
Erik Johnston
54d2b7e596 Merge pull request #2014 from Half-Shot/hs/fix-appservice-presence 
Add fallback to last_active_ts if it beats the last sync time on a presence timeout.
 2017-03-15 17:37:15 +00:00
Erik Johnston
f83ac78201 Cache set of users whose presence the other user should see 2017-03-15 15:29:19 +00:00
Will Hunt
e6032054bf Add a great comment to handle_timeout for active vs sync times. 2017-03-15 15:24:48 +00:00
Will Hunt
ebf5a6b14c Add fallback to last_active_ts if it beats the last sync time. 2017-03-15 15:17:16 +00:00
Erik Johnston
e892457a03 Comment 2017-03-15 15:01:39 +00:00
Erik Johnston
6c82de5100 Format presence events on the edges instead of reformatting them multiple times 2017-03-15 14:27:34 +00:00
Erik Johnston
7827251daf Merge pull request #1994 from matrix-org/dbkr/msisdn_signin_2 
Phone number registration / login support v2
 2017-03-15 09:59:54 +00:00
Erik Johnston
bb256ac96f Merge branch 'develop' of github.com:matrix-org/synapse into erikj/public_list_speed 2017-03-14 11:35:05 +00:00
Luke Barnard
bbeeb97f75 Implement _simple_delete_many_txn, use it to delete devices 
(But this doesn't implement the same for deleting access tokens or e2e keys.

Also respond to code review.
 2017-03-13 17:53:23 +00:00
David Baker
73a5f06652 Support registration / login with phone number 
Changes from https://github.com/matrix-org/synapse/pull/1971
 2017-03-13 17:27:51 +00:00
Richard van der Hoff
3c69f32402 Merge remote-tracking branch 'origin/develop' into rav/refactor_received_pdu 2017-03-13 12:20:47 +00:00
Erik Johnston
0162994983 Comments 2017-03-13 11:53:26 +00:00
Erik Johnston
7eae6eaa2f Revert "Support registration & login with phone number" 2017-03-13 09:59:33 +00:00
Erik Johnston
79926e016e Assume rooms likely haven't changed 2017-03-13 09:50:10 +00:00
Erik Johnston
8ffbe43ba1 Get current state by using current_state_events table 2017-03-10 17:39:35 +00:00