Matrix-Mirrors/synapse-product
1
0
Fork 0
mirror of https://git.anonymousland.org/anonymousland/synapse-product.git synced 2024-12-25 09:49:27 -05:00
Code Issues Packages Projects Releases Wiki Activity
9,597 Commits 4 Branches 609 Tags 60 MiB
f9b255cd62
Commit Graph

6945 Commits

Author SHA1 Message Date
Richard van der Hoff
e43de3ae4b Improve logging of failures in matrixfederationclient 
* don't log exception types twice
* not all exceptions have a meaningful 'message'. Use the repr rather than
  attempting to build a string ourselves.
 2017-09-28 15:38:09 +01:00
Richard van der Hoff
75e67b9ee4 Handle SERVFAILs when doing AAAA lookups for federation (#2477) 
... to cope with people with broken dnssec setups, mostly
 2017-09-28 15:24:00 +01:00
Erik Johnston
768f00dedb Up the limits on number of url cache entries to delete at one time 2017-09-28 14:27:27 +01:00
Erik Johnston
4dc07e93a8 Add old indices 2017-09-28 14:10:33 +01:00
Erik Johnston
7cc483aa0e Clear up expired url cache every 10s 2017-09-28 13:56:53 +01:00
Erik Johnston
e1e7d76cf1 Actually assign result to variable 2017-09-28 13:55:29 +01:00
Erik Johnston
93247a424a Only pull out local media that were for url cache 2017-09-28 13:48:14 +01:00
Erik Johnston
5f501ec7e2 Fix typo in url cache expiry timer 2017-09-28 12:59:01 +01:00
Erik Johnston
761d255fdf Merge pull request #2479 from matrix-org/erikj/expire_url_cache_thumbnails 
Support new and old style media id formats
 2017-09-28 12:58:13 +01:00
Erik Johnston
ace8079086 Support new and old style media id formats 2017-09-28 12:52:51 +01:00
Erik Johnston
7a44c01d89 Fix typo 2017-09-28 12:46:04 +01:00
Erik Johnston
ae79764fe5 Change expires column to expires_ts 2017-09-28 12:37:53 +01:00
Erik Johnston
77f1d24de3 More brackets 2017-09-28 12:23:15 +01:00
Erik Johnston
9ccb4226ba Delete expired url cache data 2017-09-28 12:18:06 +01:00
Erik Johnston
8090fd4664 Fix /joined_members to work with AS users 2017-09-28 10:09:32 +01:00
Erik Johnston
adec03395d Fix bug where /joined_members didn't check user was in room 2017-09-27 15:14:39 +01:00
David Baker
ef3a5ae787 Don't test is spam_checker not None 
Sometimes it's a Mock object which is not none but is still not
what we're after
 2017-09-27 11:24:19 +01:00
David Baker
8c06dd6071 Remove unintentional debugging 2017-09-27 10:31:14 +01:00
David Baker
60c78666ab pep8 2017-09-27 10:26:13 +01:00
David Baker
1786b0e768 Forgot the new file again :( 2017-09-27 10:22:54 +01:00
David Baker
6cd5fcd536 Make the spam checker a module 2017-09-26 19:20:23 +01:00
David Baker
9fd086e506 unnecessary parens 2017-09-26 17:59:46 +01:00
David Baker
0b03a97708 Add module_loader.py 2017-09-26 17:56:41 +01:00
David Baker
4824a33c31 Factor out module loading to a separate place 
So it can be reused
 2017-09-26 17:51:26 +01:00
Erik Johnston
17b8e2bd02 Add remove room API 2017-09-26 15:52:41 +01:00
Erik Johnston
a8e2a3df32 Add unique index to group_rooms table 2017-09-26 15:39:21 +01:00
Erik Johnston
95298783bb Add is_publicised to group summary 2017-09-26 11:04:37 +01:00
Erik Johnston
1a398b19fd Merge branch 'develop' of github.com:matrix-org/synapse into release-v0.23.0 2017-09-26 10:08:59 +01:00
Erik Johnston
f4c8cd5e85 Bump changelog and version 2017-09-26 10:02:48 +01:00
Erik Johnston
e3edca3b5d Refactor to speed up incremental syncs 2017-09-25 17:35:39 +01:00
Erik Johnston
44edac0497 Merge branch 'release-v0.23.0' of github.com:matrix-org/synapse into develop 2017-09-25 14:52:46 +01:00
Richard van der Hoff
94133d7ce8 Merge branch 'develop' into develop 2017-09-25 11:50:11 +01:00
Erik Johnston
ba8fdc925c Bump version and changes 2017-09-25 11:01:31 +01:00
Richard van der Hoff
79b3cf3e02 Fix logcontxt leak in keyclient (#2465) 
preserve_context_over_function doesn't do what you want it to do.
 2017-09-25 09:51:39 +01:00
Richard van der Hoff
f65e31d22f Do an AAAA lookup on SRV record targets (#2462) 
Support SRV records which point at AAAA records, as well as A records.

Fixes https://github.com/matrix-org/synapse/issues/2405
 2017-09-22 20:26:47 +01:00
Matthew Hodgson
f496399ac4 fix thinko'd docstring 2017-09-22 15:34:14 +01:00
Erik Johnston
3166ed55b2 Fix device list when rejoining room (#2461) 2017-09-22 14:44:17 +01:00
Erik Johnston
e1dec2f1a7 Remove user from group summary when the leave the group 2017-09-21 16:09:57 +01:00
Erik Johnston
bb746a9de1 Revert: Keep room_id's in group summary 2017-09-21 15:57:22 +01:00
Erik Johnston
ae8d4bb0f0 Keep room_id's in group summary 2017-09-21 15:55:18 +01:00
Erik Johnston
197d82dc07 Correctly return next token 2017-09-20 11:12:11 +01:00
Erik Johnston
069ae2df12 Fix initial sync 2017-09-20 10:52:12 +01:00
Richard van der Hoff
6de74ea6d7 Fix logcontexts in _check_sigs_and_hashes 2017-09-20 01:32:42 +01:00
Richard van der Hoff
c5c24c239b Fix logcontext handling in verify_json_objects_for_server 
preserve_context_over_fn is essentially broken, because (a) it pointlessly
drops the current logcontext before calling its wrapped function, which means
we don't get any useful logcontexts for _handle_key_deferred; (b) it wraps the
resulting deferred in a _PreservingContextDeferred, which is very dangerous
because you then can't yield on it without leaking context back into the
reactor.

Instead, let's specify that the resultant deferreds call their callbacks with
no logcontext.
 2017-09-20 01:32:42 +01:00
Richard van der Hoff
c5b0e9f485 Turn _start_key_lookups into an inlineCallbacks function 
... which means that logcontexts can be correctly preserved for the stuff it
does.

get_server_verify_keys is now called with the logcontext, so needs to
preserve_fn when it fires off its nested inlineCallbacks function.

Also renames get_server_verify_keys to reflect the fact it's meant to be
private.
 2017-09-20 01:32:42 +01:00
Richard van der Hoff
abdefb8a01 Fix potential race in _start_key_lookups 
If the verify_request.deferred has already completed, then `remove_deferreds`
will be called immediately. It therefore might resolve the server_to_deferred
deferred while there are still other requests for that server in flight.

To avoid that, we should build the complete list of requests, and *then* add the
callbacks.
 2017-09-20 01:32:42 +01:00
Richard van der Hoff
afbd773dc6 Add some comments to _start_key_lookups 2017-09-20 01:32:42 +01:00
Richard van der Hoff
2a4b9ea233 Consistency for how verify_request.deferred is called 
Define that it is run with no log context, and make sure that happens.

If we aren't careful to reset the logcontext, we can't bung the deferreds into
defer.gatherResults etc. We don't actually do that directly, but we *do*
resolve other deferreds from affected callbacks (notably the server_to_deferred
map in _start_key_lookups), and those *do* get passed into
defer.gatherResults. It turns out that this way ends up being least confusing.
 2017-09-20 01:32:42 +01:00
Richard van der Hoff
3b98439eca Factor out _start_key_lookups 
... to make it easier to see what's going on.
 2017-09-20 01:32:42 +01:00
Richard van der Hoff
fde63b880d Replace server_and_json with verify_requests 
This is a precursor to factoring some of this code out.
 2017-09-20 01:32:42 +01:00
Richard van der Hoff
2d511defd9 pull out handle_key_deferred to top level 
There's no need for this to be a nested definition; pulling it out not only
makes it more efficient, but makes it easier to check that it's not accessing
any local variables it shouldn't be.
 2017-09-20 01:32:42 +01:00
Richard van der Hoff
dd1ea9763a Fix incorrect key_ids in error message 2017-09-20 01:32:42 +01:00
Richard van der Hoff
e76d1135dd Invalidate signing key cache when we gat an update 
This might make the cache slightly more efficient.
 2017-09-20 01:32:42 +01:00
Richard van der Hoff
fcf2c0fd1a Remove redundant preserve_fn 
preserve_fn is a no-op unless the wrapped function returns a
Deferred. verify_json_objects_for_server returns a list, so this is doing
nothing.
 2017-09-20 01:32:42 +01:00
Richard van der Hoff
9864efa532 Fix concurrent server_key requests (#2458) 
Fix a bug where we could end up firing off multiple requests for server_keys
for the same server at the same time.
 2017-09-19 23:25:44 +01:00
Richard van der Hoff
aa620d09a0 Add a config option to block all room invites (#2457) 
- allows sysadmins the ability to lock down their servers so that people can't
send their users room invites.
 2017-09-19 16:08:14 +01:00
Richard van der Hoff
2eabdf3f98 add some comments to on_exchange_third_party_invite_request 2017-09-19 12:20:36 +01:00
Richard van der Hoff
5ed109d59f PoC for filtering spammy events (#2456) 
Demonstration of how you might add some hooks to filter out spammy events.
 2017-09-19 12:20:11 +01:00
Erik Johnston
93e504d04e Ensure that creator of group sees group down /sync 2017-09-19 11:08:16 +01:00
Erik Johnston
b5feaa5a49 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/groups_merged 2017-09-19 11:07:45 +01:00
Richard van der Hoff
3f405b34e9 Fix overzealous kicking of guest users (#2453) 
We should only kick guest users if the guest access event is authorised.
 2017-09-19 08:52:52 +01:00
Richard van der Hoff
290777b3d9 Clean up and document handling of logcontexts in Keyring (#2452) 
I'm still unclear on what the intended behaviour for
`verify_json_objects_for_server` is, but at least I now understand the
behaviour of most of the things it calls...
 2017-09-18 18:31:01 +01:00
Erik Johnston
77c81ca6ea Merge pull request #2451 from matrix-org/erikj/add_state_to_timeline 
Don't filter out current state events from timeline
 2017-09-18 17:22:33 +01:00
Erik Johnston
2d1b7955ae Don't filter out current state events from timeline 2017-09-18 17:13:03 +01:00
David Baker
862c8da560 Merge pull request #2450 from matrix-org/dbkr/push_event_id_only 
Add support for event_id_only push format
 2017-09-18 16:41:29 +01:00
David Baker
436ee0a2ea Also include the room_id 
as really it's part of the event ID
 2017-09-18 15:58:38 +01:00
David Baker
b393f5db51 Use .get - it's much shorter 2017-09-18 15:50:26 +01:00
David Baker
a2562f9d74 Add support for event_id_only push format 
Param in the data dict of a pusher that tells an HTTP pusher to
send just the event_id of the event it's notifying about and the
notification counts. For clients that want to go & fetch the body
of the event themselves anyway.
 2017-09-18 15:39:39 +01:00
Erik Johnston
d6dadd95ac Correctly handle leaving room in /key/changes 2017-09-18 15:38:22 +01:00
Erik Johnston
4a94eb3ea4 Fix typo 2017-09-15 09:56:54 +01:00
Erik Johnston
3a0cee28d6 Actually hook leave notifs up 2017-09-14 11:49:37 +01:00
Erik Johnston
4f845a0713 Handle joining/leaving rooms in /keys/changes 2017-09-13 16:28:08 +01:00
Erik Johnston
473700f016 Get left rooms 2017-09-13 15:13:41 +01:00
Erik Johnston
9ce866ed4f In sync handle device lists for newly joined/left rooms 2017-09-12 16:44:26 +01:00
Erik Johnston
69ef4987a6 Add left section to /keys/changes 2017-09-08 14:44:36 +01:00
Erik Johnston
53cc8ad35a Send down device list change notif when member leaves/rejoins room 2017-09-07 15:08:39 +01:00
Erik Johnston
4a9b1cf253 Add user profiles to summary from group server 2017-08-25 16:23:58 +01:00
Erik Johnston
258409ef61 Fix typos and reinherit 2017-08-25 14:45:20 +01:00
Erik Johnston
bf81f3cf2c Split out profile handler to fix tests 2017-08-25 14:34:56 +01:00
Erik Johnston
27ebc5c8f2 Add remote profile cache 2017-08-25 11:25:47 +01:00
Erik Johnston
97c544f91f Add _simple_update 2017-08-25 11:11:37 +01:00
Richard van der Hoff
a7effa8400 Merge pull request #2288 from kyrias/bcrypt 
python_dependencies: Use bcrypt module instead of py-bcrypt
 2017-08-23 14:14:56 +01:00
Erik Johnston
2800983f3e Merge pull request #2410 from matrix-org/erikj/groups_publicise 
Add ability to publicise group membership
 2017-08-21 16:51:56 +01:00
Erik Johnston
8b50fe5330 Use BOOLEAN rather than TEXT type 2017-08-21 16:37:29 +01:00
Erik Johnston
175a01f56c Groups: Fix mising json.load in initial sync 2017-08-21 14:45:56 +01:00
Richard van der Hoff
692250c6be Fix user_dir startup 
Add missing parameter to _base.start_worker_reactor
 2017-08-16 15:11:29 +01:00
Richard van der Hoff
d2352347cf Fix process startup 
escape the % that got added in 92168cb so that the process starts up ok.
 2017-08-16 14:57:35 +01:00
Matthew Hodgson
92168cbbc5 explain why CPU affinity is a good idea 2017-08-15 18:27:42 +01:00
Richard van der Hoff
10d8b701a1 Allow configuration of CPU affinity 
Make it possible to set the CPU affinity in the config file, so that we don't
need to remember to do it manually every time.
 2017-08-15 17:08:28 +01:00
Richard van der Hoff
543c794a76 Factor out common application start 
We have 10 copies of this code, and I don't really want to update each one
separately.
 2017-08-15 17:04:40 +01:00
Erik Johnston
ba3ff7918b Fixup 2017-08-11 13:42:42 +01:00
Erik Johnston
ef8e578677 Add bulk group publicised lookup API 2017-08-09 13:36:22 +01:00
Erik Johnston
b880ff190a Allow update group publicity 2017-08-08 14:19:41 +01:00
Erik Johnston
05e21285aa Store whether the user wants to publicise their membership of a group 2017-08-08 13:01:46 +01:00
hera
eae04f1952 fix english 2017-08-04 23:56:42 +01:00
hera
5699b05072 typo 2017-08-04 23:44:37 +01:00
Erik Johnston
a1e67bcb97 Remove stale TODO comments 2017-08-04 10:07:10 +01:00
Erik Johnston
09552f9d9c Reduce spammy log line in synchrotrons 2017-08-02 17:29:51 +01:00
Kenny Keslar
f18373dc5d Fix iteration of requests_missing_keys; list doesn't have .values() 
Signed-off-by: Kenny Keslar <r3dey3@r3dey3.com>
 2017-07-26 22:44:19 -05:00
Erik Johnston
966a70f1fa Update comment 2017-07-24 17:49:39 +01:00
Erik Johnston
629cdfb124 Use join rather than joined, etc. 2017-07-24 14:54:05 +01:00
Erik Johnston
ed666d3969 Fix all the typos 2017-07-24 14:05:09 +01:00
Erik Johnston
b76ef6ccb8 Include users membership in group in summary API 2017-07-24 13:55:39 +01:00
Erik Johnston
851aeae7c7 Check users/rooms are in group before adding to summary 2017-07-24 13:40:56 +01:00
Erik Johnston
d5e32c843f Correctly add joins to correct segment 2017-07-24 13:31:26 +01:00
Erik Johnston
96917d5552 Merge pull request #2378 from matrix-org/erikj/group_sync_support 
Add groups to sync stream
 2017-07-21 11:05:39 +01:00
Erik Johnston
0401604222 Merge pull request #2377 from matrix-org/erikj/group_profile_update 
Add update group profile API
 2017-07-20 17:53:39 +01:00
Erik Johnston
b238cf7f6b Remove spurious content param 2017-07-20 17:49:55 +01:00
Erik Johnston
960dae3340 Add notifier 2017-07-20 17:14:44 +01:00
Erik Johnston
2cc998fed8 Fix replication. And notify 2017-07-20 17:13:18 +01:00
Erik Johnston
139fe30f47 Remember to cast to bool 2017-07-20 16:47:35 +01:00
Erik Johnston
4d793626ff Fix bug in generating current token 2017-07-20 16:42:44 +01:00
Erik Johnston
c544188ee3 Add groups to sync stream 2017-07-20 16:36:42 +01:00
Erik Johnston
0ab153d201 Check values are strings 2017-07-20 16:24:18 +01:00
Erik Johnston
8209b5f033 Fix a storage desc 2017-07-20 16:22:22 +01:00
Erik Johnston
57826d645b Fix typo 2017-07-20 13:15:22 +01:00
Erik Johnston
6f443a74cf Add update group profile API 2017-07-20 09:46:33 +01:00
Erik Johnston
14a34f12d7 Comments 2017-07-18 17:28:42 +01:00
Erik Johnston
3431ec55dc Comments 2017-07-18 17:23:50 +01:00
Erik Johnston
6027b1992f Fix permissions 2017-07-18 16:51:25 +01:00
Erik Johnston
e884ff31d8 Add DELETE 2017-07-18 16:41:44 +01:00
Erik Johnston
05c13f6c22 Add 'args' param to post_json 2017-07-18 16:40:21 +01:00
Erik Johnston
94ecd871a0 Fix typos 2017-07-18 16:38:54 +01:00
Erik Johnston
12ed4ee48e Correctly parse query params 2017-07-18 15:33:09 +01:00
Erik Johnston
332839f6ea Update federation client pokes 2017-07-18 14:45:37 +01:00
Erik Johnston
e5ea6dd021 Add client apis 2017-07-18 14:37:06 +01:00
Erik Johnston
cccfcfa7b9 Comments 2017-07-18 10:35:18 +01:00
Erik Johnston
68f34e85ce Use transport client directly 2017-07-18 10:29:57 +01:00
Erik Johnston
3e703eb04e Comment 2017-07-18 10:17:25 +01:00
Erik Johnston
508460f240 Remove sync stuff 2017-07-18 09:55:46 +01:00
Erik Johnston
6e9f147faa Add GroupID type 2017-07-18 09:47:25 +01:00
Erik Johnston
4540730111 Remove unused tables 2017-07-18 09:38:15 +01:00
Erik Johnston
e96ee95a7e Remove sync stuff 2017-07-18 09:38:08 +01:00
Erik Johnston
2f9eafdd36 Add local group server support 2017-07-17 12:03:49 +01:00
Erik Johnston
bfde076022 Increase cache hit ratio for push 
We don't update the cache in all code paths, which causes subsequent
calls to miss the cache
 2017-07-14 16:11:26 +01:00
Erik Johnston
cb3aee8219 Ensure category and role ids are non-null 2017-07-14 14:06:55 +01:00
Erik Johnston
85fda57208 Add DEFAULT_ROLE_ID 2017-07-14 14:03:54 +01:00
Erik Johnston
4b203bdba5 Correctly increment orders 2017-07-14 14:02:31 +01:00
Erik Johnston
8d26385d76 Add more metrics to push rule evaluation 2017-07-13 14:37:30 +01:00
Erik Johnston
3b0470dba5 Remove unused functions 2017-07-13 13:54:01 +01:00
Erik Johnston
8575e3160f Comments 2017-07-13 13:52:41 +01:00
Erik Johnston
f60218ec41 Push: Don't acquire lock unless necessary 2017-07-13 11:23:53 +01:00
Erik Johnston
a78cda4baf Remove TODO 2017-07-13 11:17:07 +01:00
Erik Johnston
7a39da8cc6 Add summary APIs to federation 2017-07-13 11:13:19 +01:00
Erik Johnston
5bbb53580a raise NotImplementedError 2017-07-13 10:25:29 +01:00
Erik Johnston
26451a09eb Comments 2017-07-12 14:47:18 +01:00
Erik Johnston
8d55877c9e Simplify checking if admin 2017-07-12 11:43:39 +01:00
Erik Johnston
a62406aaa5 Add group summary APIs 2017-07-12 11:36:15 +01:00
Erik Johnston
e9aec001f4 Use less DB for device list handling in sync 2017-07-12 10:30:10 +01:00
Erik Johnston
28e8c46f29 Merge pull request #2352 from matrix-org/erikj/group_server_split 
Initial Group Server
 2017-07-12 10:26:29 +01:00
Erik Johnston
6d586dc05c Comment 2017-07-12 09:58:37 +01:00
Erik Johnston
410b4e14a1 Move comment 2017-07-11 15:44:18 +01:00
Erik Johnston
fe4e885f54 Add federation API for adding room to group 2017-07-11 14:35:07 +01:00
Erik Johnston
bbb739d24a Comment 2017-07-11 14:31:36 +01:00
Erik Johnston
26752df503 Typo 2017-07-11 14:29:03 +01:00
Erik Johnston
e52c391cd4 Rename column to attestation_json 2017-07-11 14:25:46 +01:00
Erik Johnston
0aac30d53b Comments 2017-07-11 14:23:50 +01:00
Erik Johnston
0184a97dbd Merge pull request #2354 from krombel/reduce_static_sync_reply 
encode sync-response statically
 2017-07-11 14:19:56 +01:00
Krombel
85b9f76f1d split out reducing stuff; just make encode_* static 2017-07-11 13:14:35 +02:00
Erik Johnston
6322fbbd41 Comment 2017-07-11 11:52:03 +01:00
Erik Johnston
8ba89f1050 Remove u/ requirement 2017-07-11 11:45:32 +01:00
Erik Johnston
429925a5e9 Lift out visibility parsing 2017-07-11 11:44:08 +01:00
Erik Johnston
83936293eb Comments 2017-07-11 11:42:25 +01:00
Erik Johnston
e2cb760dcc Merge pull request #2357 from matrix-org/erikj/push 
Don't compute push actions for backfilled events
 2017-07-11 10:53:22 +01:00
Erik Johnston
925b3638ff Reduce log levels in tcp replication 2017-07-11 10:04:21 +01:00
Erik Johnston
9a6fd3ef29 Don't compute push actions for backfilled events 2017-07-11 10:02:21 +01:00
Krombel
2f82de18ee fix test 2017-07-10 17:34:58 +02:00
Erik Johnston
b8ca494ee9 Initial group server implementation 2017-07-10 15:44:15 +01:00
Krombel
6e16aca8b0 encode sync-response statically; omit empty objects from sync-response 2017-07-10 16:42:17 +02:00
Erik Johnston
d4d12daed9 Include registration and as stores in frontend proxy 2017-07-07 18:36:45 +01:00
Erik Johnston
f467a8f66d Merge branch 'master' of github.com:matrix-org/synapse into develop 2017-07-07 18:26:28 +01:00
Erik Johnston
1fc4a962e4 Add a frontend proxy 2017-07-07 18:19:46 +01:00
Erik Johnston
f502b0dea1 Perf: Don't filter events for push 
We know the users are joined and we can explicitly check for if they are
ignoring the user, so lets do that.
 2017-07-07 14:04:40 +01:00
Erik Johnston
76ed3476d3 Bump version and changelog 2017-07-06 18:11:22 +01:00
Erik Johnston
58dc1f2c78 Merge pull request #2342 from matrix-org/erikj/pusher_pool_instantiate 
Fix bug where pusherpool didn't start and broke some rooms
 2017-07-06 18:08:43 +01:00
Erik Johnston
5a7f561a9b Fix bug where pusherpool didn't start and broke some rooms 
Since we didn't instansiate the PusherPool at start time it could fail
at run time, which it did for some users.

This may or may not fix things for those users, but it should happen at
start time and stop the server from starting.
 2017-07-06 17:55:51 +01:00
Erik Johnston
ed9a7f5436 Merge pull request #2309 from matrix-org/erikj/user_ip_repl 
Fix up user_ip replication commands
 2017-07-06 14:33:14 +01:00
Erik Johnston
1f64207f26 Merge branch 'master' of github.com:matrix-org/synapse into develop 2017-07-06 13:57:45 +01:00
Erik Johnston
6264cf9666 Bump version and changelog 2017-07-06 10:35:56 +01:00
Erik Johnston
f386632800 Merge pull request #2334 from matrix-org/erikj/refactor_transport_server 
Separate federation servlet into different lists
 2017-07-05 17:09:07 +01:00
Erik Johnston
5e49a57ecc Separate federation servlet into different lists 2017-07-05 14:32:24 +01:00
Richard van der Hoff
3d31b39297 Merge pull request #2332 from matrix-org/rav/fix_pushes 
Fix caching error in the push evaluator
 2017-07-05 11:10:53 +01:00
Richard van der Hoff
73cfe48031 Fix caching error in the push evaluator 
Initialising `result` to `{}` in the parameters meant that every call to
_flatten_dict used the *same* target dictionary.

I'm hopeful this will fix https://github.com/matrix-org/synapse/issues/2270,
but I suspect it won't. (This code seems to have been here since forever,
unlike the bug, and I don't really think it explains the observed
behaviour). Still, it makes it hard to investigate the problem.
 2017-07-05 00:28:43 +01:00
Erik Johnston
05538587ef Bump version and changelog 2017-07-04 14:02:21 +01:00
Erik Johnston
f92d7416d7 Merge pull request #2330 from matrix-org/erikj/cache_size_factor 
Increase default cache size
 2017-07-04 10:51:21 +01:00
Erik Johnston
495f075b41 Increase default cache factor size. 2017-07-04 09:58:32 +01:00
Erik Johnston
b5e8d529e6 Define CACHE_SIZE_FACTOR once 2017-07-04 09:56:44 +01:00
Mark Haines
3e279411fe Improve the error handling for bad invites received over federation 2017-06-30 16:20:30 +01:00
Erik Johnston
6ff14ddd2e Make into list 2017-06-29 15:47:37 +01:00
Erik Johnston
5946aa0877 Prefill forward extrems and event to state groups 2017-06-29 15:38:48 +01:00
Erik Johnston
2c365f4723 Cache macaroon parse and validation 
Turns out this can be quite expensive for requests, and is easily
cachable. We don't cache the lookup to the DB so invalidation still
works.
 2017-06-29 14:50:18 +01:00
Erik Johnston
c72058bcc6 Use an ExpiringCache for storing registration sessions 
This is because pruning them was a significant performance drain on
matrix.org
 2017-06-29 14:08:37 +01:00
Erik Johnston
27f26e48b7 Serialize user ip command as json 2017-06-27 16:25:38 +01:00
Erik Johnston
8c23221666 Fix up 2017-06-27 15:53:45 +01:00
Erik Johnston
731f3c37a0 Merge branch 'release-v0.22.0' of github.com:matrix-org/synapse into develop 2017-06-27 15:41:34 +01:00
Erik Johnston
4b444723f0 Merge pull request #2308 from matrix-org/erikj/user_ip_repl 
Make workers report to master for user ip updates
 2017-06-27 15:36:47 +01:00
Erik Johnston
816605a137 Merge pull request #2307 from matrix-org/erikj/user_ip_batch 
Batch upsert user ips
 2017-06-27 15:08:32 +01:00
Erik Johnston
78cefd78d6 Make workers report to master for user ip updates 2017-06-27 14:58:10 +01:00
Erik Johnston
a0a561ae85 Fix up client ips to read from pending data 2017-06-27 14:46:12 +01:00
Erik Johnston
ed3d0170d9 Batch upsert user ips 2017-06-27 13:37:04 +01:00
Erik Johnston
976128f368 Update version and changelog 2017-06-26 16:14:56 +01:00
Erik Johnston
d04d672a80 Merge pull request #2290 from matrix-org/erikj/ensure_round_trip 
Reject local events that don't round trip the DB
 2017-06-26 15:12:02 +01:00
Erik Johnston
1bce3e6b35 Remove unused variables 2017-06-26 14:03:27 +01:00
Erik Johnston
e3cbec10c1 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/ensure_round_trip 2017-06-26 14:02:44 +01:00
Erik Johnston
8abdd7b553 Fix up indices for users_who_share_rooms 2017-06-26 14:01:30 +01:00
Erik Johnston
ff13c5e7af Merge pull request #2301 from xwiki-labs/push-redact-content 
Add configuration parameter to allow redaction of content from push m…
 2017-06-24 13:13:51 +01:00
Caleb James DeLisle
27bd0b9a91 Change the config file generator to more descriptive explanation of push.redact_content 2017-06-24 10:32:12 +02:00
Caleb James DeLisle
bce144595c Fix TravisCI tests for PR #2301 - Fat finger mistake 2017-06-23 15:26:09 +02:00
Caleb James DeLisle
75eba3b07d Fix TravisCI tests for PR #2301 2017-06-23 15:15:18 +02:00
Caleb James DeLisle
1591eddaea Add configuration parameter to allow redaction of content from push messages for google/apple devices 2017-06-23 13:01:04 +02:00
Erik Johnston
7fe8ed1787 Store URL cache preview downloads seperately 
This makes it easier to clear old media out at a later date
 2017-06-23 11:14:11 +01:00
Erik Johnston
44c722931b Make some more params configurable 2017-06-22 14:59:52 +01:00
Erik Johnston
2d520a9826 Typo. ARGH. 2017-06-22 14:42:39 +01:00
Erik Johnston
24d894e2e2 Fix thinko in unhandled user spam 2017-06-22 14:39:05 +01:00
Matthew Hodgson
ccfcef6b59 Merge branch 'master' into develop 2017-06-22 13:03:44 +01:00
Erik Johnston
e0004aa28a Add desc 2017-06-22 10:03:48 +01:00
Erik Johnston
b668112320 Merge pull request #2296 from matrix-org/erikj/dont_appserver_shar 
Don't work out users who share room with appservice users
 2017-06-21 14:50:24 +01:00
Erik Johnston
dae9a00a28 Initialise exclusive_user_regex 2017-06-21 14:19:33 +01:00
Erik Johnston
8177563ebe Fix for workers 2017-06-21 13:57:49 +01:00
Krombel
4202fba82a Merge branch 'develop' into avoid_duplicate_filters 2017-06-21 14:48:21 +02:00
Krombel
812c030e87 replaced json.dumps with encode_canonical_json 2017-06-21 14:48:12 +02:00
Erik Johnston
1217c7da91 Don't work out users who share room with appservice users 2017-06-21 12:00:41 +01:00
Erik Johnston
385dcb7c60 Handle thumbnail urls 2017-06-19 17:48:28 +01:00
Erik Johnston
b8b936a6ea Add API to quarantine media 2017-06-19 17:39:21 +01:00
Erik Johnston
e5ae386ea4 Handle all cases of sending membership events 2017-06-19 16:07:54 +01:00
Erik Johnston
36e51aad3c Remove unused import 2017-06-19 14:42:21 +01:00
Erik Johnston
b490299a3b Change to create new room and join other users 2017-06-19 14:10:13 +01:00
Erik Johnston
5db7070dd1 Forget room 2017-06-19 12:40:29 +01:00
Erik Johnston
d7fe6b356c Add shutdown room API 2017-06-19 12:37:27 +01:00
Erik Johnston
fcf01dd88e Reject local events that don't round trip the DB 2017-06-19 11:33:40 +01:00
Johannes Löthberg
4f66312df8 python_dependencies: Use bcrypt module instead of py-bcrypt 
py-bcrypt has been unmaintained for a long while, while bcrypt is
actively maintained. And since ff8b87118d
we're compatible with the bcrypt anyway.

Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>
 2017-06-17 17:39:35 +02:00
Erik Johnston
dfeca6cf40 Merge pull request #2286 from matrix-org/erikj/split_out_user_dir 
Split out user directory to a separate process
 2017-06-16 13:01:19 +01:00
Erik Johnston
6aa5bc8635 Initial worker impl 2017-06-16 11:47:11 +01:00
Erik Johnston
d8f47d2efa Merge pull request #2280 from matrix-org/erikj/share_room_user_dir 
Include users who you share a room with in user directory
 2017-06-16 11:06:00 +01:00
Krombel
1ff419d343 allow Authorization header which handling got implemented in #1098 
Signed-off-by: Matthias Kesler <krombel@krombel.de>
 2017-06-16 11:21:14 +02:00
Erik Johnston
fdf1ca30f0 Bump version and changelog 2017-06-15 12:59:06 +01:00
Erik Johnston
052c5d19d5 Merge pull request #2281 from matrix-org/erikj/phone_home_stats 
Fix phone home stats
 2017-06-15 12:46:23 +01:00
Erik Johnston
5ddd199870 Typo 2017-06-15 10:49:10 +01:00
Erik Johnston
a9d6fa8b2b Include users who share room with requester in user directory 2017-06-15 10:17:21 +01:00
Erik Johnston
4564b05483 Implement updating users who share rooms on the fly 2017-06-15 10:17:17 +01:00
Erik Johnston
72613bc379 Implement initial population of users who share rooms table 2017-06-15 09:59:04 +01:00
Erik Johnston
ebcd55d641 Add DB schema for tracking users who share rooms 2017-06-15 09:45:48 +01:00
Erik Johnston
4b461a6931 Add some more stats 2017-06-15 09:39:39 +01:00
Erik Johnston
617304b2cf Fix phone home stats 2017-06-14 19:47:15 +01:00
Erik Johnston
d9fd937e39 Fix user directory insertion due to missing room_id 2017-06-13 11:50:24 +01:00
Erik Johnston
505e7e8b9d Fix up sql 2017-06-13 11:19:18 +01:00
Erik Johnston
6fd7e6db3d Fix user dir to not assume existence of user 2017-06-13 11:11:26 +01:00
Erik Johnston
fdca6e36ee Merge pull request #2274 from matrix-org/erikj/cache_is_host_joined 
Add cache for is_host_joined
 2017-06-13 10:58:53 +01:00
Erik Johnston
90ae0cffec Merge pull request #2275 from matrix-org/erikj/tweark_user_directory_search 
Tweak the ranking of PG user dir search
 2017-06-13 10:58:43 +01:00
Erik Johnston
de4cb50ca6 Merge pull request #2276 from matrix-org/erikj/fix_user_di 
Don't assume existence of events when updating user directory
 2017-06-13 10:55:41 +01:00
Erik Johnston
48d2949416 Throw exception when not retrying when downloading media 2017-06-13 10:23:14 +01:00
Erik Johnston
6ae8373d40 Don't assume existance of events when updating user directory 2017-06-13 10:19:26 +01:00
Erik Johnston
b58e24cc3c Tweak the ranking of PG user dir search 2017-06-13 10:16:31 +01:00
Erik Johnston
d53fe399eb Add cache for is_host_joined 2017-06-13 09:56:18 +01:00
Erik Johnston
a837765e8c Merge pull request #2266 from matrix-org/erikj/host_in_room 
Change is_host_joined to use current_state table
 2017-06-12 09:49:51 +01:00
Erik Johnston
8060974344 Fix replication 2017-06-09 16:40:52 +01:00
Erik Johnston
b0d975e216 Comments 2017-06-09 16:25:42 +01:00
Erik Johnston
e54d7d536e Cache state deltas 2017-06-09 16:24:00 +01:00
Erik Johnston
1e9b4d5a95 Merge pull request #2268 from matrix-org/erikj/entity_has_changed 
Fix has_any_entity_changed
 2017-06-09 15:30:55 +01:00
Erik Johnston
efc2b7db95 Rewrite conditional 2017-06-09 13:35:15 +01:00
Erik Johnston
bfd68019c2 Merge pull request #2267 from matrix-org/erikj/missing_notifier 
Fix removing of pushers when using workers
 2017-06-09 13:07:29 +01:00
Erik Johnston
1664948e41 Comment 2017-06-09 13:05:05 +01:00
Erik Johnston
935e588799 Tweak SQL 2017-06-09 13:01:23 +01:00
Erik Johnston
eed59dcc1e Fix has_any_entity_changed 
Occaisonally has_any_entity_changed would throw the error: "Set changed
size during iteration" when taking the max of the `sorteddict`. While
its uncertain how that happens, its quite inefficient to iterate over
the entire dict anyway so we change to using the more traditional
`bisect_*` functions.
 2017-06-09 11:44:01 +01:00
Erik Johnston
2cac7623a5 Add missing notifier 2017-06-09 11:24:41 +01:00
Erik Johnston
298d83b340 Fix replication 2017-06-09 11:01:28 +01:00
Erik Johnston
0185b75381 Change is_host_joined to use current_state table 
This bypasses a bug where using the state groups to figure out if a host
is in a room sometimes errors if the servers isn't in the room. (For
example when the server rejected an invite to a remote room)
 2017-06-09 10:52:26 +01:00
Erik Johnston
7132e5cdff Mark remote invite rejections as outliers 2017-06-09 10:08:18 +01:00
Erik Johnston
ea11ee09f3 Ensure we don't use unpersisted state group as prev group 2017-06-08 11:59:57 +01:00
Erik Johnston
c62c480dc6 Merge pull request #2259 from matrix-org/erikj/fix_state_woes 
Fix bug where state_group tables got corrupted
 2017-06-07 17:51:25 +01:00
Erik Johnston
197bd126f0 Fix bug where state_group tables got corrupted 
This is due to the fact that we prefilled caches using txn.call_after,
which always gets called including on error.

We fix this by making txn.call_after only fire when a transaction
completes successfully, which is what we want most of the time anyway.
 2017-06-07 17:39:36 +01:00
Erik Johnston
f45f07ab86 Merge pull request #2258 from matrix-org/erikj/user_dir 
Don't start user_directory handling on workers
 2017-06-07 14:04:50 +01:00
Erik Johnston
a053ff3979 Merge pull request #2248 from matrix-org/erikj/state_fixup 
Faster cache for get_joined_hosts
 2017-06-07 14:01:06 +01:00
Erik Johnston
ecdd2a3658 Don't start user_directory handling on workers 2017-06-07 12:02:53 +01:00
Erik Johnston
2f34ad31ac Add some logging to user directory 2017-06-07 11:50:44 +01:00
Erik Johnston
64ed74c01e When pruning, delete from device_lists_outbound_last_success 2017-06-07 11:20:47 +01:00
Erik Johnston
1a81a1898e Keep pruning background task 2017-06-07 11:16:56 +01:00
Erik Johnston
6ba21bf2b8 Comments 2017-06-07 11:08:36 +01:00
Erik Johnston
09e4bc0501 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/state_fixup 2017-06-07 11:05:23 +01:00
Erik Johnston
6e2a7ee1bc Remove spurious log lines 2017-06-07 11:05:17 +01:00
Erik Johnston
65f0513a33 Split up device_lists_outbound_pokes table for faster updates. 2017-06-07 11:02:38 +01:00
Erik Johnston
6f83c4537c Increase size of IP cache 2017-06-07 10:18:44 +01:00
Erik Johnston
cca94272fa Fix typo when getting app name 2017-06-06 11:50:07 +01:00
Erik Johnston
66b121b2fc Fix wrong number of arguments 2017-06-06 11:46:38 +01:00
Erik Johnston
1a01af079e Handle profile updates in user directory 2017-06-01 15:39:51 +01:00
Erik Johnston
4d039aa2ca Fix sqlite 2017-06-01 14:58:48 +01:00
Erik Johnston
21e255a8f1 Split the table in two 2017-06-01 14:50:46 +01:00
Erik Johnston
d5477c7afd Tweak search query 2017-06-01 13:28:01 +01:00
Erik Johnston
02a6108235 Tweak search query 2017-06-01 13:16:40 +01:00
Erik Johnston
7233341eac Comments 2017-06-01 13:11:38 +01:00
Erik Johnston
8be6fd95a3 Check if host is still in room 2017-06-01 13:05:39 +01:00
Erik Johnston
59dbb47065 Remove spurious inlineCallbacks 2017-06-01 11:41:29 +01:00
Erik Johnston
9c7db2491b Fix removing users 2017-06-01 11:36:50 +01:00
Erik Johnston
0fe6f3c521 Bug fixes and logging 
- Check if room is public when a user joins before adding to user dir
- Fix typo of field name "content.join_rules" -> "content.join_rule"
 2017-06-01 11:09:49 +01:00
Erik Johnston
036362ede6 Order by if they have profile info 2017-06-01 09:41:08 +01:00
Erik Johnston
a757dd4863 Use prefix matching 2017-06-01 09:40:37 +01:00
Erik Johnston
f5cc22bdc6 Comment on why arbitrary comments 2017-05-31 17:30:26 +01:00
Erik Johnston
5dd1b2c525 Use unique indices 2017-05-31 17:29:12 +01:00
Erik Johnston
cc7609aa9f Comment briefly on how we keep user_directory up to date 2017-05-31 17:11:18 +01:00
Erik Johnston
f1378aef91 Convert to int 2017-05-31 17:03:08 +01:00