Commit Graph

136 Commits

Author SHA1 Message Date
Neil Johnson
adfdd82b21
Back out perf regression from get_cross_signing_keys_from_cache. (#6494)
Back out cross-signing code added in Synapse 1.5.0, which caused a performance regression.
2019-12-09 13:59:27 +00:00
Erik Johnston
c16e192e2f Fix caching devices for remote servers in worker.
When the `/keys/query` API is hit on client_reader worker Synapse may
decide that it needs to resync some remote deivces. Usually this happens
on master, and then gets cached. However, that fails on workers and so
it falls back to fetching devices from remotes directly, which may in
turn fail if the remote is down.
2019-11-05 15:49:43 +00:00
Hubert Chathi
6f4bc6d01d Merge branch 'develop' into cross-signing_federation 2019-10-31 22:38:21 -04:00
Amber Brown
020add5099
Update black to 19.10b0 (#6304)
* update version of black and also fix the mypy config being overridden
2019-11-01 02:43:24 +11:00
Hubert Chathi
bc32f102cd black 2019-10-30 10:07:36 -04:00
Hubert Chathi
d78b1e339d apply changes as a result of PR review 2019-10-30 10:01:53 -04:00
Hubert Chathi
ff05c9b760 don't error if federation query doesn't have cross-signing keys 2019-10-24 21:46:11 -04:00
Hubert Chathi
480eac30eb black 2019-10-22 22:37:16 -04:00
Hubert Chathi
404e8c8532 vendor-prefix the EDU name until MSC1756 is merged into the spec 2019-10-22 22:33:23 -04:00
Hubert Chathi
1fabf82d50 update to work with newer code, and fix formatting 2019-10-22 21:44:58 -04:00
Hubert Chathi
41ad35b523 add missing param 2019-10-22 19:06:29 -04:00
Hubert Chathi
cfdb84422d make black happy 2019-10-22 19:06:06 -04:00
Hubert Chathi
8d3542a64e implement federation parts of cross-signing 2019-10-22 19:04:35 -04:00
Hubert Chathi
36adfaedab Merge branch 'develop' into cross-signing_sig_upload 2019-10-18 18:34:42 +01:00
Hubert Chathi
125eb45e19 fix doc strings 2019-10-18 16:56:16 +01:00
Hubert Chathi
cfc28325a6 Merge branch 'develop' into uhoreg/e2e_cross-signing_merged 2019-10-18 11:39:40 +01:00
Hubert Chathi
7a0dce9259 make sure we actually return something 2019-10-10 20:31:30 -04:00
Hubert Chathi
4908fb3b30 make storage layer in charge of interpreting the device key data 2019-10-10 19:15:30 -04:00
Hubert Chathi
c3635c9459 make isort happy 2019-09-24 16:21:03 -04:00
Hubert Chathi
f4b6d43ec3 add some comments 2019-09-24 16:19:54 -04:00
Hubert Chathi
39864f45ec drop some logger lines to debug 2019-09-24 15:26:45 -04:00
Hubert Chathi
26113fb7de make changes based on PR feedback 2019-09-24 14:12:20 -04:00
Hubert Chathi
d3f2fbcfe5 add function docs 2019-09-07 14:13:18 -04:00
Hubert Chathi
415d0a00e0 run black 2019-09-06 16:46:45 -04:00
Hubert Chathi
561cbba057 split out signature processing into separate functions 2019-09-06 16:44:24 -04:00
Hubert Chathi
369462da74 avoid modifying input parameter 2019-09-05 17:03:31 -04:00
Hubert Chathi
c8dc740a94 update with newer coding style 2019-09-04 22:30:45 -04:00
Hubert Chathi
7d6c70fc7a make black happy 2019-09-04 22:04:12 -04:00
Hubert Chathi
ac4746ac4b allow uploading signatures of master key signed by devices 2019-09-04 22:03:41 -04:00
Hubert Chathi
4bb4544784 implement device signature uploading/fetching 2019-09-04 20:02:56 -04:00
Hubert Chathi
e3d3fbf63f Merge branch 'uhoreg/e2e_cross-signing_merged' into cross-signing_keys 2019-08-28 17:36:46 -07:00
Hubert Chathi
96bda56370 black 2019-08-28 17:18:40 -07:00
Hubert Chathi
3b0b22cb05 use stream ID generator instead of timestamp 2019-08-28 17:17:21 -07:00
Jorik Schellekens
9a6f2be572
Opentrace e2e keys (#5855)
Add opentracing tags and logs for e2e keys
2019-08-22 11:28:12 +01:00
Hubert Chathi
814f253f1b make isort happy 2019-08-21 13:22:15 -07:00
Hubert Chathi
7c3abc6572 apply PR review suggestions 2019-08-21 13:19:35 -07:00
Hubert Chathi
fac1cdc562 make changes from PR review 2019-08-01 21:51:19 -04:00
Hubert Chathi
336c546d6a Merge branch 'cross-signing_hidden' into cross-signing_keys 2019-08-01 16:31:40 -04:00
Erik Johnston
1ec7d656dd Unwrap error 2019-07-30 13:09:02 +01:00
Erik Johnston
458e51df7a Fix error handling when fetching remote device keys 2019-07-30 13:07:02 +01:00
Jorik Schellekens
85b0bd8fe0 Update the device list cache when keys/query is called (#5693) 2019-07-29 16:34:44 +01:00
Hubert Chathi
c659b9f94f allow uploading keys for cross-signing 2019-07-25 11:08:24 -04:00
Amber Brown
4806651744
Replace returnValue with return (#5736) 2019-07-23 23:00:55 +10:00
Hubert Chathi
7b3bc755a3 remove unused and unnecessary check for FederationDeniedError (#5645)
FederationDeniedError is a subclass of SynapseError, which is a subclass of
CodeMessageException, so if e is a FederationDeniedError, then this check for
FederationDeniedError will never be reached since it will be caught by the
check for CodeMessageException above.  The check for CodeMessageException does
almost the same thing as this check (since FederationDeniedError initialises
with code=403 and msg="Federation denied with %s."), so may as well just keep
allowing it to handle this case.
2019-07-09 18:37:39 +01:00
Amber Brown
463b072b12
Move logging utilities out of the side drawer of util/ and into logging/ (#5606) 2019-07-04 00:07:04 +10:00
Amber Brown
32e7c9e7f2
Run Black. (#5482) 2019-06-20 19:32:02 +10:00
Schnuffle
dc5db01ff2 Replaced all occurences of e.message with str(e)
Signed-off-by: Schnuffle  <schnuffle@github.com>
2018-09-27 13:38:50 +02:00
Amber Brown
2608ebc04c
Port handlers/ to Python 3 (#3803) 2018-09-07 00:22:23 +10:00
Amber Brown
49af402019 run isort 2018-07-09 16:09:20 +10:00
Amber Brown
6350bf925e
Attempt to be more performant on PyPy (#3462) 2018-06-28 14:49:57 +01:00
Matthew Hodgson
784189b1f4 typos 2018-06-25 17:37:16 +01:00
Adrian Tschira
933bf2dd35 replace some iteritems with six
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-05-19 17:59:26 +02:00
Richard van der Hoff
fc149b4eeb Merge remote-tracking branch 'origin/develop' into rav/use_run_in_background 2018-04-27 14:31:23 +01:00
Richard van der Hoff
2a13af23bc Use run_in_background in preference to preserve_fn
While I was going through uses of preserve_fn for other PRs, I converted places
which only use the wrapped function once to use run_in_background, to avoid
creating the function object.
2018-04-27 12:55:51 +01:00
Richard van der Hoff
605defb9e4 Add missing consumeErrors
In general we want defer.gatherResults to consumeErrors, rather than having
exceptions hanging around and getting logged as CRITICAL unhandled errors.
2018-04-27 11:16:28 +01:00
Richard van der Hoff
a134c572a6 Stringify exceptions for keys/{query,claim}
Make sure we stringify any exceptions we return from keys/query and keys/claim,
to avoid a 'not JSON serializable' error later

Fixes #3010
2018-03-27 17:15:06 +01:00
Richard van der Hoff
c2a5cf2fe3 factor out exception handling for keys/claim and keys/query
this stuff is badly c&p'ed
2018-03-27 17:11:23 +01:00
Erik Johnston
fa72803490 Merge branch 'master' of github.com:matrix-org/synapse into develop 2018-03-19 11:41:01 +00:00
Erik Johnston
926ba76e23 Replace ujson with simplejson 2018-03-15 23:43:31 +00:00
Erik Johnston
cb9f8e527c s/replication_client/federation_client/ 2018-03-13 13:26:52 +00:00
Erik Johnston
265b993b8a Split replication layer into two 2018-03-13 10:55:47 +00:00
Erik Johnston
c3f79c9da5 Split out edu/query registration to a separate class 2018-03-13 10:24:27 +00:00
Matthew Hodgson
ab9f844aaf
Add federation_domain_whitelist option (#2820)
Add federation_domain_whitelist

gives a way to restrict which domains your HS is allowed to federate with.
useful mainly for gracefully preventing a private but internet-connected HS from trying to federate to the wider public Matrix network
2018-01-22 19:11:18 +01:00
Richard van der Hoff
a5213df1f7 Sanity checking for user ids
Check the user_id passed to a couple of APIs for validity, to avoid
"IndexError: list index out of range" exception which looks scary and results
in a 500 rather than a more useful error.

Fixes #1432, among other things
2018-01-17 14:28:54 +00:00
Richard van der Hoff
aedaba018f Replace some instances of preserve_context_over_deferred 2017-05-09 19:04:56 +01:00
Richard van der Hoff
de042b3b88 Do some logging when one-time-keys get claimed
might help us figure out if https://github.com/vector-im/riot-web/issues/3868
has happened.
2017-05-09 19:04:56 +01:00
Richard van der Hoff
a7e9d8762d Allow clients to upload one-time-keys with new sigs
When a client retries a key upload, don't give an error if the signature has
changed (but the key is the same).

Fixes https://github.com/vector-im/riot-android/issues/1208, hopefully.
2017-05-09 19:04:56 +01:00
Richard van der Hoff
9397edb28b Merge pull request #2050 from matrix-org/rav/federation_backoff
push federation retry limiter down to matrixfederationclient
2017-03-23 22:27:01 +00:00
Richard van der Hoff
4bd597d9fc push federation retry limiter down to matrixfederationclient
rather than having to instrument everywhere we make a federation call,
make the MatrixFederationHttpClient manage the retry limiter.
2017-03-23 09:28:46 +00:00
Richard van der Hoff
e08f81d96a Add a missing yield in device key upload
(this would only very very rarely actually be a useful thing, so the main
problem was the logcontext leak...)
2017-03-23 00:16:43 +00:00
Erik Johnston
c7a26b7c32 Fix unit tests 2017-01-30 17:11:24 +00:00
Erik Johnston
c2c9a78db9 Noop device key changes if they're the same 2017-01-30 16:55:04 +00:00
Erik Johnston
fbfad76c03 Add comments 2017-01-26 16:33:21 +00:00
Erik Johnston
c974116f19 Implement device key caching over federation 2017-01-26 16:07:24 +00:00
Erik Johnston
2367c5568c Add basic implementation of local device list changes 2017-01-25 14:27:27 +00:00
Richard van der Hoff
deca951241 Remove unused import 2016-12-12 09:24:35 +00:00
Richard van der Hoff
efa4ccfaee E2E key query: handle federation fails
Don't fail the whole request if we can't connect to a particular server.
2016-12-09 18:31:01 +00:00
Mark Haines
ad816b0add Limit how often we ask for keys from dead servers 2016-09-13 11:53:50 +01:00
Mark Haines
18ab019a4a Move the E2E key handling into the e2e handler 2016-09-13 11:35:35 +01:00
Mark Haines
949c2c5435 Add a timeout parameter for end2end key queries.
Add a timeout parameter for controlling how long synapse will wait
for responses from remote servers. For servers that fail include how
they failed to make it easier to debug.

Fetch keys from different servers in parallel rather than in series.

Set the default timeout to 10s.
2016-09-12 18:17:09 +01:00
Richard van der Hoff
f131cd9e53 keys/query: Omit device displayname if null
... which makes it more consistent with user displaynames.
2016-08-04 10:59:51 +01:00
Richard van der Hoff
68264d7404 Include device name in /keys/query response
Add an 'unsigned' section which includes the device display name.
2016-08-03 15:42:08 +01:00
Richard van der Hoff
91fa69e029 keys/query: return all users which were asked for
In the situation where all of a user's devices get deleted, we want to
indicate this to a client, so we want to return an empty dictionary, rather
than nothing at all.
2016-08-03 15:41:44 +01:00
Richard van der Hoff
a843868fe9 E2eKeysHandler: minor tweaks
PR feedback
2016-08-03 14:24:33 +01:00
Richard van der Hoff
1efee2f52b E2E keys: Make federation query share code with client query
Refactor the e2e query handler to separate out the local query, and then make
the federation handler use it.
2016-08-02 18:12:00 +01:00
Richard van der Hoff
986615b0b2 Move e2e query logic into a handler 2016-08-01 18:02:07 +01:00