Matrix-Mirrors/forked-synapse
1
0
Fork 0
mirror of https://mau.dev/maunium/synapse.git synced 2024-10-01 01:36:05 -04:00
Code Issues Packages Projects Releases Wiki Activity
23,459 Commits 5 Branches 723 Tags 124 MiB
719014d9d5
Commit Graph

271 Commits

Author SHA1 Message Date
Andrew Morgan
6824ddd93d Config option for verifying federation certificates (MSC 1711) (#4967) 2019-04-25 14:22:49 +01:00
Andrew Morgan
caa76e6021
Remove periods from copyright headers (#5046) 2019-04-11 17:08:13 +01:00
Richard van der Hoff
18b69be00f Rewrite Datastore.get_server_verify_keys 
Rewrite this so that it doesn't hammer the database.
 2019-04-09 00:00:10 +01:00
Richard van der Hoff
f88a9e6323 Remove redundant merged_keys dict 
There's no point in collecting a merged dict of keys: it is sufficient to
consider just the new keys which have been fetched by the most recent
key_fetch_fns.
 2019-04-08 22:36:18 +01:00
Richard van der Hoff
7d2a0c848e Fix from_server buglet in get_keys_from_perspectives 
make sure we store the name of the server the keys came from, rather than the
origin server, after doing a fetch-from-perspectives.
 2019-04-08 12:51:16 +01:00
Richard van der Hoff
6ae9361510 Hoist server_name check out of process_v2_response 
It's easier to check it in the caller than to complicate the interface with an
extra param.
 2019-04-04 19:12:54 +01:00
Richard van der Hoff
ef27d434d1 Clean up Keyring.process_v2_response 
Make this just return the key dict, rather than a single-entry dict mapping the
server name to the key dict. It's easy for the caller to get the server name
from from the response object anyway.
 2019-04-04 19:12:54 +01:00
Erik Johnston
78c563b77c Correctly log expected errors when fetching server keys 2019-03-11 14:11:10 +00:00
Erik Johnston
65d1003d01 raise_from already raises 2019-02-25 14:34:03 +00:00
Erik Johnston
41285ffe5b Handle errors when fetching remote server keys 2019-02-23 15:09:39 +00:00
Amber Brown
561eebe170 fix to use makeContext so that we don't need to rebuild the certificateoptions each time 2019-02-19 16:18:05 +11:00
Erik Johnston
7fc1196a36 Correctly handle RequestSendFailed exceptions 
This mainly reduces the number of exceptions we log.
 2019-02-14 14:01:04 +00:00
Richard van der Hoff
9645728619 Don't create server contexts when TLS is disabled 
we aren't going to use them anyway.
 2019-02-11 21:32:01 +00:00
Erik Johnston
554ca58ea1 Make add_hashes_and_signatures operate on dicts 2019-01-29 11:12:38 +00:00
Erik Johnston
855a151015 Refactor event signing to work on dicts 
This is in preparation for making EventBuilder format agnostic, which
means event signing should be done against the event dict rather than
the EventBuilder object.
 2019-01-28 16:42:10 +00:00
Richard van der Hoff
97fd29c019
Don't send IP addresses as SNI (#4452) 
The problem here is that we have cut-and-pasted an impl from Twisted, and then
failed to maintain it. It was fixed in Twisted in
https://github.com/twisted/twisted/pull/1047/files; let's do the same here.
 2019-01-24 09:34:44 +00:00
Richard van der Hoff
6bfa735a69
Make key fetches use regular federation client (#4426) 
All this magic is redundant.
 2019-01-22 11:04:20 +00:00
Amber Brown
23b0813599
Require ECDH key exchange & remove dh_params (#4429) 
* remove dh_params and set better cipher string
 2019-01-22 21:58:50 +11:00
Amber Brown
916efc8249
Remove fetching keys via the deprecated v1 kex method (#4120) 2018-10-31 23:14:39 +11:00
Richard van der Hoff
ef771cc4c2 Fix a number of flake8 errors 
Broadly three things here:

* disable W504 which seems a bit whacko
* remove a bunch of `as e` expressions from exception handlers that don't use
  them
* use `r""` for strings which include backslashes

Also, we don't use pep8 any more, so we can get rid of the duplicate config
there.
 2018-10-24 10:39:03 +01:00
Amber Brown
33716c4aea
Merge pull request #3826 from matrix-org/rav/logging_for_keyring 
add some logging for the keyring queue
 2018-09-12 20:43:47 +10:00
Amber Brown
8fd93b5eea
Port crypto/ to Python 3 (#3822) 2018-09-12 20:16:31 +10:00
Richard van der Hoff
806964b5de add some logging for the keyring queue 
why is it so damn slow?
 2018-09-06 18:51:06 +01:00
Erik Johnston
808d8e06aa Don't log exceptions when failing to fetch server keys 
Not being able to resolve or connect to remote servers is an expected
error, so we shouldn't log at ERROR with stacktraces.
 2018-08-21 11:19:26 +01:00
Jeroen
2e9c73e8ca more generic conversion of str/bytes to unicode 2018-08-09 21:31:26 +02:00
Jeroen
64899341dc include private functions from twisted 2018-08-09 21:04:22 +02:00
Jeroen
d5c0ce4cad updated docstring for ServerContextFactory 2018-08-08 19:25:01 +02:00
Jeroen
2903e65aff fix isort 2018-07-29 19:47:08 +02:00
Jeroen
8e3f75b39a fix accidental removal of hs 2018-07-27 12:17:31 +02:00
Jeroen
505530f36a Merge remote-tracking branch 'upstream/develop' into send_sni_for_federation_requests 
# Conflicts:
#	synapse/crypto/context_factory.py
 2018-07-14 20:24:46 +02:00
Jeroen
b5e157d895 Merge branch 'develop' into send_sni_for_federation_requests 
# Conflicts:
#	synapse/http/endpoint.py
 2018-07-09 08:51:11 +02:00
Amber Brown
49af402019 run isort 2018-07-09 16:09:20 +10:00
Amber Brown
6350bf925e
Attempt to be more performant on PyPy (#3462) 2018-06-28 14:49:57 +01:00
Jeroen
95341a8f6f take idna implementation from twisted 2018-06-26 21:15:14 +02:00
Jeroen
b7f34ee348 allow self-signed certificates 2018-06-26 20:41:05 +02:00
Jeroen
07b4f88de9 formatting changes for pep8 2018-06-25 12:31:16 +02:00
Jeroen
3d605853c8 send SNI for federation requests 2018-06-24 22:38:43 +02:00
Richard van der Hoff
e82db24a0e Try to log more helpful info when a sig verification fails 
Firstly, don't swallow the reason for the failure

Secondly, don't assume all exceptions are verification failures

Thirdly, log a bit of info about the key being used if debug is enabled
 2018-06-08 12:13:08 +01:00
Will Hunt
2ad3fc36e6 Fixes #3135 - Replace _OpenSSLECCurve with crypto.get_elliptic_curve (#3157) 
fixes #3135

Signed-off-by: Will Hunt will@half-shot.uk
 2018-04-30 16:21:11 +01:00
Richard van der Hoff
fc149b4eeb Merge remote-tracking branch 'origin/develop' into rav/use_run_in_background 2018-04-27 14:31:23 +01:00
Richard van der Hoff
2a13af23bc Use run_in_background in preference to preserve_fn 
While I was going through uses of preserve_fn for other PRs, I converted places
which only use the wrapped function once to use run_in_background, to avoid
creating the function object.
 2018-04-27 12:55:51 +01:00
Richard van der Hoff
9255a6cb17 Improve exception handling for background processes 
There were a bunch of places where we fire off a process to happen in the
background, but don't have any exception handling on it - instead relying on
the unhandled error being logged when the relevent deferred gets
garbage-collected.

This is unsatisfactory for a number of reasons:
 - logging on garbage collection is best-effort and may happen some time after
   the error, if at all
 - it can be hard to figure out where the error actually happened.
 - it is logged as a scary CRITICAL error which (a) I always forget to grep for
   and (b) it's not really CRITICAL if a background process we don't care about
   fails.

So this is an attempt to add exception handling to everything we fire off into
the background.
 2018-04-27 11:07:40 +01:00
Adrian Tschira
1515560f5c Use str(e) instead of e.message 
Doing this I learned e.message was pretty shortlived, added in 2.6,
they realized it was a bad idea and deprecated it in 2.7

Signed-off-by: Adrian Tschira <nota@notafile.com>
 2018-04-15 20:32:42 +02:00
Richard van der Hoff
7303ed65e1 Fix 500 when joining matrix-dev 
matrix-dev has an event (`$/6ANj/9QWQyd71N6DpRQPf+SDUu11+HVMeKSpMzBCwM:zemos.net`)
which has no `hashes` member.

Check for missing `hashes` element in events.
 2017-11-29 16:00:46 +00:00
Richard van der Hoff
eaaabc6c4f replace 'except:' with 'except Exception:' 
what could possibly go wrong
 2017-10-23 15:52:32 +01:00
Richard van der Hoff
94133d7ce8 Merge branch 'develop' into develop 2017-09-25 11:50:11 +01:00
Richard van der Hoff
79b3cf3e02 Fix logcontxt leak in keyclient (#2465) 
preserve_context_over_function doesn't do what you want it to do.
 2017-09-25 09:51:39 +01:00
Richard van der Hoff
c5c24c239b Fix logcontext handling in verify_json_objects_for_server 
preserve_context_over_fn is essentially broken, because (a) it pointlessly
drops the current logcontext before calling its wrapped function, which means
we don't get any useful logcontexts for _handle_key_deferred; (b) it wraps the
resulting deferred in a _PreservingContextDeferred, which is very dangerous
because you then can't yield on it without leaking context back into the
reactor.

Instead, let's specify that the resultant deferreds call their callbacks with
no logcontext.
 2017-09-20 01:32:42 +01:00
Richard van der Hoff
c5b0e9f485 Turn _start_key_lookups into an inlineCallbacks function 
... which means that logcontexts can be correctly preserved for the stuff it
does.

get_server_verify_keys is now called with the logcontext, so needs to
preserve_fn when it fires off its nested inlineCallbacks function.

Also renames get_server_verify_keys to reflect the fact it's meant to be
private.
 2017-09-20 01:32:42 +01:00
Richard van der Hoff
abdefb8a01 Fix potential race in _start_key_lookups 
If the verify_request.deferred has already completed, then `remove_deferreds`
will be called immediately. It therefore might resolve the server_to_deferred
deferred while there are still other requests for that server in flight.

To avoid that, we should build the complete list of requests, and *then* add the
callbacks.
 2017-09-20 01:32:42 +01:00
Richard van der Hoff
afbd773dc6 Add some comments to _start_key_lookups 2017-09-20 01:32:42 +01:00
Richard van der Hoff
2a4b9ea233 Consistency for how verify_request.deferred is called 
Define that it is run with no log context, and make sure that happens.

If we aren't careful to reset the logcontext, we can't bung the deferreds into
defer.gatherResults etc. We don't actually do that directly, but we *do*
resolve other deferreds from affected callbacks (notably the server_to_deferred
map in _start_key_lookups), and those *do* get passed into
defer.gatherResults. It turns out that this way ends up being least confusing.
 2017-09-20 01:32:42 +01:00
Richard van der Hoff
3b98439eca Factor out _start_key_lookups 
... to make it easier to see what's going on.
 2017-09-20 01:32:42 +01:00
Richard van der Hoff
fde63b880d Replace server_and_json with verify_requests 
This is a precursor to factoring some of this code out.
 2017-09-20 01:32:42 +01:00
Richard van der Hoff
2d511defd9 pull out handle_key_deferred to top level 
There's no need for this to be a nested definition; pulling it out not only
makes it more efficient, but makes it easier to check that it's not accessing
any local variables it shouldn't be.
 2017-09-20 01:32:42 +01:00
Richard van der Hoff
dd1ea9763a Fix incorrect key_ids in error message 2017-09-20 01:32:42 +01:00
Richard van der Hoff
9864efa532 Fix concurrent server_key requests (#2458) 
Fix a bug where we could end up firing off multiple requests for server_keys
for the same server at the same time.
 2017-09-19 23:25:44 +01:00
Richard van der Hoff
290777b3d9 Clean up and document handling of logcontexts in Keyring (#2452) 
I'm still unclear on what the intended behaviour for
`verify_json_objects_for_server` is, but at least I now understand the
behaviour of most of the things it calls...
 2017-09-18 18:31:01 +01:00
Kenny Keslar
f18373dc5d Fix iteration of requests_missing_keys; list doesn't have .values() 
Signed-off-by: Kenny Keslar <r3dey3@r3dey3.com>
 2017-07-26 22:44:19 -05:00
Richard van der Hoff
9397edb28b Merge pull request #2050 from matrix-org/rav/federation_backoff 
push federation retry limiter down to matrixfederationclient
 2017-03-23 22:27:01 +00:00
Richard van der Hoff
4bd597d9fc push federation retry limiter down to matrixfederationclient 
rather than having to instrument everywhere we make a federation call,
make the MatrixFederationHttpClient manage the retry limiter.
 2017-03-23 09:28:46 +00:00
Richard van der Hoff
64778693be fix up some key verif docstrings 2017-03-21 13:27:50 +00:00
Richard van der Hoff
c36d15d2de Add some debug to help diagnose weird federation issue 2017-03-20 15:36:14 +00:00
Erik Johnston
9219139351 Preserve some logcontexts 2016-08-24 11:58:40 +01:00
Erik Johnston
04fc8bbcb0 Update keyring Measure 2016-08-19 18:23:44 +01:00
Erik Johnston
2426c2f21a Measure keyrings 2016-08-19 18:23:44 +01:00
Erik Johnston
fa1ce4d8ad Don't print stack traces when failing to get remote keys 2016-08-10 10:44:37 +01:00
Erik Johnston
a285194021 Merge branch 'erikj/key_client_fix' of github.com:matrix-org/synapse into release-v0.17.0 2016-07-28 10:47:06 +01:00
Mark Haines
29b25d59c6 Merge branch 'develop' into markjh/verify 
Conflicts:
	synapse/crypto/keyring.py
 2016-07-27 15:11:02 +01:00
Mark Haines
884b800899 Merge pull request #955 from matrix-org/markjh/only_from2 
Add a couple more checks to the keyring
 2016-07-27 15:08:22 +01:00
Mark Haines
fe1b369946 Clean up verify_json_objects_for_server 2016-07-27 14:10:43 +01:00
Mark Haines
a4b06b619c Add a couple more checks to the keyring 2016-07-26 19:50:11 +01:00
Mark Haines
87ffd21b29 Fix a couple of bugs in the transaction and keyring code 2016-07-26 19:19:08 +01:00
Erik Johnston
d26b660aa6 Cache getPeer 2016-07-21 17:38:51 +01:00
Erik Johnston
cf94a78872 Set host not path 2016-07-21 11:45:53 +01:00
Erik Johnston
081e5d55e6 Send the correct host header when fetching keys 2016-07-21 11:14:54 +01:00
David Baker
f28643cea9 Uncommit accidentally commited edit to cipher list 2016-05-10 18:44:32 +02:00
David Baker
d46b18a00f Pass through _get_event_txn 2016-05-10 18:27:06 +02:00
Erik Johnston
2e2be463f8 Make key client send a Host header 2016-03-11 10:29:05 +00:00
Erik Johnston
2c1fbea531 Fix up logcontexts 2016-02-08 14:26:45 +00:00
Matthew Hodgson
6c28ac260c copyrights 2016-01-07 04:26:29 +00:00
Paul "LeoNerd" Evans
a6ba41e078 Actually look up required remote server key IDs 
set.union() is a side-effect-free function that returns the union of two
sets. This clearly wanted .update(), which is the side-effecting mutator
version.
 2015-12-18 21:36:42 +00:00
Erik Johnston
0eabfa55f6 Fix typo 2015-11-20 17:17:58 +00:00
Erik Johnston
6408541075 Don't limit connections to perspective servers 2015-11-20 17:15:44 +00:00
Erik Johnston
ffe8cf7e59 Fix bug where we sometimes didn't fetch all the keys requested for a 
server.
 2015-09-17 10:21:32 +01:00
Daniel Wagner-Hall
2c8f16257a Merge pull request #272 from matrix-org/daniel/insecureclient 
Allow configuration to ignore invalid SSL certs
 2015-09-15 16:52:38 +01:00
Erik Johnston
dd0867f5ba Various bug fixes to crypto.keyring 2015-09-09 17:02:39 +01:00
Daniel Wagner-Hall
81a93ddcc8 Allow configuration to ignore invalid SSL certs 
This will be useful for sytest, and sytest only, hence the aggressive
config key name.
 2015-09-09 12:02:07 +01:00
Mark Haines
78323ccdb3 Remove syutil dependency in favour of smaller single-purpose libraries 2015-08-24 16:17:38 +01:00
Erik Johnston
0b3389bcd2 Merge pull request #194 from matrix-org/erikj/bulk_verify_sigs 
Implement bulk verify_signed_json API
 2015-07-10 13:46:53 +01:00
Matthew Hodgson
fb8d2862c1 remove the tls_certificate_chain_path param and simply support tls_certificate_path pointing to a file containing a chain of certificates 2015-07-09 00:45:41 +01:00
Matthew Hodgson
f26a3df1bf oops, context.tls_certificate_chain_file() expects a file, not a certificate. 2015-07-08 21:33:02 +01:00
Matthew Hodgson
19fa3731ae typo 2015-07-08 18:53:41 +01:00
Matthew Hodgson
64afbe6ccd add new optional config for tls_certificate_chain_path for folks with intermediary SSL certs 2015-07-08 18:20:02 +01:00
Erik Johnston
f0dd568e16 Wait for previous attempts at fetching keys for a given server before trying to fetch more 2015-06-26 11:25:00 +01:00
Erik Johnston
b5f55a1d85 Implement bulk verify_signed_json API 2015-06-26 10:39:34 +01:00
Erik Johnston
291cba284b Handle the case when things return empty but non none things 2015-05-19 14:42:46 +01:00
Erik Johnston
253f76a0a5 Don't always hit get_server_verify_key_v1_direct 2015-05-19 14:42:38 +01:00
Erik Johnston
d3e09f12d0 SYN-383: Actually, we expect this value to be a dict 2015-05-19 13:12:41 +01:00
Erik Johnston
2b7120e233 SYN-383: Handle the fact the server might not have signed things 2015-05-19 12:49:38 +01:00
Erik Johnston
8b256a7296 Don't reuse var names 2015-05-19 11:58:22 +01:00
Erik Johnston
2aeee2a905 SYN-383: Fix parsing of verify_keys and catching of _DefGen_Return 2015-05-19 11:56:18 +01:00
Mark Haines
c6a03c46e6 SYN-383: Extract the response list from 'server_keys' in the response JSON as it might work better than iterating over the top level dict 2015-05-19 10:23:02 +01:00
Mark Haines
ec07dba29e Merge pull request #143 from matrix-org/erikj/SYN-375 
SYN-375 - Lots of unhandled deferred exceptions.
 2015-05-12 15:25:54 +01:00
Erik Johnston
476899295f Change the way we do logging contexts so that they survive divergences 2015-05-08 16:32:18 +01:00
Erik Johnston
fca28d243e Change the way we create observers to deferreds so that we don't get spammed by 'unhandled errors' 2015-05-08 16:28:08 +01:00
Mark Haines
1319905d7a Use a defer.gatherResults to collect results from the perspective servers 2015-04-29 13:31:14 +01:00
Mark Haines
74874ffda7 Update the query format used by keyring to match current key v2 spec 2015-04-29 12:14:08 +01:00
Mark Haines
46d200a3a1 Implement minimum_valid_until_ts in the remote key resource 2015-04-29 11:57:26 +01:00
Mark Haines
f8b865264a Merge branch 'develop' into key_distribution 
Conflicts:
	synapse/crypto/keyring.py
 2015-04-27 18:29:32 +01:00
Erik Johnston
2c70849dc3 Fix newlines 2015-04-27 14:38:29 +01:00
Erik Johnston
0a016b0525 Pull inner function out. 2015-04-27 14:37:24 +01:00
Erik Johnston
e701aec2d1 Implement locks using create_observer for fetching media and server keys 2015-04-27 14:20:26 +01:00
Mark Haines
288702170d Add config for setting the perspective servers 2015-04-24 17:01:34 +01:00
Mark Haines
4bbf7156ef Update to match the specification for key/v2 2015-04-23 16:39:13 +01:00
Mark Haines
f30d47c876 Implement remote key lookup api 2015-04-22 14:21:08 +01:00
Mark Haines
2f9157b427 Implement v2 key lookup 2015-04-20 16:23:47 +01:00
Mark Haines
8d761134c2 Fail quicker for 4xx responses in the key client, optional hit a different API path 2015-04-15 16:57:58 +01:00
Erik Johnston
3ce8540484 Don't look for an TLS private key if we have set --no-tls 2015-03-06 11:34:06 +00:00
Erik Johnston
5b5c7a28d6 Log error message when we fail to fetch remote server keys 2015-03-05 17:09:13 +00:00
Erik Johnston
9371019133 Try to only back off if we think we failed to connect to the remote 2015-02-17 18:13:34 +00:00
Erik Johnston
2b8f1a956c Add per server retry limiting. 
Factor out the pre destination retry logic from TransactionQueue so it
can be reused in both get_pdu and crypto.keyring
 2015-02-17 17:20:56 +00:00
Erik Johnston
5025305fb2 Rate limit retries when fetching server keys. 2015-02-17 15:57:42 +00:00
Erik Johnston
4ebbaf0d43 Blunty replace json with simplejson 2015-02-11 14:23:10 +00:00
Mark Haines
84a769cdb7 Fix code-style 2015-02-10 17:58:36 +00:00
Erik Johnston
823999716e Fix bug in timeout handling in keyclient 2015-01-30 11:08:01 +00:00
Mark Haines
adb04b1e57 Update copyright notices 2015-01-06 13:21:39 +00:00
Erik Johnston
95aa903ffa Try and figure out how and why signatures are being changed. 2014-12-10 11:37:47 +00:00
Erik Johnston
609c31e8df More bug fixes 2014-12-08 17:50:56 +00:00
Erik Johnston
c31dba86ec Convert rest and handlers to use new event structure 2014-12-04 15:50:01 +00:00
Erik Johnston
75b4329aaa WIP for new way of managing events. 2014-12-03 16:07:21 +00:00
David Baker
f1c7f8e813 Merge branch 'develop' into http_client_refactor 2014-11-20 17:49:48 +00:00
David Baker
e377d33652 Separate out the matrix http client completely because just about all of its code it now separate from the simple case we need for standard HTTP(S) 2014-11-20 17:41:56 +00:00
Mark Haines
dfdda2c871 Use module loggers rather than the root logger. Exceptions caused by bad clients shouldn't cause ERROR level logging. Fix sql logging to use 'repr' rather than 'str' 2014-11-20 17:10:37 +00:00
Mark Haines
32090aee16 Add a few missing yields, Move deferred lists inside PreserveLoggingContext because they don't interact well with the logging contexts 2014-11-20 16:24:00 +00:00
Mark Haines
cb4b6c844a Merge PDUs and Events into one object 2014-11-14 21:25:02 +00:00
Mark Haines
8c2b5ea7c4 Fix PDU and event signatures 2014-11-14 19:11:04 +00:00
Mark Haines
de1ec90133 Validate signatures on incoming events 2014-11-14 19:11:04 +00:00
Mark Haines
e903c941cb Merge branch 'develop' into request_logging 
Conflicts:
	setup.py
	synapse/storage/_base.py
	synapse/util/async.py
 2014-11-14 11:16:50 +00:00
Erik Johnston
6cb6cb9e69 Tidy up some of the unused sql tables 2014-11-10 10:31:00 +00:00
Erik Johnston
1c06806f90 Finish redaction algorithm. 2014-11-10 10:21:32 +00:00
Erik Johnston
97a096b507 Add hash of current state to events 2014-11-07 11:37:06 +00:00
Erik Johnston
68698e0ac8 Fix bugs in generating event signatures and hashing 2014-11-03 17:51:42 +00:00
Erik Johnston
82a6b83524 Don't assume event has hashes key already 2014-11-03 11:32:12 +00:00
Erik Johnston
ecabff7eb4 Sign evnets 2014-10-31 17:08:52 +00:00
Erik Johnston
d30d79b5be Make prev_event signing work again. 2014-10-31 15:35:39 +00:00
Erik Johnston
ef9c4476a0 Merge branch 'develop' of github.com:matrix-org/synapse into federation_authorization 2014-10-30 11:18:28 +00:00
Mark Haines
7c06399512 Merge branch 'develop' into request_logging 
Conflicts:
	synapse/config/logger.py
 2014-10-30 11:13:58 +00:00
Mark Haines
7d709542ca Fix pep8 warnings 2014-10-30 11:10:17 +00:00
Mark Haines
b29517bd01 Add a request-id to each log line 2014-10-30 01:21:33 +00:00