Commit Graph

260 Commits

Author SHA1 Message Date
Eric Eastwood
7af07f9716
Instrument _check_sigs_and_hash_and_fetch to trace time spent in child concurrent calls ()
Instrument `_check_sigs_and_hash_and_fetch` to trace time spent in child concurrent calls because I've see `_check_sigs_and_hash_and_fetch` take [10.41s to process 100 events](https://github.com/matrix-org/synapse/issues/13587)

Fix https://github.com/matrix-org/synapse/issues/13587

Part of https://github.com/matrix-org/synapse/issues/13356
2022-08-23 21:53:37 -05:00
David Robertson
e0bb268134
Fix typechecker problems exposed by signedjson 1.1.2 () 2022-03-29 21:37:50 +00:00
David Robertson
a2b00a4486
Bump black and click versions () 2022-03-29 10:41:19 +00:00
Richard van der Hoff
e24ff8ebe3
Remove HomeServer.get_datastore() ()
The presence of this method was confusing, and mostly present for backwards
compatibility. Let's get rid of it.

Part of 
2022-02-23 11:04:02 +00:00
Patrick Cloke
10a88ba91c
Use auto_attribs/native type hints for attrs classes. () 2022-01-13 13:49:28 +00:00
Richard van der Hoff
9cd13c5f63
Fix perspectives requests for multiple keys for the same server ()
If we tried to request multiple keys for the same server, we would end up
dropping some of those requests.
2021-11-29 13:15:36 +00:00
Richard van der Hoff
a6f7f84570
Fix verification of objects signed with old local keys ()
Fixes a bug introduced in : objects signed by the local server, but with
keys other than the current one, could not be successfully verified.

We need to check the key id in the signature, and track down the right key.
2021-11-19 10:55:09 +00:00
Shay
e002faee01
Fetch verify key locally rather than trying to do so over federation if origin and host are the same. ()
* add tests for fetching key locally

* add logic to check if origin server is same as host and fetch verify key locally rather than over federation

* add changelog

* slight refactor, add docstring, change changelog entry

* Make changelog entry one line

* remove verify_json_locally and push locality check to process_request, add function process_request_locally

* remove leftover code reference

* refactor to add common call to 'verify_json and associated handling code

* add type hint to process_json

* add some docstrings + very slight refactor
2021-10-28 10:27:17 -07:00
Patrick Cloke
0f9adc99ad
Add missing type hints to synapse.crypto. ()
And require type hints for this module.
2021-10-21 13:07:07 +00:00
Patrick Cloke
bb7fdd821b
Use direct references for configuration variables (part 5). () 2021-09-24 07:25:21 -04:00
Patrick Cloke
8c7a531e27
Use direct references for some configuration variables (part 2) () 2021-09-15 08:34:52 -04:00
Jonathan de Jong
bf72d10dbf
Use inline type hints in various other places (in synapse/) () 2021-07-15 11:02:43 +01:00
Erik Johnston
c842c581ed
When joining a remote room limit the number of events we concurrently check signatures/hashes for ()
If we do hundreds of thousands at once the memory overhead can easily reach 500+ MB.
2021-06-08 11:07:46 +01:00
Erik Johnston
fc3d2dc269
Rewrite the KeyRing () 2021-06-02 16:37:59 +01:00
Erik Johnston
1c6a19002c
Add Keyring.verify_events_for_server and reduce memory usage ()
Also add support for giving a callback to generate the JSON object to
verify. This should reduce memory usage, as we no longer have the event
in memory in dict form (which has a large memory footprint) for extend
periods of time.
2021-05-20 16:25:11 +01:00
Richard van der Hoff
5a153772c1
remove HomeServer.get_config ()
Every single time I want to access the config object, I have to remember
whether or not we use `get_config`. Let's just get rid of it.
2021-04-14 19:09:08 +01:00
Jonathan de Jong
4b965c862d
Remove redundant "coding: utf-8" lines ()
Part of 

Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now.

`Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`
2021-04-14 15:34:27 +01:00
Jonathan de Jong
4c3827f2c1
Enable addtional flake8-bugbear linting checks. () 2021-03-24 09:34:30 -04:00
Patrick Cloke
b7748d3c00
Import HomeServer from the proper module. () 2021-03-23 07:12:48 -04:00
Jonathan de Jong
27d2820c33
Enable flake8-bugbear, but disable most checks. ()
* Adds B00 to ignored checks.
* Fixes remaining issues.
2021-03-16 14:19:27 -04:00
Erik Johnston
54a6afeee3
Cache config options in SSL verification ()
Reading from the config object is *slow*.
2021-01-28 17:38:59 +00:00
Patrick Cloke
1c9a850562
Add type hints to the crypto module. () 2021-01-04 10:04:50 -05:00
Patrick Cloke
30fba62108
Apply an IP range blacklist to push and key revocation requests. ()
Replaces the `federation_ip_range_blacklist` configuration setting with an
`ip_range_blacklist` setting with wider scope. It now applies to:

* Federation
* Identity servers
* Push notifications
* Checking key validitity for third-party invite events

The old `federation_ip_range_blacklist` setting is still honored if present, but
with reduced scope (it only applies to federation and identity servers).
2020-12-02 11:09:24 -05:00
Patrick Cloke
34a5696f93
Fix typos and spelling errors. () 2020-10-23 12:38:40 -04:00
Richard van der Hoff
fec6f9ac17
Fix occasional "Re-starting finished log context" from keyring ()
* Fix test_verify_json_objects_for_server_awaits_previous_requests

It turns out that this wasn't really testing what it thought it was testing
(in particular, `check_context` was turning failures into success, which was
making the tests pass even though it wasn't clear they should have been.

It was also somewhat overcomplex - we can test what it was trying to test
without mocking out perspectives servers.

* Fix warnings about finished logcontexts in the keyring

We need to make sure that we finish the key fetching magic before we run the
verifying code, to ensure that we don't mess up our logcontexts.
2020-09-25 12:29:54 +01:00
Richard van der Hoff
4f3096d866 Add a comment re 2020-09-21 12:34:06 +01:00
Patrick Cloke
8a4a4186de
Simplify super() calls to Python 3 syntax. ()
This converts calls like super(Foo, self) -> super().

Generated with:

    sed -i "" -Ee 's/super\([^\(]+\)/super()/g' **/*.py
2020-09-18 09:56:44 -04:00
Patrick Cloke
c619253db8
Stop sub-classing object () 2020-09-04 06:54:56 -04:00
Patrick Cloke
76c43f086a
Do not assume calls to runInteraction return Deferreds. () 2020-08-20 06:39:55 -04:00
Richard van der Hoff
de6f892065
Add a comment about SSLv23_METHOD () 2020-08-07 15:14:29 +01:00
Patrick Cloke
2a89ce8cd4
Convert the crypto module to async/await. () 2020-08-03 08:29:01 -04:00
Patrick Cloke
c978f6c451
Convert federation client to async/await. () 2020-07-30 08:01:33 -04:00
Dagfinn Ilmari Mannsåker
a3f11567d9
Replace all remaining six usage with native Python 3 equivalents () 2020-06-16 08:51:47 -04:00
Richard van der Hoff
39230d2171
Clean up some LoggingContext stuff ()
* Pull Sentinel out of LoggingContext

... and drop a few unnecessary references to it

* Factor out LoggingContext.current_context

move `current_context` and `set_context` out to top-level functions.

Mostly this means that I can more easily trace what's actually referring to
LoggingContext, but I think it's generally neater.

* move copy-to-parent into `stop`

this really just makes `start` and `stop` more symetric. It also means that it
behaves correctly if you manually `set_log_context` rather than using the
context manager.

* Replace `LoggingContext.alive` with `finished`

Turn `alive` into `finished` and make it a bit better defined.
2020-03-24 14:45:33 +00:00
Richard van der Hoff
c37db0211e
Share SSL contexts for non-federation requests ()
Extends  etc to the SimpleHttpClient so that it also applies to non-federation requests.

Fixes .
2020-03-17 21:32:25 +00:00
Patrick Cloke
06eb5cae08
Remove special auth and redaction rules for aliases events in experimental room ver. () 2020-03-09 08:58:25 -04:00
Patrick Cloke
509e381afa
Clarify list/set/dict/tuple comprehensions and enforce via flake8 ()
Ensure good comprehension hygiene using flake8-comprehensions.
2020-02-21 07:15:07 -05:00
Richard van der Hoff
7f93eb1903
pass room_version into compute_event_signature () 2020-01-31 13:47:43 +00:00
Richard van der Hoff
2a81393a4b Pass room_version into add_hashes_and_signatures 2020-01-30 22:15:50 +00:00
Richard van der Hoff
ba897a7590
Fix some test failures when frozen_dicts are enabled ()
Fixes 
2020-01-06 15:22:46 +00:00
Richard van der Hoff
4b36b482e0
Fix exception when fetching notary server's old keys ()
Lift the restriction that *all* the keys used for signing v2 key responses be
present in verify_keys.

Fixes .
2020-01-06 12:33:56 +00:00
Michael Kaye
f85b9842f0 Don't encode object as UTF-8 string if not needed.
I believe that string formatting ~10-15 sized events will
take a proportion of CPU time.
2019-10-24 18:08:45 +01:00
Amber Brown
850dcfd2d3
Fix well-known lookups with the federation certificate whitelist () 2019-09-14 04:58:38 +10:00
Andrew Morgan
4548d1f87e
Remove unnecessary parentheses around return statements ()
Python will return a tuple whether there are parentheses around the returned values or not.

I'm just sick of my editor complaining about this all over the place :)
2019-08-30 16:28:26 +01:00
Erik Johnston
f5b50d0871
Merge pull request from matrix-org/erikj/notary_key
Add config option to sign remote key query responses with a separate key.
2019-08-27 11:51:37 +01:00
Erik Johnston
7af5a63063 Fixup review comments 2019-08-23 15:36:28 +01:00
Richard van der Hoff
ef1c524bb3
Improve error msg when key-fetch fails ()
There's no point doing a raise_from here, because the exception is always
logged at warn with no stacktrace in the caller. Instead, let's try to give
better messages to reduce confusion.

In particular, this means that we won't log 'Failed to connect to remote
server' when we don't even attempt to connect to the remote server due to
blacklisting.
2019-08-22 10:42:06 +01:00
Erik Johnston
97cbc96093 Only sign when we respond to remote key requests 2019-08-21 11:21:58 +01:00
Erik Johnston
5906be8589 Add config option for keys to use to sign keys
This allows servers to separate keys that are used to sign remote keys
when acting as a notary server.
2019-08-21 10:44:58 +01:00
Erik Johnston
a9bcae9f50 Share SSL options for well-known requests 2019-07-31 10:39:24 +01:00