257 Commits

Author SHA1 Message Date
Olivier Wilkinson (reivilibre)
4dd9ea8f4f Revert "Revert accidental fast-forward merge from v1.49.0rc1"
This reverts commit 158d73ebdd61eef33831ae5f6990acf07244fc55.
2021-12-14 14:22:01 +00:00
Sean Quah
158d73ebdd Revert accidental fast-forward merge from v1.49.0rc1
Revert "Sort internal changes in changelog"
Revert "Update CHANGES.md"
Revert "1.49.0rc1"
Revert "Revert "Move `glob_to_regex` and `re_word_boundary` to `matrix-python-common` (#11505) (#11527)"
Revert "Refactors in `_generate_sync_entry_for_rooms` (#11515)"
Revert "Correctly register shutdown handler for presence workers (#11518)"
Revert "Fix `ModuleApi.looping_background_call` for non-async functions (#11524)"
Revert "Fix 'delete room' admin api to work on incomplete rooms (#11523)"
Revert "Correctly ignore invites from ignored users (#11511)"
Revert "Fix the test breakage introduced by #11435 as a result of concurrent PRs (#11522)"
Revert "Stabilise support for MSC2918 refresh tokens as they have now been merged into the Matrix specification. (#11435)"
Revert "Save the OIDC session ID (sid) with the device on login (#11482)"
Revert "Add admin API to get some information about federation status (#11407)"
Revert "Include bundled aggregations in /sync and related fixes (#11478)"
Revert "Move `glob_to_regex` and `re_word_boundary` to `matrix-python-common` (#11505)"
Revert "Update backward extremity docs to make it clear that it does not indicate whether we have fetched an events' `prev_events` (#11469)"
Revert "Support configuring the lifetime of non-refreshable access tokens separately to refreshable access tokens. (#11445)"
Revert "Add type hints to `synapse/tests/rest/admin` (#11501)"
Revert "Revert accidental commits to develop."
Revert "Newsfile"
Revert "Give `tests.server.setup_test_homeserver` (nominally!) the same behaviour"
Revert "Move `tests.utils.setup_test_homeserver` to `tests.server`"
Revert "Convert one of the `setup_test_homeserver`s to `make_test_homeserver_synchronous`"
Revert "Disambiguate queries on `state_key` (#11497)"
Revert "Comments on the /sync tentacles (#11494)"
Revert "Clean up tests.storage.test_appservice (#11492)"
Revert "Clean up `tests.storage.test_main` to remove use of legacy code. (#11493)"
Revert "Clean up `tests.test_visibility` to remove legacy code. (#11495)"
Revert "Minor cleanup on recently ported doc pages  (#11466)"
Revert "Add most of the missing type hints to `synapse.federation`. (#11483)"
Revert "Avoid waiting for zombie processes in `synctl stop` (#11490)"
Revert "Fix media repository failing when media store path contains symlinks (#11446)"
Revert "Add type annotations to `tests.storage.test_appservice`. (#11488)"
Revert "`scripts-dev/sign_json`: support for signing events (#11486)"
Revert "Add MSC3030 experimental client and federation API endpoints to get the closest event to a given timestamp (#9445)"
Revert "Port wiki pages to documentation website (#11402)"
Revert "Add a license header and comment. (#11479)"
Revert "Clean-up get_version_string (#11468)"
Revert "Link background update controller docs to summary (#11475)"
Revert "Additional type hints for config module. (#11465)"
Revert "Register the login redirect endpoint for v3. (#11451)"
Revert "Update openid.md"
Revert "Remove mention of OIDC certification from Dex (#11470)"
Revert "Add a note about huge pages to our Postgres doc (#11467)"
Revert "Don't start Synapse master process if `worker_app` is set (#11416)"
Revert "Expose worker & homeserver as entrypoints in `setup.py` (#11449)"
Revert "Bundle relations of relations into the `/relations` result. (#11284)"
Revert "Fix `LruCache` corruption bug with a `size_callback` that can return 0 (#11454)"
Revert "Eliminate a few `Any`s in `LruCache` type hints (#11453)"
Revert "Remove unnecessary `json.dumps` from `tests.rest.admin` (#11461)"
Revert "Merge branch 'master' into develop"

This reverts commit 26b5d2320f62b5eb6262c7614fbdfc364a4dfc02.
This reverts commit bce4220f387bf5448387f0ed7d14ed1e41e40747.
This reverts commit 966b5d0fa0893c3b628c942dfc232e285417f46d.
This reverts commit 088d748f2cb51f03f3bcacc0fb3af1e0f9607737.
This reverts commit 14d593f72d10b4d8cb67e3288bb3131ee30ccf59.
This reverts commit 2a3ec6facf79f6aae011d9fb6f9ed5e43c7b6bec.
This reverts commit eccc49d7554d1fab001e1fefb0fda8ffb254b630.
This reverts commit b1ecd19c5d19815b69e425d80f442bf2877cab76.
This reverts commit 9c55dedc8c4484e6269451a8c3c10b3e314aeb4a.
This reverts commit 2d42e586a8c54be1a83643148358b1651c1ca666.
This reverts commit 2f053f3f82ca174cc1c858c75afffae51af8ce0d.
This reverts commit a15a893df8428395df7cb95b729431575001c38a.
This reverts commit 8b4b153c9e86c04c7db8c74fde4b6a04becbc461.
This reverts commit 494ebd7347ba52d702802fba4c3bb13e7bfbc2cf.
This reverts commit a77c36989785c0d5565ab9a1169f4f88e512ce8a.
This reverts commit 4eb77965cd016181d2111f37d93526e9bb0434f0.
This reverts commit 637df95de63196033a6da4a6e286e1d58ea517b6.
This reverts commit e5f426cd54609e7f05f8241d845e6e36c5f10d9a.
This reverts commit 8cd68b8102eeab1b525712097c1b2e9679c11896.
This reverts commit 6cae125e20865c52d770b24278bb7ab8fde5bc0d.
This reverts commit 7be88fbf48156b36b6daefb228e1258e7d48cae4.
This reverts commit b3fd99b74a3f6f42a9afd1b19ee4c60e38e8e91a.
This reverts commit f7ec6e7d9e0dc360d9fb41f3a1afd7bdba1475c7.
This reverts commit 5640992d176a499204a0756b1677c9b1575b0a49.
This reverts commit d26808dd854006bd26a2366c675428ce0737238c.
This reverts commit f91624a5950e14ba9007eed9bfa1c828676d4745.
This reverts commit 16d39a5490ce74c901c7a8dbb990c6e83c379207.
This reverts commit 8a4c2969874c0b7d72003f2523883eba8a348e83.
This reverts commit 49e1356ee3d5d72929c91f778b3a231726c1413c.
This reverts commit d2279f471ba8f44d9f578e62b286897a338d8aa1.
This reverts commit b50e39df578adc3f86c5efa16bee9035cfdab61b.
This reverts commit 858d80bf0f9f656a03992794874081b806e49222.
This reverts commit 435f04480728c5d982e1a63c1b2777784bf9cd26.
This reverts commit f61462e1be36a51dbf571076afa8e1930cb182f4.
This reverts commit a6f1a3abecf8e8fd3e1bff439a06b853df18f194.
This reverts commit 84dc50e160a2ec6590813374b5a1e58b97f7a18d.
This reverts commit ed635d32853ee0a3e5ec1078679b27e7844a4ac7.
This reverts commit 7b62791e001d6a4f8897ed48b3232d7f8fe6aa48.
This reverts commit 153194c7717d8016b0eb974c81b1baee7dc1917d.
This reverts commit f44d729d4ccae61bc0cdd5774acb3233eb5f7c13.
This reverts commit a265fbd397ae72b2d3ea4c9310591ff1d0f3e05c.
This reverts commit b9fef1a7cdfcc128fa589a32160e6aa7ed8964d7.
This reverts commit b0eb64ff7bf6bde42046e091f8bdea9b7aab5f04.
This reverts commit f1795463bf503a6fca909d77f598f641f9349f56.
This reverts commit 70cbb1a5e311f609b624e3fae1a1712db639c51e.
This reverts commit 42bf0204635213e2c75188b19ee66dc7e7d8a35e.
This reverts commit 379f2650cf875f50c59524147ec0e33cfd5ef60c.
This reverts commit 7ff22d6da41cd5ca80db95c18b409aea38e49fcd.
This reverts commit 5a0b652d36ae4b6d423498c1f2c82c97a49c6f75.
This reverts commit 432a174bc192740ac7a0a755009f6099b8363ad9.
This reverts commit b14f8a1baf6f500997ae4c1d6a6d72094ce14270, reversing
changes made to e713855dca17a7605bae99ea8d71bc7f8657e4b8.
2021-12-07 16:47:31 +00:00
Richard van der Hoff
9cd13c5f63
Fix perspectives requests for multiple keys for the same server (#11440)
If we tried to request multiple keys for the same server, we would end up
dropping some of those requests.
2021-11-29 13:15:36 +00:00
Richard van der Hoff
a6f7f84570
Fix verification of objects signed with old local keys (#11379)
Fixes a bug introduced in #11129: objects signed by the local server, but with
keys other than the current one, could not be successfully verified.

We need to check the key id in the signature, and track down the right key.
2021-11-19 10:55:09 +00:00
Shay
e002faee01
Fetch verify key locally rather than trying to do so over federation if origin and host are the same. (#11129)
* add tests for fetching key locally

* add logic to check if origin server is same as host and fetch verify key locally rather than over federation

* add changelog

* slight refactor, add docstring, change changelog entry

* Make changelog entry one line

* remove verify_json_locally and push locality check to process_request, add function process_request_locally

* remove leftover code reference

* refactor to add common call to 'verify_json and associated handling code

* add type hint to process_json

* add some docstrings + very slight refactor
2021-10-28 10:27:17 -07:00
Patrick Cloke
0f9adc99ad
Add missing type hints to synapse.crypto. (#11146)
And require type hints for this module.
2021-10-21 13:07:07 +00:00
Patrick Cloke
bb7fdd821b
Use direct references for configuration variables (part 5). (#10897) 2021-09-24 07:25:21 -04:00
Patrick Cloke
8c7a531e27
Use direct references for some configuration variables (part 2) (#10812) 2021-09-15 08:34:52 -04:00
Jonathan de Jong
bf72d10dbf
Use inline type hints in various other places (in synapse/) (#10380) 2021-07-15 11:02:43 +01:00
Erik Johnston
c842c581ed
When joining a remote room limit the number of events we concurrently check signatures/hashes for (#10117)
If we do hundreds of thousands at once the memory overhead can easily reach 500+ MB.
2021-06-08 11:07:46 +01:00
Erik Johnston
fc3d2dc269
Rewrite the KeyRing (#10035) 2021-06-02 16:37:59 +01:00
Erik Johnston
1c6a19002c
Add Keyring.verify_events_for_server and reduce memory usage (#10018)
Also add support for giving a callback to generate the JSON object to
verify. This should reduce memory usage, as we no longer have the event
in memory in dict form (which has a large memory footprint) for extend
periods of time.
2021-05-20 16:25:11 +01:00
Richard van der Hoff
5a153772c1
remove HomeServer.get_config (#9815)
Every single time I want to access the config object, I have to remember
whether or not we use `get_config`. Let's just get rid of it.
2021-04-14 19:09:08 +01:00
Jonathan de Jong
4b965c862d
Remove redundant "coding: utf-8" lines (#9786)
Part of #9744

Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now.

`Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`
2021-04-14 15:34:27 +01:00
Jonathan de Jong
4c3827f2c1
Enable addtional flake8-bugbear linting checks. (#9659) 2021-03-24 09:34:30 -04:00
Patrick Cloke
b7748d3c00
Import HomeServer from the proper module. (#9665) 2021-03-23 07:12:48 -04:00
Jonathan de Jong
27d2820c33
Enable flake8-bugbear, but disable most checks. (#9499)
* Adds B00 to ignored checks.
* Fixes remaining issues.
2021-03-16 14:19:27 -04:00
Erik Johnston
54a6afeee3
Cache config options in SSL verification (#9255)
Reading from the config object is *slow*.
2021-01-28 17:38:59 +00:00
Patrick Cloke
1c9a850562
Add type hints to the crypto module. (#8999) 2021-01-04 10:04:50 -05:00
Patrick Cloke
30fba62108
Apply an IP range blacklist to push and key revocation requests. (#8821)
Replaces the `federation_ip_range_blacklist` configuration setting with an
`ip_range_blacklist` setting with wider scope. It now applies to:

* Federation
* Identity servers
* Push notifications
* Checking key validitity for third-party invite events

The old `federation_ip_range_blacklist` setting is still honored if present, but
with reduced scope (it only applies to federation and identity servers).
2020-12-02 11:09:24 -05:00
Patrick Cloke
34a5696f93
Fix typos and spelling errors. (#8639) 2020-10-23 12:38:40 -04:00
Richard van der Hoff
fec6f9ac17
Fix occasional "Re-starting finished log context" from keyring (#8398)
* Fix test_verify_json_objects_for_server_awaits_previous_requests

It turns out that this wasn't really testing what it thought it was testing
(in particular, `check_context` was turning failures into success, which was
making the tests pass even though it wasn't clear they should have been.

It was also somewhat overcomplex - we can test what it was trying to test
without mocking out perspectives servers.

* Fix warnings about finished logcontexts in the keyring

We need to make sure that we finish the key fetching magic before we run the
verifying code, to ensure that we don't mess up our logcontexts.
2020-09-25 12:29:54 +01:00
Richard van der Hoff
4f3096d866 Add a comment re #1691 2020-09-21 12:34:06 +01:00
Patrick Cloke
8a4a4186de
Simplify super() calls to Python 3 syntax. (#8344)
This converts calls like super(Foo, self) -> super().

Generated with:

    sed -i "" -Ee 's/super\([^\(]+\)/super()/g' **/*.py
2020-09-18 09:56:44 -04:00
Patrick Cloke
c619253db8
Stop sub-classing object (#8249) 2020-09-04 06:54:56 -04:00
Patrick Cloke
76c43f086a
Do not assume calls to runInteraction return Deferreds. (#8133) 2020-08-20 06:39:55 -04:00
Richard van der Hoff
de6f892065
Add a comment about SSLv23_METHOD (#8043) 2020-08-07 15:14:29 +01:00
Patrick Cloke
2a89ce8cd4
Convert the crypto module to async/await. (#8003) 2020-08-03 08:29:01 -04:00
Patrick Cloke
c978f6c451
Convert federation client to async/await. (#7975) 2020-07-30 08:01:33 -04:00
Dagfinn Ilmari Mannsåker
a3f11567d9
Replace all remaining six usage with native Python 3 equivalents (#7704) 2020-06-16 08:51:47 -04:00
Richard van der Hoff
39230d2171
Clean up some LoggingContext stuff (#7120)
* Pull Sentinel out of LoggingContext

... and drop a few unnecessary references to it

* Factor out LoggingContext.current_context

move `current_context` and `set_context` out to top-level functions.

Mostly this means that I can more easily trace what's actually referring to
LoggingContext, but I think it's generally neater.

* move copy-to-parent into `stop`

this really just makes `start` and `stop` more symetric. It also means that it
behaves correctly if you manually `set_log_context` rather than using the
context manager.

* Replace `LoggingContext.alive` with `finished`

Turn `alive` into `finished` and make it a bit better defined.
2020-03-24 14:45:33 +00:00
Richard van der Hoff
c37db0211e
Share SSL contexts for non-federation requests (#7094)
Extends #5794 etc to the SimpleHttpClient so that it also applies to non-federation requests.

Fixes #7092.
2020-03-17 21:32:25 +00:00
Patrick Cloke
06eb5cae08
Remove special auth and redaction rules for aliases events in experimental room ver. (#7037) 2020-03-09 08:58:25 -04:00
Patrick Cloke
509e381afa
Clarify list/set/dict/tuple comprehensions and enforce via flake8 (#6957)
Ensure good comprehension hygiene using flake8-comprehensions.
2020-02-21 07:15:07 -05:00
Richard van der Hoff
7f93eb1903
pass room_version into compute_event_signature (#6807) 2020-01-31 13:47:43 +00:00
Richard van der Hoff
2a81393a4b Pass room_version into add_hashes_and_signatures 2020-01-30 22:15:50 +00:00
Richard van der Hoff
ba897a7590
Fix some test failures when frozen_dicts are enabled (#6642)
Fixes #4026
2020-01-06 15:22:46 +00:00
Richard van der Hoff
4b36b482e0
Fix exception when fetching notary server's old keys (#6625)
Lift the restriction that *all* the keys used for signing v2 key responses be
present in verify_keys.

Fixes #6596.
2020-01-06 12:33:56 +00:00
Michael Kaye
f85b9842f0 Don't encode object as UTF-8 string if not needed.
I believe that string formatting ~10-15 sized events will
take a proportion of CPU time.
2019-10-24 18:08:45 +01:00
Amber Brown
850dcfd2d3
Fix well-known lookups with the federation certificate whitelist (#5997) 2019-09-14 04:58:38 +10:00
Andrew Morgan
4548d1f87e
Remove unnecessary parentheses around return statements (#5931)
Python will return a tuple whether there are parentheses around the returned values or not.

I'm just sick of my editor complaining about this all over the place :)
2019-08-30 16:28:26 +01:00
Erik Johnston
f5b50d0871
Merge pull request #5895 from matrix-org/erikj/notary_key
Add config option to sign remote key query responses with a separate key.
2019-08-27 11:51:37 +01:00
Erik Johnston
7af5a63063 Fixup review comments 2019-08-23 15:36:28 +01:00
Richard van der Hoff
ef1c524bb3
Improve error msg when key-fetch fails (#5896)
There's no point doing a raise_from here, because the exception is always
logged at warn with no stacktrace in the caller. Instead, let's try to give
better messages to reduce confusion.

In particular, this means that we won't log 'Failed to connect to remote
server' when we don't even attempt to connect to the remote server due to
blacklisting.
2019-08-22 10:42:06 +01:00
Erik Johnston
97cbc96093 Only sign when we respond to remote key requests 2019-08-21 11:21:58 +01:00
Erik Johnston
5906be8589 Add config option for keys to use to sign keys
This allows servers to separate keys that are used to sign remote keys
when acting as a notary server.
2019-08-21 10:44:58 +01:00
Erik Johnston
a9bcae9f50 Share SSL options for well-known requests 2019-07-31 10:39:24 +01:00
Amber Brown
4806651744
Replace returnValue with return (#5736) 2019-07-23 23:00:55 +10:00
Richard van der Hoff
dcca56baba Add a delay to key lookup lock release to fix stack overflow
A tactical call_later here should fix #5723
2019-07-19 17:57:00 +01:00
Richard van der Hoff
c7095be913 Refactor Keyring._start_key_lookups
There's an awful lot of deferreds and dictionaries flying around here. The
whole thing can be made much simpler and achieve the same effect.
2019-07-19 17:49:19 +01:00