Commit Graph

129 Commits

Author SHA1 Message Date
Erik Johnston
d35bed8369
Don't wake up destination transaction queue if they're not due for retry. (#16223) 2023-09-04 17:14:09 +01:00
Erik Johnston
f84baecb6f
Don't reset retry timers on "valid" error codes (#16221) 2023-09-04 14:04:43 +01:00
Patrick Cloke
9ec3da06da
Bump mypy-zope & mypy. (#16188) 2023-08-29 10:38:56 -04:00
Patrick Cloke
daf11e26ef
Replace make_awaitable with AsyncMock (#16179)
Python 3.8 provides a native AsyncMock, we can replace the
homegrown version we have.
2023-08-24 19:38:46 -04:00
Eric Eastwood
1c802de626
Re-introduce the outbound federation proxy (#15913)
Allow configuring the set of workers to proxy outbound federation traffic through (`outbound_federation_restricted_to`).

This is useful when you have a worker setup with `federation_sender` instances responsible for sending outbound federation requests and want to make sure *all* outbound federation traffic goes through those instances. Before this change, the generic workers would still contact federation themselves for things like profile lookups, backfill, etc. This PR allows you to set more strict access controls/firewall for all workers and only allow the `federation_sender`'s to contact the outside world.
2023-07-18 09:49:21 +01:00
Eric Eastwood
c9bf644fa0
Revert "Federation outbound proxy" (#15910)
Revert "Federation outbound proxy (#15773)"

This reverts commit b07b14b494.
2023-07-10 11:10:20 -05:00
Eric Eastwood
b07b14b494
Federation outbound proxy (#15773)
Allow configuring the set of workers to proxy outbound federation traffic through (`outbound_federation_restricted_to`).

This is useful when you have a worker setup with `federation_sender` instances responsible for sending outbound federation requests and want to make sure *all* outbound federation traffic goes through those instances. Before this change, the generic workers would still contact federation themselves for things like profile lookups, backfill, etc. This PR allows you to set more strict access controls/firewall for all workers and only allow the `federation_sender`'s to contact the outside world.

The original code is from @erikjohnston's branches which I've gotten in-shape to merge.
2023-07-05 18:53:55 -05:00
Patrick Cloke
42aea0d8af
Add final type hint to tests.unittest. (#15072)
Adds a return type to HomeServerTestCase.make_homeserver and deal
with any variables which are no longer Any.
2023-02-14 14:03:35 -05:00
Andrew Morgan
871ff05add
Fix type hints in typing edu unit tests (#14886) 2023-01-26 10:15:50 +00:00
realtyem
854a6884d8
Modernize unit tests configuration settings for workers. (#14568)
Use the newer foo_instances configuration instead of the
deprecated flags to enable specific features (e.g. start_pushers).
2022-12-01 07:38:27 -05:00
Mathieu Velten
41461fd4d6
typing: check origin server of typing event against room's servers (#13830)
This is also using the partial state approximation if needed so we do
not block here during a fast join.

Signed-off-by: Mathieu Velten <mathieuv@matrix.org>
Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
2022-09-26 17:33:32 +02:00
reivilibre
c06b2b7142
Faster Remote Room Joins: tell remote homeservers that we are unable to authorise them if they query a room which has partial state on our server. (#13823) 2022-09-23 11:47:16 +01:00
Quentin Gliech
3dd175b628
synapse.api.auth.Auth cleanup: make permission-related methods use Requester instead of the UserID (#13024)
Part of #13019

This changes all the permission-related methods to rely on the Requester instead of the UserID. This is a first step towards enabling scoped access tokens at some point, since I expect the Requester to have scope-related informations in it.

It also changes methods which figure out the user/device/appservice out of the access token to return a Requester instead of something else. This avoids having store-related objects in the methods signatures.
2022-08-22 14:17:59 +01:00
Erik Johnston
44de53bb79
Reduce state pulled from DB due to sending typing and receipts over federation (#12964)
Reducing the amount of state we pull from the DB is useful as fetching state is expensive in terms of DB, CPU and memory.
2022-06-06 16:46:11 +01:00
Erik Johnston
888a29f412
Wait for lazy join to complete when getting current state (#12872) 2022-06-01 16:02:53 +01:00
Patrick Cloke
c52abc1cfd
Additional constants for EDU types. (#12884)
Instead of hard-coding strings in many places.
2022-05-27 07:14:36 -04:00
Sean Quah
78b99de7c2
Prefer make_awaitable over defer.succeed in tests (#12505)
When configuring the return values of mocks, prefer awaitables from
`make_awaitable` over `defer.succeed`. `Deferred`s are only awaitable
once, so it is inappropriate for a mock to return the same `Deferred`
multiple times.

Also update `run_in_background` to support functions that return
arbitrary awaitables.

Signed-off-by: Sean Quah <seanq@element.io>
2022-04-27 14:58:26 +01:00
Dirk Klimpel
9d21ecf7ce
Add type hints to tests files. (#12256) 2022-03-21 09:43:16 -04:00
Patrick Cloke
02d708568b
Replace assertEquals and friends with non-deprecated versions. (#12092) 2022-02-28 07:12:29 -05:00
Richard van der Hoff
e24ff8ebe3
Remove HomeServer.get_datastore() (#12031)
The presence of this method was confusing, and mostly present for backwards
compatibility. Let's get rid of it.

Part of #11733
2022-02-23 11:04:02 +00:00
Patrick Cloke
4054dfa409
Add type hints for event streams. (#10856) 2021-09-21 13:34:26 -04:00
Dagfinn Ilmari Mannsåker
47e28b4031
Ignore EDUs for rooms we're not in (#10317) 2021-07-06 14:31:13 +01:00
Erik Johnston
3e831f24ff
Don't hammer the database for destination retry timings every ~5mins (#10036) 2021-05-21 17:57:08 +01:00
Jonathan de Jong
4b965c862d
Remove redundant "coding: utf-8" lines (#9786)
Part of #9744

Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now.

`Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`
2021-04-14 15:34:27 +01:00
Patrick Cloke
0b3112123d
Use mock from the stdlib. (#9772) 2021-04-09 13:44:38 -04:00
Eric Eastwood
0a00b7ff14
Update black, and run auto formatting over the codebase (#9381)
- Update black version to the latest
 - Run black auto formatting over the codebase
    - Run autoformatting according to [`docs/code_style.md
`](80d6dc9783/docs/code_style.md)
 - Update `code_style.md` docs around installing black to use the correct version
2021-02-16 22:32:34 +00:00
Richard van der Hoff
394516ad1b Remove spurious "SynapseRequest" result from `make_request"
This was never used, so let's get rid of it.
2020-12-15 22:35:40 +00:00
Richard van der Hoff
ed5172852a
Merge pull request #8858 from matrix-org/rav/sso_uia
UIA: offer only available auth flows
2020-12-02 20:06:53 +00:00
Richard van der Hoff
7ea85302f3 fix up various test cases
A few test cases were relying on being able to mount non-client servlets on the
test resource. it's better to give them their own Resources.
2020-12-02 16:30:01 +00:00
Patrick Cloke
30fba62108
Apply an IP range blacklist to push and key revocation requests. (#8821)
Replaces the `federation_ip_range_blacklist` configuration setting with an
`ip_range_blacklist` setting with wider scope. It now applies to:

* Federation
* Identity servers
* Push notifications
* Checking key validitity for third-party invite events

The old `federation_ip_range_blacklist` setting is still honored if present, but
with reduced scope (it only applies to federation and identity servers).
2020-12-02 11:09:24 -05:00
Richard van der Hoff
acfe3b3065 Remove redundant HomeserverTestCase.render 2020-11-16 18:24:08 +00:00
Patrick Cloke
629a951b49
Move additional tasks to the background worker, part 4 (#8513) 2020-10-13 08:20:32 -04:00
reivilibre
576bc37d31
Catch-up after Federation Outage (split, 4): catch-up loop (#8272) 2020-09-15 09:07:19 +01:00
Erik Johnston
c9dbee50ae
Fixup pusher pool notifications (#8287)
`pusher_pool.on_new_notifications` expected a min and max stream ID, however that was not what we were passing in. Instead, let's just pass it the current max stream ID and have it track the last stream ID it got passed.

I believe that it mostly worked as we called the function for every event. However, it would break for events that got persisted out of order, i.e, that were persisted but the max stream ID wasn't incremented as not all preceding events had finished persisting, and push for that event would be delayed until another event got pushed to the effected users.
2020-09-09 16:56:08 +01:00
Erik Johnston
dc9dcdbd59 Revert "Fixup pusher pool notifications"
This reverts commit e7fd336a53.
2020-09-09 16:19:22 +01:00
Erik Johnston
e7fd336a53 Fixup pusher pool notifications 2020-09-09 16:17:50 +01:00
Patrick Cloke
cef00211c8
Allow for make_awaitable's return value to be re-used. (#8261) 2020-09-08 07:26:55 -04:00
Patrick Cloke
e00816ad98
Do not yield on awaitables in tests. (#8193) 2020-08-27 17:24:46 -04:00
Patrick Cloke
6fe12c9512
Do not propagate typing notifications from shadow-banned users. (#8176) 2020-08-26 12:05:36 -04:00
Patrick Cloke
4c6c56dc58
Convert simple_select_one and simple_select_one_onecol to async (#8162) 2020-08-26 07:19:32 -04:00
Patrick Cloke
050e20e7ca
Convert some of the general database methods to async (#8100) 2020-08-17 12:18:01 -04:00
Patrick Cloke
5ecc8b5825
Convert devices database to async/await. (#8069) 2020-08-12 10:51:42 -04:00
Patrick Cloke
d68e10f308
Convert account data, device inbox, and censor events databases to async/await (#8063) 2020-08-12 09:29:06 -04:00
Patrick Cloke
d4a7829b12
Convert synapse.api to async/await (#8031) 2020-08-06 08:30:06 -04:00
Erik Johnston
9006e125af Fix tests 2020-07-15 15:47:27 +01:00
Patrick Cloke
3630825612
Convert the typing handler to async/await. (#7679) 2020-06-17 10:37:59 -04:00
Erik Johnston
1531b214fc
Add ability to wait for replication streams (#7542)
The idea here is that if an instance persists an event via the replication HTTP API it can return before we receive that event over replication, which can lead to races where code assumes that persisting an event immediately updates various caches (e.g. current state of the room).

Most of Synapse doesn't hit such races, so we don't do the waiting automagically, instead we do so where necessary to avoid unnecessary delays. We may decide to change our minds here if it turns out there are a lot of subtle races going on.

People probably want to look at this commit by commit.
2020-05-22 14:21:54 +01:00
Richard van der Hoff
3e99528f2b
Store room version on invite (#6983)
When we get an invite over federation, store the room version in the rooms table.

The general idea here is that, when we pull the invite out again, we'll want to know what room_version it belongs to (so that we can later redact it if need be). So we need to store it somewhere...
2020-02-26 16:58:33 +00:00
Patrick Cloke
509e381afa
Clarify list/set/dict/tuple comprehensions and enforce via flake8 (#6957)
Ensure good comprehension hygiene using flake8-comprehensions.
2020-02-21 07:15:07 -05:00
Richard van der Hoff
b58d17e44f Refactor the membership check methods in Auth
these were getting a bit unwieldy, so let's combine `check_joined_room` and
`check_user_was_in_room` into a single `check_user_in_room`.
2020-02-18 23:21:44 +00:00