Commit Graph

489 Commits

Author SHA1 Message Date
Dirk Klimpel
1847d027e6
Add type hints to synapse/storage/databases/main/transactions.py (#11589) 2021-12-16 14:59:35 -05:00
Sean Quah
c7fe32edb4
Add type hints to synapse/storage/databases/main/room.py (#11575) 2021-12-15 18:00:48 +00:00
Patrick Cloke
f901f8b70e
Require Collections as the parameters for simple_* methods. (#11580)
Instead of Iterable since the generators are not allowed due
to the potential for their re-use.
2021-12-15 17:00:50 +00:00
reivilibre
17886d2603
Add experimental support for MSC3202: allowing application services to masquerade as specific devices. (#11538) 2021-12-15 10:40:52 +00:00
Sean Quah
ecfcd9bbbe
Add type hints to synapse/storage/databases/main/e2e_room_keys.py (#11549) 2021-12-14 17:46:47 +00:00
Sean Quah
a4dce5b53d
Remove redundant COALESCE()s around COUNT()s in database queries (#11570)
`COUNT()` never returns `NULL`. A `COUNT(*)` over 0 rows is 0 and a
`COUNT(NULL)` is also 0.
2021-12-14 12:34:30 +00:00
Richard van der Hoff
ff6fd52160
checks for generators in database functions (#11564)
A couple of safety-checks to hopefully stop people doing what I just did, and create a storage
function which only works the first time it is called (and not when it is re-run due to a database
concurrency error or similar).
2021-12-13 19:01:27 +00:00
Sean Quah
5305a5e881
Type hint the constructors of the data store classes (#11555) 2021-12-13 17:05:00 +00:00
Sean Quah
1abfb15f07
Add type hints to synapse/storage/databases/main/end_to_end_keys.py (#11551) 2021-12-13 16:28:26 +00:00
Sean Quah
6da8591f2e
Add type hints to synapse/storage/databases/main/account_data.py (#11546) 2021-12-13 16:28:10 +00:00
reivilibre
e5cdb9e233
Make get_device return None if the device doesn't exist rather than raising an exception. (#11565)
Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
2021-12-13 15:39:43 +00:00
Richard van der Hoff
f0562183e7
skip some dict munging in event persistence (#11560)
Create a new dict helper method `simple_insert_many_values_txn`, which takes
raw row values, rather than {key=>value} dicts. This saves us a bunch of dict
munging, and makes it easier to use generators rather than creating
intermediate lists and dicts.
2021-12-10 15:02:33 +00:00
Richard van der Hoff
86e7a6d16e
Stop populating state_events.prev_state (#11558)
this field is never read, so we may as well stop populating it.
2021-12-10 14:13:23 +00:00
Patrick Cloke
9562f0c2f1
Ensure emails are canonicalized before fetching associated user. (#11547)
This should fix pushers with an email in non-canonical form is used as
the pushkey.
2021-12-10 07:17:28 -05:00
Patrick Cloke
3b8872299a
Do not allow cross-room relations, per MSC2674. (#11516) 2021-12-09 13:16:01 -05:00
Patrick Cloke
d93362d87f
Add a constant for receipt types (m.read). (#11531)
And expand some type hints in the receipts storage module.
2021-12-08 12:26:29 -05:00
Richard van der Hoff
ff7cc17b57
Improve log messages for stream ids (#11536)
Somehow I'd managed to get my database in a pickle with stream ids. These
changes were useful to debug.
2021-12-08 14:15:14 +00:00
Hubert Chathi
8541809cb9
Send and handle cross-signing messages using the stable prefix. (#10520) 2021-12-08 10:01:38 +00:00
Quentin Gliech
a15a893df8
Save the OIDC session ID (sid) with the device on login (#11482)
As a step towards allowing back-channel logout for OIDC.
2021-12-06 12:43:06 -05:00
Dirk Klimpel
8b4b153c9e
Add admin API to get some information about federation status (#11407) 2021-12-06 16:59:50 +00:00
Richard van der Hoff
5640992d17
Disambiguate queries on state_key (#11497)
We're going to add a `state_key` column to the `events` table, so we need to
add some disambiguation to queries which use it.
2021-12-02 22:42:58 +00:00
David Robertson
d26808dd85
Comments on the /sync tentacles (#11494)
This mainly consists of docstrings and inline comments. There are one or two type annotations and variable renames thrown in while I was here.

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2021-12-02 20:58:32 +00:00
reivilibre
435f044807
Add type annotations to tests.storage.test_appservice. (#11488) 2021-12-02 15:30:05 +00:00
Eric Eastwood
a6f1a3abec
Add MSC3030 experimental client and federation API endpoints to get the closest event to a given timestamp (#9445)
MSC3030: https://github.com/matrix-org/matrix-doc/pull/3030

Client API endpoint. This will also go and fetch from the federation API endpoint if unable to find an event locally or we found an extremity with possibly a closer event we don't know about.
```
GET /_matrix/client/unstable/org.matrix.msc3030/rooms/<roomID>/timestamp_to_event?ts=<timestamp>&dir=<direction>
{
    "event_id": ...
    "origin_server_ts": ...
}
```

Federation API endpoint:
```
GET /_matrix/federation/unstable/org.matrix.msc3030/timestamp_to_event/<roomID>?ts=<timestamp>&dir=<direction>
{
    "event_id": ...
    "origin_server_ts": ...
}
```

Co-authored-by: Erik Johnston <erik@matrix.org>
2021-12-02 01:02:20 -06:00
Marcus
a9481223d1
Improved push typing (#11409)
Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
2021-11-30 11:49:20 +00:00
Eric Eastwood
fb58611d21
Refactor backfilled into specific behavior function arguments (_persist_events_and_state_updates) (#11417)
Part of https://github.com/matrix-org/synapse/issues/11300

Call stack:

 - `_persist_events_and_state_updates` (added `use_negative_stream_ordering`)
    - `_persist_events_txn`
       - `_update_room_depths_txn` (added `update_room_forward_stream_ordering`)
       - `_update_metadata_tables_txn`
          - `_store_room_members_txn` (added `inhibit_local_membership_updates`)

Using keyword-only arguments (`*`) to reduce the mistakes from `backfilled` being left as a positional argument somewhere and being interpreted wrong by our new arguments.
2021-11-29 16:01:54 -06:00
Sean Quah
ffd858aa68
Add type hints to synapse/storage/databases/main/events_worker.py (#11411)
Also refactor the stream ID trackers/generators a bit and try to
document them better.
2021-11-26 18:41:31 +00:00
reivilibre
1d8b80b334
Support expiry of refresh tokens and expiry of the overall session when refresh tokens are in use. (#11425) 2021-11-26 14:27:14 +00:00
Sean Quah
c675a18071
Track ongoing event fetches correctly (again) (#11376)
The previous fix for the ongoing event fetches counter
(8eec25a1d9) was both insufficient and
incorrect.

When the database is unreachable, `_do_fetch` never gets run and so
`_event_fetch_ongoing` is never decremented.

The previous fix also moved the `_event_fetch_ongoing` decrement outside
of the `_event_fetch_lock` which allowed race conditions to corrupt the
counter.
2021-11-26 13:47:24 +00:00
Brendan Abolivier
0d88c4f903
Improve performance of remove_{hidden,deleted}_devices_from_device_inbox (#11421)
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2021-11-25 15:14:54 +00:00
Patrick Cloke
55669bd3de
Add missing type hints to config base classes (#11377) 2021-11-23 15:21:19 +00:00
Shay
7cebaf9644
Remove code invalidated by deprecated config flag 'trust_identity_servers_for_password_resets' (#11395)
* remove background update code related to deprecated config flag

* changelog entry

* update changelog

* Delete 11394.removal

Duplicate, wrong number

* add no-op background update and change newfragment so it will be consolidated with associated work

* remove unused code

* Remove code associated with deprecated flag from legacy docker dynamic config file

Co-authored-by: reivilibre <oliverw@matrix.org>
2021-11-23 06:46:40 -08:00
Patrick Cloke
3d893b8cf2
Store arbitrary relations from events. (#11391)
Instead of only known relation types. This also reworks the background
update for thread relations to crawl events and search for any relation
type, not just threaded relations.
2021-11-22 12:01:47 -05:00
Hubert Chathi
eca7cffb73
Keep fallback key marked as used if it's re-uploaded (#11382) 2021-11-19 11:40:12 +00:00
Dirk Klimpel
81b18fe5c0
Add dedicated admin API for blocking a room (#11324) 2021-11-18 17:43:49 +00:00
reivilibre
539e441399
Use auto_attribs for RefreshTokenLookupResult (#11386) 2021-11-18 14:40:26 +00:00
Patrick Cloke
4bd54b263e
Do not allow MSC3440 threads to fork threads (#11161)
Adds validation to the Client-Server API to ensure that
the potential thread head does not relate to another event
already. This results in not allowing a thread to "fork" into
other threads.

If the target event is unknown for some reason (maybe it isn't
visible to your homeserver), but is the target of other events
it is assumed that the thread can be created from it. Otherwise,
it is rejected as an unknown event.
2021-11-18 13:43:09 +00:00
Patrick Cloke
24b61f379a
Add ability to un-shadow-ban via the admin API. (#11347) 2021-11-16 12:43:53 +00:00
David Robertson
e605e4b8f2
Database storage profile passes mypy (#11342)
It already seems to pass mypy. I wonder what changed, given that it was
on the exclusion list. So this commit consists of me ensuring
`--disallow-untyped-defs` passes and a minor fixup to a function that
returned either `True` or `None`.
2021-11-15 12:59:33 +00:00
David Robertson
5562ce6a53
Get directory db file to pass mypy (#11339) 2021-11-15 12:59:05 +00:00
David Robertson
fe58672546
Annotations for state_deltas.py (#11316)
I was sad that I couldn't do better for
`_curr_state_delta_stream_cache`. At least it's explicitly called out in
a comment with #TODO.
2021-11-12 20:24:12 +00:00
David Robertson
bea815cec8
Test room alias deletion (#11327)
* Prefer `HTTPStatus` over plain `int`

This is an Opinion that no-one has seemed to object to yet.

* `--disallow-untyped-defs` for `tests.rest.client.test_directory`
* Improve synapse's annotations for deleting aliases
* Test case for deleting a room alias
* Changelog
2021-11-12 19:56:00 +00:00
Shay
0bcae8ad56
Change display names/avatar URLs to None if they contain null bytes before storing in DB (#11230)
* change display names/avatar URLS to None if they contain null bytes

* add changelog

* add POC test, requested changes

* add a saner test and remove old one

* update test to verify that display name has been changed to None

* make test less fragile
2021-11-12 10:38:24 -08:00
Patrick Cloke
9b90b9454b
Add type hints to media repository storage module (#11311) 2021-11-12 11:05:26 -05:00
David Robertson
6f8f3d4bc5
Attempt to annotate events_forward_extremities (#11314)
* Make DataStore inherit from EventForwardExtremitiesStore before CacheInvalidationWorkerStore

the former implicitly inherits from the latter, so they should be
ordered like this when used.
2021-11-12 15:58:17 +00:00
David Robertson
c99da2d079
Annotations for user_erasure_store (#11313)
I'm not sure why this was excluded---it seemed to be passing for me. But
it's easy enough to fixup.
2021-11-11 19:22:19 +00:00
David Robertson
6a605f4a77
Get db signatures file to pass mypy (#11312) 2021-11-11 17:04:44 +00:00
David Robertson
8dc666f785
Correct type hint for room_batch.py (#11310)
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2021-11-11 16:49:28 +00:00
Patrick Cloke
64ef25391d
Add type hints to some storage classes (#11307) 2021-11-11 08:47:31 -05:00
Andrew Morgan
a026695083
Clarifications and small fixes to to-device related code (#11247)
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2021-11-09 14:31:15 +00:00