Commit Graph

77 Commits

Author SHA1 Message Date
Eric Eastwood
92d21faf12
Instrument /messages for understandable traces in Jaeger ()
In Jaeger:

 - Before: huge list of uncategorized database calls
 - After: nice and collapsible into units of work
2022-08-03 10:57:38 -05:00
Erik Johnston
1e453053cb
Rename storage classes () 2022-05-31 12:17:50 +00:00
Jacek Kuśnierz
c4f548e05d
Don't return end from /messages if there are no more events ()
Signed-off-by: Jacek Kusnierz <jacek.kusnierz@tum.de>
2022-05-30 21:03:52 +01:00
Erik Johnston
4660d9fdcf
Fix up state_store naming () 2022-05-25 12:59:04 +01:00
Brendan Abolivier
4cc4229cd7
Prevent expired events from being filtered out when retention is disabled ()
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2022-05-23 17:18:23 +00:00
Andrew Morgan
83be72d76c
Add StreamKeyType class and replace string literals with constants () 2022-05-16 15:35:31 +00:00
Eric Eastwood
84facf769e
Fix /messages throwing a 500 when querying for non-existent room ()
Fix https://github.com/matrix-org/synapse/issues/12678

Complement test added:  https://github.com/matrix-org/complement/pull/369

**Before:** 500 internal server error

**After:** According to the [spec](https://spec.matrix.org/latest/client-server-api/#get_matrixclientv3roomsroomidmessages), calling `/messages` against a non-existent `room_id` should throw a 403 forbidden (since you're not part of the room). This also matches the behavior before https://github.com/matrix-org/synapse/pull/12370 which regressed Synapse to the 500 behavior.
```json
{
    "errcode": "M_FORBIDDEN",
    "error": "User @test:my.synapse.server not in room !dne:my.synapse.server, and room previews are disabled"
}
```
2022-05-10 23:39:14 -05:00
Eric Eastwood
793d03e2c5
Generate historic pagination token for /messages when no ?from token provided () 2022-04-06 11:40:28 +01:00
Patrick Cloke
8fe930c215
Move get_bundled_aggregations to relations handler. ()
The get_bundled_aggregations code is fairly high-level and uses
a lot of store methods, we move it into the handler as that seems
like a better fit.
2022-03-18 17:49:32 +00:00
Patrick Cloke
fc9bd620ce
Add a relations handler to avoid duplication. ()
Adds a handler layer between the REST and datastore layers for relations.
2022-03-16 10:39:15 -04:00
Sean Quah
605d161d7d
Add cancellation support to ReadWriteLock ()
Also convert `ReadWriteLock` to use async context managers.

Signed-off-by: Sean Quah <seanq@element.io>
2022-03-14 18:49:07 +00:00
Patrick Cloke
1d11b452b7
Use the proper serialization format when bundling aggregations. ()
This ensures that the `latest_event` field of the bundled aggregation
for threads uses the same format as the other events in the response.
2022-03-03 10:43:06 -05:00
Richard van der Hoff
e24ff8ebe3
Remove HomeServer.get_datastore() ()
The presence of this method was confusing, and mostly present for backwards
compatibility. Let's get rid of it.

Part of 
2022-02-23 11:04:02 +00:00
Patrick Cloke
68acb0a29d
Include whether the requesting user has participated in a thread. ()
Per updates to MSC3440.

This is implement as a separate method since it needs to be cached
on a per-user basis, instead of a per-thread basis.
2022-01-18 11:38:57 -05:00
Patrick Cloke
6bf81a7a61
Bundle aggregations outside of the serialization method. ()
This makes the serialization of events synchronous (and it no
longer access the database), but we must manually calculate and
provide the bundled aggregations.

Overall this should cause no change in behavior, but is prep work
for other improvements.
2022-01-07 09:10:46 -05:00
Patrick Cloke
dd47788752
Do not bundle aggregations for APIs which shouldn't include them. ()
And make bundling aggregations opt-in, instead of opt-out to avoid
having APIs to include extraneous data (and being much heavier than
necessary).
2021-12-20 14:14:38 -05:00
Richard van der Hoff
b1ecd19c5d
Fix 'delete room' admin api to work on incomplete rooms ()
If, for some reason, we don't have the create event, we should still be able to
purge a room.
2021-12-07 11:37:54 +00:00
Dirk Klimpel
8840a7b7f1
Convert delete room admin API to async endpoint ()
Signed-off-by: Dirk Klimpel dirk@klimpel.org
2021-11-12 12:35:31 +00:00
Patrick Cloke
a19d01c3d9
Support filtering by relations per MSC3440 ()
Adds experimental support for `relation_types` and `relation_senders`
fields for filters.
2021-11-09 08:10:58 -05:00
Patrick Cloke
19d5dc6931
Refactor Filter to handle fields according to data being filtered. ()
This avoids filtering against fields which cannot exist on an
event source. E.g. presence updates don't have a room.
2021-10-27 11:26:30 -04:00
Patrick Cloke
5573133348
Move experimental & retention config out of the server module. () 2021-10-15 14:30:48 +00:00
Patrick Cloke
94b620a5ed
Use direct references for configuration variables (part 6). () 2021-09-29 06:44:15 -04:00
Patrick Cloke
eb2c7e51c4
Clean-up type hints in server config ()
By using attrs instead of dicts to store configuration.

Also updates some of the attrs classes to use proper type
hints and auto_attribs.
2021-09-28 09:24:40 -04:00
Patrick Cloke
b3590614da
Require type hints in the handlers module. ()
Adds missing type hints to methods in the synapse.handlers
module and requires all methods to have type hints there.

This also removes the unused construct_auth_difference method
from the FederationHandler.
2021-09-20 08:56:23 -04:00
Patrick Cloke
01c88a09cd
Use direct references for some configuration variables ()
Instead of proxying through the magic getter of the RootConfig
object. This should be more performant (and is more explicit).
2021-09-13 13:07:12 -04:00
Jonathan de Jong
98aec1cc9d
Use inline type hints in handlers/ and rest/. () 2021-07-16 18:22:36 +01:00
Jonathan de Jong
4b965c862d
Remove redundant "coding: utf-8" lines ()
Part of 

Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now.

`Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`
2021-04-14 15:34:27 +01:00
Patrick Cloke
7fdc6cefb3
Fix additional type hints. ()
Type hint fixes due to Twisted 21.2.0 adding type hints.
2021-03-09 07:41:32 -05:00
Eric Eastwood
0a00b7ff14
Update black, and run auto formatting over the codebase ()
- Update black version to the latest
 - Run black auto formatting over the codebase
    - Run autoformatting according to [`docs/code_style.md
`](80d6dc9783/docs/code_style.md)
 - Update `code_style.md` docs around installing black to use the correct version
2021-02-16 22:32:34 +00:00
Richard van der Hoff
a090b86209
Add force_purge option to delete-room admin api. () 2020-11-30 16:48:12 +00:00
Patrick Cloke
629a951b49
Move additional tasks to the background worker, part 4 () 2020-10-13 08:20:32 -04:00
Patrick Cloke
c9c0ad5e20
Remove the deprecated Handlers object ()
All handlers now available via get_*_handler() methods on the HomeServer.
2020-10-09 07:24:34 -04:00
Erik Johnston
7941372ec8
Make token serializing/deserializing async ()
The idea is that in future tokens will encode a mapping of instance to position. However, we don't want to include the full instance name in the string representation, so instead we'll have a mapping between instance name and an immutable integer ID in the DB that we can use instead. We'll then do the lookup when we serialize/deserialize the token (we could alternatively pass around an `Instance` type that includes both the name and ID, but that turns out to be a lot more invasive).
2020-09-30 20:29:19 +01:00
Erik Johnston
ea70f1c362
Various clean ups to room stream tokens. () 2020-09-29 21:48:33 +01:00
Patrick Cloke
babc027543
Fix a bad merge from release-v1.20.0. () 2020-09-18 12:54:04 -04:00
Patrick Cloke
00db7786de Synapse 1.20.0rc5 (2020-09-18)
==============================
 
 In addition to the below, Synapse 1.20.0rc5 also includes the bug fix that was included in 1.19.3.
 
 Features
 --------
 
 - Add flags to the `/versions` endpoint for whether new rooms default to using E2EE. ([\#8343](https://github.com/matrix-org/synapse/issues/8343))
 
 Bugfixes
 --------
 
 - Fix rate limiting of federation `/send` requests. ([\#8342](https://github.com/matrix-org/synapse/issues/8342))
 - Fix a longstanding bug where back pagination over federation could get stuck if it failed to handle a received event. ([\#8349](https://github.com/matrix-org/synapse/issues/8349))
 
 Internal Changes
 ----------------
 
 - Blacklist [MSC2753](https://github.com/matrix-org/matrix-doc/pull/2753) SyTests until it is implemented. ([\#8285](https://github.com/matrix-org/synapse/issues/8285))
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEEF3tZXk38tRDFVnUIM/xY9qcRMEgFAl9kzk8ACgkQM/xY9qcR
 MEim6A//aERkhyLGRlGpLd37lCyFQCeffTMH1rTvu04iIBQBaUZ6g7CYWOpK43zT
 U8kt379+5OShjdAXs/X4XP+ucdHVbrwsRSP3hBS/fFLiDT0fJgP8uiSf5QqO6NnT
 OqDyXYjcXvj/c6tMKglVtsdh8u4hFwNZjGPMGG68IzJu14uEhnD100cL9jSB9bLB
 ongWpsQzzdGBpJPSFRjv9dCUSeRbzyUdl1t0uqzrNqyN9s/JnzFTn7ZYo6y3lnSS
 dHGVMMo/12M2PkbBHnbJVvDY5Q/R7ZxyXlpz0gvSNOQIw8FqYFnuB0Niy5dQhXSR
 Sy5h4qbczLxqbql1x+lmzeQm4ZMORsW/Tl4C3z6yK6OYaOCJHIf9en4DplTSTqp1
 t+85JxWR2wH10d99YHBpaYKmkVovpwgchrO4YWrtXljUFAhhavzf+YiAdOHYT52s
 RDsDLsvjMbxEHsz4cHfycmshYhjzjb340wkoDXuQpj0zrO99d+Zd83xdK8pS0UQn
 OaljLRAd/5iBjTSyZPSrB1U5141OzlM3QZVJzaYAnP12yhR9eaX2twSCk+lPYOWd
 nhLJjNnj1B1XSGArthuE5NLyEiCPz6KyN2RhO0EOx5YjZN9TwH7LS9upyNFe1nN1
 GIhO5gz+jWLuBZE3xzRNjJyCx/I/LolpCwGMvKDu6638rpsbrPs=
 =tT5/
 -----END PGP SIGNATURE-----

Merge tag 'v1.20.0rc5' into develop

Synapse 1.20.0rc5 (2020-09-18)
==============================

In addition to the below, Synapse 1.20.0rc5 also includes the bug fix that was included in 1.19.3.

Features
--------

- Add flags to the `/versions` endpoint for whether new rooms default to using E2EE. ([\#8343](https://github.com/matrix-org/synapse/issues/8343))

Bugfixes
--------

- Fix rate limiting of federation `/send` requests. ([\#8342](https://github.com/matrix-org/synapse/issues/8342))
- Fix a longstanding bug where back pagination over federation could get stuck if it failed to handle a received event. ([\#8349](https://github.com/matrix-org/synapse/issues/8349))

Internal Changes
----------------

- Blacklist [MSC2753](https://github.com/matrix-org/matrix-doc/pull/2753) SyTests until it is implemented. ([\#8285](https://github.com/matrix-org/synapse/issues/8285))
2020-09-18 11:17:58 -04:00
Erik Johnston
43f2b67e4d
Intelligently select extremities used in backfill. ()
Instead of just using the most recent extremities let's pick the
ones that will give us results that the pagination request cares about,
i.e. pick extremities only if they have a smaller depth than the
pagination token.

This is useful when we fail to backfill an extremity, as we no longer
get stuck requesting that same extremity repeatedly.
2020-09-18 14:25:52 +01:00
Erik Johnston
fe8ed1b46f
Make StreamToken.room_key be a RoomStreamToken instance. () 2020-09-11 12:22:55 +01:00
Erik Johnston
1553adc831
Fix mypy error on develop () 2020-09-08 17:43:31 +01:00
Erik Johnston
0f545e6b96
Clean up types for PaginationConfig ()
This removes `SourcePaginationConfig` and `get_pagination_rows`. The reasoning behind this is that these generic classes/functions erased the types of the IDs it used (i.e. instead of passing around `StreamToken` it'd pass in e.g. `token.room_key`, which don't have uniform types).
2020-09-08 15:00:17 +01:00
Patrick Cloke
c619253db8
Stop sub-classing object () 2020-09-04 06:54:56 -04:00
Erik Johnston
be16ee59a8
Add type hints to more handlers () 2020-09-03 22:02:29 +01:00
Patrick Cloke
d2ac767de2
Convert ReadWriteLock to async/await. () 2020-08-28 16:47:11 -04:00
Brendan Abolivier
420484a334
Allow capping a room's retention policy () 2020-08-24 18:21:04 +01:00
Patrick Cloke
e19de43eb5
Convert streams to async. () 2020-08-04 07:21:47 -04:00
Patrick Cloke
98c4e35e3c
Convert the device message and pagination handlers to async/await. () 2020-06-16 08:06:17 -04:00
Patrick Cloke
bd6dc17221
Replace iteritems/itervalues/iterkeys with native versions. () 2020-06-15 07:03:36 -04:00
Richard van der Hoff
2fb7794e60
Merge pull request from matrix-org/rav/list_room_aliases_peekable
Make room alias lists peekable
2020-02-19 11:19:11 +00:00
Richard van der Hoff
a0a1fd0bec Add allow_departed_users param to check_in_room_or_world_readable
... and set it everywhere it's called.

while we're here, rename it for consistency with `check_user_in_room` (and to
help check that I haven't missed any instances)
2020-02-19 08:52:51 +00:00
Brendan Abolivier
f31a94a6dd
Fix log in message retention purge jobs 2020-02-18 17:29:57 +00:00