Commit Graph

132 Commits

Author SHA1 Message Date
Patrick Cloke
d8cc86eff4
Remove redundant types from comments. ()
Remove type hints from comments which have been added
as Python type hints. This helps avoid drift between comments
and reality, as well as removing redundant information.

Also adds some missing type hints which were simple to fill in.
2022-11-16 15:25:24 +00:00
Patrick Cloke
efd108b45d
Accept & store thread IDs for receipts (implement MSC3771). ()
Updates the `/receipts` endpoint and receipt EDU handler to parse a
`thread_id` from the body and insert it in the database.
2022-09-23 14:33:28 +00:00
reivilibre
d3d9ca156e
Cancel the processing of key query requests when they time out. () 2022-09-07 12:03:32 +01:00
David Robertson
c99b511db9
Fix destination_is errors seen in sentry. ()
* Rename test_fedclient to match its source file
* Require at least one destination to be truthy
* Explicitly validate user ID in profile endpoint GETs
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2022-06-14 18:28:26 +01:00
Patrick Cloke
e541bb9eed
Rework stream token to stop caring about groups. () 2022-05-31 07:42:50 -04:00
Patrick Cloke
49f06866e4
Remove backing code for groups/communities ()
Including handlers, configuration code, appservice support, and
the GroupID construct.
2022-05-26 09:04:34 -04:00
Brendan Abolivier
4cc4229cd7
Prevent expired events from being filtered out when retention is disabled ()
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2022-05-23 17:18:23 +00:00
Andrew Morgan
83be72d76c
Add StreamKeyType class and replace string literals with constants () 2022-05-16 15:35:31 +00:00
David Robertson
aec69d2481
Another batch of type annotations () 2022-05-13 12:35:31 +01:00
Sean Quah
31c1209c50
Make StreamToken and RoomStreamToken methods propagate cancellations ()
`StreamToken.from_string` and `RoomStreamToken.parse` are both async
methods that could be cancelled. These methods must not replace
`CancelledError`s with `SynapseError`s.

Signed-off-by: Sean Quah <seanq@element.io>
2022-04-05 16:56:52 +01:00
Eric Eastwood
5218fe7670
Explain how to decipher live and historic pagination tokens () 2022-04-05 10:57:09 +01:00
Andrew Morgan
d8d0271977
Send device list updates to application services (MSC3202) - part 1 ()
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2022-03-30 14:39:27 +01:00
David Robertson
872dbb0181
Correct check_username_for_spam annotations and docs ()
* Formally type the UserProfile in user searches
* export UserProfile in synapse.module_api
* Update docs

Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
2022-03-18 13:51:41 +00:00
Dirk Klimpel
7c82da27aa
Add type hints to synapse/storage/databases/main () 2022-02-21 16:03:06 +00:00
Patrick Cloke
10a88ba91c
Use auto_attribs/native type hints for attrs classes. () 2022-01-13 13:49:28 +00:00
Patrick Cloke
201c48c8de
Remove a Python 2-ism and improve type hints. ()
On Python 2, indexing a byte-string gives back a byte-string,
while on Python 3 it gives back the ASCII equivalent as an int.
2022-01-06 18:08:48 +00:00
Patrick Cloke
cbd82d0b2d
Convert all namedtuples to attrs. ()
To improve type hints throughout the code.
2021-12-30 18:47:12 +00:00
Sean Quah
ecfcd9bbbe
Add type hints to synapse/storage/databases/main/e2e_room_keys.py () 2021-12-14 17:46:47 +00:00
David Robertson
4c96ce396e
Misc typing fixes for tests, part 1 of N ()
* Annotate HomeserverTestCase.servlets
* Correct annotation of federation_auth_origin
* Use AnyStr custom_headers instead of a Union

This allows (str, str) and (bytes, bytes).
This disallows (str, bytes) and (bytes, str)

* DomainSpecificString.SIGIL is a ClassVar
2021-11-12 15:50:54 +00:00
Patrick Cloke
5cace20bf1
Add missing type hints to synapse.app. () 2021-11-10 15:06:54 -05:00
Andrew Morgan
aa2c027792
Remove unnecessary parentheses around tuples returned from methods () 2021-09-23 11:59:07 +01:00
Sean Quah
9391de3f37
Fix /initialSync error due to unhashable RoomStreamToken ()
The deprecated /initialSync endpoint maintains a cache of responses,
using parameter values as part of the cache key. When a `from` or `to`
parameter is specified, it gets converted into a `StreamToken`, which
contains a `RoomStreamToken` and forms part of the cache key.
`RoomStreamToken`s need to be made hashable for this to work.
2021-09-22 14:43:26 +01:00
David Robertson
319b8b6bef
Name the type of token in "Invalid token" messages ()
I had one of these error messages yesterday and assumed it was an
invalid auth token (because that was an HTTP query parameter in the
test) I was working on. In fact, it was an invalid next batch token for
syncing.
2021-09-14 11:25:05 +01:00
reivilibre
524b8ead77
Add types to synapse.util. () 2021-09-10 17:03:18 +01:00
Jason Robinson
c2000ab35b
Add get_userinfo_by_id method to ModuleApi ()
Makes it easier to fetch user details in for example spam checker modules, without needing to use api._store or figure out database interactions.

Signed-off-by: Jason Robinson <jasonr@matrix.org>
2021-08-04 10:40:25 +00:00
Jonathan de Jong
95e47b2e78
[pyupgrade] synapse/ ()
This PR is tantamount to running 
```
pyupgrade --py36-plus --keep-percent-format `find synapse/ -type f -name "*.py"`
```

Part of 
2021-07-19 15:28:05 +01:00
Jonathan de Jong
bf72d10dbf
Use inline type hints in various other places (in synapse/) () 2021-07-15 11:02:43 +01:00
Patrick Cloke
30b56f6925
Add type hints to get_domain_from_id and get_localpart_from_id. () 2021-07-13 12:08:47 -04:00
Richard van der Hoff
fcf3c7032b
Ensure that we do not cache empty sync responses after a timeout ()
Fixes  by telling the ResponseCache not to cache the /sync response if the next_batch param is the same as the since token.
2021-06-17 16:23:11 +01:00
Marcus
8070b893db
update black to 21.6b0 ()
Reformat all files with the new version.

Signed-off-by: Marcus Hoffmann <bubu@bubu1.eu>
2021-06-17 15:20:06 +01:00
Erik Johnston
a15c003e5b
Make DomainSpecificString an attrs class () 2021-04-23 15:46:29 +01:00
Richard van der Hoff
294c675033
Remove synapse.types.Collection ()
This is no longer required, since we have dropped support for Python 3.5.
2021-04-22 16:43:50 +01:00
Jonathan de Jong
4b965c862d
Remove redundant "coding: utf-8" lines ()
Part of 

Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now.

`Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`
2021-04-14 15:34:27 +01:00
Richard van der Hoff
a7a3790066
Convert Requester to attrs ()
... because namedtuples suck

Fix up a couple of other annotations to keep mypy happy.
2021-03-10 18:15:56 +00:00
Patrick Cloke
58114f8a17
Create a SynapseReactor type which incorporates the necessary reactor interfaces. ()
This helps fix some type hints when running with Twisted 21.2.0.
2021-03-08 08:25:43 -05:00
Eric Eastwood
0a00b7ff14
Update black, and run auto formatting over the codebase ()
- Update black version to the latest
 - Run black auto formatting over the codebase
    - Run autoformatting according to [`docs/code_style.md
`](80d6dc9783/docs/code_style.md)
 - Update `code_style.md` docs around installing black to use the correct version
2021-02-16 22:32:34 +00:00
Patrick Cloke
7950aa8a27 Fix some typos. 2021-02-12 11:14:12 -05:00
Richard van der Hoff
0cd2938bc8
Support icons for Identity Providers () 2021-01-20 08:15:14 -05:00
Patrick Cloke
98a64b7f7f
Add basic domain validation for DomainSpecificString.is_valid. ()
This checks that the domain given to `DomainSpecificString.is_valid` (e.g.
`UserID`, `RoomAlias`, etc.) is of a valid form. Previously some validation
was done on the localpart (e.g. the sigil), but not the domain portion.
2021-01-13 07:05:16 -05:00
Richard van der Hoff
28877fade9
Implement a username picker for synapse ()
The final part (for now) of my work to implement a username picker in synapse itself. The idea is that we allow
`UsernameMappingProvider`s to return `localpart=None`, in which case, rather than redirecting the browser
back to the client, we redirect to a username-picker resource, which allows the user to enter a username.
We *then* complete the SSO flow (including doing the client permission checks).

The static resources for the username picker itself (in 
https://github.com/matrix-org/synapse/tree/rav/username_picker/synapse/res/username_picker)
are essentially lifted wholesale from
https://github.com/matrix-org/matrix-synapse-saml-mozilla/tree/master/matrix_synapse_saml_mozilla/res. 
As the comment says, we might want to think about making them customisable, but that can be a follow-up. 

Fixes .
2020-12-18 14:19:46 +00:00
Patrick Cloke
79bfe966e0
Improve error checking for OIDC/SAML mapping providers ()
Checks that the localpart returned by mapping providers for SAML and
OIDC are valid before registering new users.

Extends the OIDC tests for existing users and invalid data.
2020-11-19 14:25:17 -05:00
Erik Johnston
f21e24ffc2
Add ability for access tokens to belong to one user but grant access to another user. ()
We do it this way round so that only the "owner" can delete the access token (i.e. `/logout/all` by the "owner" also deletes that token, but `/logout/all` by the "target user" doesn't).

A future PR will add an API for creating such a token.

When the target user and authenticated entity are different the `Processed request` log line will be logged with a: `{@admin:server as @bob:server} ...`. I'm not convinced by that format (especially since it adds spaces in there, making it harder to use `cut -d ' '` to chop off the start of log lines). Suggestions welcome.
2020-10-29 15:58:44 +00:00
Erik Johnston
52a50e8686
Use vector clocks for room stream tokens. ()
Currently when using multiple event persisters we (in the worst case) don't tell clients about events until all event persisters have persisted new events after the original event. This is a suboptimal, especially if one of the event persisters goes down.

To handle this, we encode the position of each event persister in the room tokens so that we can send events to clients immediately. To reduce the size of the token we do two things:

1. We create a unique immutable persistent mapping between instance names and a generated small integer ID, which we can encode in the tokens instead of the instance name; and
2. We encode the "persisted upto position" of the room token and then only explicitly include instances that have positions strictly greater than that.

The new tokens look something like: `m3478~1.3488~2.3489`, where the first number is the min position, and the subsequent `-` separated pairs are the instance ID to positions map. (We use `.` and `~` as separators as they're URL safe and not already used by `StreamToken`).
2020-10-07 15:15:33 +01:00
Erik Johnston
7941372ec8
Make token serializing/deserializing async ()
The idea is that in future tokens will encode a mapping of instance to position. However, we don't want to include the full instance name in the string representation, so instead we'll have a mapping between instance name and an immutable integer ID in the DB that we can use instead. We'll then do the lookup when we serialize/deserialize the token (we could alternatively pass around an `Instance` type that includes both the name and ID, but that turns out to be a lot more invasive).
2020-09-30 20:29:19 +01:00
Erik Johnston
ea70f1c362
Various clean ups to room stream tokens. () 2020-09-29 21:48:33 +01:00
Erik Johnston
ac11fcbbb8
Add EventStreamPosition type ()
The idea is to remove some of the places we pass around `int`, where it can represent one of two things:

1. the position of an event in the stream; or
2. a token that partitions the stream, used as part of the stream tokens.

The valid operations are then:

1. did a position happen before or after a token;
2. get all events that happened before or after a token; and
3. get all events between two tokens.

(Note that we don't want to allow other operations as we want to change the tokens to be vector clocks rather than simple ints)
2020-09-24 13:24:17 +01:00
Jonathan de Jong
a3f124b821
Switch metaclass initialization to python 3-compatible syntax () 2020-09-16 15:15:55 -04:00
Erik Johnston
fe8ed1b46f
Make StreamToken.room_key be a RoomStreamToken instance. () 2020-09-11 12:22:55 +01:00
Erik Johnston
63c0e9e195
Add types to StreamToken and RoomStreamToken ()
The intention here is to change `StreamToken.room_key` to be a `RoomStreamToken` in a future PR, but that is a big enough change without this refactoring too.
2020-09-08 16:48:15 +01:00
Patrick Cloke
c619253db8
Stop sub-classing object () 2020-09-04 06:54:56 -04:00