Commit Graph

199 Commits

Author SHA1 Message Date
Dirk Klimpel
9e06e22064
Add type hints to more tests files. () 2022-03-17 07:25:50 -04:00
Dirk Klimpel
5dd949bee6
Add type hints to some tests/handlers files. () 2022-03-15 09:16:37 -04:00
Dirk Klimpel
32c828d0f7
Add type hints to tests/rest. ()
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2022-03-11 12:42:22 +00:00
Erik Johnston
2ce27a24fe
Add experimental environment variable to enable asyncio reactor () 2022-03-08 13:23:18 +00:00
Dirk Klimpel
7e91107be1
Add type hints to tests/rest ()
* Add type hints to `tests/rest`

* newsfile

* change import from `SigningKey`
2022-03-03 16:05:44 +00:00
David Robertson
1fbe0316a9
Add suffices to scripts in scripts-dev ()
* Rename scripts-dev to have suffices

* Update references to `scripts-dev`

* Changelog

* These scripts don't pass mypy
2022-03-02 18:00:26 +00:00
Dirk Klimpel
2ffaf30803
Add type hints to tests/rest/client ()
* Add type hints to `tests/rest/client`

* newsfile

* fix imports

* add `test_account.py`

* Remove one type hint in `test_report_event.py`

* change `on_create_room` to `async`

* update new functions in `test_third_party_rules.py`

* Add `test_filter.py`

* add `test_rooms.py`

* change to `assertEquals` to `assertEqual`

* lint
2022-03-02 16:34:14 +00:00
David Robertson
f3f0ab10fe
Move scripts directory inside synapse, exposing as setuptools entry_points ()
* Two scripts are basically entry_points already
* Move and rename scripts/* to synapse/_scripts/*.py
* Delete sync_room_to_group.pl
* Expose entry points in setup.py
* Update linter script and config
* Fixup scripts & docs mentioning scripts that moved

Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2022-03-02 13:00:16 +00:00
Dirk Klimpel
952efd0bca
Add type hints to tests/rest/client ()
* Add type hints to `tests/rest/client`

* update `mypy.ini`

* newsfile

* add `test_register.py`
2022-02-28 18:59:00 +00:00
Dirk Klimpel
1901cb1d4a
Add type hints to tests/rest/client () 2022-02-28 17:47:37 +00:00
Dirk Klimpel
7c82da27aa
Add type hints to synapse/storage/databases/main () 2022-02-21 16:03:06 +00:00
Patrick Cloke
a121507cfe
Adds misc missing type hints () 2022-02-11 07:20:16 -05:00
Patrick Cloke
d0e78af35e
Add missing type hints to synapse.replication. () 2022-02-08 11:03:08 -05:00
Dirk Klimpel
901b264c0c
Add type hints to tests/rest/admin () 2022-01-31 14:20:05 -05:00
Dirk Klimpel
07a3b5daba
Add type hints to synapse/storage/databases/main/events_bg_updates.py () 2021-12-30 07:22:31 -05:00
reivilibre
2c7f5e74e5
Fix a type annotation in test_account_data.py and remove it from the Mypy exclusion list. ()
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2021-12-29 15:12:30 +00:00
Dirk Klimpel
15bb1c8511
Add type hints to synapse/storage/databases/main/stats.py () 2021-12-29 08:01:13 -05:00
Patrick Cloke
b6102230a7
Add type hints to event_push_actions. () 2021-12-21 13:25:34 +00:00
Shay
8ad39438fa
Add opentracing types () 2021-12-20 12:18:09 +00:00
Dirk Klimpel
8428ef66c7
Add type hints to synapse/tests/rest/admin () 2021-12-16 14:59:56 -05:00
Dirk Klimpel
1847d027e6
Add type hints to synapse/storage/databases/main/transactions.py () 2021-12-16 14:59:35 -05:00
Sean Quah
c7fe32edb4
Add type hints to synapse/storage/databases/main/room.py () 2021-12-15 18:00:48 +00:00
Sean Quah
ecfcd9bbbe
Add type hints to synapse/storage/databases/main/e2e_room_keys.py () 2021-12-14 17:46:47 +00:00
Sean Quah
0147b3de20
Add missing type hints to synapse.logging.context () 2021-12-14 17:35:28 +00:00
Patrick Cloke
2519beaad2
Add missing type hints to synapse.appservice () 2021-12-14 17:02:46 +00:00
Patrick Cloke
33abbc3278
Add missing type hints to synapse.http. () 2021-12-14 07:00:47 -05:00
Sean Quah
1abfb15f07
Add type hints to synapse/storage/databases/main/end_to_end_keys.py () 2021-12-13 16:28:26 +00:00
Sean Quah
6da8591f2e
Add type hints to synapse/storage/databases/main/account_data.py () 2021-12-13 16:28:10 +00:00
Dirk Klimpel
e5f426cd54
Add type hints to synapse/tests/rest/admin () 2021-12-03 13:57:13 +00:00
reivilibre
8a4c296987
Clean up tests.test_visibility to remove legacy code. () 2021-12-02 18:13:30 +00:00
Patrick Cloke
d2279f471b
Add most of the missing type hints to synapse.federation. ()
This skips a few methods which are difficult to type.
2021-12-02 16:18:10 +00:00
reivilibre
435f044807
Add type annotations to tests.storage.test_appservice. () 2021-12-02 15:30:05 +00:00
David Robertson
dc0a3cd596
disallow-untyped-defs for the module_api () 2021-11-29 11:28:12 +00:00
David Robertson
776ad3e5e9
Add a test case for the SendJoinParser ()
This would have caught the bug  introduced in  and fixed in .
2021-11-29 11:11:46 +00:00
Sean Quah
ffd858aa68
Add type hints to synapse/storage/databases/main/events_worker.py ()
Also refactor the stream ID trackers/generators a bit and try to
document them better.
2021-11-26 18:41:31 +00:00
Patrick Cloke
55669bd3de
Add missing type hints to config base classes () 2021-11-23 15:21:19 +00:00
Sean Quah
84fac0f814
Add type annotations to synapse.metrics () 2021-11-17 19:07:02 +00:00
Patrick Cloke
7468723697
Add most missing type hints to synapse.util () 2021-11-16 08:47:36 -05:00
David Robertson
e605e4b8f2
Database storage profile passes mypy ()
It already seems to pass mypy. I wonder what changed, given that it was
on the exclusion list. So this commit consists of me ensuring
`--disallow-untyped-defs` passes and a minor fixup to a function that
returned either `True` or `None`.
2021-11-15 12:59:33 +00:00
David Robertson
5562ce6a53
Get directory db file to pass mypy () 2021-11-15 12:59:05 +00:00
David Robertson
fe58672546
Annotations for state_deltas.py ()
I was sad that I couldn't do better for
`_curr_state_delta_stream_cache`. At least it's explicitly called out in
a comment with #TODO.
2021-11-12 20:24:12 +00:00
Patrick Cloke
3fad4e3fe5
Rollback due to wrong syntax in mypy.ini. ()
This was only checking the __init__ files in modules instead of
all files in a module, which don't pass yet.
2021-11-12 20:10:03 +00:00
David Robertson
bea815cec8
Test room alias deletion ()
* Prefer `HTTPStatus` over plain `int`

This is an Opinion that no-one has seemed to object to yet.

* `--disallow-untyped-defs` for `tests.rest.client.test_directory`
* Improve synapse's annotations for deleting aliases
* Test case for deleting a room alias
* Changelog
2021-11-12 19:56:00 +00:00
Patrick Cloke
9b90b9454b
Add type hints to media repository storage module () 2021-11-12 11:05:26 -05:00
David Robertson
6f8f3d4bc5
Attempt to annotate events_forward_extremities ()
* Make DataStore inherit from EventForwardExtremitiesStore before CacheInvalidationWorkerStore

the former implicitly inherits from the latter, so they should be
ordered like this when used.
2021-11-12 15:58:17 +00:00
Patrick Cloke
95547e5300
Generalize the disallowed_untyped_defs in mypy.ini () 2021-11-12 14:27:45 +00:00
David Robertson
c99da2d079
Annotations for user_erasure_store ()
I'm not sure why this was excluded---it seemed to be passing for me. But
it's easy enough to fixup.
2021-11-11 19:22:19 +00:00
David Robertson
6a605f4a77
Get db signatures file to pass mypy () 2021-11-11 17:04:44 +00:00
David Robertson
8dc666f785
Correct type hint for room_batch.py ()
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2021-11-11 16:49:28 +00:00
Patrick Cloke
64ef25391d
Add type hints to some storage classes () 2021-11-11 08:47:31 -05:00
Patrick Cloke
5cace20bf1
Add missing type hints to synapse.app. () 2021-11-10 15:06:54 -05:00
Patrick Cloke
66c4b774fd
Add type hints to synapse._scripts () 2021-11-10 17:55:32 +00:00
Patrick Cloke
0ef69ddbdc
Ignore missing imports for parameterized. ()
This was due to a conflict between , which changed
mypy configuration, and , a normal change.
2021-11-09 19:04:53 +00:00
Dan Callahan
3b951445a7
Require mypy for synapse/ & tests/ unless excluded ()
Signed-off-by: Dan Callahan <danc@element.io>
2021-11-09 16:22:47 +00:00
Patrick Cloke
c01bc5f43d
Add remaining type hints to synapse.events. () 2021-11-02 09:55:52 -04:00
Patrick Cloke
56e281bf6c
Additional type hints for relations database class. () 2021-10-28 14:35:12 -04:00
David Robertson
1bfd141205
Type hints for the remaining two files in synapse.http. ()
* Teach MyPy that the sentinel context is False

This means that if `ctx: LoggingContextOrSentinel`
then `bool(ctx)` narrows us to `ctx:LoggingContext`, which is a really
neat find!

* Annotate RequestMetrics

- Raise errors for sentry if we use the sentinel context
- Ensure we don't raise an error and carry on, but not recording stats
- Include stack trace in the error case to lower Sean's blood pressure

* Make mypy pass for synapse.http.request_metrics

* Make synapse.http.connectproxyclient pass mypy

Co-authored-by: reivilibre <oliverw@matrix.org>
2021-10-28 14:14:42 +01:00
Patrick Cloke
0f9adc99ad
Add missing type hints to synapse.crypto. ()
And require type hints for this module.
2021-10-21 13:07:07 +00:00
Patrick Cloke
3ab55d43bd
Add missing type hints to synapse.api. ()
* Convert UserPresenceState to attrs.
* Remove args/kwargs from error classes and explicitly pass msg/errorcode.
2021-10-18 15:01:10 -04:00
Patrick Cloke
1609ccf8fe
Fix-up some type hints in the relations tests. () 2021-10-14 09:19:35 -04:00
Patrick Cloke
1f9d0b8a7a
Add type hints to synapse.events.*. ()
Except `synapse/events/__init__.py`, which will be done in a follow-up.
2021-10-13 07:24:07 -04:00
Patrick Cloke
732bbf6737
Be more lenient when parsing the version for oEmbed responses. () 2021-10-13 07:00:07 -04:00
Sean Quah
84f5d83257
Add tests for MediaFilePaths () 2021-10-12 18:19:35 +01:00
Sean Quah
36224e056a
Add type hints to synapse.storage.databases.main.client_ips () 2021-10-12 13:50:34 +01:00
David Robertson
e0f11ae4a5
disallow-untyped-defs for synapse.push () 2021-10-11 17:42:10 +01:00
David Robertson
51a5da74cc
Annotate synapse.storage.util ()
Also mark `synapse.streams` as having has no untyped defs

Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
2021-10-08 14:25:16 +00:00
David Robertson
797ee7812d
Relax ignore-missing-imports for modules that have stubs now and update mypy ()
Updating mypy past version 0.9 means that third-party stubs are no-longer distributed with typeshed. See http://mypy-lang.blogspot.com/2021/06/mypy-0900-released.html for details.
We therefore pull in stub packages in setup.py

Additionally, some modules that we were previously ignoring import failures for now have stubs. So let's use them.

The rest of this change consists of fixups to make the newer mypy + stubs pass CI.

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2021-10-08 14:49:41 +01:00
David Robertson
e79ee48313
disallow-untyped-defs for synapse.server_notices () 2021-10-07 19:55:15 +01:00
David Robertson
f563676c09
disallow-untyped-defs for synapse.state ()
* `disallow-untyped-defs` for `synapse.state`

Much smaller than I was expecting!
2021-10-06 17:55:25 +00:00
David Robertson
f8d0f72b27
More types for synapse.util, part 1 ()
The following modules now pass `disallow_untyped_defs`:

* synapse.util.caches.cached_call 
* synapse.util.caches.lrucache
* synapse.util.caches.response_cache 
* synapse.util.caches.stream_change_cache
* synapse.util.caches.ttlcache pass
* synapse.util.daemonize
* synapse.util.patch_inline_callbacks pass `no-untyped-defs`
* synapse.util.versionstring

Additional typing in synapse.util.metrics. Didn't get this to pass `no-untyped-defs`, think I'll need to watch 
2021-10-06 11:20:49 +01:00
David Robertson
3aefc7b66d
Refactor user directory tests ()
* Pull out GetUserDirectoryTables helper
* Don't rebuild the dir in tests that don't need it

In  I changed registering a user to add directory entries under.
This means we don't have to force a directory regbuild in to tests of
the user directory search.

* Move test_initial to tests/storage
* Add type hints to both test_user_directory files

Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2021-09-30 11:04:40 +01:00
David Robertson
7f3352743e
Improve typing in user_directory files ()
* Improve typing in user_directory files

This makes the user_directory.py in storage pass most of mypy's
checks (including `no-untyped-defs`). Unfortunately that file is in the
tangled web of Store class inheritance so doesn't pass mypy at the moment.

The handlers directory has already been mypyed.

Co-authored-by: reivilibre <olivier@librepush.net>
2021-09-24 10:38:22 +01:00
Patrick Cloke
b3590614da
Require type hints in the handlers module. ()
Adds missing type hints to methods in the synapse.handlers
module and requires all methods to have type hints there.

This also removes the unused construct_auth_difference method
from the FederationHandler.
2021-09-20 08:56:23 -04:00
Patrick Cloke
3eba047d38
Add type hints to state database module. () 2021-09-15 09:54:13 -04:00
Patrick Cloke
b93259082c
Add missing type hints to non-client REST servlets. ()
Including admin, consent, key, synapse, and media. All REST servlets
(the synapse.rest module) now require typed method definitions.
2021-09-15 08:45:32 -04:00
reivilibre
8eb7cb2e0d
Make StateFilter frozen so we can hash it ()
Also enables Mypy for related tests.
2021-09-14 16:35:53 +01:00
reivilibre
524b8ead77
Add types to synapse.util. () 2021-09-10 17:03:18 +01:00
David Robertson
5e9b382505
Pull out encrypted_by_default tests from user_directory tests () 2021-09-06 11:37:54 +01:00
Patrick Cloke
ecbfa4fe4f
Additional type hints for client REST servlets (part 5) ()
Additionally this enforce type hints on all function signatures inside
of the synapse.rest.client package.
2021-09-03 09:22:22 -04:00
Richard van der Hoff
e3abc0a5cc Merge remote-tracking branch 'origin/release-v1.41' into develop 2021-08-27 16:35:02 +01:00
Richard van der Hoff
8f98260552
Fix incompatibility with Twisted < 21. ()
Turns out that the functionality added in  to skip TLS was incompatible
with older Twisted versions, so we need to be a bit more inventive.

Also, add a test to (hopefully) not break this in future. Sadly, testing TLS is
really hard.
2021-08-27 16:33:41 +01:00
Patrick Cloke
d12ba52f17
Persist room hierarchy pagination sessions to the database. () 2021-08-24 08:14:03 -04:00
David Robertson
ecd823d766
Flatten tests/rest/client/{v1,v2_alpha} too () 2021-08-20 17:50:44 +01:00
Patrick Cloke
000aa89be6
Do not include rooms with an unknown room version in a sync response. ()
A user will still see this room if it is in a local cache, but it will
not reappear if clearing the cache and reloading.
2021-08-19 11:12:55 -04:00
Dirk Klimpel
0c3565da4c
Additional type hints for the proxy agent and SRV resolver modules. () 2021-08-18 13:53:20 -04:00
Michael Telatynski
0ace38b7b3
Experimental support for MSC3266 Room Summary API. () 2021-08-16 14:49:12 +00:00
Patrick Cloke
fab352ac2c
Fix type hints in space summary tests. ()
And ensure that the file is checked via mypy.
2021-08-11 10:43:40 -04:00
Patrick Cloke
eb3beb8f12
Add type hints and comments to event auth code. () 2021-07-14 14:13:40 +01:00
Erik Johnston
7a5873277e
Add support for evicting cache entries based on last access time. () 2021-07-05 16:32:12 +01:00
Brendan Abolivier
08c8469322
Remove support for ACME v1 ()
Fixes 

ACME v1 has been fully decommissioned for existing installs on June 1st 2021(see https://community.letsencrypt.org/t/end-of-life-plan-for-acmev1/88430/27), so we can now safely remove it from Synapse.
2021-06-17 18:56:48 +01:00
Patrick Cloke
9e4610cc27
Correct type hints for parse_string(s)_from_args. () 2021-06-08 08:30:48 -04:00
Richard van der Hoff
d8be7d493d
Enable Prometheus metrics for the jaeger client library () 2021-06-04 09:25:33 +01:00
Patrick Cloke
7adcb20fc0
Add missing type hints to synapse.util () 2021-05-24 15:32:01 -04:00
Erik Johnston
64887f06fc
Use ijson to parse the response to /send_join, reducing memory usage. ()
Instead of parsing the full response to `/send_join` into Python objects (which can be huge for large rooms) and *then* parsing that into events, we instead use ijson to stream parse the response directly into `EventBase` objects.
2021-05-20 16:11:48 +01:00
Erik Johnston
ef889c98a6
Optionally track memory usage of each LruCache ()
This will double count slightly in the presence of interned strings. It's off by default as it can consume a lot of resources.
2021-05-05 16:54:36 +01:00
Andrew Morgan
fe604a022a
Remove various bits of compatibility code for Python <3.6 ()
I went through and removed a bunch of cruft that was lying around for compatibility with old Python versions. This PR also will now prevent Synapse from starting unless you're running Python 3.6+.
2021-04-27 13:13:07 +01:00
Jonathan de Jong
e2b8a90897
Update mypy configuration: no_implicit_optional = True () 2021-04-05 09:10:18 -04:00
Erik Johnston
b5efcb577e
Make it possible to use dmypy ()
Running `dmypy run` will do a `mypy` check while spinning up a daemon
that makes rerunning `dmypy run` a lot faster.

`dmypy` doesn't support `follow_imports = silent` and has
`local_partial_types` enabled, so this PR enables those options and
fixes the issues that were newly raised. Note that `local_partial_types`
will be enabled by default in upcoming mypy releases.
2021-03-26 16:49:46 +00:00
Patrick Cloke
af387cf52a
Add type hints to misc. files. () 2021-03-24 06:49:01 -04:00
Patrick Cloke
67b979bfa1
Do not ignore the unpaddedbase64 module when type checking. () 2021-03-09 14:41:02 -05:00