Nick Mills-Barrett
5d4028f217
Make all process_replication_rows methods async ( #13304 )
...
More prep work for asyncronous caching, also makes all process_replication_rows methods consistent (presence handler already is so).
Signed off by Nick @ Beeper (@Fizzadar)
2022-07-17 22:19:43 +01:00
Patrick Cloke
cf05258f76
Remove groups replication code. ( #12900 )
...
The replication logic for groups is no longer used, so the message
passing infrastructure can be removed.
2022-05-31 13:04:08 -04:00
reivilibre
39dee30f01
Send USER_IP commands on a different Redis channel, in order to reduce traffic to workers that do not process these commands. ( #12809 )
2022-05-20 15:28:23 +01:00
reivilibre
177b884ad7
Lay some foundation work to allow workers to only subscribe to some kinds of messages, reducing replication traffic. ( #12672 )
2022-05-19 16:29:08 +01:00
Andrew Morgan
83be72d76c
Add StreamKeyType class and replace string literals with constants ( #12567 )
2022-05-16 15:35:31 +00:00
Shay
d80a7ab151
Update replication.md with info on TCP module structure ( #12621 )
2022-05-09 14:46:43 -07:00
Šimon Brandner
ef86cf3d28
Update _on_new_receipts() to work with MSC2285 changes. ( #12636 )
2022-05-05 13:25:51 +00:00
Erik Johnston
c0379d6e5b
Reduce log spam when running multiple event persisters ( #12610 )
2022-05-05 10:20:23 +01:00
Erik Johnston
d1cd96ce29
Add opentracing spans to calls to external cache ( #12380 )
2022-04-07 13:18:29 +01:00
Sean Quah
800ba87cc8
Refactor and convert Linearizer to async ( #12357 )
...
Refactor and convert `Linearizer` to async. This makes a `Linearizer`
cancellation bug easier to fix.
Also refactor to use an async context manager, which eliminates an
unlikely footgun where code that doesn't immediately use the context
manager could forget to release the lock.
Signed-off-by: Sean Quah <seanq@element.io>
2022-04-05 15:43:52 +01:00
reivilibre
f871222880
Move update_client_ip background job from the main process to the background worker. ( #12251 )
2022-04-01 13:08:55 +01:00
reivilibre
4a53f35737
Improve code documentation for the typing stream over replication. ( #12211 )
2022-03-11 14:00:15 +00:00
Patrick Cloke
3e4af36bc8
Rename get_tcp_replication to get_replication_command_handler. ( #12192 )
...
Since the object it returns is a ReplicationCommandHandler.
This is clean-up from adding support to Redis where the command handler
was added as an additional layer of abstraction from the TCP protocol.
2022-03-10 13:01:56 +00:00
Patrick Cloke
d8bab6793c
Fix incorrect type hints for txredis. ( #12042 )
...
Some properties were marked as RedisProtocol instead of ConnectionHandler,
which wraps RedisProtocol instance(s).
2022-03-08 07:26:05 -05:00
Erik Johnston
423cca9efe
Spread out sending device lists to remote hosts ( #12132 )
2022-03-04 11:48:15 +00:00
Richard van der Hoff
e24ff8ebe3
Remove HomeServer.get_datastore() ( #12031 )
...
The presence of this method was confusing, and mostly present for backwards
compatibility. Let's get rid of it.
Part of #11733
2022-02-23 11:04:02 +00:00
Patrick Cloke
d0e78af35e
Add missing type hints to synapse.replication. ( #11938 )
2022-02-08 11:03:08 -05:00
Patrick Cloke
6c0984e3f0
Remove unnecessary ignores due to Twisted upgrade. ( #11939 )
...
Twisted 22.1.0 fixed some internal type hints, allowing Synapse
to remove ignore calls for parameters to connectTCP.
2022-02-08 09:15:59 -05:00
Patrick Cloke
10a88ba91c
Use auto_attribs/native type hints for attrs classes. ( #11692 )
2022-01-13 13:49:28 +00:00
Patrick Cloke
cbd82d0b2d
Convert all namedtuples to attrs. ( #11665 )
...
To improve type hints throughout the code.
2021-12-30 18:47:12 +00:00
Sean Quah
ffd858aa68
Add type hints to synapse/storage/databases/main/events_worker.py ( #11411 )
...
Also refactor the stream ID trackers/generators a bit and try to
document them better.
2021-11-26 18:41:31 +00:00
Patrick Cloke
5cace20bf1
Add missing type hints to synapse.app. ( #11287 )
2021-11-10 15:06:54 -05:00
Nick Barrett
af54167516
Enable passing typing stream writers as a list. ( #11237 )
...
This makes the typing stream writer config match the other stream writers
that only currently support a single worker.
2021-11-03 14:25:47 +00:00
Brendan Abolivier
c7a5e49664
Implement an on_new_event callback ( #11126 )
...
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2021-10-26 15:17:36 +02:00
Sean Quah
2b82ec425f
Add type hints for most HomeServer parameters ( #11095 )
2021-10-22 18:15:41 +01:00
Sean Quah
6a67f3786a
Fix logging context warnings when losing replication connection ( #10984 )
...
Instead of triggering `__exit__` manually on the replication handler's
logging context, use it as a context manager so that there is an
`__enter__` call to balance the `__exit__`.
2021-10-15 13:10:58 +01:00
Patrick Cloke
f4b1a9a527
Require direct references to configuration variables. ( #10985 )
...
This removes the magic allowing accessing configurable
variables directly from the config object. It is now required
that a specific configuration class is used (e.g. `config.foo`
must be replaced with `config.server.foo`).
2021-10-06 10:47:41 -04:00
David Robertson
29364145b2
Pass str to twisted's IReactorTCP ( #10895 )
...
This follows a correction made in twisted/twisted#1664 and should fix our Twisted Trial CI job.
Until that change is in a twisted release, we'll have to ignore the type
of the `host` argument. I've raised #10899 to remind us to review the
issue in a few months' time.
2021-09-30 12:51:47 +01:00
Patrick Cloke
94b620a5ed
Use direct references for configuration variables (part 6). ( #10916 )
2021-09-29 06:44:15 -04:00
Patrick Cloke
bb7fdd821b
Use direct references for configuration variables (part 5). ( #10897 )
2021-09-24 07:25:21 -04:00
Patrick Cloke
01c88a09cd
Use direct references for some configuration variables ( #10798 )
...
Instead of proxying through the magic getter of the RootConfig
object. This should be more performant (and is more explicit).
2021-09-13 13:07:12 -04:00
Andrew Morgan
84469bdac7
Remove the unused public_room_list_stream ( #10565 )
...
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2021-08-17 14:02:50 +01:00
Richard van der Hoff
d9cb658c78
Fix up type hints for Twisted 21.7 ( #10490 )
...
Mostly this involves decorating a few Deferred declarations with extra type hints. We wrap the types in quotes to avoid runtime errors when running against older versions of Twisted that don't have generics on Deferred.
2021-07-28 12:04:11 +00:00
Šimon Brandner
c3b037795a
Support for MSC2285 (hidden read receipts) ( #10413 )
...
Implementation of matrix-org/matrix-doc#2285
2021-07-28 10:05:11 +02:00
Jonathan de Jong
bf72d10dbf
Use inline type hints in various other places (in synapse/) ( #10380 )
2021-07-15 11:02:43 +01:00
Marcus
8070b893db
update black to 21.6b0 ( #10197 )
...
Reformat all files with the new version.
Signed-off-by: Marcus Hoffmann <bubu@bubu1.eu>
2021-06-17 15:20:06 +01:00
Richard van der Hoff
b378d98c8f
Add debug logging for issue #9533 ( #9959 )
...
Hopefully this will help us track down where to-device messages are getting
lost/delayed.
2021-05-11 11:04:03 +01:00
Erik Johnston
e3bc4617fc
Time external cache response time ( #9904 )
2021-05-04 15:14:22 +01:00
Erik Johnston
9d25a0ae65
Split presence out of master ( #9820 )
2021-04-23 12:21:55 +01:00
Richard van der Hoff
294c675033
Remove synapse.types.Collection ( #9856 )
...
This is no longer required, since we have dropped support for Python 3.5.
2021-04-22 16:43:50 +01:00
Andrew Morgan
4b2217ace2
Merge branch 'master' into develop
2021-04-21 14:55:06 +01:00
Richard van der Hoff
5d281c10dd
Stop BackgroundProcessLoggingContext making new prometheus timeseries ( #9854 )
...
This undoes part of b076bc276e
2021-04-21 10:03:31 +01:00
Andrew Morgan
6982db9651
Merge branch 'master' into develop
2021-04-20 14:55:16 +01:00
Patrick Cloke
b076bc276e
Always use the name as the log ID. ( #9829 )
...
As far as I can tell our logging contexts are meant to log the request ID, or sometimes the request ID followed by a suffix (this is generally stored in the name field of LoggingContext). There's also code to log the name@memory location, but I'm not sure this is ever used.
This simplifies the code paths to require every logging context to have a name and use that in logging. For sub-contexts (created via nested_logging_contexts, defer_to_threadpool, Measure) we use the current context's str (which becomes their name or the string "sentinel") and then potentially modify that (e.g. add a suffix).
2021-04-20 14:19:00 +01:00
Erik Johnston
de0d088adc
Add presence federation stream ( #9819 )
2021-04-20 14:11:24 +01:00
Erik Johnston
00a6db9676
Move some replication processing out of generic_worker ( #9796 )
...
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2021-04-14 17:06:06 +01:00
Jonathan de Jong
4b965c862d
Remove redundant "coding: utf-8" lines ( #9786 )
...
Part of #9744
Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now.
`Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`
2021-04-14 15:34:27 +01:00
Patrick Cloke
48d44ab142
Record more information into structured logs. ( #9654 )
...
Records additional request information into the structured logs,
e.g. the requester, IP address, etc.
2021-04-08 08:01:14 -04:00
Jonathan de Jong
e2b8a90897
Update mypy configuration: no_implicit_optional = True ( #9742 )
2021-04-05 09:10:18 -04:00
Patrick Cloke
da75d2ea1f
Add type hints for the federation sender. ( #9681 )
...
Includes an abstract base class which both the FederationSender
and the FederationRemoteSendQueue must implement.
2021-03-29 11:43:20 -04:00
