Mathieu Velten
1526ff389f
Faster joins: filter out non local events when a room doesn't have its full state ( #14404 )
...
Signed-off-by: Mathieu Velten <mathieuv@matrix.org>
2022-11-21 16:46:14 +01:00
Erik Johnston
299b00d968
Prioritize outbound to-device over device list updates ( #13922 )
...
Otherwise device list changes for large accounts can temporarily delay to-device messages.
2022-09-27 15:17:41 +01:00
Erik Johnston
a7e506ddee
Reduce amount of state we pull out when attempting to send catchup PDUs. ( #12963 )
...
* Don't pull out state for catchup
* Newsfile
* Merge newsfile
2022-06-07 14:35:56 +01:00
Patrick Cloke
c52abc1cfd
Additional constants for EDU types. ( #12884 )
...
Instead of hard-coding strings in many places.
2022-05-27 07:14:36 -04:00
Patrick Cloke
b5707ceaba
Avoid attempting to delete push actions for remote users. ( #12879 )
...
Remote users will never have push actions, so we can avoid a database
round-trip/transaction completely.
2022-05-26 07:09:16 -04:00
Erik Johnston
423cca9efe
Spread out sending device lists to remote hosts ( #12132 )
2022-03-04 11:48:15 +00:00
Richard van der Hoff
e24ff8ebe3
Remove HomeServer.get_datastore() ( #12031 )
...
The presence of this method was confusing, and mostly present for backwards
compatibility. Let's get rid of it.
Part of #11733
2022-02-23 11:04:02 +00:00
Richard van der Hoff
a85dde3445
Minor typing fixes ( #12034 )
...
These started failing in
https://github.com/matrix-org/synapse/pull/12031 ... I'm a bit mystified by how
they ever worked.
2022-02-21 18:37:04 +00:00
Patrick Cloke
10a88ba91c
Use auto_attribs/native type hints for attrs classes. ( #11692 )
2022-01-13 13:49:28 +00:00
Patrick Cloke
d2279f471b
Add most of the missing type hints to synapse.federation. ( #11483 )
...
This skips a few methods which are difficult to type.
2021-12-02 16:18:10 +00:00
Andrew Morgan
aa2c027792
Remove unnecessary parentheses around tuples returned from methods ( #10889 )
2021-09-23 11:59:07 +01:00
Erik Johnston
ac5c221208
Stagger send presence to remotes ( #10398 )
...
This is to help with performance, where trying to connect to thousands
of hosts at once can consume a lot of CPU (due to TLS etc).
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
2021-07-15 11:52:56 +01:00
Jonathan de Jong
bf72d10dbf
Use inline type hints in various other places (in synapse/) ( #10380 )
2021-07-15 11:02:43 +01:00
Richard van der Hoff
b378d98c8f
Add debug logging for issue #9533 ( #9959 )
...
Hopefully this will help us track down where to-device messages are getting
lost/delayed.
2021-05-11 11:04:03 +01:00
Andrew Morgan
4e0fd35bc9
Revert "Experimental Federation Speedup ( #9702 )"
...
This reverts commit 05e8c70c05
2021-04-28 11:38:33 +01:00
Jonathan de Jong
05e8c70c05
Experimental Federation Speedup ( #9702 )
...
This basically speeds up federation by "squeezing" each individual dual database call (to destinations and destination_rooms), which previously happened per every event, into one call for an entire batch (100 max).
Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>
2021-04-14 17:19:02 +01:00
Jonathan de Jong
4b965c862d
Remove redundant "coding: utf-8" lines ( #9786 )
...
Part of #9744
Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now.
`Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`
2021-04-14 15:34:27 +01:00
Erik Johnston
33548f37aa
Improve tracing for to device messages ( #9686 )
2021-04-01 17:08:21 +01:00
Erik Johnston
c602ba8336
Fixed undefined variable error in catchup ( #9664 )
...
Broke in #9640
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2021-03-24 16:12:47 +00:00
Erik Johnston
dd71eb0f8a
Make federation catchup send last event from any server. ( #9640 )
...
Currently federation catchup will send the last *local* event that we
failed to send to the remote. This can cause issues for large rooms
where lots of servers have sent events while the remote server was down,
as when it comes back up again it'll be flooded with events from various
points in the DAG.
Instead, let's make it so that all the servers send the most recent
events, even if its not theirs. The remote should deduplicate the
events, so there shouldn't be much overhead in doing this.
Alternatively, the servers could only send local events if they were
also extremities and hope that the other server will send the event
over, but that is a bit risky.
2021-03-18 15:52:26 +00:00
Erik Johnston
026503fa3b
Don't go into federation catch up mode so easily ( #9561 )
...
Federation catch up mode is very inefficient if the number of events
that the remote server has missed is small, since handling gaps can be
very expensive, c.f. #9492 .
Instead of going into catch up mode whenever we see an error, we instead
do so only if we've backed off from trying the remote for more than an
hour (the assumption being that in such a case it is more than a
transient failure).
2021-03-15 14:42:40 +00:00
Eric Eastwood
0a00b7ff14
Update black, and run auto formatting over the codebase ( #9381 )
...
- Update black version to the latest
- Run black auto formatting over the codebase
- Run autoformatting according to [`docs/code_style.md
`](80d6dc9783/docs/code_style.md
2021-02-16 22:32:34 +00:00
Richard van der Hoff
f31f8e6319
Remove stream ordering from Metadata dict ( #8452 )
...
There's no need for it to be in the dict as well as the events table. Instead,
we store it in a separate attribute in the EventInternalMetadata object, and
populate that on load.
This means that we can rely on it being correctly populated for any event which
has been persited to the database.
2020-10-05 14:43:14 +01:00
Richard van der Hoff
3bd3707cb9
Fix malformed log line in new federation "catch up" logic ( #8442 )
2020-10-02 11:05:29 +01:00
reivilibre
576bc37d31
Catch-up after Federation Outage (split, 4): catch-up loop ( #8272 )
2020-09-15 09:07:19 +01:00
reivilibre
17fa4c7ca7
Catch up after Federation Outage (split, 2): Track last successful stream ordering after transmission ( #8247 )
...
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2020-09-04 15:06:51 +01:00
Patrick Cloke
c619253db8
Stop sub-classing object ( #8249 )
2020-09-04 06:54:56 -04:00
reivilibre
4535e849d7
Remove obsolete order field in send_new_transaction ( #8245 )
...
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2020-09-03 19:23:07 +01:00
Patrick Cloke
ad6190c925
Convert stream database to async/await. ( #8074 )
2020-08-17 07:24:46 -04:00
reivilibre
ff0e894656
Drop federation transmission queues during a significant remote outage. ( #7864 )
...
* Empty federation transmission queues when we are backing off.
Fixes #7828 .
Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
* Address feedback
Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
* Reword newsfile
2020-08-13 12:35:04 +01:00
Erik Johnston
2c1b9d6763
Update worker docs with recent enhancements ( #7969 )
2020-07-29 23:22:13 +01:00
Erik Johnston
649a7ead5c
Add ability to run multiple pusher instances ( #7855 )
...
This reuses the same scheme as federation sender sharding
2020-07-16 14:06:28 +01:00
Erik Johnston
f299441cc6
Add ability to shard the federation sender ( #7798 )
2020-07-10 18:26:36 +01:00
Patrick Cloke
38e1fac886
Fix some spelling mistakes / typos. ( #7811 )
2020-07-09 09:52:58 -04:00
Richard van der Hoff
075375bbc9
add a comment
2020-05-21 13:25:41 +01:00
Richard van der Hoff
d5aa7d93ed
Fix catchup-on-reconnect for the Federation Stream ( #7374 )
...
looks like we managed to break this during the refactorathon.
2020-05-05 14:15:57 +01:00
Erik Johnston
b08b0a22d5
Add typing to synapse.federation.sender ( #6871 )
2020-02-07 13:56:38 +00:00
Erik Johnston
d386f2f339
Add StateMap type alias ( #6715 )
2020-01-16 13:31:22 +00:00
Hubert Chathi
6f4bc6d01d
Merge branch 'develop' into cross-signing_federation
2019-10-31 22:38:21 -04:00
Amber Brown
020add5099
Update black to 19.10b0 ( #6304 )
...
* update version of black and also fix the mypy config being overridden
2019-11-01 02:43:24 +11:00
Hubert Chathi
bb6cec27a5
rename get_devices_by_remote to get_device_updates_by_remote
2019-10-30 14:57:34 -04:00
Hubert Chathi
c40d7244f8
Merge branch 'develop' into cross-signing_federation
2019-10-24 22:31:25 -04:00
Hubert Chathi
8d3542a64e
implement federation parts of cross-signing
2019-10-22 19:04:35 -04:00
Erik Johnston
c66a06ac6b
Move storage classes into a main "data store".
...
This is in preparation for having multiple data stores that offer
different functionality, e.g. splitting out state or event storage.
2019-10-21 16:05:06 +01:00
Amber Brown
4806651744
Replace returnValue with return ( #5736 )
2019-07-23 23:00:55 +10:00
Amber Brown
32e7c9e7f2
Run Black. ( #5482 )
2019-06-20 19:32:02 +10:00
Richard van der Hoff
5c15039e06
Clean up code for sending federation EDUs. ( #5381 )
...
This code confused the hell out of me today. Split _get_new_device_messages
into its two (unrelated) parts.
2019-06-13 13:52:08 +01:00
Andrew Morgan
2d1d7b7e6f
Prevent multiple device list updates from breaking a batch send ( #5156 )
...
fixes #5153
2019-06-06 23:54:00 +01:00
Richard van der Hoff
130f932cbc
Run black on per_destination_queue
...
... mostly to fix pep8 fails
2019-05-09 16:27:02 +01:00
Quentin Dufour
11ea16777f
Limit the number of EDUs in transactions to 100 as expected by receiver ( #5138 )
...
Fixes #3951 .
2019-05-09 11:01:41 +01:00
