Erik Johnston
c80a9fe13d
When a client asks for remote keys check if should resync. ( #6797 )
...
If we detect that the remote users' keys may have changed then we should
attempt to resync against the remote server rather than using the
(potentially) stale local cache.
2020-01-30 15:06:58 +00:00
Erik Johnston
611215a49c
Delete current state when server leaves a room ( #6792 )
...
Otherwise its just stale data, which may get deleted later anyway so
can't be relied on. It's also a bit of a shotgun if we're trying to get
the current state of a room we're not in.
2020-01-29 11:01:32 +00:00
Erik Johnston
e17a110661
Detect unknown remote devices and mark cache as stale ( #6776 )
...
We just mark the fact that the cache may be stale in the database for
now.
2020-01-28 14:43:21 +00:00
Erik Johnston
33f904835a
Merge branch 'master' into develop
2020-01-28 13:39:39 +00:00
Erik Johnston
bdbeeb94ec
Fix setting mau_limit_reserved_threepids
config ( #6793 )
...
Calling the invalidation function during initialisation of the data
stores introduces a circular dependency, causing Synapse to fail to
start.
2020-01-28 13:05:24 +00:00
Erik Johnston
8df862e45d
Add rooms.room_version
column ( #6729 )
...
This is so that we don't have to rely on pulling it out from `current_state_events` table.
2020-01-27 14:30:57 +00:00
Erik Johnston
d5275fc55f
Propagate cache invalidates from workers to other workers. ( #6748 )
...
Currently if a worker invalidates a cache it will be streamed to master, which then didn't forward those to other workers.
2020-01-27 13:47:50 +00:00
Erik Johnston
fa4d609e20
Make 'event.redacts' never raise. ( #6771 )
...
There are quite a few places that we assume that a redaction event has a
corresponding `redacts` key, which is not always the case. So lets
cheekily make it so that event.redacts just returns None instead.
2020-01-23 15:19:03 +00:00
Andrew Morgan
ce84dd9e20
Remove unnecessary abstractions in admin handler ( #6751 )
2020-01-22 15:09:57 +00:00
Andrew Morgan
90a28fb475
Admin API to list, filter and sort rooms ( #6720 )
2020-01-22 13:36:43 +00:00
Neil Johnson
5e52d8563b
Allow monthly active user limiting support for worker mode, fixes #4639 . ( #6742 )
2020-01-22 11:05:14 +00:00
Erik Johnston
5d7a6ad223
Allow streaming cache invalidate all to workers. ( #6749 )
2020-01-22 10:37:00 +00:00
Erik Johnston
0e68760078
Add a DeltaState to track changes to be made to current state ( #6716 )
2020-01-20 18:07:20 +00:00
Satsuki Yanagi
722b4f302d
Fix syntax error in run_upgrade for schema 57 ( #6728 )
...
Fix #6727
Related #6655
Co-authored-by: Erik Johnston <erikj@jki.re>
2020-01-17 14:30:35 +00:00
Brendan Abolivier
3b72bb780a
Merge pull request #6714 from matrix-org/babolivier/retention_select_event
...
Fix instantiation of message retention purge jobs
2020-01-17 14:23:51 +00:00
Richard van der Hoff
14d8f342d5
move batch_iter to a separate module
2020-01-16 22:25:32 +00:00
Brendan Abolivier
dac148341b
Fixup diff
2020-01-16 20:25:09 +00:00
Brendan Abolivier
842c2cfbf1
Remove get_room_event_after_stream_ordering entirely
2020-01-16 20:24:17 +00:00
Erik Johnston
d386f2f339
Add StateMap type alias ( #6715 )
2020-01-16 13:31:22 +00:00
Brendan Abolivier
e601f35d3b
Lint
2020-01-16 09:55:11 +00:00
Brendan Abolivier
066b9f52b8
Correctly order when selecting before stream ordering
2020-01-15 19:32:47 +00:00
Brendan Abolivier
8363588237
Fix typo
2020-01-15 19:13:22 +00:00
Brendan Abolivier
855af069a4
Fix instantiation of message retention purge jobs
...
When figuring out which topological token to start a purge job at, we
need to do the following:
1. Figure out a timestamp before which events will be purged
2. Select the first stream ordering after that timestamp
3. Select info about the first event after that stream ordering
4. Build a topological token from that info
In some situations (e.g. quiet rooms with a short max_lifetime), there
might not be an event after the stream ordering at step 3, therefore we
abort the purge with the error `No event found`. To mitigate that, this
patch fetches the first event _before_ the stream ordering, instead of
after.
2020-01-15 18:56:18 +00:00
Erik Johnston
28c98e51ff
Add local_current_membership
table ( #6655 )
...
Currently we rely on `current_state_events` to figure out what rooms a
user was in and their last membership event in there. However, if the
server leaves the room then the table may be cleaned up and that
information is lost. So lets add a table that separately holds that
information.
2020-01-15 14:59:33 +00:00
Andrew Morgan
1177d3f3a3
Quarantine media by ID or user ID ( #6681 )
2020-01-13 18:10:43 +00:00
Richard van der Hoff
e97d1cf001
Modify check_database to take a connection rather than a cursor
...
We might not need the cursor at all.
2020-01-09 18:05:50 +00:00
Manuel Stahl
d2906fe666
Allow admin users to create or modify users without a shared secret ( #6495 )
...
Signed-off-by: Manuel Stahl <manuel.stahl@awesome-technologies.de>
2020-01-09 13:31:00 +00:00
Erik Johnston
7c232bd98b
Merge pull request #6664 from matrix-org/erikj/media_admin_apis
...
Fix media repo admin APIs when using a media worker.
2020-01-08 15:50:06 +00:00
Erik Johnston
d74054afda
Shuffle the code
2020-01-08 14:57:45 +00:00
Erik Johnston
bca3455b38
Comments
2020-01-08 14:27:35 +00:00
Erik Johnston
187dc6ad02
Do not rely on streaming events, as media repo doesn't
2020-01-08 14:24:28 +00:00
Manuel Stahl
7caaa29daa
Fix GET request on /_synapse/admin/v2/users endpoint ( #6563 )
...
Fixes #6552
2020-01-08 13:26:40 +00:00
Erik Johnston
3cf7d6d5b6
Move media admin store functions to worker store
2020-01-08 13:26:20 +00:00
Brendan Abolivier
0ab5853ec9
Merge pull request #6652 from matrix-org/babolivier/depth_missing_events
...
Fix conditions failing if min_depth = 0
2020-01-07 15:22:07 +01:00
Richard van der Hoff
85db7f73be
Add a background update to clear tombstoned rooms from the directory ( #6648 )
...
* Add a background update to clear tombstoned rooms from the directory
* use the ABC metaclass
2020-01-07 14:18:43 +00:00
Richard van der Hoff
1ff5491117
Merge pull request #6645 from matrix-org/rav/fix_synchrotron_error
...
Fix exceptions in the synchrotron worker log when events are rejected.
2020-01-07 14:02:14 +00:00
Brendan Abolivier
cd428a93e2
Fix conditions failing if min_depth = 0
...
This could result in Synapse not fetching prev_events for new events in the room if it has missed some events.
2020-01-07 12:08:58 +00:00
Richard van der Hoff
1807db5e73
Merge pull request #6629 from matrix-org/rav/kill_event_reference_hashes
...
Remove a bunch of unused code from event creation
2020-01-06 17:46:31 +00:00
Richard van der Hoff
bc42da4ab8
Clarify documentation on get_event* methods
...
Make it clearer how they behave in the face of rejected and/or missing events.
2020-01-06 17:12:06 +00:00
Richard van der Hoff
ba897a7590
Fix some test failures when frozen_dicts are enabled ( #6642 )
...
Fixes #4026
2020-01-06 15:22:46 +00:00
Erik Johnston
9f6c1befbb
Add experimental 'databases' config ( #6580 )
2020-01-06 14:44:01 +00:00
Richard van der Hoff
a7d2e5b37f
Remove unused get_latest_event_ids_and_hashes_in_room
2020-01-06 13:45:33 +00:00
Richard van der Hoff
dc41fbf0dd
Remove unused get_prev_events_and_hashes_for_room
2020-01-06 13:45:33 +00:00
Richard van der Hoff
15720092ac
replace get_prev_events_and_hashes_for_room with get_prev_events_for_room in create_new_client_event
2020-01-06 13:45:33 +00:00
Richard van der Hoff
5a04781643
rename get_prev_events_for_room to get_prev_events_and_hashes_for_room
...
... to make way for a new method which just returns the event ids
2020-01-06 13:45:33 +00:00
Erik Johnston
75d8f26ac8
Split state groups into a separate data store ( #6296 )
2019-12-20 10:48:24 +00:00
Erik Johnston
fa780e9721
Change EventContext to use the Storage class ( #6564 )
2019-12-20 10:32:02 +00:00
Richard van der Hoff
d6752ce5da
Clean up startup for the pusher ( #6558 )
...
* Remove redundant python2 support code
`str.decode()` doesn't exist on python3, so presumably this code was doing
nothing
* Filter out pushers with corrupt data
When we get a row with unparsable json, drop the row, rather than returning a
row with null `data`, which will then cause an explosion later on.
* Improve logging when we can't start a pusher
Log the ID to help us understand the problem
* Make email pusher setup more robust
We know we'll have a `data` member, since that comes from the database. What we
*don't* know is if that is a dict, and if that has a `brand` member, and if
that member is a string.
2019-12-18 14:26:58 +00:00
Andrew Morgan
7963ca83cb
Add delta file to fix missing default table data ( #6555 )
2019-12-18 11:13:33 +00:00
Erik Johnston
2284eb3a53
Add database config class ( #6513 )
...
This encapsulates config for a given database and is the way to get new
connections.
2019-12-18 10:45:12 +00:00