Richard van der Hoff
f31e65a749
bg update to clear out duplicate outbound_device_list_pokes ( #7193 )
...
We seem to have some duplicates, which could do with being cleared out.
2020-04-07 23:06:39 +01:00
Richard van der Hoff
2e105c156b
Remove sent outbound device list pokes from the database ( #7192 )
...
They just get in the way.
2020-04-07 15:19:19 +01:00
Richard van der Hoff
daa1ac89a0
Fix device list update stream ids going backward ( #7158 )
...
Occasionally we could get a federation device list update transaction which
looked like:
```
[
{'edu_type': 'm.device_list_update', 'content': {'user_id': '@user:test', 'device_id': 'D2', 'prev_id': [], 'stream_id': 12, 'deleted': True}},
{'edu_type': 'm.device_list_update', 'content': {'user_id': '@user:test', 'device_id': 'D1', 'prev_id': [12], 'stream_id': 11, 'deleted': True}},
{'edu_type': 'm.device_list_update', 'content': {'user_id': '@user:test', 'device_id': 'D3', 'prev_id': [11], 'stream_id': 13, 'deleted': True}}
]
```
Having `stream_ids` which are lower than `prev_ids` looks odd. It might work
(I'm not actually sure), but in any case it doesn't seem like a reasonable
thing to expect other implementations to support.
2020-04-03 10:40:22 +01:00
Andrew Morgan
b994e86e35
Only setdefault for signatures if device has key_json ( #7177 )
2020-03-31 14:51:22 +01:00
Richard van der Hoff
7966a1cde9
Rewrite prune_old_outbound_device_pokes for efficiency ( #7159 )
...
make sure we clear out all but one update for the user
2020-03-30 19:06:52 +01:00
Erik Johnston
fdb1344716
Remove concept of a non-limited stream. ( #7011 )
2020-03-20 14:40:47 +00:00
Erik Johnston
6e6476ef07
Comments from review
2020-03-18 10:13:55 +00:00
Erik Johnston
f70f44abc7
Remove handling of multiple rows per ID
2020-02-28 11:45:35 +00:00
Erik Johnston
9ce4e344a8
Change device list replication to match new semantics.
...
Instead of sending down batches of user ID/host tuples, send down a row
per entity (user ID or host).
2020-02-28 11:25:34 +00:00
Erik Johnston
f5caa1864e
Change device lists stream to have one row per id.
...
This will make it possible to process the streams more incrementally,
avoiding having to process large chunks at once.
2020-02-28 11:21:25 +00:00
Patrick Cloke
509e381afa
Clarify list/set/dict/tuple comprehensions and enforce via flake8 ( #6957 )
...
Ensure good comprehension hygiene using flake8-comprehensions.
2020-02-21 07:15:07 -05:00
Hubert Chathi
23d8a55c7a
add device signatures to device key query results
2020-02-04 00:13:12 -05:00
Erik Johnston
83b0ea047b
Fix deleting of stale marker for device lists ( #6819 )
...
We were in fact only deleting stale marker when we got an incremental
update, rather than when we did a full resync.
2020-01-31 14:04:15 +00:00
Erik Johnston
c80a9fe13d
When a client asks for remote keys check if should resync. ( #6797 )
...
If we detect that the remote users' keys may have changed then we should
attempt to resync against the remote server rather than using the
(potentially) stale local cache.
2020-01-30 15:06:58 +00:00
Erik Johnston
e17a110661
Detect unknown remote devices and mark cache as stale ( #6776 )
...
We just mark the fact that the cache may be stale in the database for
now.
2020-01-28 14:43:21 +00:00
Richard van der Hoff
14d8f342d5
move batch_iter to a separate module
2020-01-16 22:25:32 +00:00
Erik Johnston
9a4fb457cf
Change DataStores to accept 'database' param.
2019-12-06 13:30:06 +00:00
Erik Johnston
4a33a6dd19
Move background update handling out of store
2019-12-05 11:11:26 +00:00
Erik Johnston
756d4942f5
Move DB pool and helper functions into dedicated Database class
2019-12-05 10:46:37 +00:00
Erik Johnston
ee86abb2d6
Remove underscore from SQLBaseStore functions
2019-12-04 16:23:43 +00:00
Erik Johnston
1056d6885a
Move cache invalidation to main data store
2019-12-04 15:21:14 +00:00
Erik Johnston
6b2867096b
Move event fetch vars to EventWorkStore
2019-12-04 15:21:14 +00:00
Hubert Chathi
c3fc176c60
Update synapse/storage/data_stores/main/devices.py
...
Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2019-10-31 22:49:48 -04:00
Hubert Chathi
bb6cec27a5
rename get_devices_by_remote to get_device_updates_by_remote
2019-10-30 14:57:34 -04:00
Hubert Chathi
d78b1e339d
apply changes as a result of PR review
2019-10-30 10:01:53 -04:00
Hubert Chathi
c40d7244f8
Merge branch 'develop' into cross-signing_federation
2019-10-24 22:31:25 -04:00
Erik Johnston
bb6264be0b
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/refactor_stores
2019-10-22 10:41:18 +01:00
Erik Johnston
c66a06ac6b
Move storage classes into a main "data store".
...
This is in preparation for having multiple data stores that offer
different functionality, e.g. splitting out state or event storage.
2019-10-21 16:05:06 +01:00
