Commit Graph

286 Commits

Author SHA1 Message Date
Erik Johnston
c66a06ac6b Move storage classes into a main "data store".
This is in preparation for having multiple data stores that offer
different functionality, e.g. splitting out state or event storage.
2019-10-21 16:05:06 +01:00
Hubert Chathi
06fc66c81e
Merge pull request #6193 from matrix-org/uhoreg/interpret_device_key_in_storage
make storage layer in charge of interpreting the device key data
2019-10-11 15:19:06 -04:00
Neil Johnson
a0d0ba7862
Fix MAU reaping where reserved users are specified. (#6168) 2019-10-11 09:38:26 +01:00
Hubert Chathi
4908fb3b30 make storage layer in charge of interpreting the device key data 2019-10-10 19:15:30 -04:00
Erik Johnston
bc244627ac Fix postgres unit tests 2019-10-10 15:37:53 +01:00
Erik Johnston
b4fe7e19c0
Merge pull request #6146 from matrix-org/erikj/fix_destination_retry_timings
Fix errors storing large retry intervals.
2019-10-02 11:05:22 +01:00
Erik Johnston
f44f1d2e83 Fix errors storing large retry intervals.
We have set the max retry interval to a value larger than a postgres or
sqlite int can hold, which caused exceptions when updating the
destinations table.

To fix postgres we need to change the column to a bigint, and for sqlite
we lower the max interval to 2**62 (which is still incredibly long).
2019-10-02 10:36:27 +01:00
Erik Johnston
ce7a3e7e27 Fix fetching censored redactions from DB
Fetching a censored redactions caused an exception due to the code
expecting redactions to have a `redact` key, which redacted redactions
don't have.
2019-10-02 10:14:01 +01:00
Neil Johnson
034db2ba21 Fix dummy event insertion consent bug (#6053)
Fixes #5905
2019-09-26 11:47:53 +01:00
Erik Johnston
faac453f08 Test that pruning of old user IPs works 2019-09-24 15:53:17 +01:00
Erik Johnston
acb62a7cc6 Test background update 2019-09-23 16:59:45 +01:00
Erik Johnston
51d28272e2 Query devices table for last seen info.
This is a) simpler than querying user_ips directly and b) means we can
purge older entries from user_ips without losing the required info.

The storage functions now no longer return the access_token, since it
was unused.
2019-09-23 16:59:45 +01:00
Richard van der Hoff
1e19ce00bf
Add 'failure_ts' column to 'destinations' table (#6016)
Track the time that a server started failing at, for general analysis purposes.
2019-09-17 11:41:54 +01:00
Erik Johnston
470dc621ae
Merge pull request #5934 from matrix-org/erikj/censor_redactions
Censor redactions in DB after a month
2019-09-09 15:29:39 +01:00
Amber Brown
55d5b3af88
Servers-known-about statistic (#5981) 2019-09-07 01:45:51 +10:00
Erik Johnston
ad9b64b496 Fix test 2019-09-05 17:17:47 +01:00
Erik Johnston
3ff0422d2d Make redaction retention period configurable 2019-09-05 17:16:45 +01:00
Erik Johnston
a4bf72c30c Censor redactions in DB after a month 2019-08-30 11:29:17 +01:00
Half-Shot
0fb5189072 Fix registration test 2019-08-23 09:25:35 +01:00
Richard van der Hoff
fb86217553
Merge pull request #5788 from matrix-org/rav/metaredactions
Fix handling of redactions of redactions
2019-08-12 12:25:19 +01:00
Erik Johnston
63eb4a1b62
Merge pull request #5746 from matrix-org/erikj/test_bg_update_currnet_state
Add unit test for current state membership bg update
2019-07-30 10:00:02 +01:00
Richard van der Hoff
4e97eb89e5 Handle loops in redaction events 2019-07-30 00:00:34 +01:00
Richard van der Hoff
1cad8d7b6f
Convert RedactionTestCase to modern test style (#5768) 2019-07-26 07:38:55 +01:00
Erik Johnston
73bbaf2bc6 Add unit test for current state membership bg update 2019-07-23 17:00:22 +01:00
Amber Brown
4806651744
Replace returnValue with return (#5736) 2019-07-23 23:00:55 +10:00
Amber Brown
7ad1d76356
Support Prometheus_client 0.4.0+ (#5636) 2019-07-18 23:57:15 +10:00
Richard van der Hoff
5f158ec039
Implement access token expiry (#5660)
Record how long an access token is valid for, and raise a soft-logout once it
expires.
2019-07-12 17:26:02 +01:00
Richard van der Hoff
953dbb7980
Remove access-token support from RegistrationStore.register (#5642)
The 'token' param is no longer used anywhere except the tests, so let's kill
that off too.
2019-07-10 16:26:49 +01:00
Amber Brown
32e7c9e7f2
Run Black. (#5482) 2019-06-20 19:32:02 +10:00
Erik Johnston
554609288b Run as background process and fix comments 2019-06-19 11:33:03 +01:00
Erik Johnston
b42f90470f Add experimental option to reduce extremities.
Adds new config option `cleanup_extremities_with_dummy_events` which
periodically sends dummy events to rooms with more than 10 extremities.

THIS IS REALLY EXPERIMENTAL.
2019-06-18 15:02:18 +01:00
Erik Johnston
3ed595e327 Prometheus histograms are cumalative 2019-06-14 14:07:32 +01:00
Amber H. Brown
a10c8dae85 fix prometheus rendering error 2019-06-14 21:09:33 +10:00
Amber Brown
6312d6cc7c
Expose statistics on extrems to prometheus (#5384) 2019-06-13 22:40:52 +10:00
Andrew Morgan
2d1d7b7e6f Prevent multiple device list updates from breaking a batch send (#5156)
fixes #5153
2019-06-06 23:54:00 +01:00
Neil Johnson
a11865016e
Set default room version to v4. (#5379)
Set default room version to v4.
2019-06-06 20:13:47 +01:00
Erik Johnston
d7add713a8 Add test 2019-05-29 15:59:06 +01:00
Richard van der Hoff
b75537beaf Store key validity time in the storage layer
This is a first step to checking that the key is valid at the required moment.

The idea here is that, rather than passing VerifyKey objects in and out of the
storage layer, we instead pass FetchKeyResult objects, which simply wrap the
VerifyKey and add a valid_until_ts field.
2019-05-23 11:52:22 +01:00
Richard van der Hoff
2e052110ee
Rewrite store_server_verify_key to store several keys at once (#5234)
Storing server keys hammered the database a bit. This replaces the
implementation which stored a single key, with one which can do many updates at
once.
2019-05-23 11:45:39 +01:00
Erik Johnston
4fb44fb5b9 Expose DataStore._get_events as get_events_as_list
This is in preparation for reaction work which requires it.
2019-05-14 13:37:44 +01:00
Amber Brown
b36c82576e
Run Black on the tests again (#5170) 2019-05-10 00:12:11 -05:00
Richard van der Hoff
12f9d51e82
Add admin api for sending server_notices (#5121) 2019-05-02 11:59:16 +01:00
Richard van der Hoff
40e576e29c Move admin api impl to its own package
It doesn't really belong under rest/client/v1 any more.
2019-05-01 15:44:30 +01:00
Richard van der Hoff
18b69be00f Rewrite Datastore.get_server_verify_keys
Rewrite this so that it doesn't hammer the database.
2019-04-09 00:00:10 +01:00
Richard van der Hoff
0084309cd2 Rewrite test_keys as a HomeserverTestCase 2019-04-08 23:59:57 +01:00
Neil Johnson
e8419554ff
Remove presence lists (#4989)
Remove presence list support as per MSC 1819
2019-04-03 11:11:15 +01:00
Richard van der Hoff
54a87a7b08
Collect room-version variations into one place (#4969)
Collect all the things that make room-versions different to one another into
one place, so that it's easier to define new room versions.
2019-04-01 10:24:38 +01:00
Amber Brown
282c97327f
Migrate the user directory initial population to a background task (#4864) 2019-03-19 04:50:24 +11:00
Amber Brown
d306bd1b26 fixup 2019-03-12 22:38:01 +11:00
Amber Brown
f6135d06cf
Rewrite userdir to be faster (#4537) 2019-03-07 01:22:53 -08:00