Hubert Chathi
404e8c8532
vendor-prefix the EDU name until MSC1756 is merged into the spec
2019-10-22 22:33:23 -04:00
Hubert Chathi
1fabf82d50
update to work with newer code, and fix formatting
2019-10-22 21:44:58 -04:00
Hubert Chathi
cfdb84422d
make black happy
2019-10-22 19:06:06 -04:00
Hubert Chathi
a1aaf3eea6
don't crash if the user doesn't have cross-signing keys
2019-10-22 19:04:37 -04:00
Hubert Chathi
8d3542a64e
implement federation parts of cross-signing
2019-10-22 19:04:35 -04:00
Hubert Chathi
cfc28325a6
Merge branch 'develop' into uhoreg/e2e_cross-signing_merged
2019-10-18 11:39:40 +01:00
Erik Johnston
b161786c14
Replace IN usage with helper funcs
2019-10-10 13:15:49 +01:00
Brendan Abolivier
cef9f6753e
Move devices's bg updates to a dedicated store
2019-10-07 17:34:20 +01:00
Hubert Chathi
8e86f5b65c
Merge branch 'develop' into uhoreg/e2e_cross-signing_merged
2019-09-07 13:20:34 -04:00
Jorik Schellekens
5ade977d08
Opentracing context cannot be none
2019-09-05 15:06:13 +01:00
Hubert Chathi
faf72a4c40
Merge branch 'develop' into cross-signing_keys
2019-09-04 19:12:29 -04:00
Jorik Schellekens
a90d16dabc
Opentrace device lists ( #5853 )
...
Trace device list changes.
2019-09-03 10:21:30 +01:00
Andrew Morgan
4548d1f87e
Remove unnecessary parentheses around return statements ( #5931 )
...
Python will return a tuple whether there are parentheses around the returned values or not.
I'm just sick of my editor complaining about this all over the place :)
2019-08-30 16:28:26 +01:00
Hubert Chathi
e3d3fbf63f
Merge branch 'uhoreg/e2e_cross-signing_merged' into cross-signing_keys
2019-08-28 17:36:46 -07:00
Hubert Chathi
72d296a7f3
Merge branch 'develop' into uhoreg/e2e_cross-signing_merged
2019-08-28 17:33:44 -07:00
Jorik Schellekens
8767b63a82
Propagate opentracing contexts through EDUs ( #5852 )
...
Propagate opentracing contexts through EDUs
Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2019-08-22 18:21:10 +01:00
Hubert Chathi
7c3abc6572
apply PR review suggestions
2019-08-21 13:19:35 -07:00
Hubert Chathi
f63ba7a795
Cross-signing [1/4] -- hidden devices ( #5759 )
...
* allow devices to be marked as "hidden"
This is a prerequisite for cross-signing, as it allows us to create other things
that live within the device namespace, so they can be used for signatures.
2019-08-12 15:14:37 -07:00
Hubert Chathi
336c546d6a
Merge branch 'cross-signing_hidden' into cross-signing_keys
2019-08-01 16:31:40 -04:00
Hubert Chathi
d78a4fe156
don't need to return the hidden column any more
2019-08-01 02:16:09 -04:00
Hubert Chathi
73b26f827c
really fix queries to work with Postgres
...
(by going back to not using SQL directly)
2019-07-31 18:37:05 -04:00
Hubert Chathi
430ea08186
PostgreSQL, Y U no like?
2019-07-31 15:38:11 -04:00
Hubert Chathi
185188be03
remove extra SQL query param
2019-07-31 15:18:15 -04:00
Hubert Chathi
c1f0a5636c
Merge branch 'develop' into cross-signing_hidden
2019-07-30 23:33:06 -04:00
Hubert Chathi
bc95890822
Merge branch 'cross-signing_hidden' into cross-signing_keys
2019-07-30 23:21:26 -04:00
Hubert Chathi
781ade836b
apply changes from PR review
2019-07-30 23:09:50 -04:00
Hubert Chathi
c659b9f94f
allow uploading keys for cross-signing
2019-07-25 11:08:24 -04:00
Hubert Chathi
d1c7c2a98a
allow devices to be marked as "hidden"
...
This is a prerequisite for cross-signing, as it allows us to create other things
that live within the device namespace, so they can be used for signatures.
2019-07-24 23:21:52 -04:00
Amber Brown
4806651744
Replace returnValue with return ( #5736 )
2019-07-23 23:00:55 +10:00
Erik Johnston
729f5a4fb6
Review comments
2019-06-27 16:06:23 +01:00
Erik Johnston
f335e77d53
Use batch_iter and correct docstring
2019-06-26 19:39:46 +01:00
Erik Johnston
806a06daf2
Rename get_users_whose_devices_changed
2019-06-26 19:39:19 +01:00
Erik Johnston
a2f6d31a63
Refactor get_user_ids_changed to pull less from DB
...
When a client asks for users whose devices have changed since a token we
used to pull *all* users from the database since the token, which could
easily be thousands of rows for old tokens.
This PR changes this to only check for changes for users the client is
actually interested in.
Fixes #5553
2019-06-26 12:03:44 +01:00
Amber Brown
32e7c9e7f2
Run Black. ( #5482 )
2019-06-20 19:32:02 +10:00
Andrew Morgan
2d1d7b7e6f
Prevent multiple device list updates from breaking a batch send ( #5156 )
...
fixes #5153
2019-06-06 23:54:00 +01:00
Amber Brown
7efd1d87c2
Run black on the rest of the storage module ( #4996 )
2019-04-03 10:07:29 +01:00
Erik Johnston
a84b8d56c2
Fixup slave stores
2019-03-04 18:04:57 +00:00
Richard van der Hoff
350f654e7b
Add unique indexes to a couple of tables
...
The indexes on device_lists_remote_extremeties can be unique, and they
therefore should, to ensure that the db remains consistent.
2018-11-02 10:36:13 +00:00
Richard van der Hoff
50e328d1e7
Remove redundant database locks for device list updates
...
We can rely on the application-level per-user linearizer.
2018-11-02 10:36:13 +00:00
Erik Johnston
88e5ffe6fe
Deduplicate device updates sent over replication
...
We currently send several kHz of device list updates over replication
occisonally, which often causes the replications streams to lag and then
get dropped.
A lot of those updates will actually be duplicates, since we don't send
e.g. device_ids across replication, so let's deduplicate it when we pull
them out of the database.
2018-10-29 17:34:34 +00:00
Amber Brown
14e4d4f4bf
Port storage/ to Python 3 ( #3725 )
2018-08-31 00:19:58 +10:00
Richard van der Hoff
03751a6420
Fix some looping_call calls which were broken in #3604
...
It turns out that looping_call does check the deferred returned by its
callback, and (at least in the case of client_ips), we were relying on this,
and I broke it in #3604 .
Update run_as_background_process to return the deferred, and make sure we
return it to clock.looping_call.
2018-07-26 11:48:08 +01:00
Richard van der Hoff
371da42ae4
Wrap a number of things that run in the background
...
This will reduce the number of "Starting db connection from sentinel context"
warnings, and will help with our metrics.
2018-07-25 09:41:12 +01:00
Matthew Hodgson
9e40834f74
yes, we do need to invalidate the device_id_exists_cache when deleting a remote device
2018-07-19 11:15:10 +01:00
Matthew Hodgson
12ec58301f
shift to using an explicit deleted flag on m.device_list_update EDUs
...
and generally make it work.
2018-07-12 11:39:43 +01:00
Matthew Hodgson
5797f5542b
WIP to announce deleted devices over federation
...
Previously we queued up the poke correctly when the device was deleted,
but then the actual EDU wouldn't get sent, as the device was no longer known.
Instead, we now send EDUs for deleted devices too if there's a poke for them.
2018-07-12 01:32:39 +01:00
Amber Brown
49af402019
run isort
2018-07-09 16:09:20 +10:00
Amber Brown
6350bf925e
Attempt to be more performant on PyPy ( #3462 )
2018-06-28 14:49:57 +01:00
Adrian Tschira
933bf2dd35
replace some iteritems with six
...
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-05-19 17:59:26 +02:00
Erik Johnston
926ba76e23
Replace ujson with simplejson
2018-03-15 23:43:31 +00:00