4121 Commits

Author SHA1 Message Date
Erik Johnston
7a43482f19
Use execute_batch in more places (#9188)
* Use execute_batch in more places

* Newsfile
2021-01-21 14:44:12 +00:00
Erik Johnston
eee6fcf5fa
Use execute_batch instead of executemany in places (#9181)
`execute_batch` does fewer round trips in postgres than `executemany`, but does not give a correct `txn.rowcount` result after.
2021-01-21 10:22:53 +00:00
Richard van der Hoff
0cd2938bc8
Support icons for Identity Providers (#9154) 2021-01-20 08:15:14 -05:00
Erik Johnston
6633a4015a
Allow moving account data and receipts streams off master (#9104) 2021-01-18 15:47:59 +00:00
Erik Johnston
350d9923cd
Make chain cover index bg update go faster (#9124)
We do this by allowing a single iteration to process multiple rooms at a
time, as there are often a lot of really tiny rooms, which can massively
slow things down.
2021-01-15 17:18:37 +00:00
Richard van der Hoff
2de7e263ed
Ensure we store pusher data as text (#9117)
I don't think there's any need to use canonicaljson here.

Fixes: #4475.
2021-01-15 16:57:23 +00:00
Patrick Cloke
d34c6e1279
Add type hints to media rest resources. (#9093) 2021-01-15 10:57:37 -05:00
Erik Johnston
d2479c6870
Fix perf of get_cross_signing_keys (#9116) 2021-01-14 17:57:09 +00:00
Erik Johnston
659c415ed4
Fix chain cover background update to work with split out event persisters (#9115) 2021-01-14 17:19:35 +00:00
Erik Johnston
631dd06f2c
Fix get destinations to catch up query. (#9114)
t was doing a sequential scan on `destination_rooms`, which took
minutes.
2021-01-14 16:47:21 +00:00
Erik Johnston
7036e24e98
Add background update for add chain cover index (#9029) 2021-01-14 15:18:27 +00:00
Patrick Cloke
d1eb1b96e8
Register the /devices endpoint on workers. (#9092) 2021-01-13 12:35:40 -05:00
Dirk Klimpel
7a2e9b549d
Remove user's avatar URL and displayname when deactivated. (#8932)
This only applies if the user's data is to be erased.
2021-01-12 16:30:15 -05:00
Jason Robinson
c177faf5a9 Remove trailing whitespace to appease the linter
Signed-off-by: Jason Robinson <jasonr@matrix.org>
2021-01-11 23:55:44 +02:00
Jason Robinson
49c619a9a2 Simplify delete_forward_extremities_for_room_txn SQL
As per feedback.

Signed-off-by: Jason Robinson <jasonr@matrix.org>
2021-01-11 23:49:58 +02:00
Jason Robinson
da16d06301 Address pr feedback
* docs updates
* prettify SQL
* add missing copyright
* cursor_to_dict
* update touched files copyright years

Signed-off-by: Jason Robinson <jasonr@matrix.org>
2021-01-11 23:43:58 +02:00
Erik Johnston
c9195744a4
Move more encryption endpoints off master (#9068) 2021-01-11 18:01:27 +00:00
Erik Johnston
1315a2e8be
Use a chain cover index to efficiently calculate auth chain difference (#8868) 2021-01-11 16:09:22 +00:00
Erik Johnston
4e04435bda
Remove old tables after schema version bump (#9055)
These tables are unused, and can be dropped now the schema version has been bumped.
2021-01-11 13:58:19 +00:00
Erik Johnston
63f4990298
Ensure rejected events get added to some metadata tables (#9016)
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2021-01-11 13:57:33 +00:00
Jerin J Titus
c21d8f1c1d
Drop last_used column from access_tokens (#9025)
* Dropped last_used column from access_tokens

Signed-off-by: Jerin J Titus <72017981+jerinjtitus@users.noreply.github.com>
2021-01-11 10:23:49 +00:00
Jason Robinson
b52fb703f7 Don't try to use f-strings
Signed-off-by: Jason Robinson <jasonr@matrix.org>
2021-01-11 09:47:03 +02:00
Jason Robinson
2eb421b606 Merge branch 'develop' into jaywink/admin-forward-extremities 2021-01-09 22:00:04 +02:00
Jason Robinson
90ad4d443a Implement clearing cache after deleting forward extremities
Also run linter.

Signed-off-by: Jason Robinson <jasonr@matrix.org>
2021-01-09 21:57:41 +02:00
Erik Johnston
fa5f5cbc74
Fix error handling during insertion of client IPs (#9051)
You can't continue using a transaction once an exception has been
raised, so catching and dropping the error here is pointless and just
causes more errors.
2021-01-08 14:15:20 +00:00
Jason Robinson
85c0999bfb Add Rooms admin forward extremities DELETE endpoint
Signed-off-by: Jason Robinson <jasonr@matrix.org>
2021-01-08 00:12:23 +02:00
Jason Robinson
b849e46139 Add forward extremities endpoint to rooms admin API
GET /_synapse/admin/v1/rooms/<identifier>/forward_extremities now gets forward extremities for a room, returning count and the list of extremities.

Signed-off-by: Jason Robinson <jasonr@matrix.org>
2021-01-07 23:01:59 +02:00
Erik Johnston
b530eaa262
Allow running sendToDevice on workers (#9044) 2021-01-07 20:19:26 +00:00
Erik Johnston
63593134a1
Some cleanups to device inbox store. (#9041) 2021-01-07 17:20:44 +00:00
Patrick Cloke
23d701864f
Improve the performance of calculating ignored users in large rooms (#9024)
This allows for efficiently finding which users ignore a particular
user.

Co-authored-by: Erik Johnston <erik@matrix.org>
2021-01-07 13:03:38 +00:00
Erik Johnston
eee3c3c52f
Handle updating schema version without any deltas. (#9033)
This can happen when using a split out state database and we've upgraded
the schema version without there being any changes in the state schema.
2021-01-07 11:33:36 +00:00
Patrick Cloke
06fefe0bb1
Add type hints to the logging context code. (#8939) 2021-01-05 08:06:55 -05:00
Patrick Cloke
1c9a850562
Add type hints to the crypto module. (#8999) 2021-01-04 10:04:50 -05:00
Patrick Cloke
637282bb50
Add additional type hints to the storage module. (#8980) 2020-12-30 08:09:53 -05:00
Patrick Cloke
9999eb2d02
Add type hints to admin and room list handlers. (#8973) 2020-12-29 17:42:10 -05:00
Patrick Cloke
d0c3c24eb2
Drop the unused local_invites table. (#8979)
This table has been unused since Synapse v1.17.0.
2020-12-29 07:26:29 -05:00
Patrick Cloke
5d4c330ed9
Allow re-using a UI auth validation for a period of time (#8970) 2020-12-18 07:33:57 -05:00
Erik Johnston
70586aa63e
Try and drop stale extremities. (#8929)
If we see stale extremities while persisting events, and notice that
they don't change the result of state resolution, we drop them.
2020-12-18 09:49:18 +00:00
Brendan Abolivier
f2783fc201
Use the simple dictionary in full text search for the user directory (#8959)
* Use the simple dictionary in fts for the user directory

* Clarify naming
2020-12-17 14:42:30 +01:00
Dirk Klimpel
06006058d7
Make search statement in List Room and User Admin API case-insensitive (#8931) 2020-12-17 10:43:37 +00:00
Patrick Cloke
bd30cfe86a
Convert internal pusher dicts to attrs classes. (#8940)
This improves type hinting and should use less memory.
2020-12-16 11:25:30 -05:00
Patrick Cloke
be2db93b3c
Do not assume that the contents dictionary includes history_visibility. (#8945) 2020-12-16 08:46:37 -05:00
Patrick Cloke
b3a4b53587
Fix handling of stream tokens for push. (#8943)
Removes faulty assertions and fixes the logic to ensure the max
stream token is always set.
2020-12-15 10:41:34 -05:00
Dirk Klimpel
0a34cdfc66
Add number of local devices to Room Details Admin API (#8886) 2020-12-11 10:42:47 +00:00
Erik Johnston
df4b1e9c74
Pass room_id to get_auth_chain_difference (#8879)
This is so that we can choose which algorithm to use based on the room ID.
2020-12-04 15:52:49 +00:00
Richard van der Hoff
0bac276890 UIA: offer only available auth flows
During user-interactive auth, do not offer password auth to users with no
password, nor SSO auth to users with no SSO.

Fixes #7559.
2020-12-02 18:54:15 +00:00
Richard van der Hoff
9edff901d1
Add missing ordering to background updates (#8850)
It's important that we make sure our background updates happen in a defined
order, to avoid disasters like #6923.

Add an ordering to all of the background updates that have landed since #7190.
2020-12-01 15:52:49 +00:00
Richard van der Hoff
f8d13ca13d
Drop (almost) unused index on event_json (#8845) 2020-11-30 18:44:09 +00:00
Andrew Morgan
d963c69ba5
Speed up remote invite rejection database call (#8815)
This is another PR that grew out of #6739.

The existing code for checking whether a user is currently invited to a room when they want to leave the room looks like the following:

f737368a26/synapse/handlers/room_member.py (L518-L540)

It calls `get_invite_for_local_user_in_room`, which will actually query *all* rooms the user has been invited to, before iterating over them and matching via the room ID. It will then return a tuple of a lot of information which we pull the event ID out of.

I need to do a similar check for knocking, but this code wasn't very efficient. I then tried to write a different implementation using `StateHandler.get_current_state` but this actually didn't work as we haven't *joined* the room yet - we've only been invited to it. That means that only certain tables in Synapse have our desired `invite` membership state. One of those tables is `local_current_membership`.

So I wrote a store method that just queries that table instead
2020-11-25 20:06:13 +00:00
Will Hunt
51338491c9
Improve appservice handler to send only the most recent read receipts when no stream_id is stored. (#8744)
* Make this line debug (it's noisy)

* Don't include from_key for presence if we are at 0

* Limit read receipts for all rooms to 100

* changelog.d/8744.bugfix

* Allow from_key to be None

* Update 8744.bugfix

* The from_key is superflous

* Update comment
2020-11-18 18:54:09 +00:00