Richard van der Hoff
c9f11d09fc
Add missing index on users_in_public_rooms. ( #5894 )
2019-08-22 10:43:13 +01:00
Richard van der Hoff
119aa31b10
Servlet to purge old rooms ( #5845 )
2019-08-22 10:42:59 +01:00
Richard van der Hoff
4dab867288
Drop some unused tables. ( #5893 )
...
These tables are never used, so we may as well drop them.
2019-08-21 13:16:28 +01:00
Hubert Chathi
8a5f6ed130
Merge pull request #5857 from matrix-org/uhoreg/fix_e2e_room_keys_index
...
add the version field to the index for e2e_room_keys
2019-08-16 17:45:50 -07:00
Hubert Chathi
c058aeb88d
update set_e2e_room_key to agree with fixed index
2019-08-14 18:02:58 -07:00
Hubert Chathi
b7f7cc7ace
add the version field to the index for e2e_room_keys
2019-08-14 17:14:40 -07:00
Erik Johnston
af187805b3
Merge pull request #5809 from matrix-org/erikj/handle_pusher_stop
...
Handle pusher being deleted during processing.
2019-08-13 14:08:29 +01:00
Erik Johnston
fb3469f53a
Clarify docstring
...
Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2019-08-13 10:17:23 +01:00
Erik Johnston
2546f32b90
Merge pull request #5826 from matrix-org/erikj/reduce_event_pauses
...
Don't unnecessarily block notifying of new events.
2019-08-13 09:36:25 +01:00
Richard van der Hoff
fb86217553
Merge pull request #5788 from matrix-org/rav/metaredactions
...
Fix handling of redactions of redactions
2019-08-12 12:25:19 +01:00
Erik Johnston
bf4db42920
Don't unnecessarily block notifying of new events.
...
When persisting events we calculate new stream orderings up front.
Before we notify about an event all events with lower stream orderings
must have finished being persisted.
This PR moves the assignment of stream ordering till *after* calculated
the new current state and split the batch of events into separate chunks
for persistence. This means that if it takes a long time to calculate
new current state then it will not block events in other rooms being
notified about.
This should help reduce some global pauses in the events stream which
can last for tens of seconds (if not longer), caused by some
particularly expensive state resolutions.
2019-08-06 13:32:02 +01:00
Andrew Morgan
5d018d23f0
Have ClientReaderSlavedStore inherit RegistrationStore ( #5806 )
...
Fixes #5803
2019-08-01 13:54:56 +01:00
Erik Johnston
d02e41dcb2
Handle pusher being deleted during processing.
...
Instead of throwing a StoreError lets break out of processing loop and
mark the pusher as stopped.
2019-08-01 13:44:12 +01:00
Erik Johnston
123c04daa7
Don't recreate column
2019-07-30 16:42:48 +01:00
Erik Johnston
15056ca208
Fix current_state_events membership background update.
...
Turns out not all rooms are in `rooms`, so lets fetch the room list from
`current_state_events`. We move the delta file to force it to be run
again.
2019-07-30 14:51:41 +01:00
Richard van der Hoff
4e97eb89e5
Handle loops in redaction events
2019-07-30 00:00:34 +01:00
Richard van der Hoff
448bcfd0f9
recursively fetch redactions
2019-07-30 00:00:34 +01:00
Richard van der Hoff
e6a6c4fbab
split _get_events_from_db out of _enqueue_events
2019-07-29 23:15:15 +01:00
Richard van der Hoff
c9964ba600
Return dicts from _fetch_event_list
2019-07-29 23:15:15 +01:00
Erik Johnston
7c8c3b8437
Merge pull request #5774 from matrix-org/erikj/fix_rejected_membership
...
Fix room summary when rejected events are in state
2019-07-29 17:15:15 +01:00
Erik Johnston
3e013b7c8e
Merge pull request #5752 from matrix-org/erikj/forgotten_user
...
Remove some more joins on room_memberships
2019-07-29 17:15:01 +01:00
Erik Johnston
df3a5db629
Expand comment
2019-07-29 16:40:25 +01:00
Erik Johnston
105e7f6ed3
Remove lost comment
2019-07-29 16:09:48 +01:00
Erik Johnston
45df38e61b
Fix current_state bg update to work on old SQLite
2019-07-29 13:04:10 +01:00
Richard van der Hoff
d74595e2ca
Merge branch 'master' into develop
2019-07-26 12:39:33 +01:00
Erik Johnston
14c24c9037
Fix room summary when rejected events are in state
...
Annoyingly, `current_state_events` table can include rejected events,
in which case the membership column will be null. To work around this
lets just always filter out null membership for now.
2019-07-26 10:11:36 +01:00
Richard van der Hoff
0f2ecb961e
Fix DoS when there is a cycle in redaction events
...
Make sure that synapse doesn't explode when a redaction redacts itself, or
there is a larger cycle.
2019-07-26 06:36:48 +00:00
Richard van der Hoff
26d742fed6
Merge pull request #5767 from matrix-org/rav/redactions/cross_room_id
...
log when a redaction attempts to redact an event in a different room
2019-07-25 18:49:56 +01:00
Richard van der Hoff
b1605cdd23
log when a redaction attempts to redact an event in a different room
2019-07-25 18:26:20 +01:00
Richard van der Hoff
f30a71a67b
Stop trying to fetch events with event_id=None. ( #5753 )
...
`None` is not a valid event id, so queuing up a database fetch for it seems
like a silly thing to do.
I considered making `get_event` return `None` if `event_id is None`, but then
its interaction with `allow_none` seemed uninituitive, and strong typing ftw.
2019-07-24 13:16:18 +01:00
Erik Johnston
0c4a99607e
Remove join when calculating room summaries.
2019-07-24 11:49:15 +01:00
Erik Johnston
62921fb53e
Remove join on room_memberships when fetching rooms for user.
2019-07-24 11:45:58 +01:00
Erik Johnston
32768e96d4
Add function to get all forgotten rooms for user
...
This will allow us to efficiently filter out rooms that have been
forgotten in other queries without having to join against the
`room_memberships` table.
2019-07-24 11:44:23 +01:00
Amber Brown
4806651744
Replace returnValue with return ( #5736 )
2019-07-23 23:00:55 +10:00
Erik Johnston
b2a629ef49
Speed up current state background update.
...
Turns out that storing huge JSON arrays in the progress JSON isn't
something that postgres particularly likes.
2019-07-23 10:01:30 +01:00
Jason Robinson
66f5ff72fd
Add user_type
to returned fields in admin API user list endpoints
...
Mostly user type will be empty (normal user) but there is also the
"support" user type.
Signed-off-by: Jason Robinson <jasonr@matrix.org>
2019-07-22 15:29:18 +03:00
Erik Johnston
7704873cb8
Merge pull request #5720 from matrix-org/erikj/transactions_upsert
...
Use upsert when updating destination retry interval
2019-07-19 16:51:16 +01:00
Erik Johnston
d7bd9651bc
Merge pull request #5713 from matrix-org/erikj/use_cache_for_filtered_state
...
Delegate to cached version when using get_filtered_current_state_ids
2019-07-19 16:30:49 +01:00
Erik Johnston
5c07c97c09
Merge pull request #5706 from matrix-org/erikj/add_memberships_to_current_state
...
Add membership column to current_state_events table
2019-07-19 16:30:33 +01:00
Erik Johnston
2410335507
Use upsert when updating destination retry interval
2019-07-19 13:40:24 +01:00
Erik Johnston
bd2e1a2aa8
LoggingTransaction accepts None for callback lists.
...
Its a bit disingenuousto give LoggingTransaction lists to append
callbacks to if we're not going to run the callbacks.
2019-07-19 13:36:04 +01:00
Erik Johnston
ebc5ed1296
Update comment for new column
2019-07-19 13:29:02 +01:00
Erik Johnston
10523241d8
Delegate to cached version when using get_filtered_current_state_ids
...
In the case where it gets called with `StateFilter.all()`
2019-07-18 15:17:39 +01:00
Erik Johnston
8e1ada9e6f
Use the current_state_events.membership column
2019-07-18 14:16:01 +01:00
Erik Johnston
059d8c1a4e
Track if current_state_events.membership is up to date
2019-07-18 14:16:01 +01:00
Erik Johnston
c618a5d348
Add background update for current_state_events.membership column
2019-07-18 14:16:01 +01:00
Erik Johnston
6de09e07a6
Add membership column to current_state_events table.
...
It turns out that doing a join is surprisingly expensive for the DB to
do when room_membership table is larger than the disk cache.
2019-07-18 14:15:57 +01:00
Richard van der Hoff
9c70a02a9c
Ignore redactions of m.room.create events ( #5701 )
2019-07-17 19:08:02 +01:00
Richard van der Hoff
2091c91fde
More refactoring in get_events_as_list
( #5707 )
...
We can now use `_get_events_from_cache_or_db` rather than going right back to
the database, which means that (a) we can benefit from caching, and (b) it
opens the way forward to more extensive checks on the original event.
We now always require the original event to exist before we will serve up a
redaction.
2019-07-17 17:34:13 +01:00
Richard van der Hoff
375162b3c3
Fix redaction authentication ( #5700 )
...
Ensures that redactions are correctly authenticated for recent room versions.
There are a few things going on here:
* `_fetch_event_rows` is updated to return a dict rather than a list of rows.
* Rather than returning multiple copies of an event which was redacted
multiple times, it returns the redactions as a list within the dict.
* It also returns the actual rejection reason, rather than merely the fact
that it was rejected, so that we don't have to query the table again in
`_get_event_from_row`.
* The redaction handling is factored out of `_get_event_from_row`, and now
checks if any of the redactions are valid.
2019-07-17 16:52:02 +01:00