Commit Graph

3147 Commits

Author SHA1 Message Date
Hubert Chathi
3b0b22cb05 use stream ID generator instead of timestamp 2019-08-28 17:17:21 -07:00
Hubert Chathi
7c3abc6572 apply PR review suggestions 2019-08-21 13:19:35 -07:00
Hubert Chathi
fac1cdc562 make changes from PR review 2019-08-01 21:51:19 -04:00
Hubert Chathi
336c546d6a Merge branch 'cross-signing_hidden' into cross-signing_keys 2019-08-01 16:31:40 -04:00
Hubert Chathi
d78a4fe156 don't need to return the hidden column any more 2019-08-01 02:16:09 -04:00
Hubert Chathi
73b26f827c really fix queries to work with Postgres
(by going back to not using SQL directly)
2019-07-31 18:37:05 -04:00
Hubert Chathi
430ea08186 PostgreSQL, Y U no like? 2019-07-31 15:38:11 -04:00
Hubert Chathi
185188be03 remove extra SQL query param 2019-07-31 15:18:15 -04:00
Hubert Chathi
c1f0a5636c Merge branch 'develop' into cross-signing_hidden 2019-07-30 23:33:06 -04:00
Hubert Chathi
bc95890822 Merge branch 'cross-signing_hidden' into cross-signing_keys 2019-07-30 23:21:26 -04:00
Hubert Chathi
781ade836b apply changes from PR review 2019-07-30 23:09:50 -04:00
Erik Johnston
123c04daa7 Don't recreate column 2019-07-30 16:42:48 +01:00
Erik Johnston
15056ca208 Fix current_state_events membership background update.
Turns out not all rooms are in `rooms`, so lets fetch the room list from
`current_state_events`. We move the delta file to force it to be run
again.
2019-07-30 14:51:41 +01:00
Erik Johnston
7c8c3b8437
Merge pull request #5774 from matrix-org/erikj/fix_rejected_membership
Fix room summary when rejected events are in state
2019-07-29 17:15:15 +01:00
Erik Johnston
3e013b7c8e
Merge pull request #5752 from matrix-org/erikj/forgotten_user
Remove some more joins on room_memberships
2019-07-29 17:15:01 +01:00
Erik Johnston
df3a5db629 Expand comment 2019-07-29 16:40:25 +01:00
Erik Johnston
105e7f6ed3 Remove lost comment 2019-07-29 16:09:48 +01:00
Erik Johnston
45df38e61b Fix current_state bg update to work on old SQLite 2019-07-29 13:04:10 +01:00
Richard van der Hoff
d74595e2ca Merge branch 'master' into develop 2019-07-26 12:39:33 +01:00
Erik Johnston
14c24c9037 Fix room summary when rejected events are in state
Annoyingly, `current_state_events` table can include rejected events,
in which case the membership column will be null. To work around this
lets just always filter out null membership for now.
2019-07-26 10:11:36 +01:00
Richard van der Hoff
0f2ecb961e Fix DoS when there is a cycle in redaction events
Make sure that synapse doesn't explode when a redaction redacts itself, or
there is a larger cycle.
2019-07-26 06:36:48 +00:00
Richard van der Hoff
26d742fed6
Merge pull request #5767 from matrix-org/rav/redactions/cross_room_id
log when a redaction attempts to redact an event in a different room
2019-07-25 18:49:56 +01:00
Richard van der Hoff
b1605cdd23 log when a redaction attempts to redact an event in a different room 2019-07-25 18:26:20 +01:00
Hubert Chathi
c659b9f94f allow uploading keys for cross-signing 2019-07-25 11:08:24 -04:00
Hubert Chathi
d1c7c2a98a allow devices to be marked as "hidden"
This is a prerequisite for cross-signing, as it allows us to create other things
that live within the device namespace, so they can be used for signatures.
2019-07-24 23:21:52 -04:00
Richard van der Hoff
f30a71a67b
Stop trying to fetch events with event_id=None. (#5753)
`None` is not a valid event id, so queuing up a database fetch for it seems
like a silly thing to do.

I considered making `get_event` return `None` if `event_id is None`, but then
its interaction with `allow_none` seemed uninituitive, and strong typing ftw.
2019-07-24 13:16:18 +01:00
Erik Johnston
0c4a99607e Remove join when calculating room summaries. 2019-07-24 11:49:15 +01:00
Erik Johnston
62921fb53e Remove join on room_memberships when fetching rooms for user. 2019-07-24 11:45:58 +01:00
Erik Johnston
32768e96d4 Add function to get all forgotten rooms for user
This will allow us to efficiently filter out rooms that have been
forgotten in other queries without having to join against the
`room_memberships` table.
2019-07-24 11:44:23 +01:00
Amber Brown
4806651744
Replace returnValue with return (#5736) 2019-07-23 23:00:55 +10:00
Erik Johnston
b2a629ef49 Speed up current state background update.
Turns out that storing huge JSON arrays in the progress JSON isn't
something that postgres particularly likes.
2019-07-23 10:01:30 +01:00
Jason Robinson
66f5ff72fd Add user_type to returned fields in admin API user list endpoints
Mostly user type will be empty (normal user) but there is also the
"support" user type.

Signed-off-by: Jason Robinson <jasonr@matrix.org>
2019-07-22 15:29:18 +03:00
Erik Johnston
7704873cb8
Merge pull request #5720 from matrix-org/erikj/transactions_upsert
Use upsert when updating destination retry interval
2019-07-19 16:51:16 +01:00
Erik Johnston
d7bd9651bc
Merge pull request #5713 from matrix-org/erikj/use_cache_for_filtered_state
Delegate to cached version when using get_filtered_current_state_ids
2019-07-19 16:30:49 +01:00
Erik Johnston
5c07c97c09
Merge pull request #5706 from matrix-org/erikj/add_memberships_to_current_state
Add membership column to current_state_events table
2019-07-19 16:30:33 +01:00
Erik Johnston
2410335507 Use upsert when updating destination retry interval 2019-07-19 13:40:24 +01:00
Erik Johnston
bd2e1a2aa8 LoggingTransaction accepts None for callback lists.
Its a bit disingenuousto give LoggingTransaction lists to append
callbacks to if we're not going to run the callbacks.
2019-07-19 13:36:04 +01:00
Erik Johnston
ebc5ed1296 Update comment for new column 2019-07-19 13:29:02 +01:00
Erik Johnston
10523241d8 Delegate to cached version when using get_filtered_current_state_ids
In the case where it gets called with `StateFilter.all()`
2019-07-18 15:17:39 +01:00
Erik Johnston
8e1ada9e6f Use the current_state_events.membership column 2019-07-18 14:16:01 +01:00
Erik Johnston
059d8c1a4e Track if current_state_events.membership is up to date 2019-07-18 14:16:01 +01:00
Erik Johnston
c618a5d348 Add background update for current_state_events.membership column 2019-07-18 14:16:01 +01:00
Erik Johnston
6de09e07a6 Add membership column to current_state_events table.
It turns out that doing a join is surprisingly expensive for the DB to
do when room_membership table is larger than the disk cache.
2019-07-18 14:15:57 +01:00
Richard van der Hoff
9c70a02a9c
Ignore redactions of m.room.create events (#5701) 2019-07-17 19:08:02 +01:00
Richard van der Hoff
2091c91fde
More refactoring in get_events_as_list (#5707)
We can now use `_get_events_from_cache_or_db` rather than going right back to
the database, which means that (a) we can benefit from caching, and (b) it
opens the way forward to more extensive checks on the original event.

We now always require the original event to exist before we will serve up a
redaction.
2019-07-17 17:34:13 +01:00
Richard van der Hoff
375162b3c3
Fix redaction authentication (#5700)
Ensures that redactions are correctly authenticated for recent room versions.

There are a few things going on here:

 * `_fetch_event_rows` is updated to return a dict rather than a list of rows.

 * Rather than returning multiple copies of an event which was redacted
   multiple times, it returns the redactions as a list within the dict.

 * It also returns the actual rejection reason, rather than merely the fact
   that it was rejected, so that we don't have to query the table again in
   `_get_event_from_row`.

 * The redaction handling is factored out of `_get_event_from_row`, and now
   checks if any of the redactions are valid.
2019-07-17 16:52:02 +01:00
Richard van der Hoff
65c5592b8e
Refactor get_events_as_list (#5699)
A couple of changes here:

* get rid of a redundant `allow_rejected` condition - we should already have filtered out any rejected
  events before we get to that point in the code, and the redundancy is confusing. Instead, let's stick in
  an assertion just to make double-sure we aren't leaking rejected events by mistake.

* factor out a `_get_events_from_cache_or_db` method, which is going to be important for a 
  forthcoming fix to redactions.
2019-07-17 16:49:19 +01:00
Erik Johnston
d86321300a
Merge pull request #5589 from matrix-org/erikj/admin_exfiltrate_data
Add basic function to get all data for a user out of synapse
2019-07-15 10:04:02 +01:00
Richard van der Hoff
5f158ec039
Implement access token expiry (#5660)
Record how long an access token is valid for, and raise a soft-logout once it
expires.
2019-07-12 17:26:02 +01:00
Erik Johnston
db0a50bc40 Fixup docstrings 2019-07-12 16:59:59 +01:00