Commit Graph

1705 Commits

Author SHA1 Message Date
Erik Johnston
7c679b1118 Merge pull request #1106 from matrix-org/erikj/state_reindex_concurrent
Create new index concurrently
2016-09-12 14:38:40 +01:00
Erik Johnston
d080b3425c Merge pull request #1105 from matrix-org/erikj/make_notif_highlight_query_fast
Optimise /notifications query
2016-09-12 14:34:12 +01:00
Erik Johnston
03a98aff3c Create new index concurrently 2016-09-12 14:27:01 +01:00
Erik Johnston
fa20c9ce94 Change the index to be stream_ordering, highlight 2016-09-12 14:04:08 +01:00
Erik Johnston
5ef5435529 Remove unused import 2016-09-12 13:32:58 +01:00
Mark Haines
aa7b890cfe Merge pull request #1104 from matrix-org/markjh/direct_to_device_federation_sync
Fix direct to device messages recieved over federation to notify sync
2016-09-12 13:25:23 +01:00
Erik Johnston
7cd6edb947 Use register_background_index_update 2016-09-12 12:54:48 +01:00
Erik Johnston
0294c14ec4 Add back in query change 2016-09-12 12:43:56 +01:00
Erik Johnston
7fe42cf949 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/make_notif_highlight_query_fast 2016-09-12 12:37:09 +01:00
Erik Johnston
15ca0c6a4d Make reindex happen in bg 2016-09-12 12:36:36 +01:00
Mark Haines
a232e06100 Fix direct to device messages recieved over federation to notify sync 2016-09-12 12:30:46 +01:00
Mark Haines
31f85f9db9 Add comments to existing schema deltas that used "CREATE INDEX" directly 2016-09-12 11:00:26 +01:00
Erik Johnston
54417999b6 Revert "Add index to event_push_actions" 2016-09-12 10:39:55 +01:00
Erik Johnston
897d57bc58 Change state fetch query for postgres to be faster
It turns out that postgres doesn't like doing a list of OR's and is
about 1000x slower, so we just issue a query for each specific type
seperately.
2016-09-12 10:05:07 +01:00
David Baker
b91e2833b3 Merge remote-tracking branch 'origin/develop' into dbkr/make_notif_highlight_query_fast 2016-09-09 19:11:34 +01:00
David Baker
f2acc3dcf9 Add index to event_push_actions
and remove room_id caluse so it uses it

Mostly from @negativemjark
2016-09-09 18:54:54 +01:00
Mark Haines
647c724573 Use the previous MAX value if any to set the stream_id 2016-09-09 11:52:44 +01:00
Mark Haines
6a6cbfcf1e Track the max_stream_device_id in a separate table, since we delete from the inbox table 2016-09-09 11:48:23 +01:00
Mark Haines
303b6f29f0 Merge pull request #1087 from matrix-org/markjh/reapply_delta
Reapply 34/device_outbox in 35/device_outbox_again.py since the schem…
2016-09-09 11:24:27 +01:00
Erik Johnston
1fe7ca1362 Merge branch 'release-v0.17.3' of github.com:matrix-org/synapse into develop 2016-09-09 11:15:40 +01:00
Mark Haines
0877157353 Just move the schema and add some DROPs 2016-09-09 11:04:47 +01:00
Erik Johnston
2ffec928e2 Reduce batch size to be under SQL limit 2016-09-09 11:03:31 +01:00
Matthew Hodgson
b8f84f99ff Merge pull request #1081 from matrix-org/dbkr/notifications_only_highlight
Implement `only=highlight` on `/notifications`
2016-09-09 00:09:51 +01:00
Mark Haines
43b77c5d97 Only catch databas errors 2016-09-08 17:44:21 +01:00
Mark Haines
7d5b142547 Add a stub run_upgrade 2016-09-08 17:39:11 +01:00
David Baker
c3276aef25 Merge pull request #1080 from matrix-org/dbkr/fix_notifications_api_with_from
Fix /notifications API when used with `from` param
2016-09-08 17:35:35 +01:00
Mark Haines
fa722a699c Reapply 34/device_outbox in 35/device_outbox_again.py since the schema was bumped before it landed on develop 2016-09-08 17:35:16 +01:00
Erik Johnston
5c688739d6 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/reindex_state_groups 2016-09-08 16:52:09 +01:00
Erik Johnston
ebb46497ba Add delta file 2016-09-08 16:38:54 +01:00
Erik Johnston
5beda10bbd Reindex state_groups_state after pruning 2016-09-08 16:18:01 +01:00
Mark Haines
a1c8f268e5 Support wildcard device_ids for direct to device messages 2016-09-08 15:13:05 +01:00
Mark Haines
fa9d36e050 Merge branch 'develop' into markjh/direct_to_device_federation 2016-09-08 13:43:43 +01:00
David Baker
4ef222ab61 Implement only=highlight on /notifications 2016-09-08 13:43:35 +01:00
Erik Johnston
61cd9af09b Log delta files we're applying 2016-09-08 13:40:46 +01:00
David Baker
c5b49eb7ca Fix /notifications API when used with from param 2016-09-08 09:40:10 +01:00
Erik Johnston
b568ca309c Temporarily disable sequential scans for state fetching 2016-09-08 09:38:54 +01:00
Mark Haines
3c320c006c Merge pull request #1077 from matrix-org/markjh/device_logging
Log the types and values when failing to store devices
2016-09-07 18:24:24 +01:00
Mark Haines
85b51fdd6b Log the types and values when failing to store devices 2016-09-07 17:19:18 +01:00
Mark Haines
2a0159b8ae Fix the stream change cache to work over replication 2016-09-07 15:58:00 +01:00
Mark Haines
31a07d2335 Add stream change caches for device messages 2016-09-07 15:27:07 +01:00
Erik Johnston
513188aa56 Comment 2016-09-07 14:53:23 +01:00
Erik Johnston
fadb01551a Add appopriate framing clause 2016-09-07 14:39:01 +01:00
Erik Johnston
d25c20ccbe Use windowing function to make use of index 2016-09-07 14:22:22 +01:00
Mark Haines
7d893beebe Comment the add_messages storage functions 2016-09-07 12:03:37 +01:00
Mark Haines
d4a35ada28 Send device messages over federation 2016-09-06 18:16:20 +01:00
Mark Haines
e020834e4f Add storage methods for federated device messages 2016-09-06 15:12:13 +01:00
Mark Haines
2ad72da931 Add tables for federated device messages
Adds tables for storing the messages that need to be sent to a
remote device and for deduplicating messages received.
2016-09-06 15:10:40 +01:00
Mark Haines
f4164edb70 Move _add_messages_to_device_inbox_txn into a separate method 2016-09-06 11:26:37 +01:00
Erik Johnston
0595413c0f Scale the batch size so that we're not bitten by the minimum 2016-09-05 15:49:57 +01:00
Erik Johnston
a7032abb2e Correctly handle reindexing state groups that already have an edge 2016-09-05 15:07:23 +01:00