Erik Johnston
de4f798f01
Handling expiring stream extrems correctly.
2016-09-15 17:34:59 +01:00
Erik Johnston
ea6dc356b0
Merge pull request #1125 from matrix-org/erikj/public_room_cache
...
Change get_pos_of_last_change to return upper bound
2016-09-15 15:48:53 +01:00
Erik Johnston
955f34d23e
Change get_pos_of_last_change to return upper bound
2016-09-15 15:12:07 +01:00
Erik Johnston
241d7d2d62
Merge pull request #1124 from matrix-org/erikj/enable_state_caching_workers
...
Enable state caches on workers
2016-09-15 15:01:47 +01:00
Erik Johnston
4be85281f9
Enable state caches on workers
2016-09-15 14:31:22 +01:00
Erik Johnston
cb3edec6af
Use stream_change cache to make get_forward_extremeties_for_room cache more effective
2016-09-15 14:28:13 +01:00
Erik Johnston
55e6fc917c
Add cache to get_forward_extremeties_for_room
2016-09-15 14:04:28 +01:00
Erik Johnston
68c1ed4d1a
Remove default public rooms limit
2016-09-15 13:56:20 +01:00
Erik Johnston
b82fa849c8
Merge pull request #1120 from matrix-org/erikj/push_invite_cache
...
Ensure we don't mutate the cache of push rules
2016-09-15 13:27:18 +01:00
Erik Johnston
e457034e99
Merge pull request #1121 from matrix-org/erikj/public_room_paginate
...
Add pagination support to publicRooms
2016-09-15 13:27:09 +01:00
Erik Johnston
1d98cf26be
By default limit /publicRooms to 100 entries
2016-09-15 13:18:35 +01:00
Erik Johnston
211786ecd6
Stream public room changes down replication
2016-09-15 11:47:23 +01:00
Erik Johnston
4fb65a1091
Base public room list off of public_rooms stream
2016-09-15 11:27:04 +01:00
Erik Johnston
5810cffd33
Pass since/from parameters over federation
2016-09-15 10:36:19 +01:00
Erik Johnston
f3eead0660
Allow paginating both forwards and backwards
2016-09-15 10:15:37 +01:00
Erik Johnston
4131381123
Remove support for aggregate room lists
2016-09-15 09:28:15 +01:00
Erik Johnston
6a5ded5988
Ensure we don't mutate the cache of push rules
2016-09-15 09:16:13 +01:00
Erik Johnston
4f181f361d
Accept optional token to public room list
2016-09-15 09:08:57 +01:00
Erik Johnston
c566f0ee17
Calculate the public room list from a stream_ordering
2016-09-14 17:42:47 +01:00
Erik Johnston
772c6067a3
Refactor public rooms to not pull out the full state for each room
2016-09-14 17:29:25 +01:00
Erik Johnston
baffe96d95
Add a room visibility stream
2016-09-14 17:29:19 +01:00
Erik Johnston
264a48aedf
Merge pull request #1117 from matrix-org/erikj/fix_state
...
Ensure we don't mutate state cache entries
2016-09-14 16:50:37 +01:00
Erik Johnston
ed992ae6ba
Add a DB index to figure out past state at a stream ordering in a room
2016-09-14 16:20:27 +01:00
Erik Johnston
e0b6db29ed
Split out public room list into a worker process
2016-09-14 14:42:51 +01:00
Erik Johnston
a70a43bc51
Move RoomListHandler into a separate file
2016-09-14 14:07:37 +01:00
Erik Johnston
f2b2cd8eb4
Amalgamate two identical consecutive if statements
2016-09-14 11:16:22 +01:00
Erik Johnston
00f51493f5
Fix reindex
2016-09-14 10:18:30 +01:00
Erik Johnston
d5ae1f1291
Ensure we don't mutate state cache entries
2016-09-14 10:03:48 +01:00
Matthew Hodgson
1b01488d27
Merge pull request #1111 from matrix-org/matthew/device-ids
...
make device IDs more useful for human disambiguation
2016-09-14 01:18:02 +01:00
Paul Evans
0f73f0e70e
Merge pull request #1116 from matrix-org/paul/tiny-fixes
...
Fix typo "persiting"
2016-09-13 13:27:42 +01:00
Paul "LeoNerd" Evans
ca35e54d6b
Fix typo "persiting"
2016-09-13 13:26:33 +01:00
Mark Haines
ad816b0add
Limit how often we ask for keys from dead servers
2016-09-13 11:53:50 +01:00
Erik Johnston
43253c10b8
Remove redundant event_auth index
2016-09-13 11:47:48 +01:00
Mark Haines
18ab019a4a
Move the E2E key handling into the e2e handler
2016-09-13 11:35:35 +01:00
Mark Haines
76b09c29b0
Merge pull request #1110 from matrix-org/markjh/e2e_timeout
...
Add a timeout parameter for end2end key queries.
2016-09-13 10:50:45 +01:00
Erik Johnston
ba6bc2faa0
Merge pull request #1109 from matrix-org/erikj/partial_indices
...
Add WHERE clause support to index creation
2016-09-13 09:06:16 +01:00
Matthew Hodgson
edbcb4152b
make device IDs more useful for human disambiguation
2016-09-13 00:02:39 +01:00
Mark Haines
949c2c5435
Add a timeout parameter for end2end key queries.
...
Add a timeout parameter for controlling how long synapse will wait
for responses from remote servers. For servers that fail include how
they failed to make it easier to debug.
Fetch keys from different servers in parallel rather than in series.
Set the default timeout to 10s.
2016-09-12 18:17:09 +01:00
Erik Johnston
b17af156c7
Remove where clause
2016-09-12 17:05:54 +01:00
Erik Johnston
1c9da43a95
Merge pull request #1108 from matrix-org/erikj/create_dm
...
Add is_direct param to /createRoom
2016-09-12 16:57:16 +01:00
Erik Johnston
0b32bb20bb
Index contains_url for file search queries
2016-09-12 16:57:05 +01:00
Erik Johnston
c94de0ab60
Add WHERE clause support to index creation
2016-09-12 16:55:01 +01:00
Erik Johnston
502c901e11
Merge pull request #1107 from matrix-org/erikj/backfill_none
...
Fix backfill when cannot find an event.
2016-09-12 16:48:01 +01:00
Erik Johnston
48a5a7552d
Add is_direct param to /createRoom
2016-09-12 16:34:20 +01:00
Erik Johnston
706b5d76ed
Fix backfill when cannot find an event.
...
`get_pdu` can succeed but return None.
2016-09-12 14:59:51 +01:00
Erik Johnston
7c679b1118
Merge pull request #1106 from matrix-org/erikj/state_reindex_concurrent
...
Create new index concurrently
2016-09-12 14:38:40 +01:00
Erik Johnston
d080b3425c
Merge pull request #1105 from matrix-org/erikj/make_notif_highlight_query_fast
...
Optimise /notifications query
2016-09-12 14:34:12 +01:00
Erik Johnston
03a98aff3c
Create new index concurrently
2016-09-12 14:27:01 +01:00
Erik Johnston
fa20c9ce94
Change the index to be stream_ordering, highlight
2016-09-12 14:04:08 +01:00
Erik Johnston
5ef5435529
Remove unused import
2016-09-12 13:32:58 +01:00
Mark Haines
aa7b890cfe
Merge pull request #1104 from matrix-org/markjh/direct_to_device_federation_sync
...
Fix direct to device messages recieved over federation to notify sync
2016-09-12 13:25:23 +01:00
Erik Johnston
7cd6edb947
Use register_background_index_update
2016-09-12 12:54:48 +01:00
Erik Johnston
0294c14ec4
Add back in query change
2016-09-12 12:43:56 +01:00
Erik Johnston
7fe42cf949
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/make_notif_highlight_query_fast
2016-09-12 12:37:09 +01:00
Erik Johnston
15ca0c6a4d
Make reindex happen in bg
2016-09-12 12:36:36 +01:00
Mark Haines
a232e06100
Fix direct to device messages recieved over federation to notify sync
2016-09-12 12:30:46 +01:00
Mark Haines
31f85f9db9
Add comments to existing schema deltas that used "CREATE INDEX" directly
2016-09-12 11:00:26 +01:00
Erik Johnston
54417999b6
Revert "Add index to event_push_actions"
2016-09-12 10:39:55 +01:00
Erik Johnston
45dc260060
Merge pull request #1101 from matrix-org/erikj/state_types_idx
...
Change state fetch query for postgres to be faster
2016-09-12 10:20:38 +01:00
Erik Johnston
d1c217c823
Merge pull request #1097 from matrix-org/erikj/replication_typing_rest
...
Correctly handle typing stream id resetting
2016-09-12 10:10:15 +01:00
Erik Johnston
897d57bc58
Change state fetch query for postgres to be faster
...
It turns out that postgres doesn't like doing a list of OR's and is
about 1000x slower, so we just issue a query for each specific type
seperately.
2016-09-12 10:05:07 +01:00
Erik Johnston
555460ae1b
Merge pull request #1095 from matrix-org/erikj/batch_edus
...
Clobber EDUs in send queue
2016-09-12 08:04:15 +01:00
Richard van der Hoff
4162f820ff
Merge pull request #1100 from VShell/fix-cas
...
Conform better to the CAS protocol specification
2016-09-09 21:57:42 +01:00
Shell Turner
29205e9596
Conform better to the CAS protocol specification
...
Redirect to CAS's /login endpoint properly, and
don't require an <attributes> element.
Signed-off-by: Shell Turner <cam.turn@gmail.com>
2016-09-09 21:20:14 +01:00
David Baker
b91e2833b3
Merge remote-tracking branch 'origin/develop' into dbkr/make_notif_highlight_query_fast
2016-09-09 19:11:34 +01:00
David Baker
f2acc3dcf9
Add index to event_push_actions
...
and remove room_id caluse so it uses it
Mostly from @negativemjark
2016-09-09 18:54:54 +01:00
Erik Johnston
3265def8c7
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/batch_edus
2016-09-09 18:06:01 +01:00
Erik Johnston
af4701b311
Fix incorrect attribute name
2016-09-09 17:36:56 +01:00
Erik Johnston
44330a21e9
Comment
2016-09-09 17:22:07 +01:00
Erik Johnston
464ffd1b5e
Comment
2016-09-09 17:17:23 +01:00
Erik Johnston
327425764e
Add edu.type as part of key. Remove debug logging
2016-09-09 17:13:30 +01:00
Mark Haines
dbff7e9436
Merge pull request #1096 from matrix-org/markjh/get_access_token
...
Add helper function for getting access_tokens from requests
2016-09-09 17:09:27 +01:00
Erik Johnston
a4339de9de
Correctly handle typing stream id resetting
2016-09-09 16:44:26 +01:00
Mark Haines
8aee5aa068
Add helper function for getting access_tokens from requests
...
Rather than reimplementing the token parsing in the various places.
This will make it easier to change the token parsing to allow access_tokens
in HTTP headers.
2016-09-09 16:33:15 +01:00
Erik Johnston
52b2318777
Clobber EDUs in send queue
2016-09-09 15:59:08 +01:00
Paul Evans
56f38d1776
Merge pull request #1091 from matrix-org/paul/third-party-lookup
...
Improvements to 3PE lookup API
2016-09-09 15:43:11 +01:00
Paul Evans
8cb252d00c
Merge pull request #1094 from matrix-org/paul/get-state-whole-event
...
Allow clients to ask for the whole of a single state event
2016-09-09 15:26:17 +01:00
Paul "LeoNerd" Evans
776594f99d
Log if rejecting 3PE query metadata result due to type check
2016-09-09 15:11:31 +01:00
Paul "LeoNerd" Evans
ed44c475d8
Reject malformed 3PE query metadata results earlier in AS API handling code
2016-09-09 15:07:04 +01:00
Erik Johnston
ab80d5e0a9
Drop replication log levels
2016-09-09 14:56:50 +01:00
Paul "LeoNerd" Evans
f25d74f69c
Minor fixes from PR comments
2016-09-09 14:54:16 +01:00
Paul "LeoNerd" Evans
d271383e63
Filter returned events for client-facing format
2016-09-09 14:40:15 +01:00
Paul "LeoNerd" Evans
0fc0a3bdff
Allow clients to specify the format a room state event is returned in
2016-09-09 14:34:29 +01:00
Erik Johnston
6c4d582144
Deduplicate presence in _update_states
2016-09-09 14:28:22 +01:00
Erik Johnston
685da5a3b0
Merge pull request #1092 from matrix-org/erikj/transaction_queue_check
...
Check if destination is ready for retry earlier
2016-09-09 13:49:56 +01:00
Erik Johnston
a6c6750166
Check if destination is ready for retry earlier
2016-09-09 13:46:05 +01:00
Paul "LeoNerd" Evans
bdbcfc2a80
appease pep8
2016-09-09 13:31:39 +01:00
Paul "LeoNerd" Evans
6eb0c8a2e4
Python isn't JavaScript; have to quote dict keys
2016-09-09 13:31:17 +01:00
Mark Haines
6b54fa81de
Merge pull request #1089 from matrix-org/markjh/direct_to_device_stream
...
Track the max device stream_id in a separate table,
2016-09-09 13:31:07 +01:00
Paul "LeoNerd" Evans
25eb769b26
Efficiency fix for lookups of a single protocol
2016-09-09 13:25:02 +01:00
Paul "LeoNerd" Evans
3328428d05
Allow lookup of a single 3PE protocol query metadata
2016-09-09 13:19:04 +01:00
Erik Johnston
4598682b43
Fix tightloop on sending transaction
2016-09-09 13:12:53 +01:00
Paul "LeoNerd" Evans
033d43e419
Don't corrupt shared cache on subsequent protocol requests
2016-09-09 13:10:36 +01:00
Mark Haines
647c724573
Use the previous MAX value if any to set the stream_id
2016-09-09 11:52:44 +01:00
Erik Johnston
a15ba15e64
Merge pull request #1088 from matrix-org/erikj/transaction_queue_check
...
Correctly guard against multiple concurrent transactions
2016-09-09 11:49:06 +01:00
Mark Haines
6a6cbfcf1e
Track the max_stream_device_id in a separate table, since we delete from the inbox table
2016-09-09 11:48:23 +01:00
Erik Johnston
d2688d7f03
Correctly guard against multiple concurrent transactions
2016-09-09 11:44:36 +01:00
Mark Haines
303b6f29f0
Merge pull request #1087 from matrix-org/markjh/reapply_delta
...
Reapply 34/device_outbox in 35/device_outbox_again.py since the schem…
2016-09-09 11:24:27 +01:00
Erik Johnston
1fe7ca1362
Merge branch 'release-v0.17.3' of github.com:matrix-org/synapse into develop
2016-09-09 11:15:40 +01:00
Erik Johnston
66efcbbff1
Bump changelog and version
2016-09-09 11:07:39 +01:00
Mark Haines
0877157353
Just move the schema and add some DROPs
2016-09-09 11:04:47 +01:00
Erik Johnston
2ffec928e2
Reduce batch size to be under SQL limit
2016-09-09 11:03:31 +01:00
Erik Johnston
b390756150
Update last_device_stream_id_by_dest if there is nothing to send
2016-09-09 11:00:15 +01:00
Matthew Hodgson
b8f84f99ff
Merge pull request #1081 from matrix-org/dbkr/notifications_only_highlight
...
Implement `only=highlight` on `/notifications`
2016-09-09 00:09:51 +01:00
Mark Haines
43b77c5d97
Only catch databas errors
2016-09-08 17:44:21 +01:00
Paul "LeoNerd" Evans
2f267ee160
Collect up all the "instances" lists of individual AS protocol results into one combined answer to the client
2016-09-08 17:43:53 +01:00
Mark Haines
7d5b142547
Add a stub run_upgrade
2016-09-08 17:39:11 +01:00
David Baker
c3276aef25
Merge pull request #1080 from matrix-org/dbkr/fix_notifications_api_with_from
...
Fix /notifications API when used with `from` param
2016-09-08 17:35:35 +01:00
Mark Haines
fa722a699c
Reapply 34/device_outbox in 35/device_outbox_again.py since the schema was bumped before it landed on develop
2016-09-08 17:35:16 +01:00
Erik Johnston
5c688739d6
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/reindex_state_groups
2016-09-08 16:52:09 +01:00
Erik Johnston
ebb46497ba
Add delta file
2016-09-08 16:38:54 +01:00
Mark Haines
91ec972277
Merge pull request #1084 from matrix-org/markjh/direct_to_device_wildcard
...
Support wildcard device_ids for direct to device messages
2016-09-08 16:28:08 +01:00
Erik Johnston
5beda10bbd
Reindex state_groups_state after pruning
2016-09-08 16:18:01 +01:00
Erik Johnston
257025ac89
Merge pull request #1082 from matrix-org/erikj/remote_public_rooms
...
Add server param to /publicRooms
2016-09-08 16:04:22 +01:00
Erik Johnston
3f9889bfd6
Use parse_string
2016-09-08 15:51:10 +01:00
Erik Johnston
caa22334b3
Merge branch 'master' of github.com:matrix-org/synapse into develop
2016-09-08 15:33:00 +01:00
Erik Johnston
b152ee71fe
Bump version and changelog
2016-09-08 15:23:44 +01:00
Mark Haines
a1c8f268e5
Support wildcard device_ids for direct to device messages
2016-09-08 15:13:05 +01:00
Erik Johnston
8b93af662d
Check the user_id for presence/typing matches origin
2016-09-08 15:07:38 +01:00
Mark Haines
fa9d36e050
Merge branch 'develop' into markjh/direct_to_device_federation
2016-09-08 13:43:43 +01:00
David Baker
4ef222ab61
Implement only=highlight on /notifications
2016-09-08 13:43:35 +01:00
Erik Johnston
61cd9af09b
Log delta files we're applying
2016-09-08 13:40:46 +01:00
Erik Johnston
791658b576
Add server param to /publicRooms
2016-09-08 11:53:05 +01:00
David Baker
c5b49eb7ca
Fix /notifications API when used with from param
2016-09-08 09:40:10 +01:00
Erik Johnston
b568ca309c
Temporarily disable sequential scans for state fetching
2016-09-08 09:38:54 +01:00
Mark Haines
3c320c006c
Merge pull request #1077 from matrix-org/markjh/device_logging
...
Log the types and values when failing to store devices
2016-09-07 18:24:24 +01:00
Mark Haines
85b51fdd6b
Log the types and values when failing to store devices
2016-09-07 17:19:18 +01:00
Mark Haines
43954d000e
Add a new method to enqueue the device messages rather than sending a dummy EDU
2016-09-07 16:10:51 +01:00
Mark Haines
2a0159b8ae
Fix the stream change cache to work over replication
2016-09-07 15:58:00 +01:00
Mark Haines
cb98ac261b
Move the check for federated device_messages.
...
Move the check into _attempt_new_transaction.
Only delete messages if there were messages to delete.
2016-09-07 15:39:13 +01:00
Mark Haines
31a07d2335
Add stream change caches for device messages
2016-09-07 15:27:07 +01:00
Erik Johnston
513188aa56
Comment
2016-09-07 14:53:23 +01:00
Erik Johnston
fadb01551a
Add appopriate framing clause
2016-09-07 14:39:01 +01:00
Erik Johnston
d25c20ccbe
Use windowing function to make use of index
2016-09-07 14:22:22 +01:00
Mark Haines
7d893beebe
Comment the add_messages storage functions
2016-09-07 12:03:37 +01:00
Erik Johnston
94a83b534f
Merge pull request #1065 from matrix-org/erikj/state_storage
...
Move to storing state_groups_state as deltas
2016-09-07 09:39:58 +01:00
Mark Haines
d4a35ada28
Send device messages over federation
2016-09-06 18:16:20 +01:00
Mark Haines
e020834e4f
Add storage methods for federated device messages
2016-09-06 15:12:13 +01:00
Mark Haines
2ad72da931
Add tables for federated device messages
...
Adds tables for storing the messages that need to be sent to a
remote device and for deduplicating messages received.
2016-09-06 15:10:40 +01:00
Erik Johnston
3c4208a057
Record counts of state changes
2016-09-06 11:31:01 +01:00
Mark Haines
f4164edb70
Move _add_messages_to_device_inbox_txn into a separate method
2016-09-06 11:26:37 +01:00
Erik Johnston
2eed4d7af4
Merge pull request #1072 from matrix-org/erikj/presence_fiddle
...
Fiddle should_notify to better report stats
2016-09-06 10:57:48 +01:00
Erik Johnston
438ef47637
Short circuit if presence is the same
2016-09-06 10:28:35 +01:00
Erik Johnston
74a3b4a650
Fiddle should_notify to better report stats
2016-09-06 10:23:38 +01:00
Erik Johnston
d51b8a1674
Add quotes and be explicity about script-src
2016-09-05 17:35:01 +01:00
Erik Johnston
662b031a30
Allow PDF to be rendered from media repo
2016-09-05 17:25:26 +01:00
Erik Johnston
0595413c0f
Scale the batch size so that we're not bitten by the minimum
2016-09-05 15:49:57 +01:00
Erik Johnston
a7032abb2e
Correctly handle reindexing state groups that already have an edge
2016-09-05 15:07:23 +01:00
Erik Johnston
9e6d88f4e2
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/state_storage
2016-09-05 15:01:33 +01:00
Erik Johnston
8c93e0bae7
Merge pull request #1070 from matrix-org/erikj/presence_stats
...
Record why we have chosen to notify
2016-09-05 15:01:30 +01:00
