Erik Johnston
9e25443db8
Move to storing state_groups_state as deltas
2016-09-01 14:31:26 +01:00
Erik Johnston
44982606ee
Merge pull request #1060 from matrix-org/erikj/state_ids
...
Assign state groups in state handler.
2016-09-01 14:20:42 +01:00
Erik Johnston
0cfd6c3161
Use state_groups table to test existence
2016-08-31 16:25:57 +01:00
Erik Johnston
5405351b14
Lower get_linearized_receipts_for_room cache size
2016-08-31 16:19:44 +01:00
Erik Johnston
1671913287
Merge pull request #1059 from matrix-org/erikj/sent_transaction_delete
...
Clean up old sent transactions
2016-08-31 15:07:33 +01:00
Erik Johnston
c10cb581c6
Correctly handle the difference between prev and current state
2016-08-31 14:26:22 +01:00
Erik Johnston
ef0cc648cf
Clean up old sent transactions
2016-08-31 11:12:02 +01:00
Mark Haines
a662252758
Return the current stream position from add_messages_to_device_inbox
2016-08-31 10:42:52 +01:00
Mark Haines
1aa3e1d287
Add a replication stream for direct to device messages
2016-08-31 10:38:58 +01:00
Erik Johnston
1bb8ec296d
Generate state group ids in state layer
2016-08-31 10:09:46 +01:00
Erik Johnston
5dc2a702cf
Make _state_groups_id_gen a normal IdGenerator
2016-08-30 16:55:11 +01:00
Erik Johnston
55fc17cf4b
Merge pull request #1049 from matrix-org/erikj/presence_users_in_room
...
Use state handler instead of get_users_in_room/get_joined_hosts
2016-08-30 10:50:37 +01:00
Erik Johnston
bed10f9880
Use state handler instead of get_users_in_room/get_joined_hosts
2016-08-26 14:54:30 +01:00
Mark Haines
4bbef62124
Merge remote-tracking branch 'origin/develop' into markjh/direct_to_device
2016-08-26 14:35:31 +01:00
Erik Johnston
4daa397a00
Add is_host_joined to slave storage
2016-08-26 13:10:56 +01:00
Erik Johnston
1ccdc1e93a
Cache check_host_in_room
2016-08-26 10:59:40 +01:00
Erik Johnston
778fa85f47
Make sync not pull out full state
2016-08-25 18:59:44 +01:00
Mark Haines
ab34fdecb7
Merge branch 'develop' into markjh/direct_to_device
2016-08-25 18:34:46 +01:00
Mark Haines
641efb6a39
Fix the deduplication of incoming direct-to-device messages
2016-08-25 18:14:02 +01:00
Erik Johnston
721414d98a
Add desc
2016-08-25 17:49:05 +01:00
Mark Haines
e993925279
Add store-and-forward direct-to-device messaging
2016-08-25 17:35:37 +01:00
Erik Johnston
a3dc1e9cbe
Replace context.current_state with context.current_state_ids
2016-08-25 17:32:22 +01:00
Erik Johnston
17f4f14df7
Pull out event ids rather than full events for state
2016-08-25 13:42:44 +01:00
Erik Johnston
9219139351
Preserve some logcontexts
2016-08-24 11:58:40 +01:00
Erik Johnston
122c7a43c9
Merge pull request #1038 from matrix-org/erikj/receved_txn_purge
...
Delete old received_transactions rows
2016-08-23 11:02:40 +01:00
Erik Johnston
d143f211c8
Merge pull request #1028 from matrix-org/dbkr/notifications_api
...
Add the Notifications API
2016-08-22 18:23:24 +01:00
Erik Johnston
aad8a1a825
Delete old received_transactions
2016-08-22 16:29:46 +01:00
Erik Johnston
e65bc7d315
Merge pull request #1031 from matrix-org/erikj/measure_notifier
...
Add more Measure blocks
2016-08-22 12:13:07 +01:00
Matthew Hodgson
6e80c03d45
Merge branch 'develop' into dbkr/notifications_api
2016-08-20 00:16:18 +01:00
Erik Johnston
37adde32dc
Move defer.returnValue out of Measure
2016-08-19 18:23:44 +01:00
Erik Johnston
47dd8f02a1
Measure _get_event_from_row
2016-08-19 18:23:44 +01:00
Erik Johnston
e6784daf07
Merge pull request #1030 from matrix-org/erikj/cache_contexts
...
Add concept of cache contexts
2016-08-19 16:29:58 +01:00
Erik Johnston
c0d7d9d642
Rename to on_invalidate
2016-08-19 15:13:58 +01:00
Erik Johnston
dc76a3e909
Make cache_context an explicit option
2016-08-19 15:02:38 +01:00
Erik Johnston
f164fd9220
Move _bulk_get_push_rules_for_room to storage layer
2016-08-19 14:29:20 +01:00
Erik Johnston
ba214a5e32
Remove lru option
2016-08-19 14:17:11 +01:00
Erik Johnston
b770435389
Make get_new_events_for_appservice use indices
2016-08-19 10:28:42 +01:00
David Baker
1e4217c90c
Explicit join
2016-08-18 17:53:44 +01:00
David Baker
602c84cd9c
Merge remote-tracking branch 'origin/develop' into dbkr/notifications_api
2016-08-18 17:15:26 +01:00
Erik Johnston
07229bbdae
Add appservice worker
2016-08-18 14:59:55 +01:00
Erik Johnston
9da84a9a1e
Make AppserviceHandler stream events from database
...
This is for two reasons:
1. Suppresses duplicates correctly, as the notifier doesn't do any
duplicate suppression.
2. Makes it easier to connect the AppserviceHandler to the replication
stream.
2016-08-18 11:54:41 +01:00
Erik Johnston
732cf72b86
Fix push_display_name_rename schema update
2016-08-17 18:12:21 +01:00
David Baker
0d7cef0943
Merge pull request #1011 from matrix-org/dbkr/contains_display_name_override
...
Move display name rule
2016-08-17 16:15:45 +01:00
Erik Johnston
949629291c
Do it in storage function
2016-08-16 17:05:34 +01:00
David Baker
ad42322257
Add migration script
...
To port existing rule actions & enable entries to the new name
2016-08-16 16:56:30 +01:00
Erik Johnston
25c2332071
Merge pull request #1010 from matrix-org/erikj/refactor_deletions
...
Refactor user_delete_access_tokens. Invalidate get_user_by_access_token to slaves.
2016-08-16 11:37:53 +01:00
Erik Johnston
a2427981b7
Use cached get_user_by_access_token in slaves
2016-08-16 11:24:32 +01:00
Erik Johnston
6cbd1b495e
Merge branch 'fix_integrity_retry' of https://github.com/Ralith/synapse into Ralith-fix_integrity_retry
2016-08-16 10:50:24 +01:00
Erik Johnston
dc3a00f24f
Refactor user_delete_access_tokens. Invalidate get_user_by_access_token to slaves.
2016-08-15 17:04:39 +01:00
Erik Johnston
89e786bd85
Doc get_next() context manager usage
2016-08-15 13:45:26 +01:00
Erik Johnston
d9664344ec
Rename table. Add docs.
2016-08-15 11:45:57 +01:00
Erik Johnston
64e7e11853
Implement cache replication stream
2016-08-15 11:16:45 +01:00
Erik Johnston
4d70d1f80e
Add some invalidations to a cache_stream
2016-08-15 11:15:17 +01:00
Erik Johnston
99bbd90b0d
Always run txn.after_callbacks
2016-08-15 09:45:44 +01:00
Benjamin Saunders
8a57cc3123
Add missing database corruption recovery case
...
Signed-off-by: Benjamin Saunders <ben.e.saunders@gmail.com>
2016-08-14 11:50:22 -07:00
David Baker
b4ecf0b886
Merge remote-tracking branch 'origin/develop' into dbkr/notifications_api
2016-08-11 14:09:13 +01:00
Erik Johnston
a8b946decb
Raise 404 when couldn't find event
2016-08-05 15:31:02 +01:00
Erik Johnston
f0fa66f495
Delete more tables
2016-08-05 10:40:08 +01:00
Benjamin Saunders
a2b7102eea
Tweak integrity error recovery to work as intended
2016-08-04 20:38:08 -07:00
Erik Johnston
8ad8490cff
Fix typo
2016-08-04 15:21:29 +01:00
Erik Johnston
59fa91fe88
Retry event persistence on IntegrityError
...
Due to a bug in the porting script some backfilled events were not
correctly persisted, causing irrecoverable IntegrityErrors on future
attempts to persist those events.
This commit adds a retry mechanism invoked upon IntegrityError,
where when retried the tables are purged for all references to the
events being persisted.
2016-08-04 15:02:15 +01:00
Richard van der Hoff
98385888b8
PEP8
2016-08-03 15:42:08 +01:00
Richard van der Hoff
68264d7404
Include device name in /keys/query response
...
Add an 'unsigned' section which includes the device display name.
2016-08-03 15:42:08 +01:00
Richard van der Hoff
530bc862dc
Merge branch 'rav/null_default_device_displayname' into develop
2016-08-03 14:30:32 +01:00
Erik Johnston
80ad710217
Remove other bit of deduplication
2016-08-03 13:25:59 +01:00
Richard van der Hoff
4fec5e57be
Default device_display_name to null
...
It turns out that it's more useful to return a null device display name (and
let clients decide how to handle it: eg, falling back to device_id) than using
a constant string like "unknown device".
2016-08-03 11:53:00 +01:00
Erik Johnston
a8a32d2714
Ensure we only persist an event once at a time
2016-08-03 11:23:39 +01:00
Erik Johnston
5aa024e501
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/fed_reader
2016-07-29 11:24:56 +01:00
Mark Haines
8dad08a950
Fix SQL to supply arguments in the same order
2016-07-29 09:57:13 +01:00
Mark Haines
0a7d3cd00f
Create separate methods for getting messages to push
...
for the email and http pushers rather than trying to make a single
method that will work with their conflicting requirements.
The http pusher needs to get the messages in ascending stream order, and
doesn't want to miss a message.
The email pusher needs to get the messages in descending timestamp order,
and doesn't mind if it misses messages.
2016-07-28 20:24:24 +01:00
Kegan Dougal
370135ad0b
Comment get_unread_push_actions_for_user_in_range function
2016-07-28 16:47:37 +01:00
Erik Johnston
6ede23ff1b
Add more key storage funcs into slave store
2016-07-28 15:41:26 +01:00
Richard van der Hoff
40e539683c
Merge pull request #956 from matrix-org/rav/check_device_id_on_key_upload
...
Make the device id on e2e key upload optional
2016-07-27 18:16:57 +01:00
Richard van der Hoff
26cb0efa88
SQL syntax fix
2016-07-27 12:30:22 +01:00
Richard van der Hoff
d47115ff8b
Delete e2e keys on device delete
2016-07-27 12:24:52 +01:00
Richard van der Hoff
2e3d90d67c
Make the device id on e2e key upload optional
...
We should now be able to get our device_id from the access_token, so the
device_id on the upload request is optional. Where it is supplied, we should
check that it matches.
For active access_tokens without an associated device_id, we ought to register
the device in the devices table.
Also update the table on upgrade so that all of the existing e2e keys are
associated with real devices.
2016-07-26 23:38:12 +01:00
Mark Haines
87ffd21b29
Fix a couple of bugs in the transaction and keyring code
2016-07-26 19:19:08 +01:00
Richard van der Hoff
33d7776473
Fix typo
2016-07-26 13:32:15 +01:00
Richard van der Hoff
9ad8d9b17c
Merge branch 'develop' into rav/delete_refreshtoken_on_delete_device
2016-07-26 13:29:46 +01:00
Mark Haines
9c4cf83259
Merge pull request #948 from matrix-org/markjh/auth_fixes
...
Don't add rejections to the state_group, persist all rejections
2016-07-26 13:22:57 +01:00
Richard van der Hoff
8e02494166
Delete refresh tokens when deleting devices
2016-07-26 11:10:37 +01:00
Mark Haines
a6f06ce3e2
Fix how push_actions are redacted.
2016-07-26 11:05:39 +01:00
Mark Haines
efeb6176c1
Don't add rejected events if we've seen them befrore. Add some comments to explain what the code is doing mechanically
2016-07-26 10:49:52 +01:00
Richard van der Hoff
012b4c1913
Implement updating devices
...
You can update the displayname of devices now.
2016-07-26 07:35:48 +01:00
Richard van der Hoff
436bffd15f
Implement deleting devices
2016-07-26 07:35:48 +01:00
Mark Haines
1b3c3e6d68
Only update the events and event_json tables for rejected events
2016-07-25 18:44:30 +01:00
Mark Haines
8f7f4cb92b
Don't add the events to forward extremities if the event is rejected
2016-07-25 17:13:37 +01:00
Mark Haines
2623cec874
Don't add rejections to the state_group, persist all rejections
2016-07-25 16:12:16 +01:00
Mark Haines
955ef1f06c
fix: defer.returnValue takes one argument
2016-07-25 16:04:45 +01:00
Richard van der Hoff
2ee4c9ee02
background updates: fix assert again
2016-07-25 16:01:46 +01:00
Richard van der Hoff
9dbd903f41
background updates: Fix assertion to do something
2016-07-25 14:05:23 +01:00
Richard van der Hoff
465117d7ca
Fix background_update tests
...
A bit of a cleanup for background_updates, and make sure that the real
background updates have run before we start the unit tests, so that they don't
interfere with the tests.
2016-07-25 12:10:42 +01:00
Richard van der Hoff
363786845b
PEP8
2016-07-22 13:21:07 +01:00
Richard van der Hoff
ec5717caf5
Create index on user_ips in the background
...
user_ips is kinda big, so really we want to add the index in the background
once we're running. Replace the schema delta with one which will do that.
I've done this in a way that's reasonably easy to reuse as there a few other
indexes I need, and I don't suppose they will be the last.
2016-07-22 13:16:39 +01:00
Erik Johnston
aede7248ab
Split out a FederationReader process
2016-07-21 17:37:44 +01:00
Richard van der Hoff
c445f5fec7
storage/client_ips: remove some dead code
2016-07-21 11:58:47 +01:00
Richard van der Hoff
7314bf4682
Merge branch 'develop' into rav/get_devices_api
...
(pick up PR #938 in the hope of fixing the UTs)
2016-07-20 17:40:00 +01:00
Richard van der Hoff
bc8f265f0a
GET /devices endpoint
...
implement a GET /devices endpoint which lists all of the user's devices.
It also returns the last IP where we saw that device, so there is some dancing
to fish that out of the user_ips table.
2016-07-20 16:42:32 +01:00
Richard van der Hoff
ec041b335e
Record device_id in client_ips
...
Record the device_id when we add a client ip; it's somewhat redundant as we
could get it via the access_token, but it will make querying rather easier.
2016-07-20 16:41:03 +01:00
Richard van der Hoff
053e83dafb
More doc-comments
...
Fix some more comments on some things
2016-07-20 16:40:28 +01:00
