Erik Johnston
aede7248ab
Split out a FederationReader process
2016-07-21 17:37:44 +01:00
Richard van der Hoff
c445f5fec7
storage/client_ips: remove some dead code
2016-07-21 11:58:47 +01:00
Richard van der Hoff
7314bf4682
Merge branch 'develop' into rav/get_devices_api
...
(pick up PR #938 in the hope of fixing the UTs)
2016-07-20 17:40:00 +01:00
Richard van der Hoff
bc8f265f0a
GET /devices endpoint
...
implement a GET /devices endpoint which lists all of the user's devices.
It also returns the last IP where we saw that device, so there is some dancing
to fish that out of the user_ips table.
2016-07-20 16:42:32 +01:00
Richard van der Hoff
ec041b335e
Record device_id in client_ips
...
Record the device_id when we add a client ip; it's somewhat redundant as we
could get it via the access_token, but it will make querying rather easier.
2016-07-20 16:41:03 +01:00
Richard van der Hoff
053e83dafb
More doc-comments
...
Fix some more comments on some things
2016-07-20 16:40:28 +01:00
David Baker
e967bc86e7
Merge pull request #932 from matrix-org/rav/register_refactor
...
Further registration refactoring
2016-07-20 11:03:33 +01:00
Erik Johnston
1e2a7f18a1
Merge pull request #922 from matrix-org/erikj/file_api2
...
Feature: Add filter to /messages. Add 'contains_url' to filter.
2016-07-20 10:40:48 +01:00
Erik Johnston
f91faf09b3
Comment
2016-07-20 10:18:09 +01:00
Richard van der Hoff
40cbffb2d2
Further registration refactoring
...
* `RegistrationHandler.appservice_register` no longer issues an access token:
instead it is left for the caller to do it. (There are two of these, one in
`synapse/rest/client/v1/register.py`, which now simply calls
`AuthHandler.issue_access_token`, and the other in
`synapse/rest/client/v2_alpha/register.py`, which is covered below).
* In `synapse/rest/client/v2_alpha/register.py`, move the generation of
access_tokens into `_create_registration_details`. This means that the normal
flow no longer needs to call `AuthHandler.issue_access_token`; the
shared-secret flow can tell `RegistrationHandler.register` not to generate a
token; and the appservice flow continues to work despite the above change.
2016-07-19 18:46:19 +01:00
Richard van der Hoff
022b9176fe
schema fix
...
device_id should be text, not bigint.
2016-07-19 11:44:05 +01:00
Richard van der Hoff
f863a52cea
Add device_id support to /login
...
Add a 'devices' table to the storage, as well as a 'device_id' column to
refresh_tokens.
Allow the client to pass a device_id, and initial_device_display_name, to
/login. If login is successful, then register the device in the devices table
if it wasn't known already. If no device_id was supplied, make one up.
Associate the device_id with the access token and refresh token, so that we can
get at it again later. Ensure that the device_id is copied from the refresh
token to the access_token when the token is refreshed.
2016-07-18 16:39:44 +01:00
Erik Johnston
f52565de50
Fix /purge_history bug
...
This was caused by trying to insert duplicate backward extremeties
2016-07-15 14:23:15 +01:00
Erik Johnston
978fa53cc2
Pull out min stream_ordering from ex_outlier_stream
2016-07-15 10:22:30 +01:00
Erik Johnston
eec9609e96
event_backwards_extremeties may not be empty
2016-07-15 10:22:09 +01:00
Erik Johnston
d554ca5e1d
Add support for filters in paginate_room_events
2016-07-14 15:59:04 +01:00
Erik Johnston
b64aa6d687
Add sender and contains_url field to events table
2016-07-14 15:35:43 +01:00
Mark Haines
dfde67a6fe
Add a comment explaining allow_none
2016-07-08 15:57:06 +01:00
Mark Haines
10c843fcfb
Ensure that the guest user is in the database when upgrading accounts
2016-07-08 15:15:55 +01:00
Erik Johnston
58930da52b
Merge branch 'master' of github.com:matrix-org/synapse into develop
2016-07-08 14:11:37 +01:00
Erik Johnston
b92e7955be
Comment
2016-07-07 11:42:15 +01:00
Erik Johnston
c98e1479bd
Return 400 rather than 500
2016-07-07 11:41:07 +01:00
Erik Johnston
67f2c901ea
Add rest servlet. Fix SQL.
2016-07-06 15:56:59 +01:00
Erik Johnston
a17e7caeb7
Merge branch 'erikj/shared_secret' into erikj/test2
2016-07-06 14:46:31 +01:00
Erik Johnston
be3548f7e1
Remove spurious txn
2016-07-05 17:46:51 +01:00
Erik Johnston
4adf93e0f7
Fix for postgres
2016-07-05 17:34:25 +01:00
Erik Johnston
651faee698
Add an admin option to shared secret registration
2016-07-05 17:30:22 +01:00
Erik Johnston
8f8798bc0d
Add ReadWriteLock for pagination and history prune
2016-07-05 15:30:25 +01:00
Mark Haines
dd2ccee27d
Fix typo
2016-07-05 14:06:07 +01:00
Mark Haines
b6b0132ac7
Make get_events_around more efficient on sqlite3
2016-07-05 13:55:18 +01:00
Mark Haines
d44d11d864
Use true/false for boolean parameter inclusive to avoid potential for sqli, and possibly make the code clearer
2016-07-05 10:39:13 +01:00
Erik Johnston
2d21d43c34
Add purge_history API
2016-07-05 10:28:51 +01:00
Mark Haines
0fb76c71ac
Use different SQL for postgres and sqlite3 for when using multicolumn indexes
2016-07-04 19:44:55 +01:00
Erik Johnston
a67bf0b074
Add storage function to purge history for a room
2016-07-04 16:02:50 +01:00
Mark Haines
f18d7546c6
Use a query that postgresql optimises better for get_events_around
2016-07-04 15:48:25 +01:00
Erik Johnston
f328d95cef
Feature: Add deactivate account admin API
...
Allows server admins to "deactivate" accounts, which:
- Revokes all access tokens
- Removes all threepids
- Removes password
The API is a POST to `/admin/deactivate/<user_id>`
2016-06-30 15:40:58 +01:00
Mark Haines
6783534a0f
Merge pull request #886 from matrix-org/markjh/async_commit
...
Optionally make committing to postgres asynchronous.
2016-06-29 15:21:58 +01:00
Erik Johnston
a70688445d
Implement purge_media_cache admin API
2016-06-29 14:57:59 +01:00
Erik Johnston
314b146b2e
Track approximate last access time for remote media
2016-06-29 11:41:20 +01:00
David Baker
aa3a4944d5
more pep8
2016-06-24 11:45:23 +01:00
David Baker
870c45913e
Use similar naming we use in email notifs for push
...
Fixes https://github.com/vector-im/vector-web/issues/1654
2016-06-24 11:41:11 +01:00
David Baker
b5fb7458d5
Actually we need to order these properly
...
otherwise we'll end up returning the wrong 20
2016-06-22 18:07:14 +01:00
David Baker
f73fdb04a6
Style
2016-06-22 17:51:40 +01:00
David Baker
3a4120e49a
Put most recent 20 messages in notif
...
Fixes https://github.com/vector-im/vector-web/issues/1648
2016-06-22 17:47:18 +01:00
Mark Haines
d5fb561709
Optionally make committing to postgres asynchronous.
...
Useful when running tests when you don't care whether the server
will lose data that it claims that it has committed.
2016-06-20 17:53:38 +01:00
Mark Haines
41e4b2efea
Add the create_profile method back since the tests use it
2016-06-17 19:20:47 +01:00
Mark Haines
9f1800fba8
Remove registered_users from the distributor.
...
The only place that was observed was to set the profile. I've made it
so that the profile is set within store.register in the same transaction
that creates the user.
This required some slight changes to the registration code for upgrading
guest users, since it previously relied on the distributor swallowing errors
if the profile already existed.
2016-06-17 19:14:16 +01:00
Erik Johnston
1a815fb04f
Don't hit DB for noop replications queries
2016-06-08 11:33:30 +01:00
Mark Haines
0b2158719c
Remove dead code.
...
Loading push rules now happens in the datastore, so we can remove
the methods that loaded them outside the datastore.
The ``waiting_for_join_list`` in federation handler is populated by
anything, so can be removed.
The ``_get_members_events_txn`` method isn't called from anywhere
so can be removed.
2016-06-07 15:07:11 +01:00
Erik Johnston
84379062f9
Fix AS retries, but with correct ordering
2016-06-07 10:24:50 +01:00
Erik Johnston
310197bab5
Fix AS retries
2016-06-07 09:34:50 +01:00
Erik Johnston
e4134c5e13
Merge pull request #841 from matrix-org/erikj/event_counter
...
Add metric counter for number of persisted events
2016-06-06 14:17:40 +01:00
Erik Johnston
7aa778fba9
Add metric counter for number of persisted events
2016-06-06 11:58:09 +01:00
Erik Johnston
70aee0717c
Add events to cache when we persist them
2016-06-06 11:34:53 +01:00
Erik Johnston
cffe46408f
Don't rely on options when inserting event into cache
2016-06-03 18:25:21 +01:00
Erik Johnston
10ea3f46ba
Change the way we cache events
2016-06-03 17:57:50 +01:00
Erik Johnston
05e01f21d7
Remove event fetching from DB threads
2016-06-03 17:22:13 +01:00
Mark Haines
eef541a291
Move insert_client_ip to a separate class
2016-06-03 14:42:35 +01:00
Erik Johnston
d06f2a229e
Merge pull request #828 from matrix-org/erikj/joined_hosts_for_room
...
Make get_joined_hosts_for_room use get_users_in_room
2016-06-03 11:50:30 +01:00
Erik Johnston
722472b48c
Merge pull request #825 from matrix-org/erikj/cache_push_rules
...
Load push rules in storage layer so that they get cached
2016-06-03 11:44:32 +01:00
Mark Haines
b09f348530
Merge pull request #824 from matrix-org/markjh/slaved_presence_store
...
Add a slaved store for presence
2016-06-03 11:26:33 +01:00
Erik Johnston
ccb56fc24b
Make get_joined_hosts_for_room use get_users_in_room
2016-06-03 11:20:23 +01:00
Erik Johnston
6a0afa582a
Load push rules in storage layer, so that they get cached
2016-06-03 11:10:00 +01:00
Mark Haines
3ae915b27e
Add a slaved store for presence
2016-06-03 11:05:53 +01:00
Erik Johnston
9c26b390a2
Only get local users
2016-06-03 11:04:31 +01:00
Erik Johnston
065e739d6e
Merge pull request #811 from matrix-org/erikj/state_users_in_room
...
Use state to calculate get_users_in_room
2016-06-03 10:58:27 +01:00
Erik Johnston
696d7c5937
Merge pull request #809 from matrix-org/erikj/cache_receipts_in_room
...
Add get_users_with_read_receipts_in_room cache
2016-06-03 10:58:24 +01:00
Mark Haines
80f34d7b57
Fix setting the _clock in SQLBaseStore
2016-06-02 15:23:56 +01:00
Erik Johnston
a04a2d043c
Merge pull request #807 from matrix-org/erikj/push_rules_cache
...
Ensure we always return boolean in push rules
2016-06-01 18:07:48 +01:00
Erik Johnston
0f06b496d1
Merge pull request #806 from matrix-org/erikj/hash_cache
...
Cache get_event_reference_hashes
2016-06-01 18:07:42 +01:00
Erik Johnston
c8285564a3
Use state to calculate get_users_in_room
2016-06-01 15:25:25 +01:00
David Baker
d60eed0710
Limit number of notifications in an email notification
2016-06-01 11:45:43 +01:00
Erik Johnston
43db0d9f6a
Add get_users_with_read_receipts_in_room cache
2016-06-01 10:54:32 +01:00
Erik Johnston
dea9f20f8c
Force boolean
2016-05-31 17:24:30 +01:00
Mark Haines
c8c5bf950a
Fix synapse/storage/schema/delta/30/as_users.py
2016-05-31 17:10:40 +01:00
Erik Johnston
c9ca285d33
Merge pull request #805 from matrix-org/erikj/push_rules_cache
...
Fix GET /push_rules
2016-05-31 16:42:21 +01:00
Erik Johnston
1d4ee854e2
Fix typo
2016-05-31 15:45:53 +01:00
Erik Johnston
cca0093fa9
Change fix
2016-05-31 15:44:08 +01:00
Erik Johnston
aefd2d1cbc
Cache get_event_reference_hashes
2016-05-31 15:32:32 +01:00
Erik Johnston
10de8c2631
Merge pull request #804 from matrix-org/erikj/push_rules_cache
...
Add caches to bulk_get_push_rules*
2016-05-31 15:04:40 +01:00
Erik Johnston
e5b0bbcd33
Add caches to bulk_get_push_rules*
2016-05-31 13:46:58 +01:00
Erik Johnston
85b992f621
Fix to allow start with postgres
2016-05-27 10:44:44 +01:00
David Baker
37b7e84620
Include the ts the notif was received at
2016-05-24 11:33:32 +01:00
David Baker
a24bc5b2dc
Add GET /notifications API
2016-05-23 18:33:51 +01:00
David Baker
31b5395ab6
Remove debug logging
2016-05-23 16:32:01 +01:00
David Baker
c2da3406fc
Oops, missing comma
2016-05-20 18:03:31 +01:00
David Baker
ccffb0965d
Remove stale line
2016-05-20 17:59:10 +01:00
David Baker
18d68bfee4
Handle empty events table
2016-05-20 17:58:09 +01:00
David Baker
d4503e25ed
Make deleting push actions more efficient
...
There's no index on received_ts, so manually binary search using the stream_ordering index, and only update it once an hour.
2016-05-20 17:56:10 +01:00
David Baker
149fa411e2
Only delete push actions after 30 days
2016-05-20 15:25:12 +01:00
Erik Johnston
42368ea8db
Add desc to get_presence_for_users
2016-05-18 11:38:10 +01:00
Mark Haines
6a30a0bfd3
Move the functions for parsing app service config
2016-05-17 11:28:58 +01:00
Mark Haines
816df9f267
get_room_members is unused now
2016-05-16 19:51:43 +01:00
Mark Haines
1a3a2002ff
Spell "domain" correctly
...
s/domian/domain/g
2016-05-16 19:17:23 +01:00
Mark Haines
b7381d5338
Allow receipts for events we haven't seen in the db
2016-05-13 15:46:41 +01:00
Erik Johnston
ba5c616ff4
Merge pull request #778 from matrix-org/erikj/add_pusher
...
Fixup add_pusher
2016-05-13 14:43:23 +01:00
Erik Johnston
0c11c1be88
Spelling
2016-05-13 14:42:25 +01:00
Erik Johnston
e00e8f2166
Merge pull request #769 from matrix-org/erikj/push_actions_delete
...
Delete old pushers
2016-05-13 14:41:36 +01:00
Erik Johnston
fd8e921b6e
Merge pull request #779 from matrix-org/erikj/receipts
...
Use tree cache for get_linearized_receipts_for_room
2016-05-13 14:41:21 +01:00
Erik Johnston
c9aff0736c
Remove topics table
2016-05-13 14:40:38 +01:00
Mark Haines
9295fa30a8
Annotate the removed indicies with why they were removed.
2016-05-13 14:16:57 +01:00
Erik Johnston
5e50058473
Remove unused indices
...
This includes removing both unused indices and indices that are subsets
of other indices.
2016-05-13 13:28:07 +01:00
Erik Johnston
6da7f39d95
Use tree cache for get_linearized_receipts_for_room
2016-05-13 11:41:23 +01:00
Erik Johnston
13d37c3c56
Fixup add_pusher
2016-05-13 11:25:02 +01:00
Matrix
3b1930e8ec
unbreak schema
2016-05-10 16:42:37 +01:00
Matthew Hodgson
fe97b81c09
Merge pull request #759 from matrix-org/dbkr/email_notifs
...
Send email notifications for missed messages
2016-05-10 16:30:05 +02:00
David Baker
997db04648
Merge remote-tracking branch 'origin/develop' into dbkr/email_notifs
2016-05-10 14:40:19 +02:00
Erik Johnston
c9eb6dfc1b
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/ignore_user
2016-05-09 13:21:06 +01:00
Erik Johnston
3f84da139c
Merge pull request #773 from matrix-org/erikj/get_domian_from_id
...
Add and use get_domain_from_id
2016-05-09 13:21:00 +01:00
Erik Johnston
def64d6ef3
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/ignore_user
2016-05-09 13:05:09 +01:00
Erik Johnston
8715731559
Merge pull request #772 from matrix-org/erikj/get_user_cache
...
Add cache to get_user_by_id
2016-05-09 11:12:11 +01:00
Erik Johnston
08dfa8eee2
Add and use get_domian_from_id
2016-05-09 10:36:03 +01:00
Erik Johnston
4ea762c1a2
Add cache to get_user_by_id
2016-05-09 10:08:21 +01:00
Erik Johnston
012cb5416c
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/push_actions_delete
2016-05-06 15:59:20 +01:00
Erik Johnston
fcb2c3f0db
Remove unused import
2016-05-06 15:47:40 +01:00
Erik Johnston
fd85b167ec
Pull loop one level up
2016-05-06 15:38:42 +01:00
Erik Johnston
b6e0be701e
Queue events for persistence
2016-05-06 14:31:44 +01:00
Erik Johnston
d13459636f
Pull prev txn from in memory
2016-05-06 11:30:55 +01:00
Erik Johnston
1d275dba69
Don't needlessly enter transaction
2016-05-06 11:25:58 +01:00
Erik Johnston
56b5e83e36
Reduce database inserts when sending transactions
2016-05-06 11:20:18 +01:00
Mark Haines
9c272da05f
Add an openidish mechanism for proving to third parties that you own a given user_id
2016-05-05 13:42:44 +01:00
Erik Johnston
1f0f5ffa1e
Add bulk fetch storage API
2016-05-05 10:03:15 +01:00
Erik Johnston
fee1118a20
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/ignore_user
2016-05-04 19:08:27 +01:00
Erik Johnston
fcd1eb642d
Add primary key
2016-05-04 16:51:51 +01:00
Erik Johnston
8e6a163f27
Add timestamp and auto incrementing ID
2016-05-04 15:19:12 +01:00
David Baker
80be396464
Correct SQL statement for postgres
...
In standard sql, join binds tighter than comma, so we were joining on the wrong table. Postgres follows the standard (apparently).
2016-05-04 13:19:59 +01:00
Erik Johnston
984d4a2c0f
Add /report endpoint
2016-05-04 11:28:10 +01:00
Erik Johnston
a438a6d2bc
Implement basic ignore user
2016-05-04 10:16:46 +01:00
Erik Johnston
183f23f10d
Delete old pushers
2016-05-03 14:22:33 +01:00
David Baker
35b7b8e4bc
Remove unused function
2016-04-29 20:10:34 +01:00
David Baker
b0a1036d93
Use explicit join
2016-04-29 19:28:56 +01:00
David Baker
8f99cd5996
Oops, actually specify the user id
2016-04-29 19:27:03 +01:00
David Baker
60f44c098d
Remove unnecessary if
2016-04-29 19:17:10 +01:00
David Baker
50ad8005e4
Put spaces at start of line
2016-04-29 19:16:15 +01:00
David Baker
acded821c4
Merge remote-tracking branch 'origin/develop' into dbkr/email_notifs
2016-04-29 10:05:20 +01:00
Mark Haines
871357d539
Check that somethign has happend before running the selects
2016-04-27 11:54:13 +01:00
Erik Johnston
183cacac90
Simplify query and handle finishing correctly
2016-04-22 10:01:57 +01:00
Erik Johnston
ae571810f2
Order NULLs first
2016-04-21 18:14:18 +01:00
Erik Johnston
3ddbb1687c
Fix query
2016-04-21 18:02:36 +01:00
Erik Johnston
8fae3d7b1e
Use special UPDATE syntax
2016-04-21 18:01:49 +01:00
Erik Johnston
b57dcb4b51
Typo
2016-04-21 17:49:00 +01:00
Erik Johnston
26db18bc90
Need to do _background_update_progress_txn in actual transaction
2016-04-21 17:45:56 +01:00
Erik Johnston
e395eb1108
Update progress when creating index
2016-04-21 17:39:24 +01:00
Erik Johnston
3b0fa77f50
Fix SQL statement
2016-04-21 17:37:42 +01:00
Erik Johnston
129e403487
Create index must be on a conn
2016-04-21 17:35:51 +01:00
Erik Johnston
51bb339ab2
Create index concurrently
2016-04-21 17:16:11 +01:00
Erik Johnston
b743c1237e
Add missing run_upgrade
2016-04-21 17:12:04 +01:00
Erik Johnston
c877f0f034
Optimise event_search in postgres
2016-04-21 16:56:14 +01:00
David Baker
f63bd4ff47
Send a rather basic email notif
...
Also pep8 fixes
2016-04-20 13:02:01 +01:00
Erik Johnston
61c7edfd34
Add cache to _get_state_groups_from_groups
2016-04-19 17:22:03 +01:00
