Commit Graph

326 Commits

Author SHA1 Message Date
Richard van der Hoff
bc8f265f0a GET /devices endpoint
implement a GET /devices endpoint which lists all of the user's devices.

It also returns the last IP where we saw that device, so there is some dancing
to fish that out of the user_ips table.
2016-07-20 16:42:32 +01:00
Erik Johnston
1e2a7f18a1 Merge pull request #922 from matrix-org/erikj/file_api2
Feature: Add filter to /messages. Add 'contains_url' to filter.
2016-07-20 10:40:48 +01:00
Richard van der Hoff
022b9176fe schema fix
device_id should be text, not bigint.
2016-07-19 11:44:05 +01:00
Richard van der Hoff
f863a52cea Add device_id support to /login
Add a 'devices' table to the storage, as well as a 'device_id' column to
refresh_tokens.

Allow the client to pass a device_id, and initial_device_display_name, to
/login. If login is successful, then register the device in the devices table
if it wasn't known already. If no device_id was supplied, make one up.

Associate the device_id with the access token and refresh token, so that we can
get at it again later. Ensure that the device_id is copied from the refresh
token to the access_token when the token is refreshed.
2016-07-18 16:39:44 +01:00
Erik Johnston
b64aa6d687 Add sender and contains_url field to events table 2016-07-14 15:35:43 +01:00
Erik Johnston
314b146b2e Track approximate last access time for remote media 2016-06-29 11:41:20 +01:00
Mark Haines
c8c5bf950a Fix synapse/storage/schema/delta/30/as_users.py 2016-05-31 17:10:40 +01:00
Erik Johnston
c9aff0736c Remove topics table 2016-05-13 14:40:38 +01:00
Mark Haines
9295fa30a8 Annotate the removed indicies with why they were removed. 2016-05-13 14:16:57 +01:00
Erik Johnston
5e50058473 Remove unused indices
This includes removing both unused indices and indices that are subsets
of other indices.
2016-05-13 13:28:07 +01:00
Matrix
3b1930e8ec unbreak schema 2016-05-10 16:42:37 +01:00
David Baker
997db04648 Merge remote-tracking branch 'origin/develop' into dbkr/email_notifs 2016-05-10 14:40:19 +02:00
Mark Haines
9c272da05f Add an openidish mechanism for proving to third parties that you own a given user_id 2016-05-05 13:42:44 +01:00
Erik Johnston
fcd1eb642d Add primary key 2016-05-04 16:51:51 +01:00
Erik Johnston
8e6a163f27 Add timestamp and auto incrementing ID 2016-05-04 15:19:12 +01:00
Erik Johnston
984d4a2c0f Add /report endpoint 2016-05-04 11:28:10 +01:00
David Baker
acded821c4 Merge remote-tracking branch 'origin/develop' into dbkr/email_notifs 2016-04-29 10:05:20 +01:00
Erik Johnston
ae571810f2 Order NULLs first 2016-04-21 18:14:18 +01:00
Erik Johnston
8fae3d7b1e Use special UPDATE syntax 2016-04-21 18:01:49 +01:00
Erik Johnston
51bb339ab2 Create index concurrently 2016-04-21 17:16:11 +01:00
Erik Johnston
b743c1237e Add missing run_upgrade 2016-04-21 17:12:04 +01:00
Erik Johnston
c877f0f034 Optimise event_search in postgres 2016-04-21 16:56:14 +01:00
David Baker
07d765209d First bits of emailpusher
Mostly logic of when to send an email
2016-04-19 14:24:36 +01:00
Erik Johnston
96bcfb29c7 Add index 2016-04-14 11:26:33 +01:00
David Baker
2547dffccc Merge pull request #705 from matrix-org/dbkr/pushers_use_event_actions
Change pushers to use the event_actions table
2016-04-11 12:58:55 +01:00
Matthew Hodgson
6ff7a79308 move local_media_repository_url_cache.sql to schema v31 2016-04-08 19:09:02 +01:00
Matthew Hodgson
d6e7333ae4 Merge branch 'develop' into matthew/preview_urls 2016-04-07 17:26:44 +01:00
David Baker
05d044aac3 pep8 2016-04-07 16:45:38 +01:00
David Baker
2d5c693fd3 Fix port script for changes merged from develop 2016-04-07 16:43:54 +01:00
David Baker
9c99ab4572 Merge remote-tracking branch 'origin/develop' into dbkr/pushers_use_event_actions 2016-04-07 16:35:22 +01:00
David Baker
7e2c89a37f Make pushers use the event_push_actions table instead of listening on an event stream & running the rules again. Sytest passes, but remaining to do:
* Make badges work again
 * Remove old, unused code
2016-04-06 15:42:15 +01:00
Erik Johnston
8aab9d87fa Don't require config to create database 2016-04-06 14:15:45 +01:00
Erik Johnston
0c53d750e7 Docs and indents 2016-04-04 18:02:48 +01:00
Erik Johnston
92ab45a330 Add upgrade path, rename table 2016-04-04 17:07:43 +01:00
Erik Johnston
3d76b7cb2b Store invites in a separate table. 2016-04-04 16:30:15 +01:00
Matthew Hodgson
9f7dc2bef7 Merge branch 'develop' into matthew/preview_urls 2016-04-04 00:38:21 +01:00
Matthew Hodgson
7426c86eb8 add a persistent cache of URL lookups, and fix up the in-memory one to work 2016-04-03 00:31:57 +01:00
Mark Haines
1fbb094c6f Add replication streams for ex outliers and current state resets 2016-03-30 17:19:56 +01:00
Erik Johnston
b275765545 Comment about weird SQL 2016-03-23 14:15:32 +00:00
Erik Johnston
0c1a27b787 SQLite and postgres doesn't share a true literal 2016-03-23 14:10:49 +00:00
Erik Johnston
84afeb41f3 Ensure all old public rooms have aliases 2016-03-23 13:59:34 +00:00
Mark Haines
ba660ecde2 Add a comment to offer a hint to an explanation for why we have a unique constraint on (app_id, pushkey, user_id) 2016-03-16 10:35:00 +00:00
Mark Haines
b6e8420aee Add replication stream for pushers 2016-03-15 17:33:10 +00:00
Erik Johnston
9669a99d1a Update users table in a batched manner 2016-03-10 15:12:19 +00:00
Daniel Wagner-Hall
3b97797c8d Merge branch 'develop' into daniel/ick 2016-03-08 17:35:09 +00:00
Daniel Wagner-Hall
edca2d9891 Idempotent-ise schema update script
If any ASes don't have an ID, the schema will fail, and then it will
error when trying to add the column again.
2016-03-08 17:32:29 +00:00
Mark Haines
ebcbb23226 s/stream_ordering/event_stream_ordering/ in push 2016-03-04 16:15:23 +00:00
Mark Haines
ec7460b4f2 Merge branch 'develop' into markjh/pushrule_stream 2016-03-04 14:44:34 +00:00
Erik Johnston
dd463e246d Merge pull request #614 from matrix-org/erikj/alias_delete
Allow alias creators to delete aliases
2016-03-04 09:02:33 +00:00
Daniel Wagner-Hall
95481e7ba7 Merge pull request #571 from matrix-org/daniel/asids
Mark AS users with their AS's ID
2016-03-03 13:56:28 +00:00
Mark Haines
a1cf9e3bf3 Add a stream for push rule updates 2016-03-01 18:16:37 +00:00
Erik Johnston
f9af8962f8 Allow alias creators to delete aliases 2016-03-01 14:46:31 +00:00
Daniel Wagner-Hall
33300673b7 Generate guest access token on 3pid invites
This means that following the same link across multiple sessions or
devices can re-use the same guest account.

Note that this is somewhat of an abuse vector; we can't throw up
captchas on this flow, so this is a way of registering ephemeral
accounts for spam, whose sign-up we don't rate limit.
2016-02-24 14:41:25 +00:00
Erik Johnston
112283e230 Prefix TS fields with _ts 2016-02-18 10:11:43 +00:00
Erik Johnston
e5999bfb1a Initial cut 2016-02-17 15:40:50 +00:00
Daniel Wagner-Hall
763360594d Mark AS users with their AS's ID 2016-02-11 17:26:42 +00:00
Erik Johnston
8450114098 Merge pull request #554 from matrix-org/erikj/event_push
Change event_push_actions_rm_tokens schema
2016-02-03 15:02:47 +00:00
Erik Johnston
771528ab13 Change event_push_actions_rm_tokens schema 2016-02-03 10:50:49 +00:00
Erik Johnston
65e92eca49 Change the way we do public room list fetching 2016-02-02 15:21:10 +00:00
Erik Johnston
fdca8ec418 Add events index 2016-01-28 16:41:59 +00:00
Erik Johnston
86896408b0 Add index to event_push_actions 2016-01-25 15:30:32 +00:00
David Baker
d4315bbf6b Add index by user id on receipts_linearized 2016-01-20 15:33:27 +00:00
David Baker
fe56138142 Remove rogue 'admin' 2016-01-11 17:09:03 +00:00
David Baker
c232780081 Merge pull request #456 from matrix-org/store_event_actions
Send unread notification counts
2016-01-08 14:47:15 +00:00
Matthew Hodgson
6c28ac260c copyrights 2016-01-07 04:26:29 +00:00
David Baker
ae1262a241 Add schema change file for is_guest flag 2016-01-06 11:58:20 +00:00
David Baker
c79f221192 Add is_guest flag to users db to track whether a user is a guest user or not. Use this so we can run _filter_events_for_client when calculating event_push_actions. 2016-01-06 11:38:09 +00:00
David Baker
eb03625626 Merge remote-tracking branch 'origin/develop' into store_event_actions 2016-01-05 18:39:50 +00:00
Daniel Wagner-Hall
cfd07aafff Allow guests to upgrade their accounts 2016-01-05 18:01:18 +00:00
David Baker
c914d67cda Rename event-actions to event_push_actions as per PR request 2016-01-04 14:05:37 +00:00
David Baker
42ad49f5b7 still very WIP, but now sends unread_notifications_count in the room object on sync (only actually corrrect in a full sync: hardcoded to 0 in incremental syncs). 2015-12-16 18:42:09 +00:00
David Baker
aa667ee396 Save event actions to the db 2015-12-10 17:51:15 +00:00
Mark Haines
d57c5cda71 Bump schema version.
As we released version 26 in v0.11.1
2015-12-04 15:28:39 +00:00
Erik Johnston
b8680b82c3 Merge pull request #414 from matrix-org/erikj/if_not_exists
Older versions of SQLite don't like IF NOT EXISTS in virtual tables
2015-12-03 13:21:09 +00:00
David Baker
e515b48929 Just replace the table definition with the one from full_schema 16 2015-12-02 17:23:52 +00:00
David Baker
8810eb8c39 Fix schema delta 15 on postgres in the very unlikley event that anyone upgrades to 15... 2015-12-02 17:19:11 +00:00
Erik Johnston
b2def42bfd Older versions of SQLite don't like IF NOT EXISTS in virtual tables 2015-12-02 13:29:14 +00:00
Mark Haines
95f30ecd1f Add API for setting account_data globaly or on a per room basis 2015-12-01 18:41:32 +00:00
Erik Johnston
4dcaa42b6d Allow paginating search ordered by recents 2015-11-30 17:45:31 +00:00
Daniel Wagner-Hall
7dfa455508 Remove size specifier for database column
Postgres doesn't support them like this.

We don't have a bool type in common between postgres and sqlite.
2015-11-23 18:35:25 +00:00
Paul "LeoNerd" Evans
1cfda3d2d8 Merge branch 'develop' into daniel/forgetrooms 2015-11-19 16:53:13 +00:00
Mark Haines
d153f482dd Rename the database table 2015-11-18 15:33:02 +00:00
Daniel Wagner-Hall
ba26eb3d5d Allow users to forget rooms 2015-11-17 17:17:30 -05:00
Erik Johnston
e21cef9bb5 Merge pull request #359 from matrix-org/markjh/incremental_indexing
Incremental background updates for db indexes
2015-11-11 17:19:51 +00:00
Mark Haines
e1627388d1 Fix param style to work on both sqlite and postgres 2015-11-11 17:14:56 +00:00
Mark Haines
940a161192 Fix the background update 2015-11-11 13:59:40 +00:00
Daniel Wagner-Hall
cf437900e0 Return world_readable and guest_can_join in /publicRooms 2015-11-10 17:10:27 +00:00
Mark Haines
90b503216c Use a background task to update databases to use the full text search 2015-11-10 16:20:13 +00:00
Mark Haines
c6a01f2ed0 Add storage module for tracking background updates.
The progress for each background update is stored as a JSON blob in the
database. Each background update is broken up into separate batches.
The batch size is automatically tuned to try avoid blocking single
threaded databases for too long.
2015-11-09 17:26:27 +00:00
Daniel Wagner-Hall
f2c4ee41b9 Remove accidentally added ID column 2015-11-06 14:27:49 +00:00
Daniel Wagner-Hall
ca2f90742d Open up /events to anonymous users for room events only
Squash-merge of PR #345 from daniel/anonymousevents
2015-11-05 14:32:26 +00:00
Mark Haines
6797fcd9ab Merge pull request #335 from matrix-org/markjh/room_tags
Add APIs for adding and removing tags from rooms
2015-11-03 16:45:53 +00:00
Mark Haines
ddd8566f41 Store room tag content and return the content in the m.tag event 2015-11-02 15:11:31 +00:00
Daniel Wagner-Hall
621e84d9a0 Add missing column 2015-10-30 16:25:53 +00:00
Mark Haines
892e70ec84 Add APIs for adding and removing tags from rooms 2015-10-28 16:06:57 +00:00
Erik Johnston
4cf633d5e9 Pull out sender when computing search results 2015-10-23 15:41:36 +01:00
Erik Johnston
0c36098c1f Implement rank function for SQLite FTS 2015-10-23 13:23:48 +01:00
Erik Johnston
f142898f52 PEP8 2015-10-22 11:18:01 +01:00
Erik Johnston
4d25bc6c92 Move FTS to delta 25 2015-10-22 11:12:28 +01:00
Erik Johnston
22a8c91448 Split up run_upgrade 2015-10-16 11:19:44 +01:00
Erik Johnston
88971fd034 Merge branch 'erikj/store_engine' into erikj/search 2015-10-13 14:03:30 +01:00
Erik Johnston
f9340ea0d5 Merge branch 'erikj/store_engine' into erikj/search 2015-10-13 13:29:02 +01:00
Erik Johnston
1a40afa756 Add sqlite schema 2015-10-13 10:36:25 +01:00
Erik Johnston
61561b9df7 Keep FTS indexes up to date. Only search through rooms currently joined 2015-10-12 10:49:53 +01:00
Erik Johnston
c85c912562 Add basic full text search impl. 2015-10-09 15:48:31 +01:00
Daniel Wagner-Hall
7213588083 Implement configurable stats reporting
SYN-287

This requires that HS owners either opt in or out of stats reporting.

When --generate-config is passed, --report-stats must be specified
If an already-generated config is used, and doesn't have the
report_stats key, it is requested to be set.
2015-09-22 12:57:40 +01:00
Erik Johnston
dffc9c4ae0 Drop unused index 2015-09-14 14:41:37 +01:00
Mark Haines
037481a033 Remove autoincrement since we incrementing the ID in the storage layer 2015-08-24 17:48:57 +01:00
Daniel Wagner-Hall
e8cf77fa49 Merge branch 'develop' into refresh
Conflicts:
	synapse/rest/client/v1/login.py
2015-08-20 16:25:40 +01:00
Daniel Wagner-Hall
cecbd636e9 /tokenrefresh POST endpoint
This allows refresh tokens to be exchanged for (access_token,
refresh_token).

It also starts issuing them on login, though no clients currently
interpret them.
2015-08-20 16:21:35 +01:00
Mark Haines
a0b8e5f2fe Merge pull request #211 from matrix-org/email_in_use
Changes for unique emails
2015-08-20 10:04:04 +01:00
Erik Johnston
8f4165628b Add index receipts_linearized_room_stream 2015-08-17 14:43:54 +01:00
David Baker
c77048e12f Add endpoint that proxies ID server request token and errors if the given email is in use on this Home Server. 2015-08-04 14:37:09 +01:00
Erik Johnston
5989637f37 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/receipts 2015-07-13 13:50:57 +01:00
Erik Johnston
0d7f0febf4 Uniquely name unique constraint 2015-07-10 13:43:03 +01:00
Mark Haines
bf0d59ed30 Don't bother with a timeout for one time keys on the server. 2015-07-09 14:04:03 +01:00
Erik Johnston
1af188209a Change format of receipts to allow inclusion of data 2015-07-09 11:39:30 +01:00
Mark Haines
b5770f8947 Add store for client end to end keys 2015-07-06 18:46:47 +01:00
Erik Johnston
ac78e60de6 Add stream_id index 2015-07-02 13:18:41 +01:00
Erik Johnston
80a61330ee Add basic storage functions for handling of receipts 2015-07-01 17:19:12 +01:00
Erik Johnston
901f56fa63 Add tables for receipts 2015-06-30 15:29:47 +01:00
Erik Johnston
2eda996a63 Add a dummy.sql into delta/20 as pip isn't packinging the pushers.py 2015-06-13 12:21:58 +01:00
David Baker
da84946de4 pep8 2015-06-04 16:43:45 +01:00
David Baker
63a7b3ad1e Add script to (re)convert the pushers table to changing the unique key. Also give the python db upgrade scripts the database engine so they can convert parameter strings, and add *args **kwargs to the upgrade function so we can add more args in future and previous scripts will ignore them. 2015-06-04 16:16:01 +01:00
Erik Johnston
c71176858b Newline, remove debug logging 2015-05-18 10:11:14 +01:00
Erik Johnston
4f1d984e56 Add index on events 2015-05-13 17:22:26 +01:00
Mark Haines
1c7912751e Drop the old table not the new table 2015-05-08 16:04:32 +01:00
Mark Haines
9d36eb4eab Rename unique constraint 2015-05-08 16:01:55 +01:00
Mark Haines
b0f71db3ff Remove unsigned 2015-05-08 15:59:51 +01:00
Mark Haines
6538d445e8 Make the timestamps in server_keys_json bigints 2015-05-08 15:55:17 +01:00
Erik Johnston
b8940cd902 Remove some unused indexes 2015-05-01 16:14:25 +01:00
David Baker
f8c30faf25 Oops, update the contraint too 2015-04-29 16:58:42 +01:00
David Baker
61cd5d9045 Be more postgressive 2015-04-29 16:57:14 +01:00
David Baker
fb95035a65 Be postgressive 2015-04-29 16:53:41 +01:00
David Baker
4669def000 Oops, forgot the schema delta file 2015-04-29 16:50:16 +01:00
Mark Haines
e26a3d8d9e bump database schema version 2015-04-29 13:32:32 +01:00
Mark Haines
a9549fdce3 Use bytea rather than BLOB 2015-04-29 13:16:09 +01:00
Mark Haines
4ad8b45155 Merge branch 'develop' into key_distribution
Conflicts:
	synapse/config/homeserver.py
2015-04-29 13:15:14 +01:00
Erik Johnston
327ca883ec Merge branch 'develop' of github.com:matrix-org/synapse into postgres 2015-04-28 13:39:42 +01:00
Erik Johnston
18f8247701 Use TEXT instead of VARCHAR(n), since PostgreSQL treats them the same except for a limit 2015-04-28 12:41:33 +01:00
Mark Haines
9182f87664 Merge pull request #126 from matrix-org/csauth
Client / Server Auth Refactor
2015-04-28 11:00:27 +01:00
Mark Haines
b96c133034 Add server_keys.sql to the current delta rather than creating a new delta 2015-04-28 10:50:00 +01:00
Erik Johnston
6f8e2d517e Merge branch 'develop' of github.com:matrix-org/synapse into postgres 2015-04-27 14:41:40 +01:00
Mark Haines
eede182df7 Merge branch 'develop' into key_distribution 2015-04-24 10:35:49 +01:00
Mark Haines
4e2f8b8722 Copyright notices 2015-04-24 10:35:29 +01:00
David Baker
cb03fafdf1 Merge branch 'develop' into csauth 2015-04-17 13:51:10 +01:00
Erik Johnston
0ec346d942 Add unique index to room_aliases, remove duplicates on upgrade. Convert some columns back to TEXT from bytea 2015-04-17 11:10:20 +01:00
Erik Johnston
b8092fbc82 Go back to storing JSON in TEXT 2015-04-16 11:17:52 +01:00
Erik Johnston
ffad75bd62 Remove mysql/maria support 2015-04-15 17:00:50 +01:00
Mark Haines
a429515bdd Add methods for storing and retrieving the raw key json 2015-04-15 16:58:35 +01:00