Erik Johnston
77f1d24de3
More brackets
2017-09-28 12:23:15 +01:00
Erik Johnston
9ccb4226ba
Delete expired url cache data
2017-09-28 12:18:06 +01:00
Erik Johnston
a8e2a3df32
Add unique index to group_rooms table
2017-09-26 15:39:21 +01:00
Erik Johnston
27ebc5c8f2
Add remote profile cache
2017-08-25 11:25:47 +01:00
Erik Johnston
8b50fe5330
Use BOOLEAN rather than TEXT type
2017-08-21 16:37:29 +01:00
Erik Johnston
05e21285aa
Store whether the user wants to publicise their membership of a group
2017-08-08 13:01:46 +01:00
Erik Johnston
c544188ee3
Add groups to sync stream
2017-07-20 16:36:42 +01:00
Erik Johnston
508460f240
Remove sync stuff
2017-07-18 09:55:46 +01:00
Erik Johnston
4540730111
Remove unused tables
2017-07-18 09:38:15 +01:00
Erik Johnston
2f9eafdd36
Add local group server support
2017-07-17 12:03:49 +01:00
Erik Johnston
26451a09eb
Comments
2017-07-12 14:47:18 +01:00
Erik Johnston
a62406aaa5
Add group summary APIs
2017-07-12 11:36:15 +01:00
Erik Johnston
6d586dc05c
Comment
2017-07-12 09:58:37 +01:00
Erik Johnston
410b4e14a1
Move comment
2017-07-11 15:44:18 +01:00
Erik Johnston
e52c391cd4
Rename column to attestation_json
2017-07-11 14:25:46 +01:00
Erik Johnston
0aac30d53b
Comments
2017-07-11 14:23:50 +01:00
Erik Johnston
83936293eb
Comments
2017-07-11 11:42:25 +01:00
Erik Johnston
b8ca494ee9
Initial group server implementation
2017-07-10 15:44:15 +01:00
Erik Johnston
8abdd7b553
Fix up indices for users_who_share_rooms
2017-06-26 14:01:30 +01:00
Erik Johnston
7fe8ed1787
Store URL cache preview downloads seperately
...
This makes it easier to clear old media out at a later date
2017-06-23 11:14:11 +01:00
Erik Johnston
b8b936a6ea
Add API to quarantine media
2017-06-19 17:39:21 +01:00
Erik Johnston
d7fe6b356c
Add shutdown room API
2017-06-19 12:37:27 +01:00
Erik Johnston
ebcd55d641
Add DB schema for tracking users who share rooms
2017-06-15 09:45:48 +01:00
Erik Johnston
65f0513a33
Split up device_lists_outbound_pokes table for faster updates.
2017-06-07 11:02:38 +01:00
Erik Johnston
21e255a8f1
Split the table in two
2017-06-01 14:50:46 +01:00
Erik Johnston
5dd1b2c525
Use unique indices
2017-05-31 17:29:12 +01:00
Erik Johnston
5d79d728f5
Split out directory and search tables
2017-05-31 15:23:49 +01:00
Erik Johnston
350622a107
Handle the server leaving a public room
2017-05-31 15:11:36 +01:00
Erik Johnston
63fda37e20
Add comments
2017-05-31 15:00:29 +01:00
Erik Johnston
eeb2f9e546
Add user_directory to database
2017-05-31 11:51:01 +01:00
Erik Johnston
dd48f7204c
Add comment
2017-05-30 15:01:22 +01:00
Erik Johnston
04095f7581
Add clobbered event_id
2017-05-30 14:53:01 +01:00
Erik Johnston
a584a81b3e
Add current_state_delta_stream table
2017-05-30 14:44:09 +01:00
Erik Johnston
6e614e9e10
Add background task to clear out old event_auth
2017-05-24 15:23:34 +01:00
Richard van der Hoff
ff3d810ea8
Add a comment to old delta
2017-05-11 12:48:50 +01:00
Richard van der Hoff
baafb85ba4
Add an index to event_search
...
- to make the purge API quicker
2017-05-11 12:05:22 +01:00
Erik Johnston
b990b2fce5
Add per user ratelimiting overrides
2017-05-10 11:05:43 +01:00
Erik Johnston
d766343668
Add index to device_lists_stream
2017-03-01 15:56:30 +00:00
Erik Johnston
e4919b9329
Add stream_id index to device_lists_outbound_pokes
...
As this is used for replication streaming
2017-02-28 11:19:06 +00:00
David Baker
4aa29508af
Use TEXT rather than VARCHAR
...
While we're changing anyway
2017-02-17 10:51:49 +00:00
David Baker
b4017539d4
Make the pushers lang field column longer
...
To accommodate things like zh-Hans-CN
Fixes https://github.com/vector-im/riot-ios/issues/1031
2017-02-17 10:42:57 +00:00
Erik Johnston
095b45c165
Aggregate event push actions
2017-02-14 13:39:41 +00:00
Erik Johnston
21b7375778
Add an index to make membership queries faster
2017-01-31 15:15:57 +00:00
Erik Johnston
d1e1fd6210
Add ts column to device_lists_outbound_pokes
2017-01-27 15:23:48 +00:00
Erik Johnston
fbfad76c03
Add comments
2017-01-26 16:33:21 +00:00
Erik Johnston
c974116f19
Implement device key caching over federation
2017-01-26 16:07:24 +00:00
Erik Johnston
2367c5568c
Add basic implementation of local device list changes
2017-01-25 14:27:27 +00:00
Erik Johnston
c050f493dd
Add comment
2017-01-13 15:14:41 +00:00
Erik Johnston
caddadfc5a
Change device_inbox stream index to include user
...
This makes fetching the nost recently changed users much tricker, and
brings it in line with e.g. presence_stream indices.
2017-01-10 15:04:57 +00:00
Erik Johnston
1574b839e0
Merge pull request #1676 from matrix-org/erikj/room_list
...
Add new API appservice specific public room list
2016-12-12 17:00:10 +00:00
Erik Johnston
c45d8e9ba2
Add profile data to the room_membership table for joins
2016-12-08 13:08:41 +00:00
Erik Johnston
2687af82d4
Comments
2016-12-07 09:58:33 +00:00
Erik Johnston
f32fb65552
Add new API appservice specific public room list
2016-12-06 16:12:27 +00:00
Erik Johnston
46ca345b06
Don't send old events as federation
2016-11-24 12:29:02 +00:00
Erik Johnston
2ac516850b
More efficient notif count queries
2016-11-23 15:57:04 +00:00
Erik Johnston
7c9cdb2245
Store federation stream positions in the database
2016-11-21 11:33:08 +00:00
Erik Johnston
f8ee66250a
Handle sending events and device messages over federation
2016-11-17 15:48:04 +00:00
Erik Johnston
8fd4d9129f
Replace postgres GIN with GIST
...
This is because GIN can be slow to write too, especially when the table
gets large.
2016-11-03 15:00:03 +00:00
Erik Johnston
a9111786f9
Use most recently added binding, not most recently seen user.
2016-10-27 14:32:45 +01:00
Erik Johnston
1fc1bc2a51
Fix user_threepids schema delta
...
The delta `37/user_threepids.sql` aimed to update all the email
addresses to be lower case, however duplicate emails may exist in the
table already.
This commit adds a step where the delta moves the duplicate emails to a
new `medium` `email_old`. Only the most recently used account keeps the
binding intact. We move rather than delete so that we retain some record
of which emails were associated with which account.
2016-10-27 14:14:44 +01:00
David Baker
0108ed8ae6
Latest delta is now 37
2016-10-19 11:40:35 +01:00
David Baker
a7f48320b1
Merge remote-tracking branch 'origin/develop' into dbkr/password_reset_case_insensitive
2016-10-19 11:28:56 +01:00
David Baker
df2a616c7b
Convert emails to lowercase when storing
...
And db migration sql to convert existing addresses.
2016-10-19 11:13:55 +01:00
Erik Johnston
6942d68247
Bump schema version
2016-10-17 11:17:45 +01:00
Erik Johnston
b59994b454
Remove TODO
2016-10-17 11:17:02 +01:00
Erik Johnston
816988baaa
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/remove_auth
2016-10-17 11:10:37 +01:00
Erik Johnston
2869a29fd7
Drop some unused indices
2016-10-17 11:08:19 +01:00
David Baker
62073992c5
Make password reset email field case insensitive
2016-10-14 13:56:53 +01:00
Erik Johnston
8009d84364
Match against event_id, rather than room_id
2016-09-21 16:46:59 +01:00
Erik Johnston
dc78db8c56
Update correct table
2016-09-21 15:52:44 +01:00
Erik Johnston
4f78108d8c
Readd entries to public_room_list_stream that were deleted
2016-09-21 15:24:22 +01:00
Erik Johnston
baffe96d95
Add a room visibility stream
2016-09-14 17:29:19 +01:00
Erik Johnston
ed992ae6ba
Add a DB index to figure out past state at a stream ordering in a room
2016-09-14 16:20:27 +01:00
Erik Johnston
43253c10b8
Remove redundant event_auth index
2016-09-13 11:47:48 +01:00
Erik Johnston
0b32bb20bb
Index contains_url for file search queries
2016-09-12 16:57:05 +01:00
Erik Johnston
7fe42cf949
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/make_notif_highlight_query_fast
2016-09-12 12:37:09 +01:00
Erik Johnston
15ca0c6a4d
Make reindex happen in bg
2016-09-12 12:36:36 +01:00
Mark Haines
31f85f9db9
Add comments to existing schema deltas that used "CREATE INDEX" directly
2016-09-12 11:00:26 +01:00
Erik Johnston
54417999b6
Revert "Add index to event_push_actions"
2016-09-12 10:39:55 +01:00
David Baker
b91e2833b3
Merge remote-tracking branch 'origin/develop' into dbkr/make_notif_highlight_query_fast
2016-09-09 19:11:34 +01:00
David Baker
f2acc3dcf9
Add index to event_push_actions
...
and remove room_id caluse so it uses it
Mostly from @negativemjark
2016-09-09 18:54:54 +01:00
Mark Haines
647c724573
Use the previous MAX value if any to set the stream_id
2016-09-09 11:52:44 +01:00
Mark Haines
6a6cbfcf1e
Track the max_stream_device_id in a separate table, since we delete from the inbox table
2016-09-09 11:48:23 +01:00
Mark Haines
303b6f29f0
Merge pull request #1087 from matrix-org/markjh/reapply_delta
...
Reapply 34/device_outbox in 35/device_outbox_again.py since the schem…
2016-09-09 11:24:27 +01:00
Mark Haines
0877157353
Just move the schema and add some DROPs
2016-09-09 11:04:47 +01:00
Mark Haines
43b77c5d97
Only catch databas errors
2016-09-08 17:44:21 +01:00
Mark Haines
7d5b142547
Add a stub run_upgrade
2016-09-08 17:39:11 +01:00
Mark Haines
fa722a699c
Reapply 34/device_outbox in 35/device_outbox_again.py since the schema was bumped before it landed on develop
2016-09-08 17:35:16 +01:00
Erik Johnston
ebb46497ba
Add delta file
2016-09-08 16:38:54 +01:00
Mark Haines
fa9d36e050
Merge branch 'develop' into markjh/direct_to_device_federation
2016-09-08 13:43:43 +01:00
Mark Haines
d4a35ada28
Send device messages over federation
2016-09-06 18:16:20 +01:00
Mark Haines
2ad72da931
Add tables for federated device messages
...
Adds tables for storing the messages that need to be sent to a
remote device and for deduplicating messages received.
2016-09-06 15:10:40 +01:00
Erik Johnston
485d999c8a
Correctly delete old state groups in purge history API
2016-09-05 14:49:08 +01:00
Erik Johnston
a99e933550
Add upgrade script that will slowly prune state_groups_state entries
2016-09-05 10:05:36 +01:00
Erik Johnston
657847e4c6
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/state_storage
2016-09-02 11:04:48 +01:00
Erik Johnston
9e25443db8
Move to storing state_groups_state as deltas
2016-09-01 14:31:26 +01:00
Erik Johnston
ef0cc648cf
Clean up old sent transactions
2016-08-31 11:12:02 +01:00
Mark Haines
e993925279
Add store-and-forward direct-to-device messaging
2016-08-25 17:35:37 +01:00
Erik Johnston
aad8a1a825
Delete old received_transactions
2016-08-22 16:29:46 +01:00
Erik Johnston
9da84a9a1e
Make AppserviceHandler stream events from database
...
This is for two reasons:
1. Suppresses duplicates correctly, as the notifier doesn't do any
duplicate suppression.
2. Makes it easier to connect the AppserviceHandler to the replication
stream.
2016-08-18 11:54:41 +01:00
Erik Johnston
732cf72b86
Fix push_display_name_rename schema update
2016-08-17 18:12:21 +01:00
David Baker
ad42322257
Add migration script
...
To port existing rule actions & enable entries to the new name
2016-08-16 16:56:30 +01:00
Erik Johnston
d9664344ec
Rename table. Add docs.
2016-08-15 11:45:57 +01:00
Erik Johnston
4d70d1f80e
Add some invalidations to a cache_stream
2016-08-15 11:15:17 +01:00
Richard van der Hoff
4fec5e57be
Default device_display_name to null
...
It turns out that it's more useful to return a null device display name (and
let clients decide how to handle it: eg, falling back to device_id) than using
a constant string like "unknown device".
2016-08-03 11:53:00 +01:00
Richard van der Hoff
26cb0efa88
SQL syntax fix
2016-07-27 12:30:22 +01:00
Richard van der Hoff
2e3d90d67c
Make the device id on e2e key upload optional
...
We should now be able to get our device_id from the access_token, so the
device_id on the upload request is optional. Where it is supplied, we should
check that it matches.
For active access_tokens without an associated device_id, we ought to register
the device in the devices table.
Also update the table on upgrade so that all of the existing e2e keys are
associated with real devices.
2016-07-26 23:38:12 +01:00
Richard van der Hoff
436bffd15f
Implement deleting devices
2016-07-26 07:35:48 +01:00
Richard van der Hoff
ec5717caf5
Create index on user_ips in the background
...
user_ips is kinda big, so really we want to add the index in the background
once we're running. Replace the schema delta with one which will do that.
I've done this in a way that's reasonably easy to reuse as there a few other
indexes I need, and I don't suppose they will be the last.
2016-07-22 13:16:39 +01:00
Richard van der Hoff
bc8f265f0a
GET /devices endpoint
...
implement a GET /devices endpoint which lists all of the user's devices.
It also returns the last IP where we saw that device, so there is some dancing
to fish that out of the user_ips table.
2016-07-20 16:42:32 +01:00
Erik Johnston
1e2a7f18a1
Merge pull request #922 from matrix-org/erikj/file_api2
...
Feature: Add filter to /messages. Add 'contains_url' to filter.
2016-07-20 10:40:48 +01:00
Richard van der Hoff
022b9176fe
schema fix
...
device_id should be text, not bigint.
2016-07-19 11:44:05 +01:00
Richard van der Hoff
f863a52cea
Add device_id support to /login
...
Add a 'devices' table to the storage, as well as a 'device_id' column to
refresh_tokens.
Allow the client to pass a device_id, and initial_device_display_name, to
/login. If login is successful, then register the device in the devices table
if it wasn't known already. If no device_id was supplied, make one up.
Associate the device_id with the access token and refresh token, so that we can
get at it again later. Ensure that the device_id is copied from the refresh
token to the access_token when the token is refreshed.
2016-07-18 16:39:44 +01:00
Erik Johnston
b64aa6d687
Add sender and contains_url field to events table
2016-07-14 15:35:43 +01:00
Erik Johnston
314b146b2e
Track approximate last access time for remote media
2016-06-29 11:41:20 +01:00
Mark Haines
c8c5bf950a
Fix synapse/storage/schema/delta/30/as_users.py
2016-05-31 17:10:40 +01:00
Erik Johnston
c9aff0736c
Remove topics table
2016-05-13 14:40:38 +01:00
Mark Haines
9295fa30a8
Annotate the removed indicies with why they were removed.
2016-05-13 14:16:57 +01:00
Erik Johnston
5e50058473
Remove unused indices
...
This includes removing both unused indices and indices that are subsets
of other indices.
2016-05-13 13:28:07 +01:00
Matrix
3b1930e8ec
unbreak schema
2016-05-10 16:42:37 +01:00
David Baker
997db04648
Merge remote-tracking branch 'origin/develop' into dbkr/email_notifs
2016-05-10 14:40:19 +02:00
Mark Haines
9c272da05f
Add an openidish mechanism for proving to third parties that you own a given user_id
2016-05-05 13:42:44 +01:00
Erik Johnston
fcd1eb642d
Add primary key
2016-05-04 16:51:51 +01:00
Erik Johnston
8e6a163f27
Add timestamp and auto incrementing ID
2016-05-04 15:19:12 +01:00
Erik Johnston
984d4a2c0f
Add /report endpoint
2016-05-04 11:28:10 +01:00
David Baker
acded821c4
Merge remote-tracking branch 'origin/develop' into dbkr/email_notifs
2016-04-29 10:05:20 +01:00
Erik Johnston
ae571810f2
Order NULLs first
2016-04-21 18:14:18 +01:00
Erik Johnston
8fae3d7b1e
Use special UPDATE syntax
2016-04-21 18:01:49 +01:00
Erik Johnston
51bb339ab2
Create index concurrently
2016-04-21 17:16:11 +01:00
Erik Johnston
b743c1237e
Add missing run_upgrade
2016-04-21 17:12:04 +01:00
Erik Johnston
c877f0f034
Optimise event_search in postgres
2016-04-21 16:56:14 +01:00
David Baker
07d765209d
First bits of emailpusher
...
Mostly logic of when to send an email
2016-04-19 14:24:36 +01:00
Erik Johnston
96bcfb29c7
Add index
2016-04-14 11:26:33 +01:00
David Baker
2547dffccc
Merge pull request #705 from matrix-org/dbkr/pushers_use_event_actions
...
Change pushers to use the event_actions table
2016-04-11 12:58:55 +01:00
Matthew Hodgson
6ff7a79308
move local_media_repository_url_cache.sql to schema v31
2016-04-08 19:09:02 +01:00
Matthew Hodgson
d6e7333ae4
Merge branch 'develop' into matthew/preview_urls
2016-04-07 17:26:44 +01:00
David Baker
05d044aac3
pep8
2016-04-07 16:45:38 +01:00
David Baker
2d5c693fd3
Fix port script for changes merged from develop
2016-04-07 16:43:54 +01:00
David Baker
9c99ab4572
Merge remote-tracking branch 'origin/develop' into dbkr/pushers_use_event_actions
2016-04-07 16:35:22 +01:00
David Baker
7e2c89a37f
Make pushers use the event_push_actions table instead of listening on an event stream & running the rules again. Sytest passes, but remaining to do:
...
* Make badges work again
* Remove old, unused code
2016-04-06 15:42:15 +01:00
Erik Johnston
8aab9d87fa
Don't require config to create database
2016-04-06 14:15:45 +01:00
Erik Johnston
0c53d750e7
Docs and indents
2016-04-04 18:02:48 +01:00
Erik Johnston
92ab45a330
Add upgrade path, rename table
2016-04-04 17:07:43 +01:00
Erik Johnston
3d76b7cb2b
Store invites in a separate table.
2016-04-04 16:30:15 +01:00
Matthew Hodgson
9f7dc2bef7
Merge branch 'develop' into matthew/preview_urls
2016-04-04 00:38:21 +01:00
Matthew Hodgson
7426c86eb8
add a persistent cache of URL lookups, and fix up the in-memory one to work
2016-04-03 00:31:57 +01:00
Mark Haines
1fbb094c6f
Add replication streams for ex outliers and current state resets
2016-03-30 17:19:56 +01:00
Erik Johnston
b275765545
Comment about weird SQL
2016-03-23 14:15:32 +00:00
Erik Johnston
0c1a27b787
SQLite and postgres doesn't share a true literal
2016-03-23 14:10:49 +00:00
Erik Johnston
84afeb41f3
Ensure all old public rooms have aliases
2016-03-23 13:59:34 +00:00
Mark Haines
ba660ecde2
Add a comment to offer a hint to an explanation for why we have a unique constraint on (app_id, pushkey, user_id)
2016-03-16 10:35:00 +00:00
Mark Haines
b6e8420aee
Add replication stream for pushers
2016-03-15 17:33:10 +00:00
Erik Johnston
9669a99d1a
Update users table in a batched manner
2016-03-10 15:12:19 +00:00
Daniel Wagner-Hall
3b97797c8d
Merge branch 'develop' into daniel/ick
2016-03-08 17:35:09 +00:00
Daniel Wagner-Hall
edca2d9891
Idempotent-ise schema update script
...
If any ASes don't have an ID, the schema will fail, and then it will
error when trying to add the column again.
2016-03-08 17:32:29 +00:00
Mark Haines
ebcbb23226
s/stream_ordering/event_stream_ordering/ in push
2016-03-04 16:15:23 +00:00
Mark Haines
ec7460b4f2
Merge branch 'develop' into markjh/pushrule_stream
2016-03-04 14:44:34 +00:00
Erik Johnston
dd463e246d
Merge pull request #614 from matrix-org/erikj/alias_delete
...
Allow alias creators to delete aliases
2016-03-04 09:02:33 +00:00
Daniel Wagner-Hall
95481e7ba7
Merge pull request #571 from matrix-org/daniel/asids
...
Mark AS users with their AS's ID
2016-03-03 13:56:28 +00:00
Mark Haines
a1cf9e3bf3
Add a stream for push rule updates
2016-03-01 18:16:37 +00:00
Erik Johnston
f9af8962f8
Allow alias creators to delete aliases
2016-03-01 14:46:31 +00:00
Daniel Wagner-Hall
33300673b7
Generate guest access token on 3pid invites
...
This means that following the same link across multiple sessions or
devices can re-use the same guest account.
Note that this is somewhat of an abuse vector; we can't throw up
captchas on this flow, so this is a way of registering ephemeral
accounts for spam, whose sign-up we don't rate limit.
2016-02-24 14:41:25 +00:00
Erik Johnston
112283e230
Prefix TS fields with _ts
2016-02-18 10:11:43 +00:00
Erik Johnston
e5999bfb1a
Initial cut
2016-02-17 15:40:50 +00:00
Daniel Wagner-Hall
763360594d
Mark AS users with their AS's ID
2016-02-11 17:26:42 +00:00
Erik Johnston
8450114098
Merge pull request #554 from matrix-org/erikj/event_push
...
Change event_push_actions_rm_tokens schema
2016-02-03 15:02:47 +00:00
Erik Johnston
771528ab13
Change event_push_actions_rm_tokens schema
2016-02-03 10:50:49 +00:00
Erik Johnston
65e92eca49
Change the way we do public room list fetching
2016-02-02 15:21:10 +00:00
Erik Johnston
fdca8ec418
Add events index
2016-01-28 16:41:59 +00:00
Erik Johnston
86896408b0
Add index to event_push_actions
2016-01-25 15:30:32 +00:00
David Baker
d4315bbf6b
Add index by user id on receipts_linearized
2016-01-20 15:33:27 +00:00
David Baker
fe56138142
Remove rogue 'admin'
2016-01-11 17:09:03 +00:00
David Baker
c232780081
Merge pull request #456 from matrix-org/store_event_actions
...
Send unread notification counts
2016-01-08 14:47:15 +00:00
Matthew Hodgson
6c28ac260c
copyrights
2016-01-07 04:26:29 +00:00
David Baker
ae1262a241
Add schema change file for is_guest flag
2016-01-06 11:58:20 +00:00
David Baker
c79f221192
Add is_guest flag to users db to track whether a user is a guest user or not. Use this so we can run _filter_events_for_client when calculating event_push_actions.
2016-01-06 11:38:09 +00:00
David Baker
eb03625626
Merge remote-tracking branch 'origin/develop' into store_event_actions
2016-01-05 18:39:50 +00:00
Daniel Wagner-Hall
cfd07aafff
Allow guests to upgrade their accounts
2016-01-05 18:01:18 +00:00
David Baker
c914d67cda
Rename event-actions to event_push_actions as per PR request
2016-01-04 14:05:37 +00:00
David Baker
42ad49f5b7
still very WIP, but now sends unread_notifications_count in the room object on sync (only actually corrrect in a full sync: hardcoded to 0 in incremental syncs).
2015-12-16 18:42:09 +00:00
David Baker
aa667ee396
Save event actions to the db
2015-12-10 17:51:15 +00:00
Mark Haines
d57c5cda71
Bump schema version.
...
As we released version 26 in v0.11.1
2015-12-04 15:28:39 +00:00
Erik Johnston
b8680b82c3
Merge pull request #414 from matrix-org/erikj/if_not_exists
...
Older versions of SQLite don't like IF NOT EXISTS in virtual tables
2015-12-03 13:21:09 +00:00
David Baker
e515b48929
Just replace the table definition with the one from full_schema 16
2015-12-02 17:23:52 +00:00
David Baker
8810eb8c39
Fix schema delta 15 on postgres in the very unlikley event that anyone upgrades to 15...
2015-12-02 17:19:11 +00:00
Erik Johnston
b2def42bfd
Older versions of SQLite don't like IF NOT EXISTS in virtual tables
2015-12-02 13:29:14 +00:00
Mark Haines
95f30ecd1f
Add API for setting account_data globaly or on a per room basis
2015-12-01 18:41:32 +00:00
Erik Johnston
4dcaa42b6d
Allow paginating search ordered by recents
2015-11-30 17:45:31 +00:00
Daniel Wagner-Hall
7dfa455508
Remove size specifier for database column
...
Postgres doesn't support them like this.
We don't have a bool type in common between postgres and sqlite.
2015-11-23 18:35:25 +00:00
Paul "LeoNerd" Evans
1cfda3d2d8
Merge branch 'develop' into daniel/forgetrooms
2015-11-19 16:53:13 +00:00
Mark Haines
d153f482dd
Rename the database table
2015-11-18 15:33:02 +00:00
Daniel Wagner-Hall
ba26eb3d5d
Allow users to forget rooms
2015-11-17 17:17:30 -05:00
Erik Johnston
e21cef9bb5
Merge pull request #359 from matrix-org/markjh/incremental_indexing
...
Incremental background updates for db indexes
2015-11-11 17:19:51 +00:00
Mark Haines
e1627388d1
Fix param style to work on both sqlite and postgres
2015-11-11 17:14:56 +00:00
Mark Haines
940a161192
Fix the background update
2015-11-11 13:59:40 +00:00
Daniel Wagner-Hall
cf437900e0
Return world_readable and guest_can_join in /publicRooms
2015-11-10 17:10:27 +00:00
Mark Haines
90b503216c
Use a background task to update databases to use the full text search
2015-11-10 16:20:13 +00:00
Mark Haines
c6a01f2ed0
Add storage module for tracking background updates.
...
The progress for each background update is stored as a JSON blob in the
database. Each background update is broken up into separate batches.
The batch size is automatically tuned to try avoid blocking single
threaded databases for too long.
2015-11-09 17:26:27 +00:00
Daniel Wagner-Hall
f2c4ee41b9
Remove accidentally added ID column
2015-11-06 14:27:49 +00:00
Daniel Wagner-Hall
ca2f90742d
Open up /events to anonymous users for room events only
...
Squash-merge of PR #345 from daniel/anonymousevents
2015-11-05 14:32:26 +00:00
Mark Haines
6797fcd9ab
Merge pull request #335 from matrix-org/markjh/room_tags
...
Add APIs for adding and removing tags from rooms
2015-11-03 16:45:53 +00:00
Mark Haines
ddd8566f41
Store room tag content and return the content in the m.tag event
2015-11-02 15:11:31 +00:00
Daniel Wagner-Hall
621e84d9a0
Add missing column
2015-10-30 16:25:53 +00:00
Mark Haines
892e70ec84
Add APIs for adding and removing tags from rooms
2015-10-28 16:06:57 +00:00
Erik Johnston
4cf633d5e9
Pull out sender when computing search results
2015-10-23 15:41:36 +01:00
Erik Johnston
0c36098c1f
Implement rank function for SQLite FTS
2015-10-23 13:23:48 +01:00
Erik Johnston
f142898f52
PEP8
2015-10-22 11:18:01 +01:00
Erik Johnston
4d25bc6c92
Move FTS to delta 25
2015-10-22 11:12:28 +01:00
Erik Johnston
22a8c91448
Split up run_upgrade
2015-10-16 11:19:44 +01:00
Erik Johnston
88971fd034
Merge branch 'erikj/store_engine' into erikj/search
2015-10-13 14:03:30 +01:00
Erik Johnston
f9340ea0d5
Merge branch 'erikj/store_engine' into erikj/search
2015-10-13 13:29:02 +01:00
Erik Johnston
1a40afa756
Add sqlite schema
2015-10-13 10:36:25 +01:00
Erik Johnston
61561b9df7
Keep FTS indexes up to date. Only search through rooms currently joined
2015-10-12 10:49:53 +01:00
Erik Johnston
c85c912562
Add basic full text search impl.
2015-10-09 15:48:31 +01:00
Daniel Wagner-Hall
7213588083
Implement configurable stats reporting
...
SYN-287
This requires that HS owners either opt in or out of stats reporting.
When --generate-config is passed, --report-stats must be specified
If an already-generated config is used, and doesn't have the
report_stats key, it is requested to be set.
2015-09-22 12:57:40 +01:00
Erik Johnston
dffc9c4ae0
Drop unused index
2015-09-14 14:41:37 +01:00
Mark Haines
037481a033
Remove autoincrement since we incrementing the ID in the storage layer
2015-08-24 17:48:57 +01:00
Daniel Wagner-Hall
e8cf77fa49
Merge branch 'develop' into refresh
...
Conflicts:
synapse/rest/client/v1/login.py
2015-08-20 16:25:40 +01:00
Daniel Wagner-Hall
cecbd636e9
/tokenrefresh POST endpoint
...
This allows refresh tokens to be exchanged for (access_token,
refresh_token).
It also starts issuing them on login, though no clients currently
interpret them.
2015-08-20 16:21:35 +01:00
Mark Haines
a0b8e5f2fe
Merge pull request #211 from matrix-org/email_in_use
...
Changes for unique emails
2015-08-20 10:04:04 +01:00
Erik Johnston
8f4165628b
Add index receipts_linearized_room_stream
2015-08-17 14:43:54 +01:00
David Baker
c77048e12f
Add endpoint that proxies ID server request token and errors if the given email is in use on this Home Server.
2015-08-04 14:37:09 +01:00
Erik Johnston
5989637f37
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/receipts
2015-07-13 13:50:57 +01:00
Erik Johnston
0d7f0febf4
Uniquely name unique constraint
2015-07-10 13:43:03 +01:00
Mark Haines
bf0d59ed30
Don't bother with a timeout for one time keys on the server.
2015-07-09 14:04:03 +01:00
Erik Johnston
1af188209a
Change format of receipts to allow inclusion of data
2015-07-09 11:39:30 +01:00
Mark Haines
b5770f8947
Add store for client end to end keys
2015-07-06 18:46:47 +01:00
Erik Johnston
ac78e60de6
Add stream_id index
2015-07-02 13:18:41 +01:00
Erik Johnston
80a61330ee
Add basic storage functions for handling of receipts
2015-07-01 17:19:12 +01:00
Erik Johnston
901f56fa63
Add tables for receipts
2015-06-30 15:29:47 +01:00
Erik Johnston
2eda996a63
Add a dummy.sql into delta/20 as pip isn't packinging the pushers.py
2015-06-13 12:21:58 +01:00
David Baker
da84946de4
pep8
2015-06-04 16:43:45 +01:00
David Baker
63a7b3ad1e
Add script to (re)convert the pushers table to changing the unique key. Also give the python db upgrade scripts the database engine so they can convert parameter strings, and add *args **kwargs to the upgrade function so we can add more args in future and previous scripts will ignore them.
2015-06-04 16:16:01 +01:00
Erik Johnston
c71176858b
Newline, remove debug logging
2015-05-18 10:11:14 +01:00
Erik Johnston
4f1d984e56
Add index on events
2015-05-13 17:22:26 +01:00
Mark Haines
1c7912751e
Drop the old table not the new table
2015-05-08 16:04:32 +01:00
Mark Haines
9d36eb4eab
Rename unique constraint
2015-05-08 16:01:55 +01:00
Mark Haines
b0f71db3ff
Remove unsigned
2015-05-08 15:59:51 +01:00
Mark Haines
6538d445e8
Make the timestamps in server_keys_json bigints
2015-05-08 15:55:17 +01:00
Erik Johnston
b8940cd902
Remove some unused indexes
2015-05-01 16:14:25 +01:00
David Baker
f8c30faf25
Oops, update the contraint too
2015-04-29 16:58:42 +01:00
David Baker
61cd5d9045
Be more postgressive
2015-04-29 16:57:14 +01:00
David Baker
fb95035a65
Be postgressive
2015-04-29 16:53:41 +01:00
David Baker
4669def000
Oops, forgot the schema delta file
2015-04-29 16:50:16 +01:00
Erik Johnston
c8ed9bd278
pushers table requires an access_token
2015-04-29 15:33:27 +01:00
Mark Haines
e26a3d8d9e
bump database schema version
2015-04-29 13:32:32 +01:00
Mark Haines
a9549fdce3
Use bytea rather than BLOB
2015-04-29 13:16:09 +01:00
Mark Haines
4ad8b45155
Merge branch 'develop' into key_distribution
...
Conflicts:
synapse/config/homeserver.py
2015-04-29 13:15:14 +01:00
Erik Johnston
327ca883ec
Merge branch 'develop' of github.com:matrix-org/synapse into postgres
2015-04-28 13:39:42 +01:00
Erik Johnston
18f8247701
Use TEXT instead of VARCHAR(n), since PostgreSQL treats them the same except for a limit
2015-04-28 12:41:33 +01:00
Mark Haines
9182f87664
Merge pull request #126 from matrix-org/csauth
...
Client / Server Auth Refactor
2015-04-28 11:00:27 +01:00
Mark Haines
b96c133034
Add server_keys.sql to the current delta rather than creating a new delta
2015-04-28 10:50:00 +01:00
Erik Johnston
6f8e2d517e
Merge branch 'develop' of github.com:matrix-org/synapse into postgres
2015-04-27 14:41:40 +01:00
Mark Haines
eede182df7
Merge branch 'develop' into key_distribution
2015-04-24 10:35:49 +01:00
Mark Haines
4e2f8b8722
Copyright notices
2015-04-24 10:35:29 +01:00
David Baker
cb03fafdf1
Merge branch 'develop' into csauth
2015-04-17 13:51:10 +01:00
Erik Johnston
0ec346d942
Add unique index to room_aliases, remove duplicates on upgrade. Convert some columns back to TEXT from bytea
2015-04-17 11:10:20 +01:00
Erik Johnston
b8092fbc82
Go back to storing JSON in TEXT
2015-04-16 11:17:52 +01:00
Erik Johnston
ffad75bd62
Remove mysql/maria support
2015-04-15 17:00:50 +01:00
Mark Haines
a429515bdd
Add methods for storing and retrieving the raw key json
2015-04-15 16:58:35 +01:00
Erik Johnston
cf04cedf21
Change full_schemas/11 to work with postgres
2015-04-15 16:53:47 +01:00
Erik Johnston
cce5d057d3
Add index on events (topological_ordering, stream_ordering) to help with ORDER BY clauses
2015-04-15 11:43:25 +01:00
Mark Haines
32e14d8181
Return a sha256 fingerprint rather than the entire tls certificate
2015-04-14 19:10:09 +01:00
Erik Johnston
5a95cd4442
Rename user_ips.user -> user_id
2015-04-14 13:54:09 +01:00
Erik Johnston
58d8339966
Add support for postgres instead of mysql. Change sql accourdingly. blob + varbinary -> bytea. No support for UNSIGNED or CREATE INDEX IF NOT EXISTS.
2015-04-14 13:53:20 +01:00
Erik Johnston
86fc9b617c
For backwards compat, make state_groups.id have a type of int, not varchar
2015-04-13 17:03:49 +01:00
Erik Johnston
93937b2b31
Remove duplicate rows
2015-04-10 18:47:03 +01:00
Erik Johnston
4103b1c470
DROP indexes before recreating them
2015-04-10 18:46:09 +01:00
Erik Johnston
4d5b098626
Use LONGBLOB and TEXT for arbitary length rows
2015-04-10 18:45:09 +01:00
Erik Johnston
7e863c51e6
Use unsigned bigint
2015-04-10 11:22:04 +01:00
Erik Johnston
0f12772e32
SQLite wants INTEGER and not BIGINT for primary keys
2015-04-10 11:16:09 +01:00
Erik Johnston
d5d4281647
Update full_schemas/16 to match delta files. Add delta/16 scripts
2015-04-10 10:59:46 +01:00
Erik Johnston
cda4a6f93f
Revert non-trivial changes to upgrade scripts
2015-04-10 10:19:50 +01:00
Erik Johnston
e2722f58ee
Fix schema again
2015-04-10 10:16:29 +01:00
Erik Johnston
a1665c5094
Revert non-trivial schema changes and move them to a new schema version.
2015-04-10 10:05:44 +01:00
Erik Johnston
9707acfc40
Remove spurious spaces
2015-04-09 13:45:20 +01:00
Erik Johnston
4fe95094d1
Merge branch 'develop' of github.com:matrix-org/synapse into mysql
2015-04-07 18:05:39 +01:00
Erik Johnston
8574bf62dc
Add index to presence table
2015-04-07 12:09:36 +01:00
Erik Johnston
0af5f5efaf
Don't use multiple UNIQUE constraints; it will cause deadlocks
2015-04-07 12:08:35 +01:00
Erik Johnston
779f7b0f44
Fix unicode support
2015-04-02 10:06:22 +01:00
Erik Johnston
9236136f3a
Make work in both Maria and SQLite. Fix tests
2015-04-01 14:12:33 +01:00
Kegan Dougal
c217504949
Edit SQL schema to use string IDs not ints. Use token as ID. Update tests.
2015-03-31 12:07:56 +01:00
David Baker
c1a256cc4c
Allow multiple pushers for a single app ID & pushkey, honouring the 'append' flag in the API.
2015-03-25 19:06:22 +00:00
Erik Johnston
0e8f5095c7
Fix unicode database support
2015-03-25 17:15:20 +00:00
David Baker
c7023f2155
1) Pushers are now associated with an access token
...
2) Change places where we mean unauthenticated to 401, not 403, in C/S v2: hack so it stays as 403 in v1 because web client relies on it.
2015-03-24 17:24:15 +00:00
Erik Johnston
0d0610870d
Fix up schemas some more
2015-03-20 10:55:31 +00:00
Erik Johnston
58ed393235
Remove redundant key
2015-03-19 15:12:05 +00:00
Erik Johnston
fae059cc18
Fix up schemas to work with mariadb
2015-03-19 13:42:39 +00:00
Erik Johnston
89036579ed
Update schema to work with mariadb
2015-03-18 11:18:49 +00:00
Kegan Dougal
b98cd03193
Use event IDs instead of dumping event content in the txns table.
2015-03-09 17:25:20 +00:00
Kegan Dougal
0a60bbf4fa
Finish appservice txn storage impl and tests.
2015-03-09 15:53:03 +00:00
Kegan Dougal
34ce2ca62f
Merge branch 'develop' into application-services-txn-reliability
2015-03-06 17:28:49 +00:00
Kegan Dougal
01c099d9ef
Add appservice txns sql schema
2015-03-06 17:16:47 +00:00
David Baker
cf66ddc1b4
Schema change as delta in v14
2015-03-06 14:11:49 +00:00
David Baker
ed72fc3a50
Merge branch 'develop' into pushrules2
...
Conflicts:
synapse/storage/schema/pusher.sql
2015-03-04 15:24:21 +00:00
Erik Johnston
17d319a20d
s/schema_deltas/applied_schema_deltas/
2015-03-04 15:06:22 +00:00
Erik Johnston
5eefd1f618
Add unique constraint on schema_version.lock schema. Use conflict clause in sql.
2015-03-04 13:52:18 +00:00
Erik Johnston
b4c38738f4
Change to use logger in db upgrade script
2015-03-04 13:43:35 +00:00
Erik Johnston
8c8354e85a
Actually add full_schemas dir
2015-03-04 13:34:38 +00:00
Erik Johnston
c3530c3fb3
More docs. Rename 'schema/current' to 'schema/full_schemas'
2015-03-04 13:34:11 +00:00
Erik Johnston
82b34e813d
SYN-67: Finish up implementing new database schema management
2015-03-04 12:04:19 +00:00
David Baker
94fa334b01
Add enable/disable overlay for push rules (REST API not yet hooked up)
2015-02-25 19:17:07 +00:00
Erik Johnston
1a989c436c
Bump schema version
2015-02-17 15:45:55 +00:00
Mark Haines
0d872f5aa6
Merge pull request #50 from matrix-org/application-services
...
Application Services
2015-02-13 15:06:14 +00:00
Erik Johnston
83d41f25d8
Set database schema version in delta
2015-02-12 10:05:47 +00:00
Kegan Dougal
53557fc532
Merge branch 'develop' into application-services
2015-02-09 15:20:56 +00:00
Kegan Dougal
ac3183caaa
Register a user account for the AS when the AS registers. Add 'sender' column to AS table.
2015-02-09 12:03:37 +00:00
Erik Johnston
0cd66885e3
Move delta/v13.sql to delta/v12.sql
2015-02-06 14:38:04 +00:00
Kegan Dougal
a3c6010718
Add delta sql file.
2015-02-05 16:48:57 +00:00
Kegan Dougal
951690e54d
Merge branch 'develop' into application-services
2015-02-05 14:28:03 +00:00
Kegan Dougal
27091f146a
Add hs_token column and generate a different token f.e application service.
2015-02-05 10:08:12 +00:00
David Baker
dc7bb70f22
s/instance_handle/profile_tag/
2015-02-03 16:51:07 +00:00
Kegan Dougal
c059c9fea5
Merge branch 'develop' into application-services
...
Conflicts:
synapse/handlers/__init__.py
synapse/storage/__init__.py
2015-02-02 15:57:59 +00:00
Erik Johnston
fdd2ac495a
Merge branch 'develop' of github.com:matrix-org/synapse into rejections_storage
...
Conflicts:
synapse/storage/__init__.py
2015-01-30 14:57:33 +00:00
Erik Johnston
472cf532b7
Put CREATE rejections into seperate .sql
2015-01-30 14:48:03 +00:00
Erik Johnston
e0b7c521cb
Merge branch 'develop' of github.com:matrix-org/synapse into rejections_storage
...
Conflicts:
synapse/storage/__init__.py
synapse/storage/schema/delta/v12.sql
2015-01-30 14:08:28 +00:00
Mark Haines
9d8f798a3f
Merge changes from develop
2015-01-29 14:55:27 +00:00
Mark Haines
e4f50fa0aa
Move bump schema delta
2015-01-29 14:53:18 +00:00
Kegan Dougal
3773759c0f
Also edit the filter column on the delta SQL
2015-01-29 09:15:33 +00:00
Kegan Dougal
11634017f4
s/definition/filter_json/ since definition is now used to mean a component of the filter, rather than the complete json
2015-01-28 17:42:19 +00:00
David Baker
60b143a52e
Move pushers delta to v12 and bump schema version
2015-01-28 15:48:28 +00:00
Kegan Dougal
42876969b9
Add basic application_services SQL, and hook up parts of the appservice store to read from it.
2015-01-28 11:59:38 +00:00
Paul "LeoNerd" Evans
8398f19bce
Created schema delta
2015-01-27 19:00:09 +00:00
Paul "LeoNerd" Evans
06cc147012
Initial stab at real SQL storage implementation of user filter definitions
2015-01-27 18:46:03 +00:00
David Baker
ede491b4e0
Oops: second part of commit dc938606
2015-01-22 17:38:53 +00:00
Erik Johnston
b1b85753d7
Add support for storing rejected events in EventContext and data stores
2015-01-22 15:50:17 +00:00
David Baker
afb714f7be
add instance_handles to pushers so we have a way to refer to them even if the push token changes.
2015-01-20 11:49:48 +00:00
David Baker
2d2953cf5f
Require device language when adding a pusher.
...
Because this seems like it might be useful to do sooner rather
than later.
2015-01-16 11:24:10 +00:00
David Baker
2cb30767fa
Honour the 'rejected' return from push gateways
...
Add a timestamp to push tokens so we know the last time they we
got them from the device. Send it to the push gateways so it can
determine whether its failure is more recent than the token.
Stop and remove pushers that have been rejected.
2015-01-13 19:48:37 +00:00
David Baker
c06a9063e1
Merge branch 'develop' into pushers
2015-01-13 13:15:51 +00:00
Erik Johnston
a01416cf21
Add delta and bump DB version
2015-01-06 15:42:18 +00:00
Erik Johnston
f6da237c35
Add index on transaction_id to sent_transcations
2015-01-06 15:40:38 +00:00
Mark Haines
adb04b1e57
Update copyright notices
2015-01-06 13:21:39 +00:00
David Baker
b56730bb6e
Merge branch 'develop' into pushers
...
Conflicts:
synapse/api/errors.py
synapse/server.py
synapse/storage/__init__.py
2014-12-18 15:15:22 +00:00
David Baker
afa953a293
schema version is now 10
2014-12-18 15:11:06 +00:00
David Baker
fc7c5e9cd7
Rename the pusher SQL delta to v9 which the next free one
2014-12-18 14:51:29 +00:00
David Baker
9728c305a3
after a few rethinks, a working implementation of pushers.
2014-12-18 14:49:22 +00:00
Erik Johnston
42b725ce52
Fix upgrade script to run all the missing deltas.
2014-12-16 15:13:34 +00:00
Erik Johnston
882dc8dcab
Persist internal_metadata
2014-12-16 13:17:09 +00:00
Erik Johnston
cde840a82c
Merge branch 'develop' of github.com:matrix-org/synapse into events_refactor
...
Conflicts:
setup.py
2014-12-11 17:48:48 +00:00
Mark Haines
4f37c0ea9d
Merge branch 'develop' into media_repository
2014-12-10 16:55:06 +00:00
Mark Haines
61fc37e467
Merge branch 'develop' into media_repository
2014-12-10 16:14:17 +00:00
Erik Johnston
1d2a0040cf
Fix bug where we clobbered old state group values
2014-12-10 15:55:03 +00:00
Erik Johnston
02db7eb209
Fix bug when uploading state with empty state_key
2014-12-10 14:02:48 +00:00
Erik Johnston
aae8a37e63
Merge branch 'develop' of github.com:matrix-org/synapse into events_refactor
2014-12-10 13:18:40 +00:00
Matthew Hodgson
aed62a3583
track replication destination health, and perform exponential back-off when sending transactions. does *not* yet retry transactions, but drops them on the floor if waiting for a server to recover.
2014-12-07 02:26:07 +00:00
Mark Haines
a953be097f
Add a method field to thumbnail storage
2014-12-05 16:31:56 +00:00
David Baker
88af58d41d
Update to app_id / app_instance_id (partially) and mangle to be PEP8 compliant.
2014-12-03 13:37:02 +00:00
Mark Haines
279c48c8b4
Write the upload portion of version 1 of the media repository
2014-12-02 17:13:14 +00:00
David Baker
7642d95d5e
Merge branch 'develop' into pushers
2014-12-02 13:50:05 +00:00