Matrix-Mirrors/synapse-product
1
0
Fork 0
mirror of https://git.anonymousland.org/anonymousland/synapse-product.git synced 2024-10-01 08:25:44 -04:00
Code Issues Packages Projects Releases Wiki Activity
10,346 Commits 4 Branches 609 Tags 60 MiB
88964b987e
Commit Graph

443 Commits

Author SHA1 Message Date
Erik Johnston
4f78108d8c Readd entries to public_room_list_stream that were deleted 2016-09-21 15:24:22 +01:00
Erik Johnston
baffe96d95 Add a room visibility stream 2016-09-14 17:29:19 +01:00
Erik Johnston
ed992ae6ba Add a DB index to figure out past state at a stream ordering in a room 2016-09-14 16:20:27 +01:00
Erik Johnston
43253c10b8 Remove redundant event_auth index 2016-09-13 11:47:48 +01:00
Erik Johnston
0b32bb20bb Index contains_url for file search queries 2016-09-12 16:57:05 +01:00
Erik Johnston
7fe42cf949 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/make_notif_highlight_query_fast 2016-09-12 12:37:09 +01:00
Erik Johnston
15ca0c6a4d Make reindex happen in bg 2016-09-12 12:36:36 +01:00
Mark Haines
31f85f9db9 Add comments to existing schema deltas that used "CREATE INDEX" directly 2016-09-12 11:00:26 +01:00
Erik Johnston
54417999b6 Revert "Add index to event_push_actions" 2016-09-12 10:39:55 +01:00
David Baker
b91e2833b3 Merge remote-tracking branch 'origin/develop' into dbkr/make_notif_highlight_query_fast 2016-09-09 19:11:34 +01:00
David Baker
f2acc3dcf9 Add index to event_push_actions 
and remove room_id caluse so it uses it

Mostly from @negativemjark
 2016-09-09 18:54:54 +01:00
Mark Haines
647c724573 Use the previous MAX value if any to set the stream_id 2016-09-09 11:52:44 +01:00
Mark Haines
6a6cbfcf1e Track the max_stream_device_id in a separate table, since we delete from the inbox table 2016-09-09 11:48:23 +01:00
Mark Haines
303b6f29f0 Merge pull request #1087 from matrix-org/markjh/reapply_delta 
Reapply 34/device_outbox in 35/device_outbox_again.py since the schem…
 2016-09-09 11:24:27 +01:00
Mark Haines
0877157353 Just move the schema and add some DROPs 2016-09-09 11:04:47 +01:00
Mark Haines
43b77c5d97 Only catch databas errors 2016-09-08 17:44:21 +01:00
Mark Haines
7d5b142547 Add a stub run_upgrade 2016-09-08 17:39:11 +01:00
Mark Haines
fa722a699c Reapply 34/device_outbox in 35/device_outbox_again.py since the schema was bumped before it landed on develop 2016-09-08 17:35:16 +01:00
Erik Johnston
ebb46497ba Add delta file 2016-09-08 16:38:54 +01:00
Mark Haines
fa9d36e050 Merge branch 'develop' into markjh/direct_to_device_federation 2016-09-08 13:43:43 +01:00
Mark Haines
d4a35ada28 Send device messages over federation 2016-09-06 18:16:20 +01:00
Mark Haines
2ad72da931 Add tables for federated device messages 
Adds tables for storing the messages that need to be sent to a
remote device and for deduplicating messages received.
 2016-09-06 15:10:40 +01:00
Erik Johnston
485d999c8a Correctly delete old state groups in purge history API 2016-09-05 14:49:08 +01:00
Erik Johnston
a99e933550 Add upgrade script that will slowly prune state_groups_state entries 2016-09-05 10:05:36 +01:00
Erik Johnston
657847e4c6 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/state_storage 2016-09-02 11:04:48 +01:00
Erik Johnston
9e25443db8 Move to storing state_groups_state as deltas 2016-09-01 14:31:26 +01:00
Erik Johnston
ef0cc648cf Clean up old sent transactions 2016-08-31 11:12:02 +01:00
Mark Haines
e993925279 Add store-and-forward direct-to-device messaging 2016-08-25 17:35:37 +01:00
Erik Johnston
aad8a1a825 Delete old received_transactions 2016-08-22 16:29:46 +01:00
Erik Johnston
9da84a9a1e Make AppserviceHandler stream events from database 
This is for two reasons:

1. Suppresses duplicates correctly, as the notifier doesn't do any
   duplicate suppression.
2. Makes it easier to connect the AppserviceHandler to the replication
   stream.
 2016-08-18 11:54:41 +01:00
Erik Johnston
732cf72b86 Fix push_display_name_rename schema update 2016-08-17 18:12:21 +01:00
David Baker
ad42322257 Add migration script 
To port existing rule actions & enable entries to the new name
 2016-08-16 16:56:30 +01:00
Erik Johnston
d9664344ec Rename table. Add docs. 2016-08-15 11:45:57 +01:00
Erik Johnston
4d70d1f80e Add some invalidations to a cache_stream 2016-08-15 11:15:17 +01:00
Richard van der Hoff
4fec5e57be Default device_display_name to null 
It turns out that it's more useful to return a null device display name (and
let clients decide how to handle it: eg, falling back to device_id) than using
a constant string like "unknown device".
 2016-08-03 11:53:00 +01:00
Richard van der Hoff
26cb0efa88 SQL syntax fix 2016-07-27 12:30:22 +01:00
Richard van der Hoff
2e3d90d67c Make the device id on e2e key upload optional 
We should now be able to get our device_id from the access_token, so the
device_id on the upload request is optional. Where it is supplied, we should
check that it matches.

For active access_tokens without an associated device_id, we ought to register
the device in the devices table.

Also update the table on upgrade so that all of the existing e2e keys are
associated with real devices.
 2016-07-26 23:38:12 +01:00
Richard van der Hoff
436bffd15f Implement deleting devices 2016-07-26 07:35:48 +01:00
Richard van der Hoff
ec5717caf5 Create index on user_ips in the background 
user_ips is kinda big, so really we want to add the index in the background
once we're running. Replace the schema delta with one which will do that.

I've done this in a way that's reasonably easy to reuse as there a few other
indexes I need, and I don't suppose they will be the last.
 2016-07-22 13:16:39 +01:00
Richard van der Hoff
bc8f265f0a GET /devices endpoint 
implement a GET /devices endpoint which lists all of the user's devices.

It also returns the last IP where we saw that device, so there is some dancing
to fish that out of the user_ips table.
 2016-07-20 16:42:32 +01:00
Erik Johnston
1e2a7f18a1 Merge pull request #922 from matrix-org/erikj/file_api2 
Feature: Add filter to /messages. Add 'contains_url' to filter.
 2016-07-20 10:40:48 +01:00
Richard van der Hoff
022b9176fe schema fix 
device_id should be text, not bigint.
 2016-07-19 11:44:05 +01:00
Richard van der Hoff
f863a52cea Add device_id support to /login 
Add a 'devices' table to the storage, as well as a 'device_id' column to
refresh_tokens.

Allow the client to pass a device_id, and initial_device_display_name, to
/login. If login is successful, then register the device in the devices table
if it wasn't known already. If no device_id was supplied, make one up.

Associate the device_id with the access token and refresh token, so that we can
get at it again later. Ensure that the device_id is copied from the refresh
token to the access_token when the token is refreshed.
 2016-07-18 16:39:44 +01:00
Erik Johnston
b64aa6d687 Add sender and contains_url field to events table 2016-07-14 15:35:43 +01:00
Erik Johnston
314b146b2e Track approximate last access time for remote media 2016-06-29 11:41:20 +01:00
Mark Haines
c8c5bf950a Fix synapse/storage/schema/delta/30/as_users.py 2016-05-31 17:10:40 +01:00
Erik Johnston
c9aff0736c Remove topics table 2016-05-13 14:40:38 +01:00
Mark Haines
9295fa30a8 Annotate the removed indicies with why they were removed. 2016-05-13 14:16:57 +01:00
Erik Johnston
5e50058473 Remove unused indices 
This includes removing both unused indices and indices that are subsets
of other indices.
 2016-05-13 13:28:07 +01:00
Matrix
3b1930e8ec unbreak schema 2016-05-10 16:42:37 +01:00
David Baker
997db04648 Merge remote-tracking branch 'origin/develop' into dbkr/email_notifs 2016-05-10 14:40:19 +02:00
Mark Haines
9c272da05f Add an openidish mechanism for proving to third parties that you own a given user_id 2016-05-05 13:42:44 +01:00
Erik Johnston
fcd1eb642d Add primary key 2016-05-04 16:51:51 +01:00
Erik Johnston
8e6a163f27 Add timestamp and auto incrementing ID 2016-05-04 15:19:12 +01:00
Erik Johnston
984d4a2c0f Add /report endpoint 2016-05-04 11:28:10 +01:00
David Baker
acded821c4 Merge remote-tracking branch 'origin/develop' into dbkr/email_notifs 2016-04-29 10:05:20 +01:00
Erik Johnston
ae571810f2 Order NULLs first 2016-04-21 18:14:18 +01:00
Erik Johnston
8fae3d7b1e Use special UPDATE syntax 2016-04-21 18:01:49 +01:00
Erik Johnston
51bb339ab2 Create index concurrently 2016-04-21 17:16:11 +01:00
Erik Johnston
b743c1237e Add missing run_upgrade 2016-04-21 17:12:04 +01:00
Erik Johnston
c877f0f034 Optimise event_search in postgres 2016-04-21 16:56:14 +01:00
David Baker
07d765209d First bits of emailpusher 
Mostly logic of when to send an email
 2016-04-19 14:24:36 +01:00
Erik Johnston
96bcfb29c7 Add index 2016-04-14 11:26:33 +01:00
David Baker
2547dffccc Merge pull request #705 from matrix-org/dbkr/pushers_use_event_actions 
Change pushers to use the event_actions table
 2016-04-11 12:58:55 +01:00
Matthew Hodgson
6ff7a79308 move local_media_repository_url_cache.sql to schema v31 2016-04-08 19:09:02 +01:00
Matthew Hodgson
d6e7333ae4 Merge branch 'develop' into matthew/preview_urls 2016-04-07 17:26:44 +01:00
David Baker
05d044aac3 pep8 2016-04-07 16:45:38 +01:00
David Baker
2d5c693fd3 Fix port script for changes merged from develop 2016-04-07 16:43:54 +01:00
David Baker
9c99ab4572 Merge remote-tracking branch 'origin/develop' into dbkr/pushers_use_event_actions 2016-04-07 16:35:22 +01:00
David Baker
7e2c89a37f Make pushers use the event_push_actions table instead of listening on an event stream & running the rules again. Sytest passes, but remaining to do: 
* Make badges work again
 * Remove old, unused code
 2016-04-06 15:42:15 +01:00
Erik Johnston
8aab9d87fa Don't require config to create database 2016-04-06 14:15:45 +01:00
Erik Johnston
0c53d750e7 Docs and indents 2016-04-04 18:02:48 +01:00
Erik Johnston
92ab45a330 Add upgrade path, rename table 2016-04-04 17:07:43 +01:00
Erik Johnston
3d76b7cb2b Store invites in a separate table. 2016-04-04 16:30:15 +01:00
Matthew Hodgson
9f7dc2bef7 Merge branch 'develop' into matthew/preview_urls 2016-04-04 00:38:21 +01:00
Matthew Hodgson
7426c86eb8 add a persistent cache of URL lookups, and fix up the in-memory one to work 2016-04-03 00:31:57 +01:00
Mark Haines
1fbb094c6f Add replication streams for ex outliers and current state resets 2016-03-30 17:19:56 +01:00
Erik Johnston
b275765545 Comment about weird SQL 2016-03-23 14:15:32 +00:00
Erik Johnston
0c1a27b787 SQLite and postgres doesn't share a true literal 2016-03-23 14:10:49 +00:00
Erik Johnston
84afeb41f3 Ensure all old public rooms have aliases 2016-03-23 13:59:34 +00:00
Mark Haines
ba660ecde2 Add a comment to offer a hint to an explanation for why we have a unique constraint on (app_id, pushkey, user_id) 2016-03-16 10:35:00 +00:00
Mark Haines
b6e8420aee Add replication stream for pushers 2016-03-15 17:33:10 +00:00
Erik Johnston
9669a99d1a Update users table in a batched manner 2016-03-10 15:12:19 +00:00
Daniel Wagner-Hall
3b97797c8d Merge branch 'develop' into daniel/ick 2016-03-08 17:35:09 +00:00
Daniel Wagner-Hall
edca2d9891 Idempotent-ise schema update script 
If any ASes don't have an ID, the schema will fail, and then it will
error when trying to add the column again.
 2016-03-08 17:32:29 +00:00
Mark Haines
ebcbb23226 s/stream_ordering/event_stream_ordering/ in push 2016-03-04 16:15:23 +00:00
Mark Haines
ec7460b4f2 Merge branch 'develop' into markjh/pushrule_stream 2016-03-04 14:44:34 +00:00
Erik Johnston
dd463e246d Merge pull request #614 from matrix-org/erikj/alias_delete 
Allow alias creators to delete aliases
 2016-03-04 09:02:33 +00:00
Daniel Wagner-Hall
95481e7ba7 Merge pull request #571 from matrix-org/daniel/asids 
Mark AS users with their AS's ID
 2016-03-03 13:56:28 +00:00
Mark Haines
a1cf9e3bf3 Add a stream for push rule updates 2016-03-01 18:16:37 +00:00
Erik Johnston
f9af8962f8 Allow alias creators to delete aliases 2016-03-01 14:46:31 +00:00
Daniel Wagner-Hall
33300673b7 Generate guest access token on 3pid invites 
This means that following the same link across multiple sessions or
devices can re-use the same guest account.

Note that this is somewhat of an abuse vector; we can't throw up
captchas on this flow, so this is a way of registering ephemeral
accounts for spam, whose sign-up we don't rate limit.
 2016-02-24 14:41:25 +00:00
Erik Johnston
112283e230 Prefix TS fields with _ts 2016-02-18 10:11:43 +00:00
Erik Johnston
e5999bfb1a Initial cut 2016-02-17 15:40:50 +00:00
Daniel Wagner-Hall
763360594d Mark AS users with their AS's ID 2016-02-11 17:26:42 +00:00
Erik Johnston
8450114098 Merge pull request #554 from matrix-org/erikj/event_push 
Change event_push_actions_rm_tokens schema
 2016-02-03 15:02:47 +00:00
Erik Johnston
771528ab13 Change event_push_actions_rm_tokens schema 2016-02-03 10:50:49 +00:00
Erik Johnston
65e92eca49 Change the way we do public room list fetching 2016-02-02 15:21:10 +00:00
Erik Johnston
fdca8ec418 Add events index 2016-01-28 16:41:59 +00:00
Erik Johnston
86896408b0 Add index to event_push_actions 2016-01-25 15:30:32 +00:00
David Baker
d4315bbf6b Add index by user id on receipts_linearized 2016-01-20 15:33:27 +00:00
David Baker
fe56138142 Remove rogue 'admin' 2016-01-11 17:09:03 +00:00
David Baker
c232780081 Merge pull request #456 from matrix-org/store_event_actions 
Send unread notification counts
 2016-01-08 14:47:15 +00:00
Matthew Hodgson
6c28ac260c copyrights 2016-01-07 04:26:29 +00:00
David Baker
ae1262a241 Add schema change file for is_guest flag 2016-01-06 11:58:20 +00:00
David Baker
c79f221192 Add is_guest flag to users db to track whether a user is a guest user or not. Use this so we can run _filter_events_for_client when calculating event_push_actions. 2016-01-06 11:38:09 +00:00
David Baker
eb03625626 Merge remote-tracking branch 'origin/develop' into store_event_actions 2016-01-05 18:39:50 +00:00
Daniel Wagner-Hall
cfd07aafff Allow guests to upgrade their accounts 2016-01-05 18:01:18 +00:00
David Baker
c914d67cda Rename event-actions to event_push_actions as per PR request 2016-01-04 14:05:37 +00:00
David Baker
42ad49f5b7 still very WIP, but now sends unread_notifications_count in the room object on sync (only actually corrrect in a full sync: hardcoded to 0 in incremental syncs). 2015-12-16 18:42:09 +00:00
David Baker
aa667ee396 Save event actions to the db 2015-12-10 17:51:15 +00:00
Mark Haines
d57c5cda71 Bump schema version. 
As we released version 26 in v0.11.1
 2015-12-04 15:28:39 +00:00
Erik Johnston
b8680b82c3 Merge pull request #414 from matrix-org/erikj/if_not_exists 
Older versions of SQLite don't like IF NOT EXISTS in virtual tables
 2015-12-03 13:21:09 +00:00
David Baker
e515b48929 Just replace the table definition with the one from full_schema 16 2015-12-02 17:23:52 +00:00
David Baker
8810eb8c39 Fix schema delta 15 on postgres in the very unlikley event that anyone upgrades to 15... 2015-12-02 17:19:11 +00:00
Erik Johnston
b2def42bfd Older versions of SQLite don't like IF NOT EXISTS in virtual tables 2015-12-02 13:29:14 +00:00
Mark Haines
95f30ecd1f Add API for setting account_data globaly or on a per room basis 2015-12-01 18:41:32 +00:00
Erik Johnston
4dcaa42b6d Allow paginating search ordered by recents 2015-11-30 17:45:31 +00:00
Daniel Wagner-Hall
7dfa455508 Remove size specifier for database column 
Postgres doesn't support them like this.

We don't have a bool type in common between postgres and sqlite.
 2015-11-23 18:35:25 +00:00
Paul "LeoNerd" Evans
1cfda3d2d8 Merge branch 'develop' into daniel/forgetrooms 2015-11-19 16:53:13 +00:00
Mark Haines
d153f482dd Rename the database table 2015-11-18 15:33:02 +00:00
Daniel Wagner-Hall
ba26eb3d5d Allow users to forget rooms 2015-11-17 17:17:30 -05:00
Erik Johnston
e21cef9bb5 Merge pull request #359 from matrix-org/markjh/incremental_indexing 
Incremental background updates for db indexes
 2015-11-11 17:19:51 +00:00
Mark Haines
e1627388d1 Fix param style to work on both sqlite and postgres 2015-11-11 17:14:56 +00:00
Mark Haines
940a161192 Fix the background update 2015-11-11 13:59:40 +00:00
Daniel Wagner-Hall
cf437900e0 Return world_readable and guest_can_join in /publicRooms 2015-11-10 17:10:27 +00:00
Mark Haines
90b503216c Use a background task to update databases to use the full text search 2015-11-10 16:20:13 +00:00
Mark Haines
c6a01f2ed0 Add storage module for tracking background updates. 
The progress for each background update is stored as a JSON blob in the
database. Each background update is broken up into separate batches.
The batch size is automatically tuned to try avoid blocking single
threaded databases for too long.
 2015-11-09 17:26:27 +00:00
Daniel Wagner-Hall
f2c4ee41b9 Remove accidentally added ID column 2015-11-06 14:27:49 +00:00
Daniel Wagner-Hall
ca2f90742d Open up /events to anonymous users for room events only 
Squash-merge of PR #345 from daniel/anonymousevents
 2015-11-05 14:32:26 +00:00
Mark Haines
6797fcd9ab Merge pull request #335 from matrix-org/markjh/room_tags 
Add APIs for adding and removing tags from rooms
 2015-11-03 16:45:53 +00:00
Mark Haines
ddd8566f41 Store room tag content and return the content in the m.tag event 2015-11-02 15:11:31 +00:00
Daniel Wagner-Hall
621e84d9a0 Add missing column 2015-10-30 16:25:53 +00:00
Mark Haines
892e70ec84 Add APIs for adding and removing tags from rooms 2015-10-28 16:06:57 +00:00
Erik Johnston
4cf633d5e9 Pull out sender when computing search results 2015-10-23 15:41:36 +01:00
Erik Johnston
0c36098c1f Implement rank function for SQLite FTS 2015-10-23 13:23:48 +01:00
Erik Johnston
f142898f52 PEP8 2015-10-22 11:18:01 +01:00
Erik Johnston
4d25bc6c92 Move FTS to delta 25 2015-10-22 11:12:28 +01:00
Erik Johnston
22a8c91448 Split up run_upgrade 2015-10-16 11:19:44 +01:00
Erik Johnston
88971fd034 Merge branch 'erikj/store_engine' into erikj/search 2015-10-13 14:03:30 +01:00
Erik Johnston
f9340ea0d5 Merge branch 'erikj/store_engine' into erikj/search 2015-10-13 13:29:02 +01:00
Erik Johnston
1a40afa756 Add sqlite schema 2015-10-13 10:36:25 +01:00
Erik Johnston
61561b9df7 Keep FTS indexes up to date. Only search through rooms currently joined 2015-10-12 10:49:53 +01:00
Erik Johnston
c85c912562 Add basic full text search impl. 2015-10-09 15:48:31 +01:00
Daniel Wagner-Hall
7213588083 Implement configurable stats reporting 
SYN-287

This requires that HS owners either opt in or out of stats reporting.

When --generate-config is passed, --report-stats must be specified
If an already-generated config is used, and doesn't have the
report_stats key, it is requested to be set.
 2015-09-22 12:57:40 +01:00
Erik Johnston
dffc9c4ae0 Drop unused index 2015-09-14 14:41:37 +01:00
Mark Haines
037481a033 Remove autoincrement since we incrementing the ID in the storage layer 2015-08-24 17:48:57 +01:00
Daniel Wagner-Hall
e8cf77fa49 Merge branch 'develop' into refresh 
Conflicts:
	synapse/rest/client/v1/login.py
 2015-08-20 16:25:40 +01:00
Daniel Wagner-Hall
cecbd636e9 /tokenrefresh POST endpoint 
This allows refresh tokens to be exchanged for (access_token,
refresh_token).

It also starts issuing them on login, though no clients currently
interpret them.
 2015-08-20 16:21:35 +01:00
Mark Haines
a0b8e5f2fe Merge pull request #211 from matrix-org/email_in_use 
Changes for unique emails
 2015-08-20 10:04:04 +01:00