Matrix-Mirrors/forked-synapse
1
0
Fork 0
mirror of https://mau.dev/maunium/synapse.git synced 2024-10-01 01:36:05 -04:00
Code Issues Packages Projects Releases Wiki Activity
6,957 Commits 5 Branches 723 Tags 124 MiB
370135ad0b
Commit Graph

1565 Commits

Author SHA1 Message Date
Erik Johnston
6cf0ba1466 Bump get_unread_event_push_actions_by_room_for_user cache 2016-03-22 14:18:21 +00:00
Erik Johnston
76d18a5776 Bump get_aliases_for_room cache 2016-03-22 14:08:13 +00:00
Erik Johnston
fa2f96c2e3 Merge pull request #660 from matrix-org/erikj/state_cache 
Don't cache events in _state_group_cache
 2016-03-22 13:12:06 +00:00
Erik Johnston
f93304e77f Merge pull request #659 from matrix-org/erikj/state_cache_factor 
Make stateGroupCache honour CACHE_SIZE_FACTOR
 2016-03-22 13:12:01 +00:00
Erik Johnston
2c86187a1b Don't cache events in _state_group_cache 
Instead, simply cache the event ids, relying on the event cache to cache
the actual events.

The problem was that while the state groups cache was limited in the
number of groups it could hold, each individual group could consist of
thousands of events.
 2016-03-22 12:00:09 +00:00
Erik Johnston
d6ac752538 Merge pull request #657 from matrix-org/erikj/roomlist 
Add published room list edit API
 2016-03-22 11:57:39 +00:00
Erik Johnston
b591277620 Make stateGroupCache honour CACHE_SIZE_FACTOR 2016-03-22 10:32:50 +00:00
Mark Haines
5244c0b48e Remove unused backfilled parameter from persist_event 2016-03-21 18:06:08 +00:00
Erik Johnston
3e7fac0d56 Add published room list edit API 2016-03-21 15:06:07 +00:00
Erik Johnston
67ed8065db Dedupe requested event list in _get_events 2016-03-18 14:31:31 +00:00
Mark Haines
673c96ce97 Remove dead code left over from presence changes 2016-03-17 11:03:47 +00:00
Mark Haines
ba660ecde2 Add a comment to offer a hint to an explanation for why we have a unique constraint on (app_id, pushkey, user_id) 2016-03-16 10:35:00 +00:00
Mark Haines
ee32d622ce Fix a couple of errors when deleting pushers 2016-03-15 17:47:36 +00:00
Mark Haines
b6e8420aee Add replication stream for pushers 2016-03-15 17:33:10 +00:00
Erik Johnston
15122da0e2 Thats not how transactions work. 2016-03-11 16:45:27 +00:00
Erik Johnston
b13035cc91 Implement logout 2016-03-11 16:27:50 +00:00
Erik Johnston
c081228439 Fix SQL statement 2016-03-11 15:09:17 +00:00
Erik Johnston
b5afe6bc38 Merge pull request #641 from matrix-org/dbkr/fix_change_password 
Fix logout on password change
 2016-03-11 14:48:20 +00:00
David Baker
2dee03aee5 more pep8 2016-03-11 14:38:23 +00:00
David Baker
af59826a2f Make select more sensible when dseleting access tokens, rename pusher deletion to match access token deletion and make exception arg optional. 2016-03-11 14:34:09 +00:00
David Baker
f523177850 Delete old, unused methods and rename new one to just be user_delete_access_tokens with an except_token_ids argument doing what it says on the tin. 2016-03-11 14:29:01 +00:00
David Baker
57c444b3ad Dear PyCharm, please indent sensibly for me. Thx. 2016-03-11 14:25:05 +00:00
Erik Johnston
58443a022d Merge pull request #635 from matrix-org/erikj/sync_order 
Use topological orders for initial sync timeline
 2016-03-11 13:17:49 +00:00
David Baker
aa11db5f11 Fix cache invalidation so deleting access tokens (which we did when changing password) actually takes effect without HS restart. Reinstate the code to avoid logging out the session that changed the password, removed in 415c2f0549 2016-03-11 13:14:18 +00:00
Daniel Wagner-Hall
465605d616 Store appservice ID on register 2016-03-10 15:58:22 +00:00
Erik Johnston
9669a99d1a Update users table in a batched manner 2016-03-10 15:12:19 +00:00
Erik Johnston
8a88684736 Add comment 2016-03-09 16:51:22 +00:00
Erik Johnston
af2fe6110c Return the correct token form 2016-03-09 16:11:53 +00:00
Erik Johnston
3ecaabc7fd Use topological orders for initial sync timeline 2016-03-09 15:45:34 +00:00
Erik Johnston
158a322e82 Ensure integer is an integer 2016-03-09 10:20:48 +00:00
Mark Haines
5f5817ab05 Merge pull request #632 from matrix-org/markjh/py3v2 
Fix relative imports so they work in both py3 and py27
 2016-03-08 18:11:10 +00:00
Daniel Wagner-Hall
3b97797c8d Merge branch 'develop' into daniel/ick 2016-03-08 17:35:09 +00:00
Daniel Wagner-Hall
edca2d9891 Idempotent-ise schema update script 
If any ASes don't have an ID, the schema will fail, and then it will
error when trying to add the column again.
 2016-03-08 17:32:29 +00:00
Mark Haines
7076082ae6 Fix relative imports so they work in both py3 and py27 2016-03-08 11:45:50 +00:00
Mark Haines
9848b54cac Prefill from the correct stream 2016-03-04 16:20:22 +00:00
Mark Haines
deda48068c prefill the push rules stream change cache 2016-03-04 16:19:42 +00:00
Mark Haines
ebcbb23226 s/stream_ordering/event_stream_ordering/ in push 2016-03-04 16:15:23 +00:00
Mark Haines
7e9fc9b6af /FNARG/d 2016-03-04 15:54:09 +00:00
Mark Haines
ec7460b4f2 Merge branch 'develop' into markjh/pushrule_stream 2016-03-04 14:44:34 +00:00
Mark Haines
1b4f4a936f Hook up the push rules stream to account_data in /sync 2016-03-04 14:44:01 +00:00
Erik Johnston
dd463e246d Merge pull request #614 from matrix-org/erikj/alias_delete 
Allow alias creators to delete aliases
 2016-03-04 09:02:33 +00:00
Daniel Wagner-Hall
95481e7ba7 Merge pull request #571 from matrix-org/daniel/asids 
Mark AS users with their AS's ID
 2016-03-03 13:56:28 +00:00
Mark Haines
2223204eba Hook push rules up to the replication API 2016-03-02 17:26:20 +00:00
Mark Haines
a1cf9e3bf3 Add a stream for push rule updates 2016-03-01 18:16:37 +00:00
Mark Haines
a612ce6659 Merge pull request #489 from matrix-org/markjh/replication 
Add a /replication API for extracting the updates that happened on synapse.
 2016-03-01 15:08:24 +00:00
Mark Haines
60a0f81c7a Add a /replication API for extracting the updates that happened on 
synapse

This is necessary for replicating the data in synapse to be visible to a
separate service because presence and typing notifications aren't stored
in a database so won't be visible to another process.

This API can be used to either get the raw data by requesting the tables
themselves or to just receive notifications for updates by following the
streams meta-stream.

Returns updates for each table requested a JSON array of arrays with a
row for each row in the table.

Each table is prefixed by a header row with the: name of the table,
current stream_id position for the table, number of rows, number of
columns and the names of the columns.
This is followed by the rows that have been added to the server since
the requester last asked.

The API has a timeout and is hooked up to the notifier so that a slave
can long poll for updates.
 2016-03-01 14:49:41 +00:00
Erik Johnston
f9af8962f8 Allow alias creators to delete aliases 2016-03-01 14:46:31 +00:00
Mark Haines
54172924c8 Load the current id in the IdGenerator constructor 
Rather than loading them lazily. This allows us to remove all
the yield statements and spurious arguments for the get_next
methods.

It also allows us to replace all instances of get_next_txn with
get_next since get_next no longer needs to access the db.
 2016-03-01 14:32:56 +00:00
Mark Haines
de27f7fc79 Add support for changing the actions for default rules 
See matrix-org/matrix-doc#283

Works by adding dummy rules to the push rules table with a negative priority class and then using those rules to clobber the default rule actions when adding the default rules in ``list_with_base_rules``
 2016-02-26 14:28:19 +00:00
Daniel Wagner-Hall
33300673b7 Generate guest access token on 3pid invites 
This means that following the same link across multiple sessions or
devices can re-use the same guest account.

Note that this is somewhat of an abuse vector; we can't throw up
captchas on this flow, so this is a way of registering ephemeral
accounts for spam, whose sign-up we don't rate limit.
 2016-02-24 14:41:25 +00:00
Erik Johnston
6451fcd085 Create a new stream_id per presence update 2016-02-23 15:51:39 +00:00
Erik Johnston
e5ad2e5267 Merge pull request #582 from matrix-org/erikj/presence 
Rewrite presence for performance.
 2016-02-19 09:37:50 +00:00
Erik Johnston
e6c5e3f28a Close cursor 2016-02-18 16:39:28 +00:00
Erik Johnston
42109a62a4 Remove unused param from get_max_token 2016-02-18 16:37:28 +00:00
Mark Haines
b9977ea667 Remove dead code for setting device specific rules. 
It wasn't possible to hit the code from the API because of a typo
in parsing the request path. Since no-one was using the feature
we might as well remove the dead code.
 2016-02-18 16:05:13 +00:00
Erik Johnston
112283e230 Prefix TS fields with _ts 2016-02-18 10:11:43 +00:00
Daniel Wagner-Hall
7e90fb6a57 Merge branch 'develop' into daniel/roomcleanupincremental 
Conflicts:
	synapse/rest/client/v1/room.py
 2016-02-17 15:53:59 +00:00
Erik Johnston
e5999bfb1a Initial cut 2016-02-17 15:40:50 +00:00
Daniel Wagner-Hall
a4e278bfe7 Respond to federated invite with non-empty context 
Currently, we magically perform an extra database hit to find the
inviter, and use this to guess where we should send the event. Instead,
fill in a valid context, so that other callers relying on the context
actually have one.
 2016-02-17 15:25:12 +00:00
Mark Haines
a9c9868957 Make adding push rules idempotent 
Also remove the **kwargs from the add_push_rule method.

Fixes https://matrix.org/jira/browse/SYN-391
 2016-02-16 15:53:38 +00:00
Daniel Wagner-Hall
763360594d Mark AS users with their AS's ID 2016-02-11 17:26:42 +00:00
Erik Johnston
ce14c7a995 Fix SYN-627, events are in incorrect room in /sync 2016-02-11 15:02:56 +00:00
Erik Johnston
0eff740523 Return events in correct order for /events 2016-02-11 10:07:27 +00:00
Erik Johnston
5189bfdef4 Batch fetch _get_state_groups_from_groups 2016-02-10 13:24:42 +00:00
Erik Johnston
24f00a6c33 Use _simple_select_many for _get_state_group_for_events 2016-02-10 12:57:50 +00:00
Erik Johnston
f7ef5c1d57 Merge pull request #568 from matrix-org/erikj/unread_notif 
Atomically persit push actions when we persist the event
 2016-02-10 11:25:32 +00:00
Erik Johnston
0214745239 Rename functions 2016-02-10 11:09:56 +00:00
Erik Johnston
7b0d846407 Atomically persit push actions when we persist the event 2016-02-09 16:19:15 +00:00
Erik Johnston
78d6c1b5be Change a log from debug to info 2016-02-09 14:44:12 +00:00
Erik Johnston
70a8608749 Invalidate get_last_receipt_event_id_for_user cache 2016-02-09 14:27:29 +00:00
Erik Johnston
eff12e838c Don't load all ephemeral state for a room on every sync 2016-02-09 13:55:59 +00:00
Erik Johnston
2c1fbea531 Fix up logcontexts 2016-02-08 14:26:45 +00:00
Daniel Wagner-Hall
79a1c0574b Allocate guest user IDs numericcally 
The current random IDs are ugly and confusing when presented in UIs.
This makes them prettier and easier to read.

Also, disable non-automated registration of numeric IDs so that we don't
need to worry so much about people carving out our automated address
space and us needing to keep retrying ID registration.
 2016-02-05 11:22:30 +00:00
Erik Johnston
4d36e73230 Actually return something sensible 2016-02-03 16:35:00 +00:00
Erik Johnston
aa4af94c69 We return dicts now. 2016-02-03 16:29:32 +00:00
Erik Johnston
b84d59c5f0 Add descriptions 2016-02-03 16:22:35 +00:00
Erik Johnston
8450114098 Merge pull request #554 from matrix-org/erikj/event_push 
Change event_push_actions_rm_tokens schema
 2016-02-03 15:02:47 +00:00
Erik Johnston
f8aae79a72 Simplify get_rooms 2016-02-03 13:24:35 +00:00
Erik Johnston
6f52e90065 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/public_room_fix 2016-02-03 11:06:29 +00:00
Erik Johnston
771528ab13 Change event_push_actions_rm_tokens schema 2016-02-03 10:50:49 +00:00
Erik Johnston
b32121a5d1 Unused import 2016-02-03 10:30:56 +00:00
Daniel Wagner-Hall
d83d004ccd Fix flake8 warnings for new flake8 2016-02-02 17:18:50 +00:00
Erik Johnston
8a391e33ae s/get_room_changes_for_user/get_membership_changes_for_user/ 2016-02-02 16:12:10 +00:00
Erik Johnston
477b1ed6cf Fetch events in a separate transaction. 
This has a couple of benefits:

- It reduces the time of transactions, allowing other database requests
  to run.
- Fetching events is given a dedicated database thread, and so can't
  starve other database requests.
 2016-02-02 15:58:14 +00:00
Erik Johnston
65e92eca49 Change the way we do public room list fetching 2016-02-02 15:21:10 +00:00
Erik Johnston
89b40b225c Order things correctly 2016-02-01 16:32:46 +00:00
Erik Johnston
4bf448be25 Switch over /events to use per room caches 2016-02-01 16:26:51 +00:00
Erik Johnston
2d3837bec7 Merge pull request #543 from matrix-org/erikj/sync 
Cache get_room_changes_for_user
 2016-02-01 15:05:06 +00:00
Erik Johnston
ceb6b8680a Only use room_ids if in get_room_events_stream if is_guest 2016-02-01 10:33:52 +00:00
Erik Johnston
25c311eaf6 Cache get_room_changes_for_user 2016-01-29 16:52:48 +00:00
Erik Johnston
cc9c97e0dc Invalidate _account_data_stream_cache correctly 2016-01-29 16:41:51 +00:00
Erik Johnston
8da95b6f1b Comment. Remove superfluous order by 2016-01-29 15:39:17 +00:00
Erik Johnston
b5dbced938 Don't prefill account data 2016-01-29 14:53:59 +00:00
Erik Johnston
3d60686c0c Actually use cache 2016-01-29 14:49:11 +00:00
Erik Johnston
45488e0ffa Max is not a function 2016-01-29 14:42:01 +00:00
Erik Johnston
f67d60496a Convert param style 2016-01-29 14:41:16 +00:00
Erik Johnston
18579534ea Prefill stream change caches 2016-01-29 14:37:59 +00:00
Erik Johnston
ebc5f00efe Bump AccountDataAndTagsChangeCache size 2016-01-29 13:37:40 +00:00
Erik Johnston
b18114e19e Merge pull request #536 from matrix-org/erikj/sync 
Make /sync "better".
 2016-01-29 13:04:51 +00:00
Erik Johnston
467c27a1f9 Amalgamate tags and account data stream caches 2016-01-28 18:20:00 +00:00
Erik Johnston
03b2c2577c Don't use defer.returnValue 2016-01-28 17:29:24 +00:00
Erik Johnston
8fe8951a8d Cache filters 2016-01-28 17:09:09 +00:00
Erik Johnston
fdca8ec418 Add events index 2016-01-28 16:41:59 +00:00
Erik Johnston
45cf827c8f Change name and doc has_entity_changed 2016-01-28 16:39:18 +00:00
Erik Johnston
00cb3eb24b Cache tags and account data 2016-01-28 16:37:41 +00:00
Erik Johnston
c23a8c7833 Ensure keys to RoomStreamChangeCache are ints 2016-01-28 15:55:26 +00:00
Erik Johnston
e1941442d4 Invalidate caches properly. Remove unused arg 2016-01-28 15:02:41 +00:00
Erik Johnston
19fd425928 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/sync 2016-01-28 14:46:08 +00:00
Erik Johnston
167d1df699 Merge pull request #534 from matrix-org/erikj/setup 
Add a Homeserver.setup method
 2016-01-28 14:45:36 +00:00
Erik Johnston
7ed2bbeb11 Clean up a bit. Add comment 2016-01-28 14:32:05 +00:00
Erik Johnston
4e7948b47a Allow paginating backwards from stream token 2016-01-28 11:52:34 +00:00
Erik Johnston
ba8931829b Return correct type of token 2016-01-28 11:34:17 +00:00
Erik Johnston
61eaa6ec64 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/sync 2016-01-28 10:50:57 +00:00
Erik Johnston
c5e7c0e436 Up get_rooms_for_user cache size 2016-01-28 09:58:45 +00:00
Erik Johnston
e7febf4fbb PEP8 2016-01-27 17:33:27 +00:00
Erik Johnston
aca3193efb Use the same path for incremental with gap or without gap 2016-01-27 17:33:27 +00:00
Erik Johnston
b97f6626b6 Add cache to room stream 2016-01-27 17:33:26 +00:00
Erik Johnston
a955cbfa49 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/setup 2016-01-27 17:22:35 +00:00
Mark Haines
8c94833b72 Fix adding push rules relative to other rules 2016-01-27 10:24:20 +00:00
Erik Johnston
87f9477b10 Add a Homeserver.setup method. 
This is for setting up dependencies that require work on startup. This
is useful for the DataStore that wants to read a bunch from the database
before initiliazing.
 2016-01-26 15:51:06 +00:00
Erik Johnston
8431f62ebb Merge pull request #525 from matrix-org/erikj/select_many 
Implement a `_simple_select_many_batch`
 2016-01-25 16:30:36 +00:00
Erik Johnston
f091b73e69 Merge pull request #527 from matrix-org/erikj/push_cache 
Push: Use storage apis that are cached
 2016-01-25 16:16:34 +00:00
Erik Johnston
aea5da0ef6 Guard against empty iterables 2016-01-25 15:59:29 +00:00
Erik Johnston
1ebf5e3d03 Correct docstring 2016-01-25 15:53:36 +00:00
Erik Johnston
86896408b0 Add index to event_push_actions 2016-01-25 15:30:32 +00:00
Erik Johnston
53cb173663 Push: Use storage apis that are cached 2016-01-25 13:55:18 +00:00
Erik Johnston
ddd25def01 Implement a _simple_select_many_batch 2016-01-25 13:36:02 +00:00
David Baker
52bdd1b834 Overlay the push_rules_enabled map for users, otherwise they won't be able to disable server default rules. 2016-01-22 14:58:19 +00:00
David Baker
10f76dc5da Make LRU cache not default to treecache & add options to use it 2016-01-22 12:10:33 +00:00
David Baker
42eae4634f Use new invalidate_many cache invalidation to invalidate the event_push_actions cache appropriately. 2016-01-22 11:22:48 +00:00
Erik Johnston
8f66fe6392 Cache get_unread_event_push_actions_by_room_for_user 2016-01-21 15:02:07 +00:00
David Baker
367cfab4e6 peppate 2016-01-20 16:05:09 +00:00
David Baker
d4315bbf6b Add index by user id on receipts_linearized 2016-01-20 15:33:27 +00:00
David Baker
3fa344c037 Add storage function to get all receipts for a user. Also add some cache invalidation to the receipts storage because there wasn't any, and remove a method that was unused. 2016-01-20 15:30:31 +00:00
Erik Johnston
5a7d1ecffc Add regex cache. Only caculate push actions for users that have sent read receipts, and are on that server 2016-01-19 16:01:05 +00:00
Erik Johnston
3adcc4c86a Return highlight_count in /sync 2016-01-19 11:35:50 +00:00
Erik Johnston
5cd2126a6a Remove dead code 2016-01-18 16:49:46 +00:00
Erik Johnston
2c176e02ae Make unit tests work 2016-01-18 14:48:50 +00:00
Erik Johnston
f59b564507 Make notifications go quicker 2016-01-18 14:48:29 +00:00
Erik Johnston
cc66a9a5e3 Allow filtering events for multiple users at once 2016-01-18 14:43:50 +00:00
Daniel Wagner-Hall
2680043bc6 Require ID and as_token be unique for ASs 
Defaults ID to as_token if not specified. This will change
when IDs are fully supported.
 2016-01-14 14:34:01 +00:00
Daniel Wagner-Hall
27927463a1 Merge pull request #494 from matrix-org/daniel/2 
Don't start server if ASes are invalidly configured
 2016-01-14 11:06:19 +00:00
Daniel Wagner-Hall
fcb6df45e5 Merge pull request #493 from matrix-org/daniel/1 
Delete unused code
 2016-01-14 11:06:14 +00:00
Daniel Wagner-Hall
f6fcff3602 Don't start server if ASes are invalidly configured 2016-01-13 17:09:24 +00:00
Daniel Wagner-Hall
244b356a37 Delete unused code 2016-01-13 17:03:58 +00:00
Mark Haines
9c1f853d58 Rename 'user_name' to 'user_id' in push to make it consistent with the rest of the code 2016-01-13 13:32:59 +00:00
Mark Haines
f4dad9f639 Merge remote-tracking branch 'origin/erikj/bulk_get_push_rules' into markjh/table_name 
Conflicts:
	synapse/storage/push_rule.py
 2016-01-13 11:46:07 +00:00
Erik Johnston
8740e4e94a bulk_get_push_rules should handle empty lists 2016-01-13 11:37:17 +00:00
Mark Haines
c0a279e808 Delete the table objects from TransactionStore 2016-01-13 11:15:20 +00:00
Mark Haines
96e400fee5 Remove the RoomsTable object 2016-01-13 11:07:32 +00:00
Mark Haines
a8e9e0b916 Remove the PushersTable and EventPushActionsTable objects 2016-01-12 14:41:26 +00:00
Mark Haines
31de2953a3 Remove the PushRuleTable and PushRuleEnableTable objects 2016-01-12 14:36:16 +00:00
David Baker
b5d33a656f Postgres doesn't like booleans 2016-01-11 17:13:52 +00:00
David Baker
fe56138142 Remove rogue 'admin' 2016-01-11 17:09:03 +00:00
David Baker
c232780081 Merge pull request #456 from matrix-org/store_event_actions 
Send unread notification counts
 2016-01-08 14:47:15 +00:00
David Baker
daadcf36c0 This comma is actually important 2016-01-07 10:15:35 +00:00
Matthew Hodgson
6c28ac260c copyrights 2016-01-07 04:26:29 +00:00
David Baker
442fcc02f7 Merge remote-tracking branch 'origin/develop' into store_event_actions 2016-01-06 17:28:55 +00:00
David Baker
b6a585348a Adding is_guest here won't work because it just constructs a dict of uid -> password hash 2016-01-06 17:16:02 +00:00
Daniel Wagner-Hall
5880de186b Log when we skip daily messages 2016-01-06 13:52:16 +00:00
David Baker
992928304f Delete notifications for redacted events 2016-01-06 11:58:46 +00:00
David Baker
ae1262a241 Add schema change file for is_guest flag 2016-01-06 11:58:20 +00:00
David Baker
c79f221192 Add is_guest flag to users db to track whether a user is a guest user or not. Use this so we can run _filter_events_for_client when calculating event_push_actions. 2016-01-06 11:38:09 +00:00
David Baker
eb03625626 Merge remote-tracking branch 'origin/develop' into store_event_actions 2016-01-05 18:39:50 +00:00
Daniel Wagner-Hall
cfd07aafff Allow guests to upgrade their accounts 2016-01-05 18:01:18 +00:00
David Baker
4eb7b950c8 = not == in sql 2016-01-04 18:11:17 +00:00
David Baker
c914d67cda Rename event-actions to event_push_actions as per PR request 2016-01-04 14:05:37 +00:00
David Baker
3051c9d002 Address minor PR issues 2016-01-04 13:39:29 +00:00
David Baker
4c8f6a7e42 Insert push actions in a single db query rather than one per user/profile_tag 2015-12-22 17:04:31 +00:00
David Baker
65c451cb38 Add bulk push rule evaluator which actually still evaluates rules one by one, but does far fewer db queries to fetch the rules 2015-12-22 15:19:34 +00:00
David Baker
c061b47c57 Merge remote-tracking branch 'origin/develop' into store_event_actions 2015-12-21 15:30:26 +00:00
David Baker
091c545c4f pep8 2015-12-21 10:14:57 +00:00
David Baker
413d0d6a24 Make unread notification count sending work: put the correct count in incremental syncs too, where necessary, and fix silly bugs like only select the event actions for that user... 2015-12-18 17:47:00 +00:00
Richard van der Hoff
a64f9bbfe0 Fix 500 error when back-paginating search results 
We were mistakenly adding pagination clauses to the count query, which then
failed because the count query doesn't join to the events table.
 2015-12-17 12:50:46 +00:00
David Baker
42ad49f5b7 still very WIP, but now sends unread_notifications_count in the room object on sync (only actually corrrect in a full sync: hardcoded to 0 in incremental syncs). 2015-12-16 18:42:09 +00:00
Mark Haines
4fab578b43 Merge branch 'release-v0.12.0' into develop 2015-12-16 11:18:45 +00:00
Mark Haines
dcfc70e8ed Allow users to change which account a 3pid is bound to 2015-12-15 17:02:21 +00:00
Mark Haines
15e9885197 Merge branch 'release-v0.12.0' into develop 2015-12-14 14:46:55 +00:00
Mark Haines
98dfa7d24f Skip events that where the body, name or topic isn't a string when back populating the FTS index 2015-12-14 13:55:46 +00:00
Daniel Wagner-Hall
76e69cc8de Fix typo 2015-12-14 12:38:55 +00:00
Mark Haines
bfc52a2342 Fix typo in sql for full text search on sqlite3 2015-12-14 11:38:11 +00:00
Mark Haines
deeebbfcb7 Merge branch 'release-v0.12.0' into develop 2015-12-12 14:21:49 +00:00
Erik Johnston
d9a5c56930 Include approximate count of search results 2015-12-11 11:40:23 +00:00
Erik Johnston
51fb590c0e Use more efficient query form 2015-12-11 11:12:57 +00:00
David Baker
aa667ee396 Save event actions to the db 2015-12-10 17:51:15 +00:00
Mark Haines
7d6b313312 Add caches for whether a room has been forgotten by a user 2015-12-10 17:49:34 +00:00
David Baker
21f135ba76 Very first cut of calculating actions for events as they come in. Doesn't store them yet. Not very efficient. 2015-12-10 16:26:08 +00:00
Mark Haines
6a5ff5f223 Track the time spent in the database per request. 
and track the number of transactions that request started.
 2015-12-07 17:56:11 +00:00
Mark Haines
d57c5cda71 Bump schema version. 
As we released version 26 in v0.11.1
 2015-12-04 15:28:39 +00:00
Erik Johnston
b8680b82c3 Merge pull request #414 from matrix-org/erikj/if_not_exists 
Older versions of SQLite don't like IF NOT EXISTS in virtual tables
 2015-12-03 13:21:09 +00:00
David Baker
e515b48929 Just replace the table definition with the one from full_schema 16 2015-12-02 17:23:52 +00:00
David Baker
8810eb8c39 Fix schema delta 15 on postgres in the very unlikley event that anyone upgrades to 15... 2015-12-02 17:19:11 +00:00
Erik Johnston
f721fdbf87 Merge pull request #412 from matrix-org/erikj/search 
Search: Add prefix matching support
 2015-12-02 13:56:41 +00:00
Erik Johnston
976cb5aaa8 Throw if unrecognized DB type 2015-12-02 13:50:43 +00:00
Erik Johnston
b2def42bfd Older versions of SQLite don't like IF NOT EXISTS in virtual tables 2015-12-02 13:29:14 +00:00
Erik Johnston
b9acef5301 Fix so highlight matching works again 2015-12-02 13:28:13 +00:00
Erik Johnston
7dd6e5efca Remove deuplication. Add comment about regex. 2015-12-02 13:09:37 +00:00
Erik Johnston
477da77b46 Search: Add prefix matching support 2015-12-02 11:40:52 +00:00
Mark Haines
95f30ecd1f Add API for setting account_data globaly or on a per room basis 2015-12-01 18:41:32 +00:00
Erik Johnston
4dcaa42b6d Allow paginating search ordered by recents 2015-11-30 17:45:31 +00:00
Erik Johnston
76936f43ae Return words to highlight in search results 2015-11-27 16:40:42 +00:00
Daniel Wagner-Hall
df7cf6c0eb Fix SQL for postgres again 2015-11-23 18:54:41 +00:00
Daniel Wagner-Hall
3e573a5c6b Fix SQL for postgres 2015-11-23 18:48:53 +00:00
Daniel Wagner-Hall
7dfa455508 Remove size specifier for database column 
Postgres doesn't support them like this.

We don't have a bool type in common between postgres and sqlite.
 2015-11-23 18:35:25 +00:00
Daniel Wagner-Hall
95c3306798 Merge branch 'daniel/forgetrooms' of github.com:matrix-org/synapse into daniel/forgetrooms 2015-11-19 15:00:14 -05:00
Daniel Wagner-Hall
df6824a008 Ignore forgotten rooms in v2 sync 2015-11-19 14:54:47 -05:00
Paul "LeoNerd" Evans
1cfda3d2d8 Merge branch 'develop' into daniel/forgetrooms 2015-11-19 16:53:13 +00:00
Daniel Wagner-Hall
9da4c5340d Simplify code 2015-11-19 10:07:21 -05:00
Daniel Wagner-Hall
bed7889703 Apply forgetting properly to historical events 2015-11-18 18:11:12 -05:00
Mark Haines
d153f482dd Rename the database table 2015-11-18 15:33:02 +00:00
Mark Haines
1c960fbb80 s/private_user_data/account_data/ 2015-11-18 15:31:04 +00:00
Daniel Wagner-Hall
ba26eb3d5d Allow users to forget rooms 2015-11-17 17:17:30 -05:00
Richard van der Hoff
e4d622aaaf Implementation of state rollback in /sync 
Implementation of SPEC-254: roll back the state dictionary to how it looked at
the start of the timeline.

Merged PR https://github.com/matrix-org/synapse/pull/373
 2015-11-13 10:58:56 +00:00
Richard van der Hoff
fddedd51d9 Fix a few race conditions in the state calculation 
Be a bit more careful about how we calculate the state to be returned by
/sync. In a few places, it was possible for /sync to return slightly later
state than that represented by the next_batch token and the timeline. In
particular, the following cases were susceptible:

* On a full state sync, for an active room
* During a per-room incremental sync with a timeline gap
* When the user has just joined a room. (Refactor check_joined_room to make it
  less magical)

Also, use store.get_state_for_events() (and thus the existing stategroups) to
calculate the state corresponding to a particular sync position, rather than
state_handler.compute_event_context(), which recalculates from first principles
(and tends to miss some state).

Merged from PR https://github.com/matrix-org/synapse/pull/372
 2015-11-13 10:39:09 +00:00
Erik Johnston
3de46c7755 Trailing whitespace 2015-11-12 15:36:43 +00:00
Erik Johnston
8fd8e72cec Expand comment 2015-11-12 15:33:47 +00:00
Erik Johnston
764e79d051 Comment 2015-11-12 15:19:56 +00:00
Erik Johnston
320408ef47 Fix SQL syntax 2015-11-12 15:09:45 +00:00
Erik Johnston
14a9d805b9 Use a (hopefully) more efficient SQL query for doing recency based room search 2015-11-12 14:48:39 +00:00
Erik Johnston
39de87869c Fix bug where assumed dict was namedtuple 2015-11-12 14:47:48 +00:00
Erik Johnston
e21cef9bb5 Merge pull request #359 from matrix-org/markjh/incremental_indexing 
Incremental background updates for db indexes
 2015-11-11 17:19:51 +00:00
Mark Haines
e1627388d1 Fix param style to work on both sqlite and postgres 2015-11-11 17:14:56 +00:00
Mark Haines
940a161192 Fix the background update 2015-11-11 13:59:40 +00:00
Daniel Wagner-Hall
cf437900e0 Return world_readable and guest_can_join in /publicRooms 2015-11-10 17:10:27 +00:00
Mark Haines
90b503216c Use a background task to update databases to use the full text search 2015-11-10 16:20:13 +00:00
Mark Haines
a412b9a465 Run the background updates when starting synapse. 2015-11-10 15:50:58 +00:00
Mark Haines
2ede7aa8a1 Add background update task for reindexing event search 2015-11-09 19:29:32 +00:00
Mark Haines
c6a01f2ed0 Add storage module for tracking background updates. 
The progress for each background update is stored as a JSON blob in the
database. Each background update is broken up into separate batches.
The batch size is automatically tuned to try avoid blocking single
threaded databases for too long.
 2015-11-09 17:26:27 +00:00
Mark Haines
bbe10e8be7 Merge branch 'develop' into markjh/SYN-513 
Conflicts:
	synapse/storage/tags.py
 2015-11-09 15:01:59 +00:00
Mark Haines
c4135d85e1 SYN-513: Include updates for rooms that have had all their tags deleted 2015-11-09 14:53:08 +00:00
Matthew Hodgson
dd40fb68e4 fix comedy important missing comma breaking recent-ordered FTS on sqlite 2015-11-08 16:04:37 +00:00
Matthew Hodgson
767c20a869 add a key existence check to tags_by_room to avoid /events 500'ing when testing against vector 2015-11-06 20:49:57 +01:00
Daniel Wagner-Hall
f2c4ee41b9 Remove accidentally added ID column 2015-11-06 14:27:49 +00:00
Erik Johnston
3640ddfbf6 Error handling 2015-11-05 16:10:54 +00:00
Erik Johnston
729ea933ea Merge branch 'develop' of github.com:matrix-org/synapse into erikj/search 2015-11-05 15:43:52 +00:00
Erik Johnston
7301e05122 Implement basic pagination for search results 2015-11-05 15:04:08 +00:00
Daniel Wagner-Hall
ca2f90742d Open up /events to anonymous users for room events only 
Squash-merge of PR #345 from daniel/anonymousevents
 2015-11-05 14:32:26 +00:00
Erik Johnston
05c326d445 Implement order and group by 2015-11-04 17:57:44 +00:00
Daniel Wagner-Hall
4e62ffdb21 Merge branch 'develop' of github.com:matrix-org/synapse into develop 2015-11-04 17:31:01 +00:00
Daniel Wagner-Hall
f522f50a08 Allow guests to register and call /events?room_id= 
This follows the same flows-based flow as regular registration, but as
the only implemented flow has no requirements, it auto-succeeds. In the
future, other flows (e.g. captcha) may be required, so clients should
treat this like the regular registration flow choices.
 2015-11-04 17:29:07 +00:00
Mark Haines
33b3e04049 Merge branch 'develop' into daniel/removesomelies 
Conflicts:
	synapse/notifier.py
 2015-11-04 16:01:00 +00:00
Erik Johnston
6a3a840b19 Merge pull request #343 from matrix-org/erikj/fix_retries 
Fix broken cache for getting retry times.
 2015-11-03 17:51:49 +00:00
Mark Haines
6797fcd9ab Merge pull request #335 from matrix-org/markjh/room_tags 
Add APIs for adding and removing tags from rooms
 2015-11-03 16:45:53 +00:00
Erik Johnston
7ce264ce5f Fix broken cache for getting retry times. This meant we retried remote destinations way more frequently than we should 2015-11-03 16:24:03 +00:00
Mark Haines
5897e773fd Spell "deferred" more correctly 2015-11-03 14:27:35 +00:00
Daniel Wagner-Hall
771ca56c88 Remove more unused parameters 2015-11-02 15:31:57 +00:00
Mark Haines
ddd8566f41 Store room tag content and return the content in the m.tag event 2015-11-02 15:11:31 +00:00
Mark Haines
0e36756383 Merge branch 'develop' into markjh/room_tags 2015-11-02 10:57:00 +00:00
Daniel Wagner-Hall
621e84d9a0 Add missing column 2015-10-30 16:25:53 +00:00
Mark Haines
f40b0ed5e1 Inform the client of new room tags using v1 /events 2015-10-29 15:21:09 +00:00
Mark Haines
a89b86dc47 Fix pyflakes errors 2015-10-28 16:45:57 +00:00
Mark Haines
892e70ec84 Add APIs for adding and removing tags from rooms 2015-10-28 16:06:57 +00:00
Erik Johnston
56dbcd1524 Docs 2015-10-28 14:05:50 +00:00
Erik Johnston
5cb298c934 Add room context api 2015-10-28 13:45:56 +00:00
Erik Johnston
4cf633d5e9 Pull out sender when computing search results 2015-10-23 15:41:36 +01:00
Erik Johnston
0c36098c1f Implement rank function for SQLite FTS 2015-10-23 13:23:48 +01:00
Erik Johnston
671ac699f1 Actually filter results 2015-10-22 16:54:56 +01:00
Erik Johnston
fb0fecd0b9 LESS THAN 2015-10-22 16:18:35 +01:00
Erik Johnston
232beb3a3c Use namedtuple as return value 2015-10-22 15:02:35 +01:00
Erik Johnston
ba02bba88c Limit max number of SQL vars 2015-10-22 13:25:27 +01:00
Erik Johnston
8a98f0dc5b Merge branch 'develop' of github.com:matrix-org/synapse into erikj/search 2015-10-22 13:16:35 +01:00
Erik Johnston
f142898f52 PEP8 2015-10-22 11:18:01 +01:00
Erik Johnston
4d25bc6c92 Move FTS to delta 25 2015-10-22 11:12:28 +01:00
Mark Haines
e94ffd89d6 Merge pull request #316 from matrix-org/markjh/v2_sync_archived 
Add rooms that the user has left under archived in v2 sync.
 2015-10-21 15:46:41 +01:00
Mark Haines
e3d75f564a Include banned rooms in the archived section of v2 sync 2015-10-21 11:15:48 +01:00
Mark Haines
68b7fc3e2b Add rooms that the user has left under archived in v2 sync. 2015-10-19 17:26:18 +01:00
Erik Johnston
46d39343d9 Explicitly check for Sqlite3Engine 2015-10-16 16:58:00 +01:00
Erik Johnston
f2d698cb52 Typing 2015-10-16 16:46:48 +01:00
Erik Johnston
33646eb000 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/search 2015-10-16 15:35:35 +01:00
Mark Haines
fc012aa8dc Fix FilteringStore.get_user_filter to work with postgres 2015-10-16 15:28:43 +01:00
Erik Johnston
12122bfc36 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/search 2015-10-16 14:46:32 +01:00
Erik Johnston
edb998ba23 Explicitly check for Sqlite3Engine 2015-10-16 14:37:14 +01:00
Erik Johnston
3cf9948b8d Add docstring 2015-10-16 11:28:12 +01:00
Erik Johnston
73260ad01f Comment on the LIMIT 500 2015-10-16 11:24:02 +01:00
Erik Johnston
22a8c91448 Split up run_upgrade 2015-10-16 11:19:44 +01:00
Mark Haines
6296590bf7 Encode the filter JSON as UTF-8 before storing in the database. 
Because we are using a binary column type to store the filter JSON.
 2015-10-16 10:50:32 +01:00
Erik Johnston
bcfb653816 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/search 2015-10-15 16:37:32 +01:00
Mark Haines
a059760954 Merge pull request #305 from matrix-org/markjh/v2_sync_api 
Update the v2 sync API to work as specified in the current spec.
 2015-10-14 13:56:23 +01:00
Mark Haines
f50c43464c Merge branch 'develop' into markjh/v2_sync_api 2015-10-14 11:40:45 +01:00
Erik Johnston
99c7fbfef7 Fix to work with SQLite 2015-10-14 09:52:40 +01:00
Erik Johnston
858634e1d0 Remove unused room_id arg 2015-10-14 09:31:20 +01:00
Erik Johnston
7ecd11accb Add paranoia limit 2015-10-13 15:50:56 +01:00
Erik Johnston
3e2a1297b5 Remove constraints in preperation of using filters 2015-10-13 15:22:14 +01:00
Mark Haines
cacf0688c6 Add a get_invites_for_user method to the storage to find out the rooms a user is invited to 2015-10-13 14:08:38 +01:00
Erik Johnston
88971fd034 Merge branch 'erikj/store_engine' into erikj/search 2015-10-13 14:03:30 +01:00
Erik Johnston
17c80c8a3d rename schema_prepare to prepare_database 2015-10-13 13:56:22 +01:00
Erik Johnston
cfd39d6b55 Add SQLite support 2015-10-13 13:47:50 +01:00
Erik Johnston
f9340ea0d5 Merge branch 'erikj/store_engine' into erikj/search 2015-10-13 13:29:02 +01:00
Erik Johnston
ec398af41c Expose error more nicely 2015-10-13 11:43:43 +01:00
Erik Johnston
40b6a5aad1 Split out the schema preparation and update logic into its own module 2015-10-13 11:38:48 +01:00
Erik Johnston
1a40afa756 Add sqlite schema 2015-10-13 10:36:25 +01:00
Erik Johnston
ca53ad7425 Filter events to only thsoe that the user is allowed to see 2015-10-12 15:52:55 +01:00
Erik Johnston
927004e349 Remove unused room_id parameter 2015-10-12 15:06:14 +01:00
Erik Johnston
61561b9df7 Keep FTS indexes up to date. Only search through rooms currently joined 2015-10-12 10:49:53 +01:00
Erik Johnston
c85c912562 Add basic full text search impl. 2015-10-09 15:48:31 +01:00
Mark Haines
301141515a Merge pull request #288 from matrix-org/markjh/unused_definitions 
Remove some of the unused definitions from synapse
 2015-09-28 14:22:44 +01:00
Mark Haines
cf1100887b Fix order of ON constraints in _get_rooms_for_user_where_membership_is_txn 2015-09-24 17:35:10 +01:00
Mark Haines
1cd65a8d1e synapse/storage/state.py: _make_group_id was unused 2015-09-23 10:37:58 +01:00
Mark Haines
973ebb66ba Remove unused functions from synapse/storage/signatures.py 2015-09-23 10:36:33 +01:00
Mark Haines
e51aa4be96 synapse/storage/roommember.py:_get_members_query was unused 2015-09-23 10:35:10 +01:00