Richard van der Hoff
fddedd51d9
Fix a few race conditions in the state calculation
...
Be a bit more careful about how we calculate the state to be returned by
/sync. In a few places, it was possible for /sync to return slightly later
state than that represented by the next_batch token and the timeline. In
particular, the following cases were susceptible:
* On a full state sync, for an active room
* During a per-room incremental sync with a timeline gap
* When the user has just joined a room. (Refactor check_joined_room to make it
less magical)
Also, use store.get_state_for_events() (and thus the existing stategroups) to
calculate the state corresponding to a particular sync position, rather than
state_handler.compute_event_context(), which recalculates from first principles
(and tends to miss some state).
Merged from PR https://github.com/matrix-org/synapse/pull/372
2015-11-13 10:39:09 +00:00
Erik Johnston
3de46c7755
Trailing whitespace
2015-11-12 15:36:43 +00:00
Erik Johnston
8fd8e72cec
Expand comment
2015-11-12 15:33:47 +00:00
Erik Johnston
764e79d051
Comment
2015-11-12 15:19:56 +00:00
Erik Johnston
320408ef47
Fix SQL syntax
2015-11-12 15:09:45 +00:00
Erik Johnston
14a9d805b9
Use a (hopefully) more efficient SQL query for doing recency based room search
2015-11-12 14:48:39 +00:00
Erik Johnston
39de87869c
Fix bug where assumed dict was namedtuple
2015-11-12 14:47:48 +00:00
Erik Johnston
e21cef9bb5
Merge pull request #359 from matrix-org/markjh/incremental_indexing
...
Incremental background updates for db indexes
2015-11-11 17:19:51 +00:00
Mark Haines
e1627388d1
Fix param style to work on both sqlite and postgres
2015-11-11 17:14:56 +00:00
Mark Haines
940a161192
Fix the background update
2015-11-11 13:59:40 +00:00
Daniel Wagner-Hall
cf437900e0
Return world_readable and guest_can_join in /publicRooms
2015-11-10 17:10:27 +00:00
Mark Haines
90b503216c
Use a background task to update databases to use the full text search
2015-11-10 16:20:13 +00:00
Mark Haines
a412b9a465
Run the background updates when starting synapse.
2015-11-10 15:50:58 +00:00
Mark Haines
2ede7aa8a1
Add background update task for reindexing event search
2015-11-09 19:29:32 +00:00
Mark Haines
c6a01f2ed0
Add storage module for tracking background updates.
...
The progress for each background update is stored as a JSON blob in the
database. Each background update is broken up into separate batches.
The batch size is automatically tuned to try avoid blocking single
threaded databases for too long.
2015-11-09 17:26:27 +00:00
Mark Haines
bbe10e8be7
Merge branch 'develop' into markjh/SYN-513
...
Conflicts:
synapse/storage/tags.py
2015-11-09 15:01:59 +00:00
Mark Haines
c4135d85e1
SYN-513: Include updates for rooms that have had all their tags deleted
2015-11-09 14:53:08 +00:00
Matthew Hodgson
dd40fb68e4
fix comedy important missing comma breaking recent-ordered FTS on sqlite
2015-11-08 16:04:37 +00:00
Matthew Hodgson
767c20a869
add a key existence check to tags_by_room to avoid /events 500'ing when testing against vector
2015-11-06 20:49:57 +01:00
Daniel Wagner-Hall
f2c4ee41b9
Remove accidentally added ID column
2015-11-06 14:27:49 +00:00
Erik Johnston
3640ddfbf6
Error handling
2015-11-05 16:10:54 +00:00
Erik Johnston
729ea933ea
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/search
2015-11-05 15:43:52 +00:00
Erik Johnston
7301e05122
Implement basic pagination for search results
2015-11-05 15:04:08 +00:00
Daniel Wagner-Hall
ca2f90742d
Open up /events to anonymous users for room events only
...
Squash-merge of PR #345 from daniel/anonymousevents
2015-11-05 14:32:26 +00:00
Erik Johnston
05c326d445
Implement order and group by
2015-11-04 17:57:44 +00:00
Daniel Wagner-Hall
4e62ffdb21
Merge branch 'develop' of github.com:matrix-org/synapse into develop
2015-11-04 17:31:01 +00:00
Daniel Wagner-Hall
f522f50a08
Allow guests to register and call /events?room_id=
...
This follows the same flows-based flow as regular registration, but as
the only implemented flow has no requirements, it auto-succeeds. In the
future, other flows (e.g. captcha) may be required, so clients should
treat this like the regular registration flow choices.
2015-11-04 17:29:07 +00:00
Mark Haines
33b3e04049
Merge branch 'develop' into daniel/removesomelies
...
Conflicts:
synapse/notifier.py
2015-11-04 16:01:00 +00:00
Erik Johnston
6a3a840b19
Merge pull request #343 from matrix-org/erikj/fix_retries
...
Fix broken cache for getting retry times.
2015-11-03 17:51:49 +00:00
Mark Haines
6797fcd9ab
Merge pull request #335 from matrix-org/markjh/room_tags
...
Add APIs for adding and removing tags from rooms
2015-11-03 16:45:53 +00:00
Erik Johnston
7ce264ce5f
Fix broken cache for getting retry times. This meant we retried remote destinations way more frequently than we should
2015-11-03 16:24:03 +00:00
Mark Haines
5897e773fd
Spell "deferred" more correctly
2015-11-03 14:27:35 +00:00
Daniel Wagner-Hall
771ca56c88
Remove more unused parameters
2015-11-02 15:31:57 +00:00
Mark Haines
ddd8566f41
Store room tag content and return the content in the m.tag event
2015-11-02 15:11:31 +00:00
Mark Haines
0e36756383
Merge branch 'develop' into markjh/room_tags
2015-11-02 10:57:00 +00:00
Daniel Wagner-Hall
621e84d9a0
Add missing column
2015-10-30 16:25:53 +00:00
Mark Haines
f40b0ed5e1
Inform the client of new room tags using v1 /events
2015-10-29 15:21:09 +00:00
Mark Haines
a89b86dc47
Fix pyflakes errors
2015-10-28 16:45:57 +00:00
Mark Haines
892e70ec84
Add APIs for adding and removing tags from rooms
2015-10-28 16:06:57 +00:00
Erik Johnston
56dbcd1524
Docs
2015-10-28 14:05:50 +00:00
Erik Johnston
5cb298c934
Add room context api
2015-10-28 13:45:56 +00:00
Erik Johnston
4cf633d5e9
Pull out sender when computing search results
2015-10-23 15:41:36 +01:00
Erik Johnston
0c36098c1f
Implement rank function for SQLite FTS
2015-10-23 13:23:48 +01:00
Erik Johnston
671ac699f1
Actually filter results
2015-10-22 16:54:56 +01:00
Erik Johnston
fb0fecd0b9
LESS THAN
2015-10-22 16:18:35 +01:00
Erik Johnston
232beb3a3c
Use namedtuple as return value
2015-10-22 15:02:35 +01:00
Erik Johnston
ba02bba88c
Limit max number of SQL vars
2015-10-22 13:25:27 +01:00
Erik Johnston
8a98f0dc5b
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/search
2015-10-22 13:16:35 +01:00
Erik Johnston
f142898f52
PEP8
2015-10-22 11:18:01 +01:00
Erik Johnston
4d25bc6c92
Move FTS to delta 25
2015-10-22 11:12:28 +01:00
Mark Haines
e94ffd89d6
Merge pull request #316 from matrix-org/markjh/v2_sync_archived
...
Add rooms that the user has left under archived in v2 sync.
2015-10-21 15:46:41 +01:00
Mark Haines
e3d75f564a
Include banned rooms in the archived section of v2 sync
2015-10-21 11:15:48 +01:00
Mark Haines
68b7fc3e2b
Add rooms that the user has left under archived in v2 sync.
2015-10-19 17:26:18 +01:00
Erik Johnston
46d39343d9
Explicitly check for Sqlite3Engine
2015-10-16 16:58:00 +01:00
Erik Johnston
f2d698cb52
Typing
2015-10-16 16:46:48 +01:00
Erik Johnston
33646eb000
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/search
2015-10-16 15:35:35 +01:00
Mark Haines
fc012aa8dc
Fix FilteringStore.get_user_filter to work with postgres
2015-10-16 15:28:43 +01:00
Erik Johnston
12122bfc36
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/search
2015-10-16 14:46:32 +01:00
Erik Johnston
edb998ba23
Explicitly check for Sqlite3Engine
2015-10-16 14:37:14 +01:00
Erik Johnston
3cf9948b8d
Add docstring
2015-10-16 11:28:12 +01:00
Erik Johnston
73260ad01f
Comment on the LIMIT 500
2015-10-16 11:24:02 +01:00
Erik Johnston
22a8c91448
Split up run_upgrade
2015-10-16 11:19:44 +01:00
Mark Haines
6296590bf7
Encode the filter JSON as UTF-8 before storing in the database.
...
Because we are using a binary column type to store the filter JSON.
2015-10-16 10:50:32 +01:00
Erik Johnston
bcfb653816
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/search
2015-10-15 16:37:32 +01:00
Mark Haines
a059760954
Merge pull request #305 from matrix-org/markjh/v2_sync_api
...
Update the v2 sync API to work as specified in the current spec.
2015-10-14 13:56:23 +01:00
Mark Haines
f50c43464c
Merge branch 'develop' into markjh/v2_sync_api
2015-10-14 11:40:45 +01:00
Erik Johnston
99c7fbfef7
Fix to work with SQLite
2015-10-14 09:52:40 +01:00
Erik Johnston
858634e1d0
Remove unused room_id arg
2015-10-14 09:31:20 +01:00
Erik Johnston
7ecd11accb
Add paranoia limit
2015-10-13 15:50:56 +01:00
Erik Johnston
3e2a1297b5
Remove constraints in preperation of using filters
2015-10-13 15:22:14 +01:00
Mark Haines
cacf0688c6
Add a get_invites_for_user method to the storage to find out the rooms a user is invited to
2015-10-13 14:08:38 +01:00
Erik Johnston
88971fd034
Merge branch 'erikj/store_engine' into erikj/search
2015-10-13 14:03:30 +01:00
Erik Johnston
17c80c8a3d
rename schema_prepare to prepare_database
2015-10-13 13:56:22 +01:00
Erik Johnston
cfd39d6b55
Add SQLite support
2015-10-13 13:47:50 +01:00
Erik Johnston
f9340ea0d5
Merge branch 'erikj/store_engine' into erikj/search
2015-10-13 13:29:02 +01:00
Erik Johnston
ec398af41c
Expose error more nicely
2015-10-13 11:43:43 +01:00
Erik Johnston
40b6a5aad1
Split out the schema preparation and update logic into its own module
2015-10-13 11:38:48 +01:00
Erik Johnston
1a40afa756
Add sqlite schema
2015-10-13 10:36:25 +01:00
Erik Johnston
ca53ad7425
Filter events to only thsoe that the user is allowed to see
2015-10-12 15:52:55 +01:00
Erik Johnston
927004e349
Remove unused room_id parameter
2015-10-12 15:06:14 +01:00
Erik Johnston
61561b9df7
Keep FTS indexes up to date. Only search through rooms currently joined
2015-10-12 10:49:53 +01:00
Erik Johnston
c85c912562
Add basic full text search impl.
2015-10-09 15:48:31 +01:00
Mark Haines
301141515a
Merge pull request #288 from matrix-org/markjh/unused_definitions
...
Remove some of the unused definitions from synapse
2015-09-28 14:22:44 +01:00
Mark Haines
cf1100887b
Fix order of ON constraints in _get_rooms_for_user_where_membership_is_txn
2015-09-24 17:35:10 +01:00
Mark Haines
1cd65a8d1e
synapse/storage/state.py: _make_group_id was unused
2015-09-23 10:37:58 +01:00
Mark Haines
973ebb66ba
Remove unused functions from synapse/storage/signatures.py
2015-09-23 10:36:33 +01:00
Mark Haines
e51aa4be96
synapse/storage/roommember.py:_get_members_query was unused
2015-09-23 10:35:10 +01:00
Mark Haines
92d8d724c5
Remove unused functions from synapse/storage/events.py
2015-09-23 10:33:06 +01:00
Mark Haines
c292dba70c
Remove unused functions from synapse/storage/event_federation.py
2015-09-23 10:31:25 +01:00
Mark Haines
396834f1c0
synapse/storage/_base.py:_simple_max_id was unused
2015-09-23 10:30:38 +01:00
Mark Haines
1d9036aff2
synapse/storage/_base.py:_simple_delete was unused
2015-09-23 10:30:25 +01:00
Mark Haines
1ee3d26432
synapse/storage/_base.py:_simple_selectupdate_one was unused
2015-09-23 10:30:03 +01:00
Mark Haines
3559a835a2
synapse/storage/event_federation.py:_get_auth_events is unused
2015-09-22 18:39:46 +01:00
Mark Haines
7dd4f79c49
synapse/storage/_base.py:_execute_and_decode was unused
2015-09-22 18:37:07 +01:00
Mark Haines
527d95dea0
synapse/storage/_base.py:Table was unused
2015-09-22 18:14:15 +01:00
Daniel Wagner-Hall
6d59ffe1ce
Add some docstrings
2015-09-22 13:47:40 +01:00
Daniel Wagner-Hall
eb011cd99b
Add docstring
2015-09-22 13:29:36 +01:00
Daniel Wagner-Hall
7213588083
Implement configurable stats reporting
...
SYN-287
This requires that HS owners either opt in or out of stats reporting.
When --generate-config is passed, --report-stats must be specified
If an already-generated config is used, and doesn't have the
report_stats key, it is requested to be set.
2015-09-22 12:57:40 +01:00
Mark Haines
ee2d722f0f
Merge pull request #276 from matrix-org/markjh/history_for_rooms_that_have_been_left
...
SPEC-216: Allow users to view the history of rooms that they have left.
2015-09-21 14:38:13 +01:00
Erik Johnston
2048388cfd
Merge pull request #281 from matrix-org/erikj/update_extremeties
...
When updating a stored event from outlier to non-outlier, remember to update the extremeties
2015-09-15 16:57:25 +01:00
