Erik Johnston
11d62f43c9
Invalidate cache
2017-10-03 14:12:28 +01:00
Erik Johnston
e4ab96021e
Update comments
2017-10-03 14:10:41 +01:00
Erik Johnston
30848c0fcd
Ignore incoming events for rooms that we have left
...
When synapse receives an event for a room its not in over federation, it
double checks with the remote server to see if it is in fact in the
room. This is done so that if the server has forgotten about the room
(usually as a result of the database being dropped) it can recover from
it.
However, in the presence of state resets in large rooms, this can cause
a lot of work for servers that have legitimately left. As a hacky
solution that supports both cases we drop incoming events for rooms that
we have explicitly left.
This means that we no longer support the case of servers having
forgotten that they've rejoined a room, but that is sufficiently rare
that we're not going to support it for now.
2017-10-03 11:18:21 +01:00
David Baker
27955056e0
Merge branch 'develop' into erikj/groups_merged
2017-10-02 16:20:41 +01:00
Erik Johnston
768f00dedb
Up the limits on number of url cache entries to delete at one time
2017-09-28 14:27:27 +01:00
Erik Johnston
4dc07e93a8
Add old indices
2017-09-28 14:10:33 +01:00
Erik Johnston
93247a424a
Only pull out local media that were for url cache
2017-09-28 13:48:14 +01:00
Erik Johnston
7a44c01d89
Fix typo
2017-09-28 12:46:04 +01:00
Erik Johnston
ae79764fe5
Change expires column to expires_ts
2017-09-28 12:37:53 +01:00
Erik Johnston
77f1d24de3
More brackets
2017-09-28 12:23:15 +01:00
Erik Johnston
9ccb4226ba
Delete expired url cache data
2017-09-28 12:18:06 +01:00
Erik Johnston
17b8e2bd02
Add remove room API
2017-09-26 15:52:41 +01:00
Erik Johnston
a8e2a3df32
Add unique index to group_rooms table
2017-09-26 15:39:21 +01:00
Erik Johnston
e1dec2f1a7
Remove user from group summary when the leave the group
2017-09-21 16:09:57 +01:00
Erik Johnston
197d82dc07
Correctly return next token
2017-09-20 11:12:11 +01:00
Erik Johnston
069ae2df12
Fix initial sync
2017-09-20 10:52:12 +01:00
Richard van der Hoff
e76d1135dd
Invalidate signing key cache when we gat an update
...
This might make the cache slightly more efficient.
2017-09-20 01:32:42 +01:00
Erik Johnston
4a9b1cf253
Add user profiles to summary from group server
2017-08-25 16:23:58 +01:00
Erik Johnston
27ebc5c8f2
Add remote profile cache
2017-08-25 11:25:47 +01:00
Erik Johnston
97c544f91f
Add _simple_update
2017-08-25 11:11:37 +01:00
Erik Johnston
2800983f3e
Merge pull request #2410 from matrix-org/erikj/groups_publicise
...
Add ability to publicise group membership
2017-08-21 16:51:56 +01:00
Erik Johnston
8b50fe5330
Use BOOLEAN rather than TEXT type
2017-08-21 16:37:29 +01:00
Erik Johnston
175a01f56c
Groups: Fix mising json.load in initial sync
2017-08-21 14:45:56 +01:00
Erik Johnston
ef8e578677
Add bulk group publicised lookup API
2017-08-09 13:36:22 +01:00
Erik Johnston
b880ff190a
Allow update group publicity
2017-08-08 14:19:41 +01:00
Erik Johnston
05e21285aa
Store whether the user wants to publicise their membership of a group
2017-08-08 13:01:46 +01:00
Erik Johnston
966a70f1fa
Update comment
2017-07-24 17:49:39 +01:00
Erik Johnston
629cdfb124
Use join rather than joined, etc.
2017-07-24 14:54:05 +01:00
Erik Johnston
ed666d3969
Fix all the typos
2017-07-24 14:05:09 +01:00
Erik Johnston
b76ef6ccb8
Include users membership in group in summary API
2017-07-24 13:55:39 +01:00
Erik Johnston
851aeae7c7
Check users/rooms are in group before adding to summary
2017-07-24 13:40:56 +01:00
Erik Johnston
96917d5552
Merge pull request #2378 from matrix-org/erikj/group_sync_support
...
Add groups to sync stream
2017-07-21 11:05:39 +01:00
Erik Johnston
0401604222
Merge pull request #2377 from matrix-org/erikj/group_profile_update
...
Add update group profile API
2017-07-20 17:53:39 +01:00
Erik Johnston
2cc998fed8
Fix replication. And notify
2017-07-20 17:13:18 +01:00
Erik Johnston
c544188ee3
Add groups to sync stream
2017-07-20 16:36:42 +01:00
Erik Johnston
8209b5f033
Fix a storage desc
2017-07-20 16:22:22 +01:00
Erik Johnston
57826d645b
Fix typo
2017-07-20 13:15:22 +01:00
Erik Johnston
6f443a74cf
Add update group profile API
2017-07-20 09:46:33 +01:00
Erik Johnston
94ecd871a0
Fix typos
2017-07-18 16:38:54 +01:00
Erik Johnston
3e703eb04e
Comment
2017-07-18 10:17:25 +01:00
Erik Johnston
508460f240
Remove sync stuff
2017-07-18 09:55:46 +01:00
Erik Johnston
4540730111
Remove unused tables
2017-07-18 09:38:15 +01:00
Erik Johnston
e96ee95a7e
Remove sync stuff
2017-07-18 09:38:08 +01:00
Erik Johnston
2f9eafdd36
Add local group server support
2017-07-17 12:03:49 +01:00
Erik Johnston
85fda57208
Add DEFAULT_ROLE_ID
2017-07-14 14:03:54 +01:00
Erik Johnston
4b203bdba5
Correctly increment orders
2017-07-14 14:02:31 +01:00
Erik Johnston
3b0470dba5
Remove unused functions
2017-07-13 13:54:01 +01:00
Erik Johnston
8575e3160f
Comments
2017-07-13 13:52:41 +01:00
Erik Johnston
26451a09eb
Comments
2017-07-12 14:47:18 +01:00
Erik Johnston
a62406aaa5
Add group summary APIs
2017-07-12 11:36:15 +01:00
Erik Johnston
6d586dc05c
Comment
2017-07-12 09:58:37 +01:00
Erik Johnston
410b4e14a1
Move comment
2017-07-11 15:44:18 +01:00
Erik Johnston
e52c391cd4
Rename column to attestation_json
2017-07-11 14:25:46 +01:00
Erik Johnston
0aac30d53b
Comments
2017-07-11 14:23:50 +01:00
Erik Johnston
83936293eb
Comments
2017-07-11 11:42:25 +01:00
Erik Johnston
b8ca494ee9
Initial group server implementation
2017-07-10 15:44:15 +01:00
Erik Johnston
f502b0dea1
Perf: Don't filter events for push
...
We know the users are joined and we can explicitly check for if they are
ignoring the user, so lets do that.
2017-07-07 14:04:40 +01:00
Erik Johnston
ed9a7f5436
Merge pull request #2309 from matrix-org/erikj/user_ip_repl
...
Fix up user_ip replication commands
2017-07-06 14:33:14 +01:00
Erik Johnston
b5e8d529e6
Define CACHE_SIZE_FACTOR once
2017-07-04 09:56:44 +01:00
Erik Johnston
6ff14ddd2e
Make into list
2017-06-29 15:47:37 +01:00
Erik Johnston
5946aa0877
Prefill forward extrems and event to state groups
2017-06-29 15:38:48 +01:00
Erik Johnston
8c23221666
Fix up
2017-06-27 15:53:45 +01:00
Erik Johnston
a0a561ae85
Fix up client ips to read from pending data
2017-06-27 14:46:12 +01:00
Erik Johnston
ed3d0170d9
Batch upsert user ips
2017-06-27 13:37:04 +01:00
Erik Johnston
8abdd7b553
Fix up indices for users_who_share_rooms
2017-06-26 14:01:30 +01:00
Erik Johnston
7fe8ed1787
Store URL cache preview downloads seperately
...
This makes it easier to clear old media out at a later date
2017-06-23 11:14:11 +01:00
Erik Johnston
e0004aa28a
Add desc
2017-06-22 10:03:48 +01:00
Erik Johnston
b668112320
Merge pull request #2296 from matrix-org/erikj/dont_appserver_shar
...
Don't work out users who share room with appservice users
2017-06-21 14:50:24 +01:00
Erik Johnston
dae9a00a28
Initialise exclusive_user_regex
2017-06-21 14:19:33 +01:00
Krombel
4202fba82a
Merge branch 'develop' into avoid_duplicate_filters
2017-06-21 14:48:21 +02:00
Krombel
812c030e87
replaced json.dumps with encode_canonical_json
2017-06-21 14:48:12 +02:00
Erik Johnston
385dcb7c60
Handle thumbnail urls
2017-06-19 17:48:28 +01:00
Erik Johnston
b8b936a6ea
Add API to quarantine media
2017-06-19 17:39:21 +01:00
Erik Johnston
d7fe6b356c
Add shutdown room API
2017-06-19 12:37:27 +01:00
Erik Johnston
dfeca6cf40
Merge pull request #2286 from matrix-org/erikj/split_out_user_dir
...
Split out user directory to a separate process
2017-06-16 13:01:19 +01:00
Erik Johnston
6aa5bc8635
Initial worker impl
2017-06-16 11:47:11 +01:00
Erik Johnston
d8f47d2efa
Merge pull request #2280 from matrix-org/erikj/share_room_user_dir
...
Include users who you share a room with in user directory
2017-06-16 11:06:00 +01:00
Erik Johnston
052c5d19d5
Merge pull request #2281 from matrix-org/erikj/phone_home_stats
...
Fix phone home stats
2017-06-15 12:46:23 +01:00
Erik Johnston
a9d6fa8b2b
Include users who share room with requester in user directory
2017-06-15 10:17:21 +01:00
Erik Johnston
4564b05483
Implement updating users who share rooms on the fly
2017-06-15 10:17:17 +01:00
Erik Johnston
72613bc379
Implement initial population of users who share rooms table
2017-06-15 09:59:04 +01:00
Erik Johnston
ebcd55d641
Add DB schema for tracking users who share rooms
2017-06-15 09:45:48 +01:00
Erik Johnston
4b461a6931
Add some more stats
2017-06-15 09:39:39 +01:00
Erik Johnston
617304b2cf
Fix phone home stats
2017-06-14 19:47:15 +01:00
Erik Johnston
d9fd937e39
Fix user directory insertion due to missing room_id
2017-06-13 11:50:24 +01:00
Erik Johnston
505e7e8b9d
Fix up sql
2017-06-13 11:19:18 +01:00
Erik Johnston
6fd7e6db3d
Fix user dir to not assume existence of user
2017-06-13 11:11:26 +01:00
Erik Johnston
fdca6e36ee
Merge pull request #2274 from matrix-org/erikj/cache_is_host_joined
...
Add cache for is_host_joined
2017-06-13 10:58:53 +01:00
Erik Johnston
b58e24cc3c
Tweak the ranking of PG user dir search
2017-06-13 10:16:31 +01:00
Erik Johnston
d53fe399eb
Add cache for is_host_joined
2017-06-13 09:56:18 +01:00
Erik Johnston
a837765e8c
Merge pull request #2266 from matrix-org/erikj/host_in_room
...
Change is_host_joined to use current_state table
2017-06-12 09:49:51 +01:00
Erik Johnston
8060974344
Fix replication
2017-06-09 16:40:52 +01:00
Erik Johnston
b0d975e216
Comments
2017-06-09 16:25:42 +01:00
Erik Johnston
e54d7d536e
Cache state deltas
2017-06-09 16:24:00 +01:00
Erik Johnston
935e588799
Tweak SQL
2017-06-09 13:01:23 +01:00
Erik Johnston
0185b75381
Change is_host_joined to use current_state table
...
This bypasses a bug where using the state groups to figure out if a host
is in a room sometimes errors if the servers isn't in the room. (For
example when the server rejected an invite to a remote room)
2017-06-09 10:52:26 +01:00
Erik Johnston
ea11ee09f3
Ensure we don't use unpersisted state group as prev group
2017-06-08 11:59:57 +01:00
Erik Johnston
c62c480dc6
Merge pull request #2259 from matrix-org/erikj/fix_state_woes
...
Fix bug where state_group tables got corrupted
2017-06-07 17:51:25 +01:00
Erik Johnston
197bd126f0
Fix bug where state_group tables got corrupted
...
This is due to the fact that we prefilled caches using txn.call_after,
which always gets called including on error.
We fix this by making txn.call_after only fire when a transaction
completes successfully, which is what we want most of the time anyway.
2017-06-07 17:39:36 +01:00
Erik Johnston
a053ff3979
Merge pull request #2248 from matrix-org/erikj/state_fixup
...
Faster cache for get_joined_hosts
2017-06-07 14:01:06 +01:00
Erik Johnston
64ed74c01e
When pruning, delete from device_lists_outbound_last_success
2017-06-07 11:20:47 +01:00
Erik Johnston
1a81a1898e
Keep pruning background task
2017-06-07 11:16:56 +01:00
Erik Johnston
6ba21bf2b8
Comments
2017-06-07 11:08:36 +01:00
Erik Johnston
09e4bc0501
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/state_fixup
2017-06-07 11:05:23 +01:00
Erik Johnston
6e2a7ee1bc
Remove spurious log lines
2017-06-07 11:05:17 +01:00
Erik Johnston
65f0513a33
Split up device_lists_outbound_pokes table for faster updates.
2017-06-07 11:02:38 +01:00
Erik Johnston
6f83c4537c
Increase size of IP cache
2017-06-07 10:18:44 +01:00
Erik Johnston
1a01af079e
Handle profile updates in user directory
2017-06-01 15:39:51 +01:00
Erik Johnston
4d039aa2ca
Fix sqlite
2017-06-01 14:58:48 +01:00
Erik Johnston
21e255a8f1
Split the table in two
2017-06-01 14:50:46 +01:00
Erik Johnston
d5477c7afd
Tweak search query
2017-06-01 13:28:01 +01:00
Erik Johnston
02a6108235
Tweak search query
2017-06-01 13:16:40 +01:00
Erik Johnston
59dbb47065
Remove spurious inlineCallbacks
2017-06-01 11:41:29 +01:00
Erik Johnston
0fe6f3c521
Bug fixes and logging
...
- Check if room is public when a user joins before adding to user dir
- Fix typo of field name "content.join_rules" -> "content.join_rule"
2017-06-01 11:09:49 +01:00
Erik Johnston
036362ede6
Order by if they have profile info
2017-06-01 09:41:08 +01:00
Erik Johnston
a757dd4863
Use prefix matching
2017-06-01 09:40:37 +01:00
Erik Johnston
f5cc22bdc6
Comment on why arbitrary comments
2017-05-31 17:30:26 +01:00
Erik Johnston
5dd1b2c525
Use unique indices
2017-05-31 17:29:12 +01:00
Erik Johnston
f1378aef91
Convert to int
2017-05-31 17:03:08 +01:00
Erik Johnston
f091061711
Fix tests
2017-05-31 16:34:40 +01:00
Erik Johnston
4abcff0177
Fix typo
2017-05-31 16:22:36 +01:00
Erik Johnston
63c58c2a3f
Limit number of things we fetch out of the db
2017-05-31 16:17:58 +01:00
Erik Johnston
304880d185
Add stream change cache
2017-05-31 15:46:36 +01:00
Erik Johnston
5d79d728f5
Split out directory and search tables
2017-05-31 15:23:49 +01:00
Erik Johnston
dc51af3d03
Pull max id from correct table
2017-05-31 15:13:49 +01:00
Erik Johnston
350622a107
Handle the server leaving a public room
2017-05-31 15:11:36 +01:00
Erik Johnston
63fda37e20
Add comments
2017-05-31 15:00:29 +01:00
Erik Johnston
293ef29655
Weight differently
2017-05-31 14:29:32 +01:00
Erik Johnston
3b5f22ca40
Add search
2017-05-31 14:00:01 +01:00
Erik Johnston
b5db4ed5f6
Update room column when room becomes unpublic
2017-05-31 13:40:28 +01:00
Erik Johnston
eeb2f9e546
Add user_directory to database
2017-05-31 11:51:01 +01:00
Erik Johnston
dd48f7204c
Add comment
2017-05-30 15:01:22 +01:00
Erik Johnston
04095f7581
Add clobbered event_id
2017-05-30 14:53:01 +01:00
Erik Johnston
a584a81b3e
Add current_state_delta_stream table
2017-05-30 14:44:09 +01:00
Erik Johnston
619e8ecd0c
Handle None state group correctly
2017-05-26 10:46:03 +01:00
Erik Johnston
dfbda5e025
Faster cache for get_joined_hosts
2017-05-25 17:24:44 +01:00
Erik Johnston
11f139a647
Merge pull request #2247 from matrix-org/erikj/auth_event
...
Only store event_auth for state events
2017-05-24 16:46:34 +01:00
Erik Johnston
6e614e9e10
Add background task to clear out old event_auth
2017-05-24 15:23:34 +01:00
Erik Johnston
c049472b8a
Only store event_auth for state events
2017-05-24 15:23:31 +01:00
Erik Johnston
8cf9f0a3e7
Remove redundant invalidation
2017-05-23 09:46:59 +01:00
Erik Johnston
e6618ece2d
Missed an invalidation
2017-05-23 09:36:52 +01:00
Erik Johnston
74bf4ee7bf
Stream count_e2e_one_time_keys cache invalidation
2017-05-22 16:19:22 +01:00
Erik Johnston
58ebb96cce
Fix invalidation of get_users_with_read_receipts_in_room
2017-05-19 14:38:50 +01:00
Erik Johnston
107ac7ac96
Increase size of push rule caches
2017-05-18 17:17:53 +01:00
Erik Johnston
760625acba
Make get_if_app_services_interested_in_user faster
2017-05-18 16:34:44 +01:00
Erik Johnston
d9e3a4b5db
Merge pull request #2230 from matrix-org/erikj/speed_up_get_state
...
Make get_state_groups_from_groups faster.
2017-05-17 17:23:04 +01:00
Erik Johnston
ace23463c5
Merge pull request #2216 from slipeer/app_services_interested_in_user
...
Fix users claimed non-exclusively by an app service don't get notific…
2017-05-17 16:28:50 +01:00
Erik Johnston
bbfe4e996c
Make get_state_groups_from_groups faster.
...
Most of the time was spent copying a dict to filter out sentinel values
that indicated that keys did not exist in the dict. The sentinel values
were added to ensure that we cached the non-existence of keys.
By updating DictionaryCache to keep track of which keys were known to
not exist itself we can remove a dictionary copy.
2017-05-17 15:12:15 +01:00
Erik Johnston
85e8092cca
Comment
2017-05-17 10:03:09 +01:00
Erik Johnston
ad53fc3cf4
Short circuit when we have delta ids
2017-05-17 09:57:34 +01:00
