Commit Graph

5839 Commits

Author SHA1 Message Date
Erik Johnston
ebfaff84c9 Merge pull request #1870 from matrix-org/erikj/cache_get_all_new_events
Add a small cache get_all_new_events
2017-02-01 13:22:02 +00:00
Erik Johnston
73d676dc8b Comment 2017-02-01 13:17:17 +00:00
Erik Johnston
62f6b86ba7 Merge pull request #1868 from matrix-org/erikj/replication_cache
Only invalidate membership caches based on the cache stream
2017-02-01 13:12:30 +00:00
Erik Johnston
f6124311fd Add m.room.member type to query 2017-02-01 11:59:17 +00:00
Erik Johnston
88a4d54883 Merge pull request #1867 from matrix-org/erikj/member_index
Add an index to make membership queries faster
2017-02-01 11:44:27 +00:00
Erik Johnston
368c88c487 Add a small cache get_all_new_events 2017-02-01 10:50:44 +00:00
Erik Johnston
5deaf9e30b Up get_latest_event_ids_in_room cache 2017-02-01 10:39:41 +00:00
Erik Johnston
acb501c46d Comment 2017-02-01 10:32:49 +00:00
Erik Johnston
97479d0c54 Implement /keys/changes 2017-02-01 10:30:03 +00:00
Erik Johnston
06567ec513 Merge pull request #1866 from matrix-org/erikj/device_list_fixes
Better handle 404 response for federation /send/
2017-02-01 09:44:14 +00:00
Erik Johnston
458b6f4733 Only invalidate membership caches based on the cache stream
Before we completely invalidated get_users_in_room whenever we updated
any current_state_events table. This was way too aggressive.
2017-01-31 16:09:03 +00:00
Erik Johnston
fe08db2713 Remove explicit < 400 check as apparently this is confusing 2017-01-31 15:21:32 +00:00
Erik Johnston
21b7375778 Add an index to make membership queries faster 2017-01-31 15:15:57 +00:00
Erik Johnston
4c0ec15bdc Comment 2017-01-31 13:53:46 +00:00
Erik Johnston
85c590105f Comment 2017-01-31 13:46:38 +00:00
Erik Johnston
ae7a132f38 Better handle 404 response for federation /send/ 2017-01-31 13:40:09 +00:00
Erik Johnston
ac001dabdc Merge pull request #1864 from matrix-org/erikj/device_list_fixes
Fix clearing out old device list outbound pokes
2017-01-31 13:35:35 +00:00
Erik Johnston
bfb3d255b1 Merge pull request #1862 from matrix-org/erikj/presence_update
Use DB cache of joined users for presence
2017-01-31 13:23:24 +00:00
Erik Johnston
ab55794b6f Fix deletion of old sent devices correctly 2017-01-31 13:22:41 +00:00
Erik Johnston
d3169e8d28 Only fetch with row ts and count > 1 2017-01-31 11:20:03 +00:00
Erik Johnston
05b9f48ee5 Fix clearing out old device list outbound pokes 2017-01-31 10:08:55 +00:00
Erik Johnston
4b3403ca9b Stream cache invalidations for room membership storage functions 2017-01-30 17:28:22 +00:00
Erik Johnston
1c13c9f6b6 Don't have such a large cache 2017-01-30 17:12:14 +00:00
Erik Johnston
c7a26b7c32 Fix unit tests 2017-01-30 17:11:24 +00:00
Erik Johnston
fd1c18c088 Use DB cache of joined users for presence 2017-01-30 17:00:24 +00:00
Erik Johnston
c2c9a78db9 Noop device key changes if they're the same 2017-01-30 16:55:04 +00:00
Erik Johnston
e75a779d9e Fix query 2017-01-30 16:38:20 +00:00
Erik Johnston
828db669ec Use get_users_in_room and declare it iterable 2017-01-30 16:37:22 +00:00
Erik Johnston
9636b2407d Merge pull request #1857 from matrix-org/erikj/device_list_stream
Implement device lists updates over federation
2017-01-30 14:35:21 +00:00
Erik Johnston
3670025e64 Rename func 2017-01-30 14:11:31 +00:00
Erik Johnston
4ac363a168 Remove debug logging 2017-01-30 14:10:12 +00:00
Erik Johnston
d360c97ae1 Clear out old destination pokes. 2017-01-30 10:14:37 +00:00
Erik Johnston
76100203ab Always use the latest stream_id, sent or unsent 2017-01-30 10:14:25 +00:00
Erik Johnston
d1e1fd6210 Add ts column to device_lists_outbound_pokes 2017-01-27 15:23:48 +00:00
Erik Johnston
252b503fc8 Hook device list updates to replication 2017-01-27 14:31:35 +00:00
Erik Johnston
84a35f32c7 Comment 2017-01-27 10:35:12 +00:00
Erik Johnston
c517a19c2d Comment 2017-01-27 10:33:26 +00:00
Erik Johnston
738a2867c8 SQL param ordering 2017-01-27 10:31:29 +00:00
Erik Johnston
755adff0e4 User if rather than for 2017-01-27 10:31:06 +00:00
Erik Johnston
888c59c955 Better name 2017-01-27 10:29:47 +00:00
Erik Johnston
f25a4a4692 Remove unused param 2017-01-27 10:27:39 +00:00
Erik Johnston
31aca5589c Fix on sqlite: use left rather than outer join 2017-01-26 16:55:50 +00:00
Erik Johnston
76d40f4904 Handle users leaving rooms 2017-01-26 16:39:33 +00:00
Erik Johnston
fbfad76c03 Add comments 2017-01-26 16:33:21 +00:00
Erik Johnston
c974116f19 Implement device key caching over federation 2017-01-26 16:07:24 +00:00
Erik Johnston
51e9fe36e4 Fix up sending of m.device_list_update edus 2017-01-25 16:55:21 +00:00
Erik Johnston
2367c5568c Add basic implementation of local device list changes 2017-01-25 14:27:27 +00:00
Paul "LeoNerd" Evans
10e48d8310 Don't clobber a displayname or avatar_url if provided by an m.room.member event 2017-01-24 18:06:07 +00:00
Erik Johnston
ba8e144554 Merge branch 'erikj/current_state_fix' into develop 2017-01-23 16:15:10 +00:00
Erik Johnston
f5b46482f4 Merge pull request #1840 from matrix-org/erikj/current_state_fix
Insert delta of current_state_events to be more efficient
2017-01-23 16:14:34 +00:00
Erik Johnston
fdf2a31a51 Typo 2017-01-23 16:14:14 +00:00
Erik Johnston
c77b24c092 Refactor to calculate state delta outside transaction 2017-01-23 14:51:33 +00:00
Erik Johnston
5d2134d485 Comments 2017-01-20 17:13:24 +00:00
Erik Johnston
a55fa2047f Insert delta of current_state_events to be more efficient 2017-01-20 17:10:18 +00:00
Erik Johnston
3d9d48fffb Merge pull request #1836 from matrix-org/erikj/current_state_fix
Derive current_state_events from state groups
2017-01-20 15:14:05 +00:00
Erik Johnston
d0897dead5 Spelling 2017-01-20 15:05:11 +00:00
Erik Johnston
567aa35b67 Update all call sites after rename 2017-01-20 14:40:31 +00:00
Erik Johnston
f2f40e64a9 Comments 2017-01-20 14:38:13 +00:00
Erik Johnston
4c6a31cd6e Calculate the forward extremeties once 2017-01-20 14:28:53 +00:00
Erik Johnston
86063d4321 Merge pull request #1835 from matrix-org/erikj/fix_workers
Make worker listener config backwards compat
2017-01-20 11:55:56 +00:00
Erik Johnston
09eb08f910 Derive current_state_events from state groups 2017-01-20 11:52:51 +00:00
Erik Johnston
97efe99ae9 Make worker listener config backwards compat 2017-01-20 11:45:29 +00:00
Marvin Steadfast
86e6165687 Added default config for turn username and password 2017-01-19 14:35:55 +01:00
Marvin Steadfast
1e38be3a7a Added username and password for turn server
It makes it possible to use a turn server that needs a username and
password instead of a token.
2017-01-19 14:08:20 +01:00
Erik Johnston
841c228533 Merge pull request #1828 from matrix-org/erikj/iterable_cache_size
Update LruCache size estimate on clear
2017-01-18 14:57:54 +00:00
Erik Johnston
c430111d0e Update LruCache size estimate on clear 2017-01-18 14:55:23 +00:00
David Baker
97d3918377 Merge pull request #1811 from aperezdc/unhardcode-riot-urls
Allow configuring the Riot URL used in notification emails
2017-01-18 14:38:49 +00:00
David Baker
8c5009b628 Lowercase all email addresses before querying db
Since we store all emails in the DB in lowercase
(https://github.com/matrix-org/synapse/pull/1170)
2017-01-18 13:25:56 +00:00
Erik Johnston
ae7b4da4cc Merge pull request #1823 from matrix-org/erikj/load_events_logs
Remove loading events logs
2017-01-18 11:07:58 +00:00
Erik Johnston
fc7cae8aa3 Merge pull request #1824 from matrix-org/erikj/retry_host_log
Lower the not retrying host log line to debug
2017-01-18 11:07:51 +00:00
Erik Johnston
f9058ca785 Merge pull request #1822 from matrix-org/erikj/statE_logging
Change resolve_state_groups call site logging to DEBUG
2017-01-18 11:02:03 +00:00
Erik Johnston
f648313f98 Merge pull request #1821 from matrix-org/erikj/cache_metrics_string_intern
Measure metrics of string_cache
2017-01-18 10:57:39 +00:00
Erik Johnston
15f012032c Merge pull request #1818 from matrix-org/erikj/state_auth_splitout_split
Optimise state resolution
2017-01-18 10:53:00 +00:00
Erik Johnston
4ec1cf49e2 Lower loading events log to DEBUG 2017-01-17 17:28:32 +00:00
Erik Johnston
f878f64f43 Lower the not retrying host log line to debug 2017-01-17 17:20:39 +00:00
Erik Johnston
5f027d1fc5 Change resolve_state_groups call site logging to DEBUG 2017-01-17 17:07:15 +00:00
Erik Johnston
380dba1020 Measure metrics of string_cache 2017-01-17 17:04:46 +00:00
Erik Johnston
ed4d176152 PEP8 2017-01-17 15:27:28 +00:00
Mark Haines
c6064a7ba6 Only construct sets when necessary 2017-01-17 15:23:07 +00:00
Erik Johnston
a8594fd19f Use better names 2017-01-17 14:59:03 +00:00
Erik Johnston
7fae460402 Merge pull request #1820 from matrix-org/erikj/push_tools
Get state at event rather than for room in push
2017-01-17 14:58:26 +00:00
Erik Johnston
37b4c7d8a9 Fix typo in return type 2017-01-17 14:43:32 +00:00
Erik Johnston
e5d2df9c34 Use better variable name 2017-01-17 14:32:53 +00:00
Erik Johnston
04006bb7f0 Get state at event rather than for room in push 2017-01-17 14:31:21 +00:00
Erik Johnston
ce59a2faad Correctly handle case of rejected events in state res 2017-01-17 14:18:53 +00:00
Erik Johnston
633f97151c Check event is in state_map 2017-01-17 13:33:54 +00:00
Erik Johnston
e6153e1bd1 Fix couple of federation state bugs 2017-01-17 13:22:34 +00:00
Erik Johnston
5d6bad1b3c Optimise state resolution 2017-01-17 13:22:19 +00:00
Erik Johnston
e8ecbb6f20 Merge pull request #1812 from matrix-org/erikj/state_auth_splitout_split
Split out static state methods from StateHandler
2017-01-17 11:55:18 +00:00
Erik Johnston
d6c75cb7c2 Rename and comment tree_to_leaves_iterator 2017-01-17 11:47:03 +00:00
Erik Johnston
1ccd5676e3 Remove needless call to evict() 2017-01-17 11:42:26 +00:00
Erik Johnston
d906206049 Increase state_group_cache_size 2017-01-17 11:31:08 +00:00
Erik Johnston
f85b6ca494 Speed up cache size calculation
Instead of calculating the size of the cache repeatedly, which can take
a long time now that it can use a callback, instead cache the size and
update that on insertion and deletion.

This requires changing the cache descriptors to have two caches, one for
pending deferreds and the other for the actual values. There's no reason
to evict from the pending deferreds as they won't take up any more
memory.
2017-01-17 11:18:13 +00:00
Erik Johnston
6d00213e80 Use OrderedDict in ExpiringCache 2017-01-16 15:33:22 +00:00
Erik Johnston
897f8752da Up cache max entries for state 2017-01-16 15:08:17 +00:00
Erik Johnston
beda469bc6 Put staticmethods at module level 2017-01-16 15:05:24 +00:00
Erik Johnston
46aebbbcbf Add support for 'iterable' to ExpiringCache 2017-01-16 14:57:23 +00:00
Erik Johnston
01521299c7 Increase cache size limit 2017-01-16 11:56:51 +00:00
Erik Johnston
2fae34bd2c Optionally measure size of cache by sum of length of values 2017-01-13 17:46:17 +00:00
Erik Johnston
95a22ae194 Merge pull request #1810 from matrix-org/erikj/state_auth_splitout_split
Split out static auth methods from Auth object
2017-01-13 16:32:27 +00:00