Commit Graph

208 Commits

Author SHA1 Message Date
Erik Johnston
935e588799 Tweak SQL 2017-06-09 13:01:23 +01:00
Erik Johnston
0185b75381 Change is_host_joined to use current_state table
This bypasses a bug where using the state groups to figure out if a host
is in a room sometimes errors if the servers isn't in the room. (For
example when the server rejected an invite to a remote room)
2017-06-09 10:52:26 +01:00
Erik Johnston
6ba21bf2b8 Comments 2017-06-07 11:08:36 +01:00
Erik Johnston
6e2a7ee1bc Remove spurious log lines 2017-06-07 11:05:17 +01:00
Erik Johnston
619e8ecd0c Handle None state group correctly 2017-05-26 10:46:03 +01:00
Erik Johnston
dfbda5e025 Faster cache for get_joined_hosts 2017-05-25 17:24:44 +01:00
Erik Johnston
85e8092cca Comment 2017-05-17 10:03:09 +01:00
Erik Johnston
ad53fc3cf4 Short circuit when we have delta ids 2017-05-17 09:57:34 +01:00
Erik Johnston
13f540ef1b Speed up get_joined_hosts 2017-05-16 16:05:22 +01:00
Erik Johnston
ffad4fe35b Don't update event cache hit ratio from get_joined_users
Otherwise the hit ration of plain get_events gets completely skewed by
calls to get_joined_users* functions.
2017-05-08 16:06:17 +01:00
Erik Johnston
66f371e8b8 Merge pull request #2176 from matrix-org/erikj/faster_get_joined
Make get_joined_users faster
2017-05-05 10:59:55 +01:00
Erik Johnston
e3aa8a7aa8 Merge pull request #2185 from matrix-org/erikj/smaller_caches
Optimise caches for single key
2017-05-05 10:19:05 +01:00
Erik Johnston
cf589f2c1e Fixes 2017-05-05 10:17:56 +01:00
Erik Johnston
aa93cb9f44 Add comment 2017-05-04 14:59:28 +01:00
Erik Johnston
537dbadea0 Intern host strings 2017-05-04 14:55:28 +01:00
Erik Johnston
07a07588a0 Make caches bigger 2017-05-04 14:52:28 +01:00
Erik Johnston
dfaa58f72d Fix comment and num args 2017-05-04 14:50:24 +01:00
Erik Johnston
5d8290429c Reduce size of get_users_in_room 2017-05-04 13:43:19 +01:00
Erik Johnston
7ebf518c02 Make get_joined_users faster 2017-05-03 15:55:54 +01:00
Erik Johnston
7166854f41 Add cache for get_current_hosts_in_room 2017-05-02 10:36:35 +01:00
Erik Johnston
f144365281 Comment 2017-04-25 15:18:26 +01:00
Erik Johnston
d9aa645f86 Reduce size of joined_user cache
The _get_joined_users_from_context cache stores a mapping from user_id
to avatar_url and display_name. Instead of storing those in a dict,
store them in a namedtuple as that uses much less memory.

We also try converting the string to ascii to further reduce the size.
2017-04-25 14:38:51 +01:00
Erik Johnston
7fc1f1e2b6 Cache hosts in room 2017-03-24 11:46:24 +00:00
Erik Johnston
d58b1ffe94 Replace some calls to cursor_to_dict
cursor_to_dict can be surprisinglh expensive for large result sets, so lets
only call it when we need to.
2017-03-24 11:07:02 +00:00
Erik Johnston
da146657c9 Comments 2017-03-16 13:04:07 +00:00
Erik Johnston
6957bfdca6 Don't recreate so many sets 2017-03-16 11:54:26 +00:00
Erik Johnston
38258a0976 Bump cache sizes for common membership queries 2017-02-02 18:45:55 +00:00
Erik Johnston
832e9c52ca Comment 2017-02-02 13:09:56 +00:00
Erik Johnston
54a79c1d37 Make presence.get_new_events a bit faster
We do this by caching the set of users a user shares rooms with.
2017-02-02 13:07:18 +00:00
Erik Johnston
62f6b86ba7 Merge pull request #1868 from matrix-org/erikj/replication_cache
Only invalidate membership caches based on the cache stream
2017-02-01 13:12:30 +00:00
Erik Johnston
f6124311fd Add m.room.member type to query 2017-02-01 11:59:17 +00:00
Erik Johnston
458b6f4733 Only invalidate membership caches based on the cache stream
Before we completely invalidated get_users_in_room whenever we updated
any current_state_events table. This was way too aggressive.
2017-01-31 16:09:03 +00:00
Erik Johnston
21b7375778 Add an index to make membership queries faster 2017-01-31 15:15:57 +00:00
Erik Johnston
1c13c9f6b6 Don't have such a large cache 2017-01-30 17:12:14 +00:00
Erik Johnston
828db669ec Use get_users_in_room and declare it iterable 2017-01-30 16:37:22 +00:00
Erik Johnston
897f8752da Up cache max entries for state 2017-01-16 15:08:17 +00:00
Erik Johnston
01521299c7 Increase cache size limit 2017-01-16 11:56:51 +00:00
Erik Johnston
2fae34bd2c Optionally measure size of cache by sum of length of values 2017-01-13 17:46:17 +00:00
Matthew Hodgson
8cfc0165e9 fix annoying typos 2017-01-05 13:39:43 +00:00
Erik Johnston
c3208e45c9 Fixup membership query 2016-12-14 10:46:58 +00:00
Erik Johnston
7ec2bf9b77 Fix background update that prematurely stopped 2016-12-12 16:54:58 +00:00
Erik Johnston
52d12ca782 Add /room/<room_id>/joined_members API
This returns the currently joined members in the room with their display
names and avatar urls. This is more efficient than /members for large
rooms where you don't need the full events.
2016-12-08 13:32:07 +00:00
Erik Johnston
c45d8e9ba2 Add profile data to the room_membership table for joins 2016-12-08 13:08:41 +00:00
Erik Johnston
2ffec928e2 Reduce batch size to be under SQL limit 2016-09-09 11:03:31 +01:00
Erik Johnston
c10cb581c6 Correctly handle the difference between prev and current state 2016-08-31 14:26:22 +01:00
Erik Johnston
bed10f9880 Use state handler instead of get_users_in_room/get_joined_hosts 2016-08-26 14:54:30 +01:00
Erik Johnston
4daa397a00 Add is_host_joined to slave storage 2016-08-26 13:10:56 +01:00
Erik Johnston
1ccdc1e93a Cache check_host_in_room 2016-08-26 10:59:40 +01:00
Erik Johnston
721414d98a Add desc 2016-08-25 17:49:05 +01:00
Erik Johnston
a3dc1e9cbe Replace context.current_state with context.current_state_ids 2016-08-25 17:32:22 +01:00
Erik Johnston
4d70d1f80e Add some invalidations to a cache_stream 2016-08-15 11:15:17 +01:00
Mark Haines
0b2158719c Remove dead code.
Loading push rules now happens in the datastore, so we can remove
the methods that loaded them outside the datastore.

The ``waiting_for_join_list`` in federation handler is populated by
anything, so can be removed.

The ``_get_members_events_txn`` method isn't called from anywhere
so can be removed.
2016-06-07 15:07:11 +01:00
Erik Johnston
ccb56fc24b Make get_joined_hosts_for_room use get_users_in_room 2016-06-03 11:20:23 +01:00
Erik Johnston
c8285564a3 Use state to calculate get_users_in_room 2016-06-01 15:25:25 +01:00
Mark Haines
816df9f267 get_room_members is unused now 2016-05-16 19:51:43 +01:00
Mark Haines
1a3a2002ff Spell "domain" correctly
s/domian/domain/g
2016-05-16 19:17:23 +01:00
Erik Johnston
08dfa8eee2 Add and use get_domian_from_id 2016-05-09 10:36:03 +01:00
David Baker
2547dffccc Merge pull request #705 from matrix-org/dbkr/pushers_use_event_actions
Change pushers to use the event_actions table
2016-04-11 12:58:55 +01:00
Mark Haines
7e2f971c08 Remove some unused functions (#711)
* Remove some unused functions

* get_room_events_stream is only used in tests

* is_exclusive_room might actually be something we want
2016-04-08 14:01:56 +01:00
David Baker
0fd1cd2400 pep8 2016-04-06 16:50:47 +01:00
David Baker
7e2c89a37f Make pushers use the event_push_actions table instead of listening on an event stream & running the rules again. Sytest passes, but remaining to do:
* Make badges work again
 * Remove old, unused code
2016-04-06 15:42:15 +01:00
Erik Johnston
df727f2126 Fix stuck invites
If rejecting a remote invite fails with an error response don't fail
the entire request; instead mark the invite as locally rejected.

This fixes the bug where users can get stuck invites which they can
neither accept nor reject.
2016-04-05 11:13:24 +01:00
Erik Johnston
0c53d750e7 Docs and indents 2016-04-04 18:02:48 +01:00
Erik Johnston
92ab45a330 Add upgrade path, rename table 2016-04-04 17:07:43 +01:00
Erik Johnston
3d76b7cb2b Store invites in a separate table. 2016-04-04 16:30:15 +01:00
Erik Johnston
b6507869cd Make get_invites return RoomsForUser 2016-03-23 10:32:10 +00:00
Mark Haines
673c96ce97 Remove dead code left over from presence changes 2016-03-17 11:03:47 +00:00
Erik Johnston
25c311eaf6 Cache get_room_changes_for_user 2016-01-29 16:52:48 +00:00
Erik Johnston
c5e7c0e436 Up get_rooms_for_user cache size 2016-01-28 09:58:45 +00:00
Erik Johnston
53cb173663 Push: Use storage apis that are cached 2016-01-25 13:55:18 +00:00
Erik Johnston
cc66a9a5e3 Allow filtering events for multiple users at once 2016-01-18 14:43:50 +00:00
Matthew Hodgson
6c28ac260c copyrights 2016-01-07 04:26:29 +00:00
Daniel Wagner-Hall
76e69cc8de Fix typo 2015-12-14 12:38:55 +00:00
Mark Haines
7d6b313312 Add caches for whether a room has been forgotten by a user 2015-12-10 17:49:34 +00:00
Daniel Wagner-Hall
df7cf6c0eb Fix SQL for postgres again 2015-11-23 18:54:41 +00:00
Daniel Wagner-Hall
3e573a5c6b Fix SQL for postgres 2015-11-23 18:48:53 +00:00
Daniel Wagner-Hall
df6824a008 Ignore forgotten rooms in v2 sync 2015-11-19 14:54:47 -05:00
Daniel Wagner-Hall
9da4c5340d Simplify code 2015-11-19 10:07:21 -05:00
Daniel Wagner-Hall
bed7889703 Apply forgetting properly to historical events 2015-11-18 18:11:12 -05:00
Daniel Wagner-Hall
ba26eb3d5d Allow users to forget rooms 2015-11-17 17:17:30 -05:00
Mark Haines
e3d75f564a Include banned rooms in the archived section of v2 sync 2015-10-21 11:15:48 +01:00
Mark Haines
68b7fc3e2b Add rooms that the user has left under archived in v2 sync. 2015-10-19 17:26:18 +01:00
Mark Haines
cacf0688c6 Add a get_invites_for_user method to the storage to find out the rooms a user is invited to 2015-10-13 14:08:38 +01:00
Mark Haines
301141515a Merge pull request #288 from matrix-org/markjh/unused_definitions
Remove some of the unused definitions from synapse
2015-09-28 14:22:44 +01:00
Mark Haines
cf1100887b Fix order of ON constraints in _get_rooms_for_user_where_membership_is_txn 2015-09-24 17:35:10 +01:00
Mark Haines
e51aa4be96 synapse/storage/roommember.py:_get_members_query was unused 2015-09-23 10:35:10 +01:00
Mark Haines
417485eefa Include the event_id and stream_ordering of membership events when looking up which rooms a user is in 2015-09-08 18:14:54 +01:00
Erik Johnston
2df8dd9b37 Move all the caches into their own package, synapse.util.caches 2015-08-11 18:00:59 +01:00
Erik Johnston
b8e386db59 Change Cache to not use *args in its interface 2015-08-07 11:52:21 +01:00
Erik Johnston
1e62a3d3a9 Up the cache size for 'get_joined_hosts_for_room' and 'get_users_in_room' 2015-08-05 15:40:40 +01:00
Erik Johnston
5130d80d79 Add bulk insert events API 2015-06-25 17:29:34 +01:00
Erik Johnston
4429e720ae Merge branch 'master' of github.com:matrix-org/synapse into develop 2015-05-22 10:33:00 +01:00
Mark Haines
53447e9cd3 Add caches for things requested by the pushers 2015-05-21 16:41:39 +01:00
Erik Johnston
f6f902d459 Move fetching of events into their own transactions 2015-05-14 13:45:48 +01:00
Mark Haines
d18f37e026 Collect the invalidate callbacks on the transaction object rather than passing around a separate list 2015-05-05 17:32:21 +01:00
Mark Haines
a9aea68fd5 Invalidate the caches from the correct thread 2015-05-05 14:57:08 +01:00
Erik Johnston
61d05daab1 More join conditions 2015-04-30 18:41:05 +01:00
Erik Johnston
6ead27ddda Add more conditions on JOINs to make postgres go a little faster. 2015-04-30 18:32:03 +01:00
Erik Johnston
69d4063651 Add get_rooms_for_user cache 2015-04-30 16:47:51 +01:00
Erik Johnston
0bfa78b39b PEP8 2015-04-07 12:16:05 +01:00