Erik Johnston
7ebf518c02
Make get_joined_users faster
2017-05-03 15:55:54 +01:00
Erik Johnston
7166854f41
Add cache for get_current_hosts_in_room
2017-05-02 10:36:35 +01:00
Erik Johnston
f144365281
Comment
2017-04-25 15:18:26 +01:00
Erik Johnston
d9aa645f86
Reduce size of joined_user cache
...
The _get_joined_users_from_context cache stores a mapping from user_id
to avatar_url and display_name. Instead of storing those in a dict,
store them in a namedtuple as that uses much less memory.
We also try converting the string to ascii to further reduce the size.
2017-04-25 14:38:51 +01:00
Erik Johnston
7fc1f1e2b6
Cache hosts in room
2017-03-24 11:46:24 +00:00
Erik Johnston
d58b1ffe94
Replace some calls to cursor_to_dict
...
cursor_to_dict can be surprisinglh expensive for large result sets, so lets
only call it when we need to.
2017-03-24 11:07:02 +00:00
Erik Johnston
da146657c9
Comments
2017-03-16 13:04:07 +00:00
Erik Johnston
6957bfdca6
Don't recreate so many sets
2017-03-16 11:54:26 +00:00
Erik Johnston
38258a0976
Bump cache sizes for common membership queries
2017-02-02 18:45:55 +00:00
Erik Johnston
832e9c52ca
Comment
2017-02-02 13:09:56 +00:00
Erik Johnston
54a79c1d37
Make presence.get_new_events a bit faster
...
We do this by caching the set of users a user shares rooms with.
2017-02-02 13:07:18 +00:00
Erik Johnston
62f6b86ba7
Merge pull request #1868 from matrix-org/erikj/replication_cache
...
Only invalidate membership caches based on the cache stream
2017-02-01 13:12:30 +00:00
Erik Johnston
f6124311fd
Add m.room.member type to query
2017-02-01 11:59:17 +00:00
Erik Johnston
458b6f4733
Only invalidate membership caches based on the cache stream
...
Before we completely invalidated get_users_in_room whenever we updated
any current_state_events table. This was way too aggressive.
2017-01-31 16:09:03 +00:00
Erik Johnston
21b7375778
Add an index to make membership queries faster
2017-01-31 15:15:57 +00:00
Erik Johnston
1c13c9f6b6
Don't have such a large cache
2017-01-30 17:12:14 +00:00
Erik Johnston
828db669ec
Use get_users_in_room and declare it iterable
2017-01-30 16:37:22 +00:00
Erik Johnston
897f8752da
Up cache max entries for state
2017-01-16 15:08:17 +00:00
Erik Johnston
01521299c7
Increase cache size limit
2017-01-16 11:56:51 +00:00
Erik Johnston
2fae34bd2c
Optionally measure size of cache by sum of length of values
2017-01-13 17:46:17 +00:00
Matthew Hodgson
8cfc0165e9
fix annoying typos
2017-01-05 13:39:43 +00:00
Erik Johnston
c3208e45c9
Fixup membership query
2016-12-14 10:46:58 +00:00
Erik Johnston
7ec2bf9b77
Fix background update that prematurely stopped
2016-12-12 16:54:58 +00:00
Erik Johnston
52d12ca782
Add /room/<room_id>/joined_members API
...
This returns the currently joined members in the room with their display
names and avatar urls. This is more efficient than /members for large
rooms where you don't need the full events.
2016-12-08 13:32:07 +00:00
Erik Johnston
c45d8e9ba2
Add profile data to the room_membership table for joins
2016-12-08 13:08:41 +00:00
Erik Johnston
2ffec928e2
Reduce batch size to be under SQL limit
2016-09-09 11:03:31 +01:00
Erik Johnston
c10cb581c6
Correctly handle the difference between prev and current state
2016-08-31 14:26:22 +01:00
Erik Johnston
bed10f9880
Use state handler instead of get_users_in_room/get_joined_hosts
2016-08-26 14:54:30 +01:00
Erik Johnston
4daa397a00
Add is_host_joined to slave storage
2016-08-26 13:10:56 +01:00
Erik Johnston
1ccdc1e93a
Cache check_host_in_room
2016-08-26 10:59:40 +01:00
Erik Johnston
721414d98a
Add desc
2016-08-25 17:49:05 +01:00
Erik Johnston
a3dc1e9cbe
Replace context.current_state with context.current_state_ids
2016-08-25 17:32:22 +01:00
Erik Johnston
4d70d1f80e
Add some invalidations to a cache_stream
2016-08-15 11:15:17 +01:00
Mark Haines
0b2158719c
Remove dead code.
...
Loading push rules now happens in the datastore, so we can remove
the methods that loaded them outside the datastore.
The ``waiting_for_join_list`` in federation handler is populated by
anything, so can be removed.
The ``_get_members_events_txn`` method isn't called from anywhere
so can be removed.
2016-06-07 15:07:11 +01:00
Erik Johnston
ccb56fc24b
Make get_joined_hosts_for_room use get_users_in_room
2016-06-03 11:20:23 +01:00
Erik Johnston
c8285564a3
Use state to calculate get_users_in_room
2016-06-01 15:25:25 +01:00
Mark Haines
816df9f267
get_room_members is unused now
2016-05-16 19:51:43 +01:00
Mark Haines
1a3a2002ff
Spell "domain" correctly
...
s/domian/domain/g
2016-05-16 19:17:23 +01:00
Erik Johnston
08dfa8eee2
Add and use get_domian_from_id
2016-05-09 10:36:03 +01:00
David Baker
2547dffccc
Merge pull request #705 from matrix-org/dbkr/pushers_use_event_actions
...
Change pushers to use the event_actions table
2016-04-11 12:58:55 +01:00
Mark Haines
7e2f971c08
Remove some unused functions ( #711 )
...
* Remove some unused functions
* get_room_events_stream is only used in tests
* is_exclusive_room might actually be something we want
2016-04-08 14:01:56 +01:00
David Baker
0fd1cd2400
pep8
2016-04-06 16:50:47 +01:00
David Baker
7e2c89a37f
Make pushers use the event_push_actions table instead of listening on an event stream & running the rules again. Sytest passes, but remaining to do:
...
* Make badges work again
* Remove old, unused code
2016-04-06 15:42:15 +01:00
Erik Johnston
df727f2126
Fix stuck invites
...
If rejecting a remote invite fails with an error response don't fail
the entire request; instead mark the invite as locally rejected.
This fixes the bug where users can get stuck invites which they can
neither accept nor reject.
2016-04-05 11:13:24 +01:00
Erik Johnston
0c53d750e7
Docs and indents
2016-04-04 18:02:48 +01:00
Erik Johnston
92ab45a330
Add upgrade path, rename table
2016-04-04 17:07:43 +01:00
Erik Johnston
3d76b7cb2b
Store invites in a separate table.
2016-04-04 16:30:15 +01:00
Erik Johnston
b6507869cd
Make get_invites return RoomsForUser
2016-03-23 10:32:10 +00:00
Mark Haines
673c96ce97
Remove dead code left over from presence changes
2016-03-17 11:03:47 +00:00
Erik Johnston
25c311eaf6
Cache get_room_changes_for_user
2016-01-29 16:52:48 +00:00
Erik Johnston
c5e7c0e436
Up get_rooms_for_user cache size
2016-01-28 09:58:45 +00:00
Erik Johnston
53cb173663
Push: Use storage apis that are cached
2016-01-25 13:55:18 +00:00
Erik Johnston
cc66a9a5e3
Allow filtering events for multiple users at once
2016-01-18 14:43:50 +00:00
Matthew Hodgson
6c28ac260c
copyrights
2016-01-07 04:26:29 +00:00
Daniel Wagner-Hall
76e69cc8de
Fix typo
2015-12-14 12:38:55 +00:00
Mark Haines
7d6b313312
Add caches for whether a room has been forgotten by a user
2015-12-10 17:49:34 +00:00
Daniel Wagner-Hall
df7cf6c0eb
Fix SQL for postgres again
2015-11-23 18:54:41 +00:00
Daniel Wagner-Hall
3e573a5c6b
Fix SQL for postgres
2015-11-23 18:48:53 +00:00
Daniel Wagner-Hall
df6824a008
Ignore forgotten rooms in v2 sync
2015-11-19 14:54:47 -05:00
Daniel Wagner-Hall
9da4c5340d
Simplify code
2015-11-19 10:07:21 -05:00
Daniel Wagner-Hall
bed7889703
Apply forgetting properly to historical events
2015-11-18 18:11:12 -05:00
Daniel Wagner-Hall
ba26eb3d5d
Allow users to forget rooms
2015-11-17 17:17:30 -05:00
Mark Haines
e3d75f564a
Include banned rooms in the archived section of v2 sync
2015-10-21 11:15:48 +01:00
Mark Haines
68b7fc3e2b
Add rooms that the user has left under archived in v2 sync.
2015-10-19 17:26:18 +01:00
Mark Haines
cacf0688c6
Add a get_invites_for_user method to the storage to find out the rooms a user is invited to
2015-10-13 14:08:38 +01:00
Mark Haines
301141515a
Merge pull request #288 from matrix-org/markjh/unused_definitions
...
Remove some of the unused definitions from synapse
2015-09-28 14:22:44 +01:00
Mark Haines
cf1100887b
Fix order of ON constraints in _get_rooms_for_user_where_membership_is_txn
2015-09-24 17:35:10 +01:00
Mark Haines
e51aa4be96
synapse/storage/roommember.py:_get_members_query was unused
2015-09-23 10:35:10 +01:00
Mark Haines
417485eefa
Include the event_id and stream_ordering of membership events when looking up which rooms a user is in
2015-09-08 18:14:54 +01:00
Erik Johnston
2df8dd9b37
Move all the caches into their own package, synapse.util.caches
2015-08-11 18:00:59 +01:00
Erik Johnston
b8e386db59
Change Cache to not use *args in its interface
2015-08-07 11:52:21 +01:00
Erik Johnston
1e62a3d3a9
Up the cache size for 'get_joined_hosts_for_room' and 'get_users_in_room'
2015-08-05 15:40:40 +01:00
Erik Johnston
5130d80d79
Add bulk insert events API
2015-06-25 17:29:34 +01:00
Erik Johnston
4429e720ae
Merge branch 'master' of github.com:matrix-org/synapse into develop
2015-05-22 10:33:00 +01:00
Mark Haines
53447e9cd3
Add caches for things requested by the pushers
2015-05-21 16:41:39 +01:00
Erik Johnston
f6f902d459
Move fetching of events into their own transactions
2015-05-14 13:45:48 +01:00
Mark Haines
d18f37e026
Collect the invalidate callbacks on the transaction object rather than passing around a separate list
2015-05-05 17:32:21 +01:00
Mark Haines
a9aea68fd5
Invalidate the caches from the correct thread
2015-05-05 14:57:08 +01:00
Erik Johnston
61d05daab1
More join conditions
2015-04-30 18:41:05 +01:00
Erik Johnston
6ead27ddda
Add more conditions on JOINs to make postgres go a little faster.
2015-04-30 18:32:03 +01:00
Erik Johnston
69d4063651
Add get_rooms_for_user cache
2015-04-30 16:47:51 +01:00
Erik Johnston
0bfa78b39b
PEP8
2015-04-07 12:16:05 +01:00
Erik Johnston
5a447098dd
Don't use room hosts table
2015-03-24 16:19:24 +00:00
Erik Johnston
9a7f496298
Sanitize RoomMemberStore
2015-03-23 15:29:04 +00:00
Erik Johnston
f6583796fe
Merge branch 'develop' of github.com:matrix-org/synapse into mysql
2015-03-20 16:31:48 +00:00
Erik Johnston
dc0c989ef4
Give sensible names for '_simple_...' transactions
2015-03-20 15:59:18 +00:00
Erik Johnston
d7a0496f3e
Convert storage layer to be mysql compatible
2015-03-19 15:59:48 +00:00
Kegan Dougal
cb97ea3ec2
PEP8
2015-03-02 11:23:46 +00:00
Kegan Dougal
377ae369c1
Wrap all of get_app_service_rooms in a txn.
2015-03-02 11:20:51 +00:00
Paul "LeoNerd" Evans
357fba2c24
RoomMemberStore no longer needs a _user_rooms_cache member
2015-02-23 15:57:41 +00:00
Paul "LeoNerd" Evans
ebc3db295b
Take named arguments to @cached() decorator, add a 'max_entries' limit
2015-02-19 18:36:02 +00:00
Paul "LeoNerd" Evans
077d200342
Move @cached decorator out into synapse.storage._base; add minimal docs
2015-02-19 17:29:39 +00:00
Paul "LeoNerd" Evans
61959928bb
Pull out the 'get_rooms_for_user' cache logic into a reüsable @cached decorator
2015-02-19 14:58:07 +00:00
Erik Johnston
72a4de2ce6
Use consumeErrors=True on all DeferredLists.
...
This is so that the DeferredLists actually consume the error instead of
propogating down the non-existent errback chain. This should reduce the
number of unhandled errors we are seeing.
2015-02-17 10:07:01 +00:00
Paul "LeoNerd" Evans
7f47ba7383
Added another TODO note
2015-02-11 16:18:21 +00:00
Paul "LeoNerd" Evans
45b56609ae
Cache the result of a get_rooms_for_user query, to make user_rooms_intersect() much lighter in the read-common case
2015-02-11 16:04:08 +00:00
Paul "LeoNerd" Evans
7be0f6594e
First step of making user_rooms_intersect() faster - implement in intersection logic in Python code terms of a DB query that is cacheable per user
2015-02-11 15:53:56 +00:00
Mark Haines
5759bec43c
Replace hs.parse_userid with UserID.from_string
2015-01-23 11:47:15 +00:00
Erik Johnston
8a12df8cf3
Merge branch 'erikj-perf' of github.com:matrix-org/synapse into develop
2015-01-06 14:45:57 +00:00
Erik Johnston
96707ed718
Name 'user_rooms_intersect' transaction
2015-01-06 14:44:27 +00:00
