Commit Graph

2802 Commits

Author SHA1 Message Date
Neil Johnson
61b05727fa guest users should not be part of mau total 2018-09-05 22:30:36 +01:00
Neil Johnson
0b01281e77 move threepid checker to config, add missing yields 2018-08-31 17:11:11 +01:00
Neil Johnson
09f3cf1a7e ensure post registration auth checks do not fail erroneously 2018-08-31 15:42:51 +01:00
Amber Brown
14e4d4f4bf
Port storage/ to Python 3 (#3725) 2018-08-31 00:19:58 +10:00
Hubert Chathi
83caead95a
Merge branch 'develop' into e2e_backups 2018-08-24 11:44:26 -04:00
Erik Johnston
cd77270a66 Implement trail users 2018-08-23 19:17:19 +01:00
Erik Johnston
7a0da69eee Add missing yield 2018-08-23 10:28:12 +01:00
Erik Johnston
764030cf63
Merge pull request #3659 from matrix-org/erikj/split_profiles
Allow profile updates to happen on workers
2018-08-22 11:35:55 +01:00
Erik Johnston
3bf8bab8f9
Merge pull request #3673 from matrix-org/erikj/refactor_state_handler
Refactor state module to support multiple room versions
2018-08-22 10:04:55 +01:00
Matthew Hodgson
bb81e78ec6
Split the state_group_cache in two (#3726)
Splits the state_group_cache in two.

One half contains normal state events; the other contains member events.

The idea is that the lazyloading common case of: "I want a subset of member events plus all of the other state" can be accomplished efficiently by splitting the cache into two, and asking for "all events" from the non-members cache, and "just these keys" from the members cache.  This means we can avoid having to make DictionaryCache aware of these sort of complicated queries, whilst letting LL requests benefit from the caching.

Previously we were unable to sensibly use the caching and had to pull all state from the DB irrespective of the filtering, which made things slow.  Hopefully fixes https://github.com/matrix-org/synapse/issues/3720.
2018-08-22 00:56:37 +02:00
Hubert Chathi
42a394caa2 allow session_data to be any JSON instead of just a string 2018-08-21 14:51:34 -04:00
Hubert Chathi
8550a7e9c2 allow auth_data to be any JSON instead of a string 2018-08-21 10:38:00 -04:00
Erik Johnston
4d664278af Merge branch 'develop' of github.com:matrix-org/synapse into erikj/refactor_state_handler 2018-08-20 14:49:43 +01:00
Erik Johnston
782689bd40 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/split_profiles 2018-08-17 14:15:48 +01:00
Erik Johnston
38f708a2bb Remote profile cache should remain in master worker 2018-08-17 11:37:42 +01:00
Neil Johnson
bcfeb44afe call reap on start up and fix under reaping bug 2018-08-16 22:55:32 +01:00
Matthew Hodgson
3f543dc021 initial cut at a room summary API (#3574) 2018-08-16 09:46:50 +01:00
Matthew Hodgson
2f78f432c4 speed up /members and add at= and membership params (#3568) 2018-08-15 16:35:22 +01:00
Erik Johnston
ef184caf30 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/split_federation 2018-08-15 14:25:46 +01:00
Neil Johnson
9ecbaf8ba8 adding missing yield 2018-08-14 16:55:28 +01:00
Neil Johnson
414d54b61a
Merge pull request #3670 from matrix-org/neilj/mau_sync_block
Block ability to read via sync if mau limit exceeded
2018-08-14 15:21:31 +00:00
Amber Brown
591bf87c6a Merge remote-tracking branch 'origin/develop' into neilj/fix_reap_users_in_postgres 2018-08-14 20:56:23 +10:00
Amber Brown
99dd975dae
Run tests under PostgreSQL (#3423) 2018-08-13 16:47:46 +10:00
Matthew Hodgson
f0cede5556 missing import 2018-08-12 19:14:31 -04:00
Matthew Hodgson
66a4ca1d28 404 nicely if you try to interact with a missing current version 2018-08-12 19:14:31 -04:00
Matthew Hodgson
edc427a351 flake8 2018-08-12 19:14:31 -04:00
Matthew Hodgson
fe87890b18 implement remaining tests and make them work 2018-08-12 19:14:31 -04:00
Matthew Hodgson
15d513f16f fix idiocies and so make tests pass 2018-08-12 19:14:31 -04:00
Matthew Hodgson
b5eee511c7 don't needlessly return user_id 2018-08-12 19:14:31 -04:00
Matthew Hodgson
982edca380 fix flakes 2018-08-12 19:14:31 -04:00
Matthew Hodgson
234611f347 fix typos 2018-08-12 19:14:31 -04:00
Matthew Hodgson
9f0791b7bd add a tonne of docstring; make upload_room_keys properly assert version 2018-08-12 19:14:31 -04:00
Matthew Hodgson
8d14598e90 add storage docstring; remove unused set_e2e_room_keys 2018-08-12 19:14:31 -04:00
Matthew Hodgson
cac0253799 rename room_key_version table correctly, and fix opt args 2018-08-12 19:14:31 -04:00
Matthew Hodgson
0abb205b47 blindly incorporate PR review - needs testing & fixing 2018-08-12 19:14:31 -04:00
Matthew Hodgson
69e51c7ba4 make /room_keys/version work 2018-08-12 19:14:31 -04:00
Matthew Hodgson
8ae64b270f implement /room_keys/version too (untested) 2018-08-12 19:14:31 -04:00
Matthew Hodgson
6b8c07abc2 make it work and fix pep8 2018-08-12 19:13:09 -04:00
Matthew Hodgson
0bc4627a73 interim WIP checkin; doesn't build yet 2018-08-12 18:23:10 -04:00
Matthew Hodgson
53ace904b2 total WIP skeleton for /room_keys API 2018-08-12 18:23:10 -04:00
Neil Johnson
31fa743567 fix sqlite/postgres incompatibility in reap_monthly_active_users 2018-08-11 22:38:34 +01:00
Amber Brown
b37c472419
Rename async to async_helpers because async is a keyword on Python 3.7 (#3678) 2018-08-10 23:50:21 +10:00
Neil Johnson
885ea9c602 rename _user_last_seen_monthly_active 2018-08-09 18:02:12 +01:00
Erik Johnston
ce6db0e547 Choose state algorithm based on room version 2018-08-09 14:58:47 +01:00
Erik Johnston
484a0ebdfc Merge branch 'develop' of github.com:matrix-org/synapse into erikj/split_profiles 2018-08-09 10:16:29 +01:00
Erik Johnston
5785b93711 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/split_federation 2018-08-09 10:16:16 +01:00
Richard van der Hoff
8521ae13e3
Merge pull request #3654 from matrix-org/rav/room_versions
Support for room versioning
2018-08-08 17:10:53 +01:00
Neil Johnson
be59910b93 Merge branch 'develop' of github.com:matrix-org/synapse into neilj/reserved_users 2018-08-08 13:45:21 +01:00
Neil Johnson
990fe9fc23
Merge pull request #3633 from matrix-org/neilj/mau_tracker
API for monthly_active_users table
2018-08-08 12:44:15 +00:00
Neil Johnson
312ae74746 typos 2018-08-08 13:33:16 +01:00
Neil Johnson
ef3589063a prevent total number of reserved users being too large 2018-08-07 17:59:27 +01:00
Neil Johnson
e8eba2b4e3 implement reserved users for mau limits 2018-08-07 17:49:43 +01:00
Erik Johnston
cd9765805e Allow ratelimiting on workers 2018-08-07 10:50:28 +01:00
Erik Johnston
495cb100d1 Allow profile changes to happen on workers 2018-08-07 10:50:26 +01:00
Richard van der Hoff
ca9bc1f4fe Fix occasional glitches in the synapse_event_persisted_position metric
Every so often this metric glitched to a negative number. I'm assuming it was
due to backfilled events.
2018-08-07 10:00:03 +01:00
Neil Johnson
a74b25faaa WIP building out mau reserved users 2018-08-06 23:25:25 +01:00
Neil Johnson
e54794f5b6 Fix postgres compatibility bug 2018-08-06 21:55:54 +01:00
Neil Johnson
16d78be315 make use of _simple_select_one_onecol, improved comments 2018-08-06 17:51:15 +01:00
Erik Johnston
96a9a29645 Pull in necessary stores in federation_reader 2018-08-06 15:23:57 +01:00
Erik Johnston
62ace05c45 Move necessary storage functions to worker classes 2018-08-06 15:23:38 +01:00
Neil Johnson
886be75ad1 bug fixes 2018-08-03 22:29:03 +01:00
Neil Johnson
e10830e976 wip commit - tests failing 2018-08-03 17:55:50 +01:00
Richard van der Hoff
0ca459ea33 Basic support for room versioning
This is the first tranche of support for room versioning. It includes:
 * setting the default room version in the config file
 * new room_version param on the createRoom API
 * storing the version of newly-created rooms in the m.room.create event
 * fishing the version of existing rooms out of the m.room.create event
2018-08-03 16:08:32 +01:00
Neil Johnson
950807d93a fix caching and tests 2018-08-03 13:49:53 +01:00
Neil Johnson
897c51d274 Merge branch 'develop' of github.com:matrix-org/synapse into neilj/mau_tracker 2018-08-03 13:40:47 +01:00
Neil Johnson
5e2f7b8084 typo 2018-08-02 22:47:48 +01:00
Neil Johnson
4ecb4bdac9 wip attempt at caching 2018-08-02 22:41:05 +01:00
Neil Johnson
74b1d46ad9 do mau checks based on monthly_active_users table 2018-08-02 16:57:35 +01:00
Neil Johnson
9180061b49 remove unused count_monthly_users 2018-08-02 15:55:29 +01:00
Richard van der Hoff
704c3e6239 Merge branch 'master' into develop 2018-08-02 15:43:30 +01:00
Richard van der Hoff
14fa9d4d92 Avoid extra db lookups
Since we're about to look up the events themselves anyway, we can skip the
extra db queries here.
2018-08-02 13:55:51 +01:00
Richard van der Hoff
0a65450d04 Validation for events/rooms in fed requests
When we get a federation request which refers to an event id, make sure that
said event is in the room the caller claims it is in.

(patch supplied by @turt2live)
2018-08-02 13:48:40 +01:00
Neil Johnson
00f99f74b1 insertion into monthly_active_users 2018-08-02 13:47:19 +01:00
Neil Johnson
4a6725d9d1 Merge branch 'neilj/mau_tracker' of github.com:matrix-org/synapse into neilj/mau_tracker 2018-08-02 11:04:18 +01:00
Neil Johnson
165e067033 Revert "change monthly_active_users table to be a single column"
This reverts commit ec716a35b2.
2018-08-02 10:59:58 +01:00
Erik Johnston
40c1c59cf4
Merge pull request #3621 from matrix-org/erikj/split_fed_store
Split out DB writes in federation handler
2018-08-02 10:41:42 +01:00
Neil Johnson
08281fe6b7 self.db_conn unused 2018-08-01 23:26:24 +01:00
Neil Johnson
c21d82bab3 normalise reaping query 2018-08-01 23:24:38 +01:00
Neil Johnson
ec716a35b2 change monthly_active_users table to be a single column 2018-08-01 17:54:37 +01:00
Neil Johnson
d766f26de9 Merge branch 'develop' of github.com:matrix-org/synapse into neilj/mau_tracker 2018-08-01 17:49:41 +01:00
Neil Johnson
085435e13a
Merge pull request #3630 from matrix-org/neilj/mau_sign_in_log_in_limits
Initial impl of capping MAU
2018-08-01 15:58:45 +00:00
Neil Johnson
7ff44d9215 improve clarity 2018-08-01 16:17:00 +01:00
Amber Brown
da7785147d
Python 3: Convert some unicode/bytes uses (#3569) 2018-08-02 00:54:06 +10:00
Neil Johnson
303f1c851f Merge branch 'develop' of github.com:matrix-org/synapse into neilj/mau_sign_in_log_in_limits 2018-08-01 13:42:50 +01:00
Erik Johnston
a6d7b74915 update docs 2018-08-01 13:39:14 +01:00
Neil Johnson
4e5ac901dd clean up 2018-08-01 12:03:57 +01:00
Neil Johnson
f9f5559971 fix comment 2018-08-01 12:03:42 +01:00
Neil Johnson
6023cdd227 remove errant print 2018-08-01 10:27:17 +01:00
Neil Johnson
7931393495 make count_monthly_users async synapse/handlers/auth.py 2018-08-01 10:21:56 +01:00
Neil Johnson
6ef983ce5c api into monthly_active_users table 2018-07-31 16:36:24 +01:00
Richard van der Hoff
5de936caa1
Merge pull request #3612 from matrix-org/rav/store_heirarchy
Make EventStore inherit from EventFederationStore
2018-07-31 13:44:04 +01:00
Neil Johnson
df2235e7fa coding style 2018-07-31 13:16:20 +01:00
Richard van der Hoff
0bc9b9e397 reinstate explicit include of EventsWorkerStore 2018-07-31 13:11:04 +01:00
Neil Johnson
fef7e58ac6 actually close conn 2018-07-30 22:29:44 +01:00
Neil Johnson
9b13817e06 factor out metrics from __init__ to app/homeserver 2018-07-30 22:07:07 +01:00
Neil Johnson
251e6c1210 limit register and sign in on number of monthly users 2018-07-30 15:55:57 +01:00
Erik Johnston
143f1a2532 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/split_fed_store 2018-07-30 09:56:18 +01:00
Matthew Hodgson
e9b2d047f6
make /context lazyload & filter aware (#3567)
make /context lazyload & filter aware.
2018-07-27 15:12:50 +01:00
Richard van der Hoff
65c8dee900
Merge pull request #3614 from matrix-org/rav/stop_populating_event_content
Stop populating events.content
2018-07-26 22:54:08 +01:00
Richard van der Hoff
9e68b1bd2d
Merge pull request #3613 from matrix-org/rav/stop_using_event_edges_room_id
Remove some redundant joins on event_edges.room_id
2018-07-26 22:31:01 +01:00