7785 Commits

Author	SHA1	Message	Date
David Baker	d4553818a0	Merge pull request #2627 from matrix-org/rav/custom_rest_endpoints ... Add a hook for custom rest endpoints	2017-11-02 15:18:37 +00:00
David Baker	6b6f03ae05	Merge pull request #2626 from matrix-org/rav/refactor_module_api ... Factor _AccountHandler proxy out to ModuleApi	2017-11-02 15:15:30 +00:00
Richard van der Hoff	6b60f7dca0	Add more hooks to ModuleApi ... add `get_user_by_req` and `invalidate_access_token`	2017-11-02 14:37:39 +00:00
Richard van der Hoff	fcdfc911ee	Add a hook for custom rest endpoints ... Let the user specify custom modules which can be used for implementing extra endpoints.	2017-11-02 14:36:55 +00:00
Richard van der Hoff	1189be43a2	Factor _AccountHandler proxy out to ModuleApi ... We're going to need to use this from places that aren't password auth, so let's move it to a proper class.	2017-11-02 14:36:11 +00:00
Richard van der Hoff	6650a07ede	Factor out _configure_named_resource ... This was a bit of a code vomit, so let's factor it out to preserve some sanity	2017-11-02 14:33:37 +00:00
David Baker	b19d9e2174	Merge pull request #2624 from matrix-org/rav/password_provider_notify_logout ... Notify auth providers on logout	2017-11-02 10:55:17 +00:00
David Baker	1f080a6c97	Merge pull request #2623 from matrix-org/rav/callbacks_for_auth_providers ... Allow password_auth_providers to return a callback	2017-11-02 10:49:03 +00:00
David Baker	04897c9dc1	Merge pull request #2622 from matrix-org/rav/db_access_for_auth_providers ... Let auth providers get to the database	2017-11-02 10:41:25 +00:00
Richard van der Hoff	979eed4362	Fix user-interactive password auth ... this got broken in the previous commit	2017-11-01 17:03:20 +00:00
Richard van der Hoff	bc8a5c0330	Notify auth providers on logout ... Provide a hook by which auth providers can be notified of logouts.	2017-11-01 16:51:51 +00:00
Richard van der Hoff	4c8f94ac94	Allow password_auth_providers to return a callback ... ... so that they have a way to record access tokens.	2017-11-01 16:51:03 +00:00
Richard van der Hoff	846a94fbc9	Merge pull request #2620 from matrix-org/rav/auth_non_password ... Let password auth providers handle arbitrary login types	2017-11-01 16:45:33 +00:00
Richard van der Hoff	3cd6b22c7b	Let password auth providers handle arbitrary login types ... Provide a hook where password auth providers can say they know about other login types, and get passed the relevant parameters	2017-11-01 16:43:57 +00:00
David Baker	c9b9ef575b	Merge pull request #2621 from matrix-org/rav/refactor_accesstoken_delete ... Move access token deletion into auth handler	2017-11-01 16:26:06 +00:00
Matthew Hodgson	275826f234	Merge pull request #2617 from matrix-org/matthew/auto-displayname ... automatically set default displayname on register	2017-11-01 16:21:16 +00:00
David Baker	4f0488b307	Merge remote-tracking branch 'origin/develop' into rav/refactor_accesstoken_delete	2017-11-01 16:20:19 +00:00
David Baker	e5e930aec3	Merge pull request #2615 from matrix-org/rav/break_auth_device_dep ... Break dependency of auth_handler on device_handler	2017-11-01 16:06:31 +00:00
David Baker	fbbacb284e	Merge pull request #2613 from matrix-org/rav/kill_refresh_tokens ... Remove the last vestiges of refresh_tokens	2017-11-01 15:57:35 +00:00
Matthew Hodgson	9f7a555b4e	switch to setting default displayname in the storage layer ... to avoid clobbering guest user displaynames on registration	2017-11-01 15:51:30 +00:00
Richard van der Hoff	dd13310fb8	Move access token deletion into auth handler ... Also move duplicated deactivation code into the auth handler. I want to add some hooks when we deactivate an access token, so let's bring it all in here so that there's somewhere to put it.	2017-11-01 15:46:22 +00:00
David Baker	0bb253f37b	Apparently this is python	2017-11-01 14:02:52 +00:00
David Baker	59e7e62c4b	Log login requests ... Carefully though, to avoid logging passwords	2017-11-01 13:58:01 +00:00
Matthew Hodgson	f8420d6279	automatically set default displayname on register ... to avoid leaking ugly MXIDs and cluttering up the timeline with displayname changes as well as membership joins for autojoin rooms (e.g. the status autojoin rooms), automatically set the displayname to match the localpart of the mxid upon registration.	2017-11-01 13:15:41 +00:00
Luke Barnard	99354b430e	Merge pull request #2612 from matrix-org/luke/groups-room-relationship-is-public ... Modify group room association API to allow modification of is_public	2017-11-01 11:08:36 +00:00
Richard van der Hoff	74c56f794c	Break dependency of auth_handler on device_handler ... I'm going to need to make the device_handler depend on the auth_handler, so I need to break this dependency to avoid a cycle. It turns out that the auth_handler was only using the device_handler in one place which was an edge case which we can more elegantly handle by throwing an error rather than fixing it up.	2017-11-01 10:27:06 +00:00
Luke Barnard	318a249c8b	Leave is_public as required argument of update_room_group_association	2017-11-01 09:36:01 +00:00
Luke Barnard	207fabbc6a	Update docs for updating room group association	2017-11-01 09:35:15 +00:00
Richard van der Hoff	356bcafc44	Remove the last vestiges of refresh_tokens	2017-10-31 20:35:58 +00:00
Richard van der Hoff	3e0aaad190	Let auth providers get to the database ... Somewhat open to abuse, but also somewhat unavoidable :/	2017-10-31 17:22:29 +00:00
Luke Barnard	13b3d7b4a0	Flake8	2017-10-31 17:20:11 +00:00
Luke Barnard	20fe347906	Modify group room association API to allow modification of is_public ... also includes renamings to make things more consistent.	2017-10-31 17:04:28 +00:00
Richard van der Hoff	1650eb5847	DB schema interface for password auth providers ... Provide an interface by which password auth providers can register db schema files to be run at startup	2017-10-31 14:01:53 +00:00
David Baker	c31a7c3ff6	Merge pull request #2609 from matrix-org/rav/refactor_login ... Refactor some logic from LoginRestServlet into AuthHandler	2017-10-31 13:51:36 +00:00
Richard van der Hoff	1b65ae00ac	Refactor some logic from LoginRestServlet into AuthHandler ... I'm going to need some more flexibility in handling login types in password auth providers, so as a first step, move some stuff from LoginRestServlet into AuthHandler. In particular, we pass everything other than SAML, JWT and token logins down to the AuthHandler, which now has responsibility for checking the login type and fishing the password out of the login dictionary, as well as qualifying the user_id if need be. Ideally SAML, JWT and token would go that way too, but there's no real need for it right now and I'm trying to minimise impact. This commit *should* be non-functional.	2017-10-31 10:48:41 +00:00
Richard van der Hoff	ffc574a6f9	Clean up backwards-compat hacks for ldap ... try to make the backwards-compat flows follow the same code paths as the modern impl. This commit should be non-functional.	2017-10-31 10:47:02 +00:00
Luke Barnard	9bc17fc5fb	Fix wording on group creation error	2017-10-30 15:17:23 +00:00
Matthew Hodgson	208a6647f1	fix typo	2017-10-29 20:54:20 +00:00
Erik Johnston	d0abb4e8e6	Fix typo when checking if user is invited to group	2017-10-27 16:57:19 +01:00
Erik Johnston	977078f06d	Fix bad merge	2017-10-27 15:10:50 +01:00
Erik Johnston	6980c4557e	Merge branch 'erikj/attestation_jitter' of github.com:matrix-org/synapse into develop	2017-10-27 15:09:05 +01:00
Erik Johnston	af92f5b00f	Revert "Add jitter to validity period of attestations"	2017-10-27 15:07:21 +01:00
Erik Johnston	4ab8abbc2b	Merge branch 'erikj/attestation_local_fix' of github.com:matrix-org/synapse into develop	2017-10-27 15:07:08 +01:00
Erik Johnston	6af3656deb	Merge pull request #2595 from matrix-org/erikj/attestation_commnet ... Add comment about attestations	2017-10-27 14:20:19 +01:00
Richard van der Hoff	4d83632009	Merge pull request #2591 from matrix-org/rav/device_delete_auth ... Device deletion: check UI auth matches access token	2017-10-27 12:30:10 +01:00
Richard van der Hoff	110b373e9c	Merge pull request #2589 from matrix-org/rav/as_deactivate_account ... Allow ASes to deactivate their own users	2017-10-27 12:29:32 +01:00
Erik Johnston	ca571b0ec3	Add jitter to validity period of attestations ... This helps ensure that the renewals of attestations are spread out more evenly.	2017-10-27 11:57:27 +01:00
Luke Barnard	d8c26162a1	Merge pull request #2582 from matrix-org/luke/group-is-public ... Add is_public to groups table to allow for private groups	2017-10-27 11:41:13 +01:00
Erik Johnston	c067088747	Add comment about attestations	2017-10-27 11:35:41 +01:00
Luke Barnard	5451cc7792	Request is_public from database	2017-10-27 11:27:43 +01:00
Luke Barnard	124314672f	group is dict	2017-10-27 11:08:19 +01:00
Luke Barnard	6362298fa5	Create groups with is_public = True	2017-10-27 11:04:20 +01:00
Richard van der Hoff	8b56977b6f	Merge pull request #2586 from matrix-org/rav/frontend_proxy_auth_header ... Front-end proxy: pass through auth header	2017-10-27 11:01:50 +01:00
Richard van der Hoff	173567a7f2	Docstring for post_urlencoded_get_json	2017-10-27 10:59:50 +01:00
Luke Barnard	c7d9f25d22	Fix create_group to pass requester_user_id	2017-10-27 10:57:20 +01:00
Erik Johnston	e27b76d117	Import logger	2017-10-27 10:54:02 +01:00
Richard van der Hoff	8854c039f2	Merge pull request #2585 from matrix-org/rav/unstable_to_r0 ... Support /keys/upload on /r0 as well as /unstable	2017-10-27 10:53:48 +01:00
Richard van der Hoff	14f581abc2	Merge pull request #2584 from matrix-org/rav/fix_httpclient_logcontexts ... Fix logcontext leaks in httpclient	2017-10-27 10:53:29 +01:00
Luke Barnard	2ca46c7afc	Correct logic for checking private group membership	2017-10-27 10:48:01 +01:00
Erik Johnston	82d8c1bacb	Fixup	2017-10-27 10:30:21 +01:00
Erik Johnston	2fd9831f7c	Merge pull request #2574 from matrix-org/erikj/room_list_fixes ... Add logging and fix log contexts for publicRooms	2017-10-27 10:01:23 +01:00
Erik Johnston	195abfe7a5	Remove incorrect attestations	2017-10-27 09:58:13 +01:00
Erik Johnston	d8dde19f04	Log if we try to do attestations for our own user and group	2017-10-27 09:55:01 +01:00
Erik Johnston	585972b51a	Don't generate group attestations for local users	2017-10-27 09:46:56 +01:00
Richard van der Hoff	7a6546228b	Device deletion: check UI auth matches access token ... (otherwise there's no point in the UI auth)	2017-10-27 00:04:31 +01:00
Richard van der Hoff	785bd7fd75	Allow ASes to deactivate their own users	2017-10-27 00:01:00 +01:00
Richard van der Hoff	c89e6aadff	Merge pull request #2581 from matrix-org/rav/fix_init_with_no_logfile ... Fix error when running synapse with no logfile	2017-10-26 22:16:57 +01:00
Richard van der Hoff	54a2525133	Front-end proxy: pass through auth header ... So that access-token-in-an-auth-header works.	2017-10-26 18:19:01 +01:00
Richard van der Hoff	0a5866bec9	Support /keys/upload on /r0 as well as /unstable ... (So that we can stop riot relying on it in /unstable)	2017-10-26 18:18:23 +01:00
Richard van der Hoff	0d8e3ad48b	Fix logcontext leaks in httpclient ... `preserve_context_over_fn` is borked	2017-10-26 18:17:10 +01:00
Richard van der Hoff	12ef02dc3d	SimpleHTTPClient: add support for headers ... Sometimes we need to pass headers into these methods	2017-10-26 17:59:50 +01:00
Luke Barnard	69e8a05f35	Make it work	2017-10-26 17:55:58 +01:00
Luke Barnard	007cd48af6	Recreate groups table instead of adding column ... Adding a column with non-constant default not possible in sqlite3	2017-10-26 17:55:22 +01:00
Luke Barnard	713e60b9b6	Awful hack to get default true	2017-10-26 17:38:14 +01:00
Luke Barnard	cfa4e658e0	Bump schema version to 46	2017-10-26 17:23:49 +01:00
Luke Barnard	595fe67f01	delint	2017-10-26 17:20:24 +01:00
Luke Barnard	9b2feef9eb	Add is_public to groups table to allow for private groups ... Prevent group API access to non-members for private groups Also make all the group code paths consistent with `requester_user_id` always being the User ID of the requesting user.	2017-10-26 16:51:32 +01:00
Richard van der Hoff	f7f90e0c8d	Fix error when running synapse with no logfile ... Fixes 'UnboundLocalError: local variable 'sighup' referenced before assignment'	2017-10-26 16:45:20 +01:00
Krombel	8299b323ee	add release endpoints for /thirdparty	2017-10-26 16:58:20 +02:00
Krombel	9b436c8b4c	register some /unstable endpoints in /r0 as well	2017-10-26 15:22:50 +02:00
Erik Johnston	566e21eac8	Update room_list.py	2017-10-26 11:39:54 +01:00
Maxime Vaillancourt	5287e57c86	Ignore noscript tags when generating URL previews	2017-10-25 20:44:34 -04:00
Erik Johnston	2a7e9faeec	Do logcontexts outside ResponseCache	2017-10-25 15:21:08 +01:00
Erik Johnston	1ad1ba9e6a	Merge branch 'master' of github.com:matrix-org/synapse into develop	2017-10-25 10:27:23 +01:00
Erik Johnston	33a9026cdf	Add logging and fix log contexts for publicRooms	2017-10-25 10:26:06 +01:00
Matthew Hodgson	efd0f5a3c5	tip for generating tls_fingerprints	2017-10-24 18:49:49 +01:00
Erik Johnston	6ba4fabdb9	Bump version and changelog	2017-10-24 14:15:27 +01:00
Erik Johnston	39dc52157d	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/group_fed_update_profile	2017-10-24 09:16:20 +01:00
Richard van der Hoff	0be99858f3	fix vars named l ... E741 says "do not use variables named ‘l’, ‘O’, or ‘I’".	2017-10-23 15:56:38 +01:00
Richard van der Hoff	eaaabc6c4f	replace 'except:' with 'except Exception:' ... what could possibly go wrong	2017-10-23 15:52:32 +01:00
Erik Johnston	ce6d4914f4	Correctly wire in update group profile over federation	2017-10-23 15:21:24 +01:00
Richard van der Hoff	ecf198aab8	Merge pull request #2566 from matrix-org/rav/media_logcontext_leak ... Fix a logcontext leak in the media repo	2017-10-23 14:47:49 +01:00
Richard van der Hoff	3267b81b81	Merge pull request #2561 from matrix-org/rav/id_checking ... Updates to ID checking	2017-10-23 14:39:20 +01:00
Richard van der Hoff	d03cfc4258	Fix a logcontext leak in the media repo	2017-10-23 14:34:27 +01:00
Erik Johnston	1de557975f	Merge branch 'master' of github.com:matrix-org/synapse into develop	2017-10-23 13:18:12 +01:00
Erik Johnston	13e16cf302	Bump version and changelog	2017-10-23 13:13:31 +01:00
Richard van der Hoff	1135193dfd	Validate group ids when parsing ... May as well do it whenever we parse a Group ID. We check the sigil and basic structure here so it makes sense to check the grammar in the same place.	2017-10-21 00:30:39 +01:00
Richard van der Hoff	29812c628b	Allow = in mxids and groupids ... ... because the spec says we should.	2017-10-20 23:42:53 +01:00
Richard van der Hoff	58fbbe0f1d	Disallow capital letters in userids ... Factor out a common function for checking user ids and group ids, which forbids capitals.	2017-10-20 23:37:22 +01:00
Richard van der Hoff	631d7b87b5	Remove pointless create() method ... It just calls the constructor, so we may as well kill it rather than having random codepaths.	2017-10-20 22:14:55 +01:00
Erik Johnston	6070647774	Correctly bump version	2017-10-19 16:40:20 +01:00
Erik Johnston	0ef0aeceac	Bump version and changelog	2017-10-19 13:48:33 +01:00
Erik Johnston	c7d46510d7	Flake8	2017-10-19 13:36:06 +01:00
Erik Johnston	ffd3f1a783	Add missing file...	2017-10-19 12:17:30 +01:00
Erik Johnston	29bafe2f7e	Add config to enable group creation	2017-10-19 12:13:44 +01:00
Erik Johnston	513c23bfd9	Enforce sensible group IDs	2017-10-19 12:01:01 +01:00
Erik Johnston	011d03a0f6	Fix typo	2017-10-19 11:22:48 +01:00
Erik Johnston	9ab859f27b	Fix typo in group attestation handling	2017-10-19 10:55:52 +01:00
Erik Johnston	bd5718d0ad	Fix typo in thumbnail generation	2017-10-19 10:27:18 +01:00
Erik Johnston	161a862ffb	Fix typo	2017-10-19 10:17:43 +01:00
Richard van der Hoff	69994c385a	Merge pull request #2553 from matrix-org/rav/fix_500_on_event_send ... Fix 500 error when we get an error handling a PDU	2017-10-18 11:25:44 +01:00
Richard van der Hoff	b5dbbac308	Merge pull request #2552 from matrix-org/rav/fix_500_on_dodgy_powerlevels ... Fix 500 error when fields missing from power_levels event	2017-10-17 20:53:30 +01:00
Richard van der Hoff	582bd19ee9	Fix 500 error when we get an error handling a PDU ... FederationServer doesn't have a send_failure (and nor does its subclass, ReplicationLayer), so this was failing. I'm not really sure what the idea behind send_failure is, given (a) we don't do anything at the other end with it except log it, and (b) we also send back the failure via the transaction response. I suspect there's a whole lot of dead code around it, but for now I'm just removing the broken bit.	2017-10-17 20:52:40 +01:00
Richard van der Hoff	74f99f227c	Doc some more dynamic Homeserver methods	2017-10-17 20:51:29 +01:00
Richard van der Hoff	c2bd177ea0	Fix 500 error when fields missing from power_levels event ... If the users or events keys were missing from a power_levels event, then we would throw 500s when trying to auth them.	2017-10-17 17:05:42 +01:00
Erik Johnston	fe6e9f580b	Merge pull request #2550 from krombel/fix_thumbnail_2548 ... fix thumbnailing (#2548)	2017-10-17 15:35:18 +01:00
Richard van der Hoff	7216c76654	Improve error handling for missing files (#2551) ... `os.path.exists` doesn't allow us to distinguish between permissions errors and the path actually not existing, which repeatedly confuses people. It also means that we try to overwrite existing key files, which is super-confusing. (cf issues #2455, #2379). Use os.stat instead. Also, don't recomemnd the the use of --generate-config, which screws everything up if you're using debian (cf #2455).	2017-10-17 14:46:17 +01:00
Richard van der Hoff	4cc8bb0767	Merge pull request #2549 from matrix-org/rav/event_persist_logcontexts ... Fix logcontext handling for persist_events	2017-10-17 13:20:35 +01:00
David Baker	4e242b3e20	Merge pull request #2545 from matrix-org/dbkr/auto_join_rooms ... Add config option to auto-join new users to rooms	2017-10-17 11:45:49 +01:00
Krombel	a6245478c8	fix thumbnailing (#2548) ... in commit 0e28281a the code for thumbnailing got refactored and the renaming of this variables was not done correctly. Signed-Off-by: Matthias Kesler <krombel@krombel.de>	2017-10-17 12:45:33 +02:00
Richard van der Hoff	2e9f5ea31a	Fix logcontext handling for persist_events ... * don't use preserve_context_over_deferred, which is known broken. * remove a redundant preserve_fn. * add/improve some comments	2017-10-17 10:59:30 +01:00
David Baker	a9c2e930ac	pep8	2017-10-17 10:13:13 +01:00
David Baker	c05e6015cc	Add config option to auto-join new users to rooms ... New users who register on the server will be dumped into all rooms in auto_join_rooms in the config.	2017-10-16 17:57:27 +01:00
Luke Barnard	85f5674e44	Delint	2017-10-16 15:52:17 +01:00
Luke Barnard	c43e8a9736	Make it work. Warn about lack of user profile	2017-10-16 15:50:39 +01:00
Luke Barnard	a3ac4f6b0a	_create_rererouter for get_invited_users_in_group	2017-10-16 15:41:03 +01:00
Luke Barnard	5dfd0350c7	Merge branch 'develop' into luke/groups-invited-users	2017-10-16 15:34:36 +01:00
Luke Barnard	ca96d609e4	Merge pull request #2543 from matrix-org/luke/fix-on-group-invite-no-profile ... Log a warning when no profile for invited member	2017-10-16 16:33:53 +02:00
Luke Barnard	2c5972f87f	Implement GET /groups/$groupId/invited_users	2017-10-16 15:31:11 +01:00
Luke Barnard	6079d0027a	Log a warning when no profile for invited member ... And return empty profile	2017-10-16 14:20:45 +01:00
David Baker	9342bcfce0	Omit the *s for @room notifications ... They're just redundant	2017-10-16 13:38:10 +01:00
Erik Johnston	1b6b0b1e66	Add try/finally block to close t_byte_source	2017-10-13 15:34:08 +01:00
Erik Johnston	6b725cf56a	Remove old comment	2017-10-13 15:23:41 +01:00
Erik Johnston	2b24416e90	Don't reuse source but instead copy from primary media store to backup	2017-10-13 14:11:34 +01:00
Erik Johnston	b92a8e6e4a	PEP8	2017-10-13 13:58:57 +01:00
Erik Johnston	31aa7bd8d1	Move type into key	2017-10-13 13:47:38 +01:00
Erik Johnston	ad1911bbf4	Comment	2017-10-13 13:47:05 +01:00
Erik Johnston	c021c39cbd	Remove spurious addition	2017-10-13 13:46:53 +01:00
Erik Johnston	1f43d22397	Don't needlessly rename variable	2017-10-13 11:42:07 +01:00
Erik Johnston	a675bd08bd	Add paths back in...	2017-10-13 11:41:06 +01:00
Erik Johnston	4d7e1dde70	Remove unnecessary diff	2017-10-13 11:36:32 +01:00
Erik Johnston	ae5d18617a	Make things be absolute paths again	2017-10-13 11:35:44 +01:00
Erik Johnston	9732ec6797	s/write_to_file/write_to_file_and_backup/	2017-10-13 11:34:41 +01:00
Erik Johnston	0e28281a02	Fix up	2017-10-13 11:33:49 +01:00
Erik Johnston	505371414f	Fix up thumbnailing function	2017-10-13 11:23:53 +01:00
Erik Johnston	e3428d26ca	Fix typo	2017-10-13 10:39:59 +01:00
Erik Johnston	35332298ef	Fix up comments	2017-10-13 10:39:32 +01:00
Erik Johnston	64db043a71	Move makedirs to thread	2017-10-13 10:25:01 +01:00
Erik Johnston	b60859d6cc	Use make_deferred_yieldable	2017-10-13 10:24:19 +01:00
Erik Johnston	d76621a47b	Fix comments	2017-10-12 18:16:25 +01:00
Erik Johnston	4ae85ae121	Don't close prematurely..	2017-10-12 17:57:31 +01:00
Erik Johnston	cc505b4b5e	getvalue closes buffer	2017-10-12 17:52:30 +01:00
Erik Johnston	1259a76047	Get len before close	2017-10-12 17:39:23 +01:00
Erik Johnston	802ca12d05	Don't close file prematurely	2017-10-12 17:37:21 +01:00
Erik Johnston	e283b555b1	Copy everything to backup	2017-10-12 17:31:24 +01:00
Erik Johnston	b77a13812c	Typo	2017-10-12 15:32:32 +01:00
Erik Johnston	6dfde6d485	Remove dead code	2017-10-12 15:30:26 +01:00
Erik Johnston	c8eeef6947	Fix typos	2017-10-12 15:28:24 +01:00
Erik Johnston	67cb89fbdf	Fix typo	2017-10-12 15:23:41 +01:00
Erik Johnston	bf4fb1fb40	Basic implementation of backup media store	2017-10-12 15:20:59 +01:00
hera	f807f7f804	log when we get an exception handling replication updates	2017-10-12 11:51:24 +01:00
David Baker	b8d8ed1ba9	Merge pull request #2531 from matrix-org/dbkr/spamcheck_error_messages ... Allow error strings from spam checker	2017-10-12 10:31:03 +01:00
Richard van der Hoff	cc794d60e7	Merge pull request #2532 from matrix-org/rav/fix_linearizer ... Fix stackoverflow and logcontexts from linearizer	2017-10-11 17:29:32 +01:00
Richard van der Hoff	8dd0c85ac5	Merge pull request #2529 from matrix-org/rav/fix_transaction_failure_handling ... log pdu_failures from incoming transactions	2017-10-11 17:29:14 +01:00
Richard van der Hoff	76fa695241	Merge pull request #2515 from matrix-org/rav/fix_receipt_logcontext ... A logformatter which includes the stack when the exception was caught when logging exceptions.	2017-10-11 17:28:01 +01:00
Richard van der Hoff	f30c4ed2bc	logformatter: fix AttributeError ... make sure we have the relevant fields before we try to log them.	2017-10-11 17:26:17 +01:00
Erik Johnston	b752507b48	Fix fetching remote summaries	2017-10-11 16:59:18 +01:00
Erik Johnston	818b08d0e4	peeeeeeeeep8888888888888888888888888888	2017-10-11 15:54:00 +01:00
Erik Johnston	ea18996f54	Fix group stream replication ... The stream update functions expect the storage function to return a list of tuples.	2017-10-11 15:44:39 +01:00
Richard van der Hoff	68fd82e840	Merge pull request #2530 from matrix-org/rav/fix_receipt_logcontext ... fix a logcontext leak in read receipt handling	2017-10-11 15:08:53 +01:00
Richard van der Hoff	4fad8efbfb	Fix stackoverflow and logcontexts from linearizer ... 1. make it not blow out the stack when there are more than 50 things waiting for a lock. Fixes https://github.com/matrix-org/synapse/issues/2505. 2. Make it not mess up the log contexts.	2017-10-11 15:05:05 +01:00
David Baker	b78bae2d51	fix isinstance	2017-10-11 14:49:09 +01:00
Erik Johnston	271f5601f3	Fix typo in invite to group	2017-10-11 14:45:33 +01:00
David Baker	c3b7a45e84	Allow error strings from spam checker	2017-10-11 14:39:22 +01:00
Richard van der Hoff	c3e190ce67	fix a logcontext leak in read receipt handling	2017-10-11 14:37:20 +01:00
Richard van der Hoff	b75d443caf	log pdu_failures from incoming transactions ... ... even if we have no EDUs. This appears to have been introduced in 476899295f.	2017-10-11 14:36:13 +01:00
Erik Johnston	27e727a146	Fix typo	2017-10-11 14:32:40 +01:00
Erik Johnston	4ce4379235	Fix attestations to check correct server name	2017-10-11 14:11:43 +01:00
Erik Johnston	c2c47550f9	Fix schema delta versions	2017-10-11 13:23:15 +01:00
Erik Johnston	535cc49f27	Merge pull request #2466 from matrix-org/erikj/groups_merged ... Initial Group Implementation	2017-10-11 13:20:07 +01:00
Erik Johnston	dfbf73408c	Merge pull request #2501 from matrix-org/dbkr/channel_notifications ... Support for channel notifications	2017-10-11 13:19:29 +01:00
Erik Johnston	bc7f3eb32f	Merge pull request #2483 from jeremycline/unfreeze-ujson-dump ... Unfreeze event before serializing with ujson	2017-10-11 13:18:52 +01:00
Erik Johnston	ec954f47fb	Validate room ids	2017-10-11 13:15:44 +01:00
David Baker	81a5e0073c	pep8	2017-10-10 15:53:34 +01:00
David Baker	ab1bc9bf5f	Don't KeyError if no power_levels event	2017-10-10 15:34:05 +01:00
David Baker	0f1eb3e914	Use notification levels in power_levels ... Rather than making the condition directly require a specific power level. This way the level require to notify a room can be configured per room.	2017-10-10 15:23:00 +01:00
Erik Johnston	84e27a592d	Merge pull request #2490 from matrix-org/erikj/drop_left_room_events ... Ignore incoming events for rooms that we have left	2017-10-10 11:58:32 +01:00
David Baker	c9f034b4ac	There was already a constant for this ... also update copyright	2017-10-10 11:47:10 +01:00
David Baker	a9f9d68631	More optimisation	2017-10-10 11:38:31 +01:00
David Baker	707374d5dc	What year is it!? Who's the president!?	2017-10-10 11:21:41 +01:00
David Baker	89fa00ddff	Merge branch 'develop' into dbkr/channel_notifications	2017-10-10 11:20:17 +01:00
Richard van der Hoff	6a6cc27aee	fed server: process PDUs for different rooms in parallel ... With luck, this will give a real-time improvement when there are many rooms and the server ends up calling out to fetch missing events.	2017-10-09 18:30:31 +01:00
Richard van der Hoff	4c7c4d4061	Fed server: use a linearizer for ongoing transactions ... We don't want to process the same transaction multiple times concurrently, so use a linearizer.	2017-10-09 18:30:10 +01:00
Richard van der Hoff	ba5b9b80a5	fed server: refactor on_incoming_transaction ... Move as much as possible to after the have_responded check, and reduce the number of times we iterate over the pdu list.	2017-10-09 18:10:53 +01:00
Richard van der Hoff	a6e3222fe5	Fed server: Move origin-check code to _handle_received_pdu ... The response-building code expects there to be an entry in the `results` list for each entry in the pdu_list, so the early `continue` was messing this up. That doesn't really matter, because all that the federation client does is log any errors, but it's pretty poor form.	2017-10-09 17:53:32 +01:00
Richard van der Hoff	3cc852d339	Fancy logformatter to format exceptions better ... This is a bit of an experimental change at this point; the idea is to see if it helps us track down where our stack overflows are coming from by logging the stack when the exception was caught and turned into a Failure. (We'll also need edf2704420). If we deploy this, we'll be able to enable it via the log config yaml.	2017-10-09 17:44:42 +01:00
Richard van der Hoff	0eeaa25694	Merge pull request #2508 from matrix-org/rav/federation_queue_logcontexts ... Fix up logcontext handling in (federation) TransactionQueue	2017-10-09 17:43:48 +01:00
Richard van der Hoff	aa3fac8057	Merge pull request #2507 from matrix-org/rav/execute_concurrently_log_contexts ... Fix logcontext handling for concurrently_execute	2017-10-09 17:43:32 +01:00
Erik Johnston	e8496efe84	Fix up comment	2017-10-09 15:17:34 +01:00
Richard van der Hoff	01bbacf3c4	Fix up logcontext handling in (federation) TransactionQueue ... Avoid using preserve_context_over_function, which has problems with respect to logcontexts.	2017-10-06 22:39:25 +01:00
Richard van der Hoff	148428ce76	Fix logcontext handling for concurrently_execute ... Avoid preserve_context_over_deferred, which is broken.	2017-10-06 22:24:28 +01:00
Richard van der Hoff	c8f568ddf9	Fix up deferred handling in federation.py ... * Avoid preserve_context_over_deferred, which is broken * set consumeErrors=True on defer.gatherResults, to avoid spurious "unhandled failure" erros	2017-10-06 22:14:24 +01:00
Richard van der Hoff	3ddda939d3	some comments in the state res code	2017-10-05 14:58:17 +01:00
David Baker	f878e6f8af	Spam checking: add the invitee to user_may_invite	2017-10-05 14:02:28 +01:00
David Baker	269af961e9	Make be faster	2017-10-05 13:27:12 +01:00
David Baker	ed80c6b6cc	Add fastpath optimisation	2017-10-05 13:20:22 +01:00
David Baker	e433393c4f	pep8	2017-10-05 13:08:02 +01:00
David Baker	985ce80375	They're called rooms	2017-10-05 13:03:44 +01:00
David Baker	b9b9714fd5	Get rule type right	2017-10-05 13:02:19 +01:00
David Baker	fa969cfdde	Support for channel notifications ... Add condition type to check the sender's power level and add a base rule using it for @channel notifications.	2017-10-05 12:39:18 +01:00
David Baker	0c8da8b519	Use better method for word boundary searching ... From ebc95667b8	2017-10-05 11:57:43 +01:00
Erik Johnston	eaaa837e00	Don't corrupt cache	2017-10-05 11:43:22 +01:00
David Baker	cbe3c3fdd4	pep8	2017-10-05 11:43:10 +01:00
David Baker	6748f0a579	Fix notif kws that start/end with non-word chars ... Only prepend / append word bounary characters if the search expression starts or ends with a word character, otherwise they don't work because there's no word bounary between whitespace and a non-word char.	2017-10-05 11:33:30 +01:00
David Baker	d8ce68b09b	spam check room publishing	2017-10-04 14:29:33 +01:00
David Baker	78d4ced829	un-double indent	2017-10-04 12:44:27 +01:00
David Baker	197c14dbcf	Add room creation checks to spam checker ... Lets the spam checker deny attempts to create rooms and add aliases to them.	2017-10-04 10:47:54 +01:00
David Baker	1e2ac54351	s/roomid/room_id/	2017-10-03 17:41:38 +01:00
David Baker	1e375468de	pass room id too	2017-10-03 17:13:14 +01:00
David Baker	c2c188b699	Federation was passing strings anyway ... so pass string everywhere	2017-10-03 15:46:19 +01:00
David Baker	c46a0d7eb4	this shouldn't be debug	2017-10-03 15:20:14 +01:00
David Baker	bd769a81e1	better logging	2017-10-03 15:16:40 +01:00
David Baker	537088e7dc	Actually write warpper function	2017-10-03 14:28:12 +01:00
David Baker	41fd9989a2	Skip spam check for admin users	2017-10-03 14:17:44 +01:00
Erik Johnston	11d62f43c9	Invalidate cache	2017-10-03 14:12:28 +01:00
Erik Johnston	e4ab96021e	Update comments	2017-10-03 14:10:41 +01:00
David Baker	2a7ed700d5	Fix param name & lint	2017-10-03 14:04:10 +01:00
David Baker	84716d267c	Allow spam checker to reject invites too	2017-10-03 13:56:43 +01:00
Erik Johnston	f2da6df568	Remove spurious line feed	2017-10-03 11:31:06 +01:00
Erik Johnston	30848c0fcd	Ignore incoming events for rooms that we have left ... When synapse receives an event for a room its not in over federation, it double checks with the remote server to see if it is in fact in the room. This is done so that if the server has forgotten about the room (usually as a result of the database being dropped) it can recover from it. However, in the presence of state resets in large rooms, this can cause a lot of work for servers that have legitimately left. As a hacky solution that supports both cases we drop incoming events for rooms that we have explicitly left. This means that we no longer support the case of servers having forgotten that they've rejoined a room, but that is sufficiently rare that we're not going to support it for now.	2017-10-03 11:18:21 +01:00
Erik Johnston	e585c83209	Merge branch 'master' of github.com:matrix-org/synapse into develop	2017-10-02 18:11:24 +01:00
Erik Johnston	6c1bb1601e	Bump version and changelog	2017-10-02 18:05:17 +01:00
Erik Johnston	ea87cb1ba5	Make 'affinity' package optional	2017-10-02 18:03:59 +01:00
Erik Johnston	3fed5bb25f	Move quit_with_error	2017-10-02 17:59:34 +01:00
David Baker	27955056e0	Merge branch 'develop' into erikj/groups_merged	2017-10-02 16:20:41 +01:00
Erik Johnston	90d70af269	Merge branch 'master' of github.com:matrix-org/synapse into develop	2017-10-02 16:20:23 +01:00
Erik Johnston	e4a709eda3	Bump version and change log	2017-10-02 13:51:38 +01:00
Jeremy Cline	cafb8de132	Unfreeze event before serializing with ujson ... In newer versions of https://github.com/esnme/ultrajson, ujson does not serialize frozendicts (introduced in esnme/ultrajson@53f85b1). Although the PyPI version is still 1.35, Fedora ships with a build from commit esnme/ultrajson@2f1d487. This causes the serialization to fail if the distribution-provided package is used. This runs the event through the unfreeze utility before serializing it. Thanks to @ignatenkobrain for tracking down the root cause. fixes #2351 Signed-off-by: Jeremy Cline <jeremy@jcline.org>	2017-09-30 11:22:37 -04:00
Richard van der Hoff	d5325d7ef1	Merge pull request #2480 from matrix-org/rav/federation_client_logging ... Improve logging of failures in matrixfederationclient	2017-09-29 17:32:53 +01:00
Erik Johnston	d5694ac5fa	Only log if we've removed media	2017-09-28 16:08:08 +01:00
Richard van der Hoff	e43de3ae4b	Improve logging of failures in matrixfederationclient ... * don't log exception types twice * not all exceptions have a meaningful 'message'. Use the repr rather than attempting to build a string ourselves.	2017-09-28 15:38:09 +01:00
Richard van der Hoff	75e67b9ee4	Handle SERVFAILs when doing AAAA lookups for federation (#2477) ... ... to cope with people with broken dnssec setups, mostly	2017-09-28 15:24:00 +01:00
Erik Johnston	768f00dedb	Up the limits on number of url cache entries to delete at one time	2017-09-28 14:27:27 +01:00
Erik Johnston	4dc07e93a8	Add old indices	2017-09-28 14:10:33 +01:00
Erik Johnston	7cc483aa0e	Clear up expired url cache every 10s	2017-09-28 13:56:53 +01:00
Erik Johnston	e1e7d76cf1	Actually assign result to variable	2017-09-28 13:55:29 +01:00
Erik Johnston	93247a424a	Only pull out local media that were for url cache	2017-09-28 13:48:14 +01:00
Erik Johnston	5f501ec7e2	Fix typo in url cache expiry timer	2017-09-28 12:59:01 +01:00
Erik Johnston	761d255fdf	Merge pull request #2479 from matrix-org/erikj/expire_url_cache_thumbnails ... Support new and old style media id formats	2017-09-28 12:58:13 +01:00
Erik Johnston	ace8079086	Support new and old style media id formats	2017-09-28 12:52:51 +01:00
Erik Johnston	7a44c01d89	Fix typo	2017-09-28 12:46:04 +01:00
Erik Johnston	ae79764fe5	Change expires column to expires_ts	2017-09-28 12:37:53 +01:00
Erik Johnston	77f1d24de3	More brackets	2017-09-28 12:23:15 +01:00
Erik Johnston	9ccb4226ba	Delete expired url cache data	2017-09-28 12:18:06 +01:00
Erik Johnston	8090fd4664	Fix /joined_members to work with AS users	2017-09-28 10:09:32 +01:00
Erik Johnston	adec03395d	Fix bug where /joined_members didn't check user was in room	2017-09-27 15:14:39 +01:00
David Baker	ef3a5ae787	Don't test is spam_checker not None ... Sometimes it's a Mock object which is not none but is still not what we're after	2017-09-27 11:24:19 +01:00
David Baker	8c06dd6071	Remove unintentional debugging	2017-09-27 10:31:14 +01:00
David Baker	60c78666ab	pep8	2017-09-27 10:26:13 +01:00
David Baker	1786b0e768	Forgot the new file again :(	2017-09-27 10:22:54 +01:00
David Baker	6cd5fcd536	Make the spam checker a module	2017-09-26 19:20:23 +01:00
David Baker	9fd086e506	unnecessary parens	2017-09-26 17:59:46 +01:00
David Baker	0b03a97708	Add module_loader.py	2017-09-26 17:56:41 +01:00
David Baker	4824a33c31	Factor out module loading to a separate place ... So it can be reused	2017-09-26 17:51:26 +01:00
Erik Johnston	17b8e2bd02	Add remove room API	2017-09-26 15:52:41 +01:00
Erik Johnston	a8e2a3df32	Add unique index to group_rooms table	2017-09-26 15:39:21 +01:00
Erik Johnston	95298783bb	Add is_publicised to group summary	2017-09-26 11:04:37 +01:00
Erik Johnston	1a398b19fd	Merge branch 'develop' of github.com:matrix-org/synapse into release-v0.23.0	2017-09-26 10:08:59 +01:00
Erik Johnston	f4c8cd5e85	Bump changelog and version	2017-09-26 10:02:48 +01:00
Erik Johnston	e3edca3b5d	Refactor to speed up incremental syncs	2017-09-25 17:35:39 +01:00
Erik Johnston	44edac0497	Merge branch 'release-v0.23.0' of github.com:matrix-org/synapse into develop	2017-09-25 14:52:46 +01:00
Richard van der Hoff	94133d7ce8	Merge branch 'develop' into develop	2017-09-25 11:50:11 +01:00
Erik Johnston	ba8fdc925c	Bump version and changes	2017-09-25 11:01:31 +01:00
Richard van der Hoff	79b3cf3e02	Fix logcontxt leak in keyclient (#2465) ... preserve_context_over_function doesn't do what you want it to do.	2017-09-25 09:51:39 +01:00
Richard van der Hoff	f65e31d22f	Do an AAAA lookup on SRV record targets (#2462) ... Support SRV records which point at AAAA records, as well as A records. Fixes https://github.com/matrix-org/synapse/issues/2405	2017-09-22 20:26:47 +01:00
Matthew Hodgson	f496399ac4	fix thinko'd docstring	2017-09-22 15:34:14 +01:00
Erik Johnston	3166ed55b2	Fix device list when rejoining room (#2461)	2017-09-22 14:44:17 +01:00
Erik Johnston	e1dec2f1a7	Remove user from group summary when the leave the group	2017-09-21 16:09:57 +01:00
Erik Johnston	bb746a9de1	Revert: Keep room_id's in group summary	2017-09-21 15:57:22 +01:00
Erik Johnston	ae8d4bb0f0	Keep room_id's in group summary	2017-09-21 15:55:18 +01:00
Erik Johnston	197d82dc07	Correctly return next token	2017-09-20 11:12:11 +01:00
Erik Johnston	069ae2df12	Fix initial sync	2017-09-20 10:52:12 +01:00
Richard van der Hoff	6de74ea6d7	Fix logcontexts in _check_sigs_and_hashes	2017-09-20 01:32:42 +01:00
Richard van der Hoff	c5c24c239b	Fix logcontext handling in verify_json_objects_for_server ... preserve_context_over_fn is essentially broken, because (a) it pointlessly drops the current logcontext before calling its wrapped function, which means we don't get any useful logcontexts for _handle_key_deferred; (b) it wraps the resulting deferred in a _PreservingContextDeferred, which is very dangerous because you then can't yield on it without leaking context back into the reactor. Instead, let's specify that the resultant deferreds call their callbacks with no logcontext.	2017-09-20 01:32:42 +01:00
Richard van der Hoff	c5b0e9f485	Turn _start_key_lookups into an inlineCallbacks function ... ... which means that logcontexts can be correctly preserved for the stuff it does. get_server_verify_keys is now called with the logcontext, so needs to preserve_fn when it fires off its nested inlineCallbacks function. Also renames get_server_verify_keys to reflect the fact it's meant to be private.	2017-09-20 01:32:42 +01:00
Richard van der Hoff	abdefb8a01	Fix potential race in _start_key_lookups ... If the verify_request.deferred has already completed, then `remove_deferreds` will be called immediately. It therefore might resolve the server_to_deferred deferred while there are still other requests for that server in flight. To avoid that, we should build the complete list of requests, and *then* add the callbacks.	2017-09-20 01:32:42 +01:00
Richard van der Hoff	afbd773dc6	Add some comments to _start_key_lookups	2017-09-20 01:32:42 +01:00
Richard van der Hoff	2a4b9ea233	Consistency for how verify_request.deferred is called ... Define that it is run with no log context, and make sure that happens. If we aren't careful to reset the logcontext, we can't bung the deferreds into defer.gatherResults etc. We don't actually do that directly, but we *do* resolve other deferreds from affected callbacks (notably the server_to_deferred map in _start_key_lookups), and those *do* get passed into defer.gatherResults. It turns out that this way ends up being least confusing.	2017-09-20 01:32:42 +01:00
Richard van der Hoff	3b98439eca	Factor out _start_key_lookups ... ... to make it easier to see what's going on.	2017-09-20 01:32:42 +01:00
Richard van der Hoff	fde63b880d	Replace server_and_json with verify_requests ... This is a precursor to factoring some of this code out.	2017-09-20 01:32:42 +01:00
Richard van der Hoff	2d511defd9	pull out handle_key_deferred to top level ... There's no need for this to be a nested definition; pulling it out not only makes it more efficient, but makes it easier to check that it's not accessing any local variables it shouldn't be.	2017-09-20 01:32:42 +01:00
Richard van der Hoff	dd1ea9763a	Fix incorrect key_ids in error message	2017-09-20 01:32:42 +01:00
Richard van der Hoff	e76d1135dd	Invalidate signing key cache when we gat an update ... This might make the cache slightly more efficient.	2017-09-20 01:32:42 +01:00
Richard van der Hoff	fcf2c0fd1a	Remove redundant preserve_fn ... preserve_fn is a no-op unless the wrapped function returns a Deferred. verify_json_objects_for_server returns a list, so this is doing nothing.	2017-09-20 01:32:42 +01:00
Richard van der Hoff	9864efa532	Fix concurrent server_key requests (#2458) ... Fix a bug where we could end up firing off multiple requests for server_keys for the same server at the same time.	2017-09-19 23:25:44 +01:00
Richard van der Hoff	aa620d09a0	Add a config option to block all room invites (#2457) ... - allows sysadmins the ability to lock down their servers so that people can't send their users room invites.	2017-09-19 16:08:14 +01:00
Richard van der Hoff	2eabdf3f98	add some comments to on_exchange_third_party_invite_request	2017-09-19 12:20:36 +01:00
Richard van der Hoff	5ed109d59f	PoC for filtering spammy events (#2456) ... Demonstration of how you might add some hooks to filter out spammy events.	2017-09-19 12:20:11 +01:00
Erik Johnston	93e504d04e	Ensure that creator of group sees group down /sync	2017-09-19 11:08:16 +01:00
Erik Johnston	b5feaa5a49	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/groups_merged	2017-09-19 11:07:45 +01:00
Richard van der Hoff	3f405b34e9	Fix overzealous kicking of guest users (#2453) ... We should only kick guest users if the guest access event is authorised.	2017-09-19 08:52:52 +01:00
Richard van der Hoff	290777b3d9	Clean up and document handling of logcontexts in Keyring (#2452) ... I'm still unclear on what the intended behaviour for `verify_json_objects_for_server` is, but at least I now understand the behaviour of most of the things it calls...	2017-09-18 18:31:01 +01:00
Erik Johnston	77c81ca6ea	Merge pull request #2451 from matrix-org/erikj/add_state_to_timeline ... Don't filter out current state events from timeline	2017-09-18 17:22:33 +01:00
Erik Johnston	2d1b7955ae	Don't filter out current state events from timeline	2017-09-18 17:13:03 +01:00
David Baker	862c8da560	Merge pull request #2450 from matrix-org/dbkr/push_event_id_only ... Add support for event_id_only push format	2017-09-18 16:41:29 +01:00
David Baker	436ee0a2ea	Also include the room_id ... as really it's part of the event ID	2017-09-18 15:58:38 +01:00
David Baker	b393f5db51	Use .get - it's much shorter	2017-09-18 15:50:26 +01:00
David Baker	a2562f9d74	Add support for event_id_only push format ... Param in the data dict of a pusher that tells an HTTP pusher to send just the event_id of the event it's notifying about and the notification counts. For clients that want to go & fetch the body of the event themselves anyway.	2017-09-18 15:39:39 +01:00
Erik Johnston	d6dadd95ac	Correctly handle leaving room in /key/changes	2017-09-18 15:38:22 +01:00
Erik Johnston	4a94eb3ea4	Fix typo	2017-09-15 09:56:54 +01:00
Erik Johnston	3a0cee28d6	Actually hook leave notifs up	2017-09-14 11:49:37 +01:00
Erik Johnston	4f845a0713	Handle joining/leaving rooms in /keys/changes	2017-09-13 16:28:08 +01:00
Erik Johnston	473700f016	Get left rooms	2017-09-13 15:13:41 +01:00
Erik Johnston	9ce866ed4f	In sync handle device lists for newly joined/left rooms	2017-09-12 16:44:26 +01:00
Erik Johnston	69ef4987a6	Add left section to /keys/changes	2017-09-08 14:44:36 +01:00
Erik Johnston	53cc8ad35a	Send down device list change notif when member leaves/rejoins room	2017-09-07 15:08:39 +01:00
Erik Johnston	4a9b1cf253	Add user profiles to summary from group server	2017-08-25 16:23:58 +01:00
Erik Johnston	258409ef61	Fix typos and reinherit	2017-08-25 14:45:20 +01:00
Erik Johnston	bf81f3cf2c	Split out profile handler to fix tests	2017-08-25 14:34:56 +01:00
Erik Johnston	27ebc5c8f2	Add remote profile cache	2017-08-25 11:25:47 +01:00
Erik Johnston	97c544f91f	Add _simple_update	2017-08-25 11:11:37 +01:00
Richard van der Hoff	a7effa8400	Merge pull request #2288 from kyrias/bcrypt ... python_dependencies: Use bcrypt module instead of py-bcrypt	2017-08-23 14:14:56 +01:00
Erik Johnston	2800983f3e	Merge pull request #2410 from matrix-org/erikj/groups_publicise ... Add ability to publicise group membership	2017-08-21 16:51:56 +01:00
Erik Johnston	8b50fe5330	Use BOOLEAN rather than TEXT type	2017-08-21 16:37:29 +01:00
Erik Johnston	175a01f56c	Groups: Fix mising json.load in initial sync	2017-08-21 14:45:56 +01:00
Richard van der Hoff	692250c6be	Fix user_dir startup ... Add missing parameter to _base.start_worker_reactor	2017-08-16 15:11:29 +01:00
Richard van der Hoff	d2352347cf	Fix process startup ... escape the % that got added in 92168cb so that the process starts up ok.	2017-08-16 14:57:35 +01:00
Matthew Hodgson	92168cbbc5	explain why CPU affinity is a good idea	2017-08-15 18:27:42 +01:00
Richard van der Hoff	10d8b701a1	Allow configuration of CPU affinity ... Make it possible to set the CPU affinity in the config file, so that we don't need to remember to do it manually every time.	2017-08-15 17:08:28 +01:00
Richard van der Hoff	543c794a76	Factor out common application start ... We have 10 copies of this code, and I don't really want to update each one separately.	2017-08-15 17:04:40 +01:00
Erik Johnston	ba3ff7918b	Fixup	2017-08-11 13:42:42 +01:00
Erik Johnston	ef8e578677	Add bulk group publicised lookup API	2017-08-09 13:36:22 +01:00
Erik Johnston	b880ff190a	Allow update group publicity	2017-08-08 14:19:41 +01:00
Erik Johnston	05e21285aa	Store whether the user wants to publicise their membership of a group	2017-08-08 13:01:46 +01:00
hera	eae04f1952	fix english	2017-08-04 23:56:42 +01:00
hera	5699b05072	typo	2017-08-04 23:44:37 +01:00
Erik Johnston	a1e67bcb97	Remove stale TODO comments	2017-08-04 10:07:10 +01:00
Erik Johnston	09552f9d9c	Reduce spammy log line in synchrotrons	2017-08-02 17:29:51 +01:00
Kenny Keslar	f18373dc5d	Fix iteration of requests_missing_keys; list doesn't have .values() ... Signed-off-by: Kenny Keslar <r3dey3@r3dey3.com>	2017-07-26 22:44:19 -05:00
Erik Johnston	966a70f1fa	Update comment	2017-07-24 17:49:39 +01:00
Erik Johnston	629cdfb124	Use join rather than joined, etc.	2017-07-24 14:54:05 +01:00
Erik Johnston	ed666d3969	Fix all the typos	2017-07-24 14:05:09 +01:00
Erik Johnston	b76ef6ccb8	Include users membership in group in summary API	2017-07-24 13:55:39 +01:00
Erik Johnston	851aeae7c7	Check users/rooms are in group before adding to summary	2017-07-24 13:40:56 +01:00
Erik Johnston	d5e32c843f	Correctly add joins to correct segment	2017-07-24 13:31:26 +01:00
Erik Johnston	96917d5552	Merge pull request #2378 from matrix-org/erikj/group_sync_support ... Add groups to sync stream	2017-07-21 11:05:39 +01:00
Erik Johnston	0401604222	Merge pull request #2377 from matrix-org/erikj/group_profile_update ... Add update group profile API	2017-07-20 17:53:39 +01:00
Erik Johnston	b238cf7f6b	Remove spurious content param	2017-07-20 17:49:55 +01:00
Erik Johnston	960dae3340	Add notifier	2017-07-20 17:14:44 +01:00
Erik Johnston	2cc998fed8	Fix replication. And notify	2017-07-20 17:13:18 +01:00
Erik Johnston	139fe30f47	Remember to cast to bool	2017-07-20 16:47:35 +01:00
Erik Johnston	4d793626ff	Fix bug in generating current token	2017-07-20 16:42:44 +01:00
Erik Johnston	c544188ee3	Add groups to sync stream	2017-07-20 16:36:42 +01:00
Erik Johnston	0ab153d201	Check values are strings	2017-07-20 16:24:18 +01:00
Erik Johnston	8209b5f033	Fix a storage desc	2017-07-20 16:22:22 +01:00
Erik Johnston	57826d645b	Fix typo	2017-07-20 13:15:22 +01:00
Erik Johnston	6f443a74cf	Add update group profile API	2017-07-20 09:46:33 +01:00
Erik Johnston	14a34f12d7	Comments	2017-07-18 17:28:42 +01:00
Erik Johnston	3431ec55dc	Comments	2017-07-18 17:23:50 +01:00
Erik Johnston	6027b1992f	Fix permissions	2017-07-18 16:51:25 +01:00
Erik Johnston	e884ff31d8	Add DELETE	2017-07-18 16:41:44 +01:00
Erik Johnston	05c13f6c22	Add 'args' param to post_json	2017-07-18 16:40:21 +01:00
Erik Johnston	94ecd871a0	Fix typos	2017-07-18 16:38:54 +01:00
Erik Johnston	12ed4ee48e	Correctly parse query params	2017-07-18 15:33:09 +01:00
Erik Johnston	332839f6ea	Update federation client pokes	2017-07-18 14:45:37 +01:00
Erik Johnston	e5ea6dd021	Add client apis	2017-07-18 14:37:06 +01:00
Erik Johnston	cccfcfa7b9	Comments	2017-07-18 10:35:18 +01:00
Erik Johnston	68f34e85ce	Use transport client directly	2017-07-18 10:29:57 +01:00
Erik Johnston	3e703eb04e	Comment	2017-07-18 10:17:25 +01:00
Erik Johnston	508460f240	Remove sync stuff	2017-07-18 09:55:46 +01:00
Erik Johnston	6e9f147faa	Add GroupID type	2017-07-18 09:47:25 +01:00
Erik Johnston	4540730111	Remove unused tables	2017-07-18 09:38:15 +01:00
Erik Johnston	e96ee95a7e	Remove sync stuff	2017-07-18 09:38:08 +01:00
Erik Johnston	2f9eafdd36	Add local group server support	2017-07-17 12:03:49 +01:00
Erik Johnston	bfde076022	Increase cache hit ratio for push ... We don't update the cache in all code paths, which causes subsequent calls to miss the cache	2017-07-14 16:11:26 +01:00
Erik Johnston	cb3aee8219	Ensure category and role ids are non-null	2017-07-14 14:06:55 +01:00
Erik Johnston	85fda57208	Add DEFAULT_ROLE_ID	2017-07-14 14:03:54 +01:00
Erik Johnston	4b203bdba5	Correctly increment orders	2017-07-14 14:02:31 +01:00
Erik Johnston	8d26385d76	Add more metrics to push rule evaluation	2017-07-13 14:37:30 +01:00
Erik Johnston	3b0470dba5	Remove unused functions	2017-07-13 13:54:01 +01:00
Erik Johnston	8575e3160f	Comments	2017-07-13 13:52:41 +01:00
Erik Johnston	f60218ec41	Push: Don't acquire lock unless necessary	2017-07-13 11:23:53 +01:00
Erik Johnston	a78cda4baf	Remove TODO	2017-07-13 11:17:07 +01:00
Erik Johnston	7a39da8cc6	Add summary APIs to federation	2017-07-13 11:13:19 +01:00
Erik Johnston	5bbb53580a	raise NotImplementedError	2017-07-13 10:25:29 +01:00
Erik Johnston	26451a09eb	Comments	2017-07-12 14:47:18 +01:00
Erik Johnston	8d55877c9e	Simplify checking if admin	2017-07-12 11:43:39 +01:00
Erik Johnston	a62406aaa5	Add group summary APIs	2017-07-12 11:36:15 +01:00
Erik Johnston	e9aec001f4	Use less DB for device list handling in sync	2017-07-12 10:30:10 +01:00
Erik Johnston	28e8c46f29	Merge pull request #2352 from matrix-org/erikj/group_server_split ... Initial Group Server	2017-07-12 10:26:29 +01:00
Erik Johnston	6d586dc05c	Comment	2017-07-12 09:58:37 +01:00
Erik Johnston	410b4e14a1	Move comment	2017-07-11 15:44:18 +01:00
Erik Johnston	fe4e885f54	Add federation API for adding room to group	2017-07-11 14:35:07 +01:00
Erik Johnston	bbb739d24a	Comment	2017-07-11 14:31:36 +01:00
Erik Johnston	26752df503	Typo	2017-07-11 14:29:03 +01:00
Erik Johnston	e52c391cd4	Rename column to attestation_json	2017-07-11 14:25:46 +01:00
Erik Johnston	0aac30d53b	Comments	2017-07-11 14:23:50 +01:00
Erik Johnston	0184a97dbd	Merge pull request #2354 from krombel/reduce_static_sync_reply ... encode sync-response statically	2017-07-11 14:19:56 +01:00
Krombel	85b9f76f1d	split out reducing stuff; just make encode_* static	2017-07-11 13:14:35 +02:00
Erik Johnston	6322fbbd41	Comment	2017-07-11 11:52:03 +01:00
Erik Johnston	8ba89f1050	Remove u/ requirement	2017-07-11 11:45:32 +01:00
Erik Johnston	429925a5e9	Lift out visibility parsing	2017-07-11 11:44:08 +01:00
Erik Johnston	83936293eb	Comments	2017-07-11 11:42:25 +01:00
Erik Johnston	e2cb760dcc	Merge pull request #2357 from matrix-org/erikj/push ... Don't compute push actions for backfilled events	2017-07-11 10:53:22 +01:00
Erik Johnston	925b3638ff	Reduce log levels in tcp replication	2017-07-11 10:04:21 +01:00
Erik Johnston	9a6fd3ef29	Don't compute push actions for backfilled events	2017-07-11 10:02:21 +01:00
Krombel	2f82de18ee	fix test	2017-07-10 17:34:58 +02:00
Erik Johnston	b8ca494ee9	Initial group server implementation	2017-07-10 15:44:15 +01:00
Krombel	6e16aca8b0	encode sync-response statically; omit empty objects from sync-response	2017-07-10 16:42:17 +02:00
Erik Johnston	d4d12daed9	Include registration and as stores in frontend proxy	2017-07-07 18:36:45 +01:00
Erik Johnston	f467a8f66d	Merge branch 'master' of github.com:matrix-org/synapse into develop	2017-07-07 18:26:28 +01:00
Erik Johnston	1fc4a962e4	Add a frontend proxy	2017-07-07 18:19:46 +01:00
Erik Johnston	f502b0dea1	Perf: Don't filter events for push ... We know the users are joined and we can explicitly check for if they are ignoring the user, so lets do that.	2017-07-07 14:04:40 +01:00
Erik Johnston	76ed3476d3	Bump version and changelog	2017-07-06 18:11:22 +01:00
Erik Johnston	58dc1f2c78	Merge pull request #2342 from matrix-org/erikj/pusher_pool_instantiate ... Fix bug where pusherpool didn't start and broke some rooms	2017-07-06 18:08:43 +01:00
Erik Johnston	5a7f561a9b	Fix bug where pusherpool didn't start and broke some rooms ... Since we didn't instansiate the PusherPool at start time it could fail at run time, which it did for some users. This may or may not fix things for those users, but it should happen at start time and stop the server from starting.	2017-07-06 17:55:51 +01:00
Erik Johnston	ed9a7f5436	Merge pull request #2309 from matrix-org/erikj/user_ip_repl ... Fix up user_ip replication commands	2017-07-06 14:33:14 +01:00
Erik Johnston	1f64207f26	Merge branch 'master' of github.com:matrix-org/synapse into develop	2017-07-06 13:57:45 +01:00
Erik Johnston	6264cf9666	Bump version and changelog	2017-07-06 10:35:56 +01:00
Erik Johnston	f386632800	Merge pull request #2334 from matrix-org/erikj/refactor_transport_server ... Separate federation servlet into different lists	2017-07-05 17:09:07 +01:00
Erik Johnston	5e49a57ecc	Separate federation servlet into different lists	2017-07-05 14:32:24 +01:00
Richard van der Hoff	3d31b39297	Merge pull request #2332 from matrix-org/rav/fix_pushes ... Fix caching error in the push evaluator	2017-07-05 11:10:53 +01:00
Richard van der Hoff	73cfe48031	Fix caching error in the push evaluator ... Initialising `result` to `{}` in the parameters meant that every call to _flatten_dict used the *same* target dictionary. I'm hopeful this will fix https://github.com/matrix-org/synapse/issues/2270, but I suspect it won't. (This code seems to have been here since forever, unlike the bug, and I don't really think it explains the observed behaviour). Still, it makes it hard to investigate the problem.	2017-07-05 00:28:43 +01:00
Erik Johnston	05538587ef	Bump version and changelog	2017-07-04 14:02:21 +01:00
Erik Johnston	f92d7416d7	Merge pull request #2330 from matrix-org/erikj/cache_size_factor ... Increase default cache size	2017-07-04 10:51:21 +01:00
Erik Johnston	495f075b41	Increase default cache factor size.	2017-07-04 09:58:32 +01:00
Erik Johnston	b5e8d529e6	Define CACHE_SIZE_FACTOR once	2017-07-04 09:56:44 +01:00
Mark Haines	3e279411fe	Improve the error handling for bad invites received over federation	2017-06-30 16:20:30 +01:00
Erik Johnston	6ff14ddd2e	Make into list	2017-06-29 15:47:37 +01:00
Erik Johnston	5946aa0877	Prefill forward extrems and event to state groups	2017-06-29 15:38:48 +01:00
Erik Johnston	2c365f4723	Cache macaroon parse and validation ... Turns out this can be quite expensive for requests, and is easily cachable. We don't cache the lookup to the DB so invalidation still works.	2017-06-29 14:50:18 +01:00
Erik Johnston	c72058bcc6	Use an ExpiringCache for storing registration sessions ... This is because pruning them was a significant performance drain on matrix.org	2017-06-29 14:08:37 +01:00
Erik Johnston	27f26e48b7	Serialize user ip command as json	2017-06-27 16:25:38 +01:00
Erik Johnston	8c23221666	Fix up	2017-06-27 15:53:45 +01:00
Erik Johnston	731f3c37a0	Merge branch 'release-v0.22.0' of github.com:matrix-org/synapse into develop	2017-06-27 15:41:34 +01:00
Erik Johnston	4b444723f0	Merge pull request #2308 from matrix-org/erikj/user_ip_repl ... Make workers report to master for user ip updates	2017-06-27 15:36:47 +01:00
Erik Johnston	816605a137	Merge pull request #2307 from matrix-org/erikj/user_ip_batch ... Batch upsert user ips	2017-06-27 15:08:32 +01:00
Erik Johnston	78cefd78d6	Make workers report to master for user ip updates	2017-06-27 14:58:10 +01:00
Erik Johnston	a0a561ae85	Fix up client ips to read from pending data	2017-06-27 14:46:12 +01:00
Erik Johnston	ed3d0170d9	Batch upsert user ips	2017-06-27 13:37:04 +01:00
Erik Johnston	976128f368	Update version and changelog	2017-06-26 16:14:56 +01:00
Erik Johnston	d04d672a80	Merge pull request #2290 from matrix-org/erikj/ensure_round_trip ... Reject local events that don't round trip the DB	2017-06-26 15:12:02 +01:00
Erik Johnston	1bce3e6b35	Remove unused variables	2017-06-26 14:03:27 +01:00
Erik Johnston	e3cbec10c1	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/ensure_round_trip	2017-06-26 14:02:44 +01:00
Erik Johnston	8abdd7b553	Fix up indices for users_who_share_rooms	2017-06-26 14:01:30 +01:00
Erik Johnston	ff13c5e7af	Merge pull request #2301 from xwiki-labs/push-redact-content ... Add configuration parameter to allow redaction of content from push m…	2017-06-24 13:13:51 +01:00
Caleb James DeLisle	27bd0b9a91	Change the config file generator to more descriptive explanation of push.redact_content	2017-06-24 10:32:12 +02:00
Caleb James DeLisle	bce144595c	Fix TravisCI tests for PR #2301 - Fat finger mistake	2017-06-23 15:26:09 +02:00
Caleb James DeLisle	75eba3b07d	Fix TravisCI tests for PR #2301	2017-06-23 15:15:18 +02:00
Caleb James DeLisle	1591eddaea	Add configuration parameter to allow redaction of content from push messages for google/apple devices	2017-06-23 13:01:04 +02:00
Erik Johnston	7fe8ed1787	Store URL cache preview downloads seperately ... This makes it easier to clear old media out at a later date	2017-06-23 11:14:11 +01:00
Erik Johnston	44c722931b	Make some more params configurable	2017-06-22 14:59:52 +01:00
Erik Johnston	2d520a9826	Typo. ARGH.	2017-06-22 14:42:39 +01:00
Erik Johnston	24d894e2e2	Fix thinko in unhandled user spam	2017-06-22 14:39:05 +01:00
Matthew Hodgson	ccfcef6b59	Merge branch 'master' into develop	2017-06-22 13:03:44 +01:00
Erik Johnston	e0004aa28a	Add desc	2017-06-22 10:03:48 +01:00
Erik Johnston	b668112320	Merge pull request #2296 from matrix-org/erikj/dont_appserver_shar ... Don't work out users who share room with appservice users	2017-06-21 14:50:24 +01:00
Erik Johnston	dae9a00a28	Initialise exclusive_user_regex	2017-06-21 14:19:33 +01:00
Erik Johnston	8177563ebe	Fix for workers	2017-06-21 13:57:49 +01:00
Krombel	4202fba82a	Merge branch 'develop' into avoid_duplicate_filters	2017-06-21 14:48:21 +02:00
Krombel	812c030e87	replaced json.dumps with encode_canonical_json	2017-06-21 14:48:12 +02:00
Erik Johnston	1217c7da91	Don't work out users who share room with appservice users	2017-06-21 12:00:41 +01:00
Erik Johnston	385dcb7c60	Handle thumbnail urls	2017-06-19 17:48:28 +01:00
Erik Johnston	b8b936a6ea	Add API to quarantine media	2017-06-19 17:39:21 +01:00
Erik Johnston	e5ae386ea4	Handle all cases of sending membership events	2017-06-19 16:07:54 +01:00
Erik Johnston	36e51aad3c	Remove unused import	2017-06-19 14:42:21 +01:00
Erik Johnston	b490299a3b	Change to create new room and join other users	2017-06-19 14:10:13 +01:00
Erik Johnston	5db7070dd1	Forget room	2017-06-19 12:40:29 +01:00
Erik Johnston	d7fe6b356c	Add shutdown room API	2017-06-19 12:37:27 +01:00
Erik Johnston	fcf01dd88e	Reject local events that don't round trip the DB	2017-06-19 11:33:40 +01:00
Johannes Löthberg	4f66312df8	python_dependencies: Use bcrypt module instead of py-bcrypt ... py-bcrypt has been unmaintained for a long while, while bcrypt is actively maintained. And since ff8b87118d we're compatible with the bcrypt anyway. Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>	2017-06-17 17:39:35 +02:00
Erik Johnston	dfeca6cf40	Merge pull request #2286 from matrix-org/erikj/split_out_user_dir ... Split out user directory to a separate process	2017-06-16 13:01:19 +01:00
Erik Johnston	6aa5bc8635	Initial worker impl	2017-06-16 11:47:11 +01:00
Erik Johnston	d8f47d2efa	Merge pull request #2280 from matrix-org/erikj/share_room_user_dir ... Include users who you share a room with in user directory	2017-06-16 11:06:00 +01:00
Krombel	1ff419d343	allow Authorization header which handling got implemented in #1098 ... Signed-off-by: Matthias Kesler <krombel@krombel.de>	2017-06-16 11:21:14 +02:00
Erik Johnston	fdf1ca30f0	Bump version and changelog	2017-06-15 12:59:06 +01:00
Erik Johnston	052c5d19d5	Merge pull request #2281 from matrix-org/erikj/phone_home_stats ... Fix phone home stats	2017-06-15 12:46:23 +01:00
Erik Johnston	5ddd199870	Typo	2017-06-15 10:49:10 +01:00
Erik Johnston	a9d6fa8b2b	Include users who share room with requester in user directory	2017-06-15 10:17:21 +01:00
Erik Johnston	4564b05483	Implement updating users who share rooms on the fly	2017-06-15 10:17:17 +01:00
Erik Johnston	72613bc379	Implement initial population of users who share rooms table	2017-06-15 09:59:04 +01:00
Erik Johnston	ebcd55d641	Add DB schema for tracking users who share rooms	2017-06-15 09:45:48 +01:00
Erik Johnston	4b461a6931	Add some more stats	2017-06-15 09:39:39 +01:00
Erik Johnston	617304b2cf	Fix phone home stats	2017-06-14 19:47:15 +01:00
Erik Johnston	d9fd937e39	Fix user directory insertion due to missing room_id	2017-06-13 11:50:24 +01:00
Erik Johnston	505e7e8b9d	Fix up sql	2017-06-13 11:19:18 +01:00
Erik Johnston	6fd7e6db3d	Fix user dir to not assume existence of user	2017-06-13 11:11:26 +01:00
Erik Johnston	fdca6e36ee	Merge pull request #2274 from matrix-org/erikj/cache_is_host_joined ... Add cache for is_host_joined	2017-06-13 10:58:53 +01:00
Erik Johnston	90ae0cffec	Merge pull request #2275 from matrix-org/erikj/tweark_user_directory_search ... Tweak the ranking of PG user dir search	2017-06-13 10:58:43 +01:00
Erik Johnston	de4cb50ca6	Merge pull request #2276 from matrix-org/erikj/fix_user_di ... Don't assume existence of events when updating user directory	2017-06-13 10:55:41 +01:00
Erik Johnston	48d2949416	Throw exception when not retrying when downloading media	2017-06-13 10:23:14 +01:00
Erik Johnston	6ae8373d40	Don't assume existance of events when updating user directory	2017-06-13 10:19:26 +01:00
Erik Johnston	b58e24cc3c	Tweak the ranking of PG user dir search	2017-06-13 10:16:31 +01:00
Erik Johnston	d53fe399eb	Add cache for is_host_joined	2017-06-13 09:56:18 +01:00
Erik Johnston	a837765e8c	Merge pull request #2266 from matrix-org/erikj/host_in_room ... Change is_host_joined to use current_state table	2017-06-12 09:49:51 +01:00
Erik Johnston	8060974344	Fix replication	2017-06-09 16:40:52 +01:00
Erik Johnston	b0d975e216	Comments	2017-06-09 16:25:42 +01:00
Erik Johnston	e54d7d536e	Cache state deltas	2017-06-09 16:24:00 +01:00
Erik Johnston	1e9b4d5a95	Merge pull request #2268 from matrix-org/erikj/entity_has_changed ... Fix has_any_entity_changed	2017-06-09 15:30:55 +01:00
Erik Johnston	efc2b7db95	Rewrite conditional	2017-06-09 13:35:15 +01:00
Erik Johnston	bfd68019c2	Merge pull request #2267 from matrix-org/erikj/missing_notifier ... Fix removing of pushers when using workers	2017-06-09 13:07:29 +01:00
Erik Johnston	1664948e41	Comment	2017-06-09 13:05:05 +01:00
Erik Johnston	935e588799	Tweak SQL	2017-06-09 13:01:23 +01:00
Erik Johnston	eed59dcc1e	Fix has_any_entity_changed ... Occaisonally has_any_entity_changed would throw the error: "Set changed size during iteration" when taking the max of the `sorteddict`. While its uncertain how that happens, its quite inefficient to iterate over the entire dict anyway so we change to using the more traditional `bisect_*` functions.	2017-06-09 11:44:01 +01:00
Erik Johnston	2cac7623a5	Add missing notifier	2017-06-09 11:24:41 +01:00
Erik Johnston	298d83b340	Fix replication	2017-06-09 11:01:28 +01:00
Erik Johnston	0185b75381	Change is_host_joined to use current_state table ... This bypasses a bug where using the state groups to figure out if a host is in a room sometimes errors if the servers isn't in the room. (For example when the server rejected an invite to a remote room)	2017-06-09 10:52:26 +01:00
Erik Johnston	7132e5cdff	Mark remote invite rejections as outliers	2017-06-09 10:08:18 +01:00
Erik Johnston	ea11ee09f3	Ensure we don't use unpersisted state group as prev group	2017-06-08 11:59:57 +01:00
Erik Johnston	c62c480dc6	Merge pull request #2259 from matrix-org/erikj/fix_state_woes ... Fix bug where state_group tables got corrupted	2017-06-07 17:51:25 +01:00
Erik Johnston	197bd126f0	Fix bug where state_group tables got corrupted ... This is due to the fact that we prefilled caches using txn.call_after, which always gets called including on error. We fix this by making txn.call_after only fire when a transaction completes successfully, which is what we want most of the time anyway.	2017-06-07 17:39:36 +01:00
Erik Johnston	f45f07ab86	Merge pull request #2258 from matrix-org/erikj/user_dir ... Don't start user_directory handling on workers	2017-06-07 14:04:50 +01:00
Erik Johnston	a053ff3979	Merge pull request #2248 from matrix-org/erikj/state_fixup ... Faster cache for get_joined_hosts	2017-06-07 14:01:06 +01:00
Erik Johnston	ecdd2a3658	Don't start user_directory handling on workers	2017-06-07 12:02:53 +01:00
Erik Johnston	2f34ad31ac	Add some logging to user directory	2017-06-07 11:50:44 +01:00
Erik Johnston	64ed74c01e	When pruning, delete from device_lists_outbound_last_success	2017-06-07 11:20:47 +01:00
Erik Johnston	1a81a1898e	Keep pruning background task	2017-06-07 11:16:56 +01:00
Erik Johnston	6ba21bf2b8	Comments	2017-06-07 11:08:36 +01:00
Erik Johnston	09e4bc0501	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/state_fixup	2017-06-07 11:05:23 +01:00
Erik Johnston	6e2a7ee1bc	Remove spurious log lines	2017-06-07 11:05:17 +01:00
Erik Johnston	65f0513a33	Split up device_lists_outbound_pokes table for faster updates.	2017-06-07 11:02:38 +01:00
Erik Johnston	6f83c4537c	Increase size of IP cache	2017-06-07 10:18:44 +01:00
Erik Johnston	cca94272fa	Fix typo when getting app name	2017-06-06 11:50:07 +01:00
Erik Johnston	66b121b2fc	Fix wrong number of arguments	2017-06-06 11:46:38 +01:00
Erik Johnston	1a01af079e	Handle profile updates in user directory	2017-06-01 15:39:51 +01:00
Erik Johnston	4d039aa2ca	Fix sqlite	2017-06-01 14:58:48 +01:00
Erik Johnston	21e255a8f1	Split the table in two	2017-06-01 14:50:46 +01:00
Erik Johnston	d5477c7afd	Tweak search query	2017-06-01 13:28:01 +01:00
Erik Johnston	02a6108235	Tweak search query	2017-06-01 13:16:40 +01:00
Erik Johnston	7233341eac	Comments	2017-06-01 13:11:38 +01:00
Erik Johnston	8be6fd95a3	Check if host is still in room	2017-06-01 13:05:39 +01:00
Erik Johnston	59dbb47065	Remove spurious inlineCallbacks	2017-06-01 11:41:29 +01:00
Erik Johnston	9c7db2491b	Fix removing users	2017-06-01 11:36:50 +01:00
Erik Johnston	0fe6f3c521	Bug fixes and logging ... - Check if room is public when a user joins before adding to user dir - Fix typo of field name "content.join_rules" -> "content.join_rule"	2017-06-01 11:09:49 +01:00
Erik Johnston	036362ede6	Order by if they have profile info	2017-06-01 09:41:08 +01:00
Erik Johnston	a757dd4863	Use prefix matching	2017-06-01 09:40:37 +01:00
Erik Johnston	f5cc22bdc6	Comment on why arbitrary comments	2017-05-31 17:30:26 +01:00
Erik Johnston	5dd1b2c525	Use unique indices	2017-05-31 17:29:12 +01:00
Erik Johnston	cc7609aa9f	Comment briefly on how we keep user_directory up to date	2017-05-31 17:11:18 +01:00
Erik Johnston	f1378aef91	Convert to int	2017-05-31 17:03:08 +01:00
Erik Johnston	b2d8d07109	Lifts things into separate function	2017-05-31 17:00:24 +01:00
Erik Johnston	f9791498ae	Typos	2017-05-31 16:50:57 +01:00
Erik Johnston	f091061711	Fix tests	2017-05-31 16:34:40 +01:00
Erik Johnston	4abcff0177	Fix typo	2017-05-31 16:22:36 +01:00
Erik Johnston	63c58c2a3f	Limit number of things we fetch out of the db	2017-05-31 16:17:58 +01:00
Erik Johnston	304880d185	Add stream change cache	2017-05-31 15:46:36 +01:00
Erik Johnston	5d79d728f5	Split out directory and search tables	2017-05-31 15:23:49 +01:00
Erik Johnston	dc51af3d03	Pull max id from correct table	2017-05-31 15:13:49 +01:00
Erik Johnston	350622a107	Handle the server leaving a public room	2017-05-31 15:11:36 +01:00
Erik Johnston	63fda37e20	Add comments	2017-05-31 15:00:29 +01:00
Erik Johnston	293ef29655	Weight differently	2017-05-31 14:29:32 +01:00
Erik Johnston	535c99f157	Use POST	2017-05-31 14:15:45 +01:00
Erik Johnston	45a5df5914	Add REST API	2017-05-31 14:11:55 +01:00
Erik Johnston	3b5f22ca40	Add search	2017-05-31 14:00:01 +01:00
Erik Johnston	b5db4ed5f6	Update room column when room becomes unpublic	2017-05-31 13:40:28 +01:00
Erik Johnston	168524543f	Add call later	2017-05-31 11:59:36 +01:00
Erik Johnston	3e123b8497	Start later	2017-05-31 11:56:27 +01:00
Erik Johnston	42137efde7	Don't go round in circles	2017-05-31 11:55:13 +01:00
Erik Johnston	eeb2f9e546	Add user_directory to database	2017-05-31 11:51:01 +01:00
Erik Johnston	dd48f7204c	Add comment	2017-05-30 15:01:22 +01:00
Erik Johnston	04095f7581	Add clobbered event_id	2017-05-30 14:53:01 +01:00
Erik Johnston	a584a81b3e	Add current_state_delta_stream table	2017-05-30 14:44:09 +01:00
Erik Johnston	619e8ecd0c	Handle None state group correctly	2017-05-26 10:46:03 +01:00
Erik Johnston	23da638360	Fix typing tests	2017-05-26 10:02:04 +01:00
Erik Johnston	dfbda5e025	Faster cache for get_joined_hosts	2017-05-25 17:24:44 +01:00
Erik Johnston	2b03751c3c	Don't return weird prev_group	2017-05-25 14:47:39 +01:00
Erik Johnston	dbc0dfd2d5	Remove unused options	2017-05-25 14:28:34 +01:00
Erik Johnston	11f139a647	Merge pull request #2247 from matrix-org/erikj/auth_event ... Only store event_auth for state events	2017-05-24 16:46:34 +01:00
Erik Johnston	6e614e9e10	Add background task to clear out old event_auth	2017-05-24 15:23:34 +01:00
Erik Johnston	c049472b8a	Only store event_auth for state events	2017-05-24 15:23:31 +01:00
Erik Johnston	9a804b2812	Merge pull request #2243 from matrix-org/matthew/fix-url-preview-length-again ... actually trim oversize og:description meta	2017-05-23 13:26:28 +01:00
Erik Johnston	fbbc40f385	Merge pull request #2237 from matrix-org/erikj/sync_key_count ... Add count of one time keys to sync stream	2017-05-23 11:18:13 +01:00
Erik Johnston	8cf9f0a3e7	Remove redundant invalidation	2017-05-23 09:46:59 +01:00
Erik Johnston	e6618ece2d	Missed an invalidation	2017-05-23 09:36:52 +01:00
Matthew Hodgson	836d5c44b6	actually trim oversize og:description meta	2017-05-22 21:14:20 +01:00
Erik Johnston	11c2a3655f	Only load jinja2 templates once ... Instead of every time a new email pusher is created, as loading jinja2 templates is slow.	2017-05-22 17:48:58 +01:00
Erik Johnston	539aa4d333	Merge pull request #2241 from matrix-org/erikj/fix_notifs ... Correctly calculate push rules for member events	2017-05-22 16:46:58 +01:00
Erik Johnston	f85a415279	Add missing storage function to slave store	2017-05-22 16:31:24 +01:00
Erik Johnston	6489455bed	Comment	2017-05-22 16:22:04 +01:00
Erik Johnston	d668caa79c	Remove spurious log level guards	2017-05-22 16:21:06 +01:00
Erik Johnston	74bf4ee7bf	Stream count_e2e_one_time_keys cache invalidation	2017-05-22 16:19:22 +01:00
Erik Johnston	33ba90c6e9	Merge pull request #2240 from matrix-org/erikj/cache_list_fix ... Update list cache to handle one arg case	2017-05-22 16:10:46 +01:00
Erik Johnston	bd7bb5df71	Pull out if statement from for loop	2017-05-22 15:12:19 +01:00
Erik Johnston	e3417a06e2	Update list cache to handle one arg case ... We update the normal cache descriptors to handle caches with a single argument specially so that the key wasn't a 1-tuple. We need to update the cache list to be aware of this.	2017-05-22 15:04:42 +01:00
Erik Johnston	7fb80b5eae	Check if current event is a membership event	2017-05-22 15:02:12 +01:00
Erik Johnston	2d17b09a6d	Add debug logging	2017-05-22 15:01:36 +01:00
Erik Johnston	24c8f38784	Comment	2017-05-22 14:59:27 +01:00
Erik Johnston	25f03cf8e9	Use tuple unpacking	2017-05-22 14:58:22 +01:00
Erik Johnston	270e1c904a	Speed up calculating push rules	2017-05-19 16:51:05 +01:00
Erik Johnston	b4f59c7e27	Add count of one time keys to sync stream	2017-05-19 15:47:55 +01:00
Erik Johnston	58ebb96cce	Fix invalidation of get_users_with_read_receipts_in_room	2017-05-19 14:38:50 +01:00
Erik Johnston	1c1c0257f4	Move invalidation cb to its own structure	2017-05-19 11:44:11 +01:00
Erik Johnston	cafe659f72	Store ActionGenerator in HomeServer	2017-05-19 10:09:56 +01:00
Erik Johnston	72ed8196b3	Don't push users who have left	2017-05-18 17:48:36 +01:00
Erik Johnston	107ac7ac96	Increase size of push rule caches	2017-05-18 17:17:53 +01:00
Erik Johnston	760625acba	Make get_if_app_services_interested_in_user faster	2017-05-18 16:34:44 +01:00
Erik Johnston	c57789d138	Remove size of push get_rules cache	2017-05-18 16:17:23 +01:00
Erik Johnston	f33df30732	Merge branch 'master' of github.com:matrix-org/synapse into develop	2017-05-18 13:56:37 +01:00
Erik Johnston	a5425b2e5b	Bump changelog and version	2017-05-18 13:53:48 +01:00
Erik Johnston	6e381180ae	Merge pull request #2177 from matrix-org/erikj/faster_push_rules ... Make calculating push actions faster	2017-05-18 11:46:18 +01:00
Erik Johnston	056ba9b795	Add comment	2017-05-18 11:45:56 +01:00
Erik Johnston	d9e3a4b5db	Merge pull request #2230 from matrix-org/erikj/speed_up_get_state ... Make get_state_groups_from_groups faster.	2017-05-17 17:23:04 +01:00
Erik Johnston	66d8ffabbd	Faster push rule calculation via push specific cache ... We add a push rule specific cache that ensures that we can reuse calculated push rules appropriately when a user join/leaves.	2017-05-17 16:55:40 +01:00
Erik Johnston	ace23463c5	Merge pull request #2216 from slipeer/app_services_interested_in_user ... Fix users claimed non-exclusively by an app service don't get notific…	2017-05-17 16:28:50 +01:00
Erik Johnston	bbfe4e996c	Make get_state_groups_from_groups faster. ... Most of the time was spent copying a dict to filter out sentinel values that indicated that keys did not exist in the dict. The sentinel values were added to ensure that we cached the non-existence of keys. By updating DictionaryCache to keep track of which keys were known to not exist itself we can remove a dictionary copy.	2017-05-17 15:12:15 +01:00
Erik Johnston	a8bc7cae56	Merge branch 'develop' of github.com:matrix-org/synapse into release-v0.21.0	2017-05-17 13:11:43 +01:00
Erik Johnston	3a431a126d	Bump changelog and version	2017-05-17 11:26:57 +01:00
Erik Johnston	85e8092cca	Comment	2017-05-17 10:03:09 +01:00
Erik Johnston	ad53fc3cf4	Short circuit when we have delta ids	2017-05-17 09:57:34 +01:00
Erik Johnston	6fa8148ccb	Merge pull request #2228 from matrix-org/erikj/speed_up_get_hosts ... Speed up get_joined_hosts	2017-05-16 17:40:55 +01:00
Erik Johnston	7c69849a0d	Merge pull request #2227 from matrix-org/erikj/presence_caches ... Make presence use cached users/hosts in room	2017-05-16 17:40:47 +01:00
Erik Johnston	13f540ef1b	Speed up get_joined_hosts	2017-05-16 16:05:22 +01:00
Erik Johnston	ec5c4499f4	Make presence use cached users/hosts in room	2017-05-16 16:01:43 +01:00
Erik Johnston	f2a5b6dbfd	Speed up get_domain_from_id	2017-05-16 15:59:37 +01:00
Erik Johnston	b8492b6c2f	Merge pull request #2224 from matrix-org/erikj/prefill_state ... Prefill state caches	2017-05-16 15:50:11 +01:00
Erik Johnston	331570ea6f	Remove spurious merge artifacts	2017-05-16 15:33:07 +01:00
Krombel	55af207321	Merge branch 'develop' into avoid_duplicate_filters	2017-05-16 15:29:59 +02:00
Richard van der Hoff	d648f65aaf	Merge pull request #2218 from matrix-org/rav/event_search_index ... Add an index to event_search	2017-05-16 13:26:07 +01:00
Erik Johnston	608b5a6317	Take a copy before prefilling, as it may be a frozendict	2017-05-16 12:55:29 +01:00
Krombel	64953c8ed2	avoid access-error if no filter_id matches	2017-05-15 18:36:37 +02:00
Erik Johnston	f451b64c8f	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/prefill_state	2017-05-15 16:09:32 +01:00
Erik Johnston	2c9475b58e	Merge pull request #2221 from psaavedra/sync_timeline_limit_filter_by_name ... Configurable maximum number of events requested by /sync and /messages	2017-05-15 16:08:46 +01:00
Erik Johnston	d12ae7fd1c	Don't log exceptions for NotRetryingDestination	2017-05-15 15:42:18 +01:00
Pablo Saavedra	224137fcf9	Fixed syntax nits	2017-05-15 16:21:02 +02:00
Erik Johnston	e4435b014e	Update comment	2017-05-15 15:11:30 +01:00
Erik Johnston	871605f4e2	Comments	2017-05-15 15:11:30 +01:00
Erik Johnston	e0d2f6d5b0	Add more granular event send metrics	2017-05-15 15:11:30 +01:00
Erik Johnston	bfbc907cec	Prefill state caches	2017-05-15 15:11:13 +01:00
Pablo Saavedra	627e6ea2b0	Fixed implementation errors ... * Added HS as property in SyncRestServlet * Fixed set_timeline_upper_limit function implementat¡ion	2017-05-15 14:51:43 +02:00
Pablo Saavedra	9da4316ca5	Configurable maximum number of events requested by /sync and /messages (#2220) ... Set the limit on the returned events in the timeline in the get and sync operations. The default value is -1, means no upper limit. For example, using `filter_timeline_limit: 5000`: POST /_matrix/client/r0/user/user:id/filter { room: { timeline: { limit: 1000000000000000000 } } } GET /_matrix/client/r0/user/user:id/filter/filter:id { room: { timeline: { limit: 5000 } } } The server cuts down the room.timeline.limit.	2017-05-13 18:17:54 +02:00
Krombel	eb7cbf27bc	insert whitespace to fix travis build	2017-05-12 12:09:42 +02:00
Krombel	6b95e35e96	add check to only add a new filter if the same filter does not exist previously ... Signed-off-by: Matthias Kesler <krombel@krombel.de>	2017-05-11 16:05:30 +02:00
Richard van der Hoff	ff3d810ea8	Add a comment to old delta	2017-05-11 12:48:50 +01:00
Richard van der Hoff	34194aaff7	Don't create event_search index on sqlite ... ... because the table is virtual	2017-05-11 12:46:55 +01:00
Richard van der Hoff	114f290947	Add more logging for purging ... Log the number of events we will be deleting at info.	2017-05-11 12:08:47 +01:00
Richard van der Hoff	baafb85ba4	Add an index to event_search ... - to make the purge API quicker	2017-05-11 12:05:22 +01:00
Richard van der Hoff	29ded770b1	Merge pull request #2214 from matrix-org/rav/hurry_up_purge ... When purging, don't de-delta state groups we're about to delete	2017-05-11 12:04:25 +01:00
Richard van der Hoff	dc026bb16f	Tidy purge code and add some comments ... Try to make this clearer with more comments and some variable renames	2017-05-11 10:56:12 +01:00
Slipeer	328378f9cb	Fix users claimed non-exclusively by an app service don't get notifications #2211	2017-05-11 11:42:08 +03:00
Richard van der Hoff	8e345ce465	Don't de-delta state groups we're about to delete	2017-05-10 18:44:22 +01:00
Richard van der Hoff	b64d312421	add some logging to purge_history	2017-05-10 18:44:22 +01:00
Luke Barnard	ccad2ed824	Modify condition on empty localpart	2017-05-10 17:34:30 +01:00
Luke Barnard	369195caa5	Modify register/available to be GET with query param ... - GET is now the method for register/available - a query parameter "username" is now used Also, empty usernames are now handled with an error message on registration or via register/available: `User ID cannot be empty`	2017-05-10 17:23:55 +01:00
Erik Johnston	a3648f84b2	Merge pull request #2208 from matrix-org/erikj/ratelimit_overrid ... Add per user ratelimiting overrides	2017-05-10 15:54:48 +01:00
Richard van der Hoff	5331cd150a	Merge pull request #2206 from matrix-org/rav/one_time_key_upload_change_sig ... Allow clients to upload one-time-keys with new sigs	2017-05-10 14:18:40 +01:00
Luke Barnard	f7278e612e	Change register/available to POST (from GET)	2017-05-10 11:40:18 +01:00
Erik Johnston	b990b2fce5	Add per user ratelimiting overrides	2017-05-10 11:05:43 +01:00
Richard van der Hoff	aedaba018f	Replace some instances of preserve_context_over_deferred	2017-05-09 19:04:56 +01:00
Richard van der Hoff	de042b3b88	Do some logging when one-time-keys get claimed ... might help us figure out if https://github.com/vector-im/riot-web/issues/3868 has happened.	2017-05-09 19:04:56 +01:00
Richard van der Hoff	a7e9d8762d	Allow clients to upload one-time-keys with new sigs ... When a client retries a key upload, don't give an error if the signature has changed (but the key is the same). Fixes https://github.com/vector-im/riot-android/issues/1208, hopefully.	2017-05-09 19:04:56 +01:00
Erik Johnston	ca238bc023	Merge branch 'release-v0.21.0' of github.com:matrix-org/synapse into develop	2017-05-08 17:35:11 +01:00
Erik Johnston	40dcf0d856	Merge pull request #2203 from matrix-org/erikj/event_cache_hit_ratio ... Don't update event cache hit ratio from get_joined_users	2017-05-08 16:52:39 +01:00
Erik Johnston	d3c3026496	Merge pull request #2201 from matrix-org/erikj/store_device_cache ... Cache check to see if device exists	2017-05-08 16:23:04 +01:00
Erik Johnston	093f7e47cc	Expand docstring a bit	2017-05-08 16:14:46 +01:00
Erik Johnston	6a12998a83	Add missing yields	2017-05-08 16:10:51 +01:00
Erik Johnston	ffad4fe35b	Don't update event cache hit ratio from get_joined_users ... Otherwise the hit ration of plain get_events gets completely skewed by calls to get_joined_users* functions.	2017-05-08 16:06:17 +01:00
Erik Johnston	94e6ad71f5	Invalidate cache on device deletion	2017-05-08 15:55:59 +01:00
Erik Johnston	8571f864d2	Cache one time key counts	2017-05-08 15:34:27 +01:00
Erik Johnston	fc6d4974a6	Comment	2017-05-08 15:33:57 +01:00
Erik Johnston	738ccf61c0	Cache check to see if device exists	2017-05-08 15:32:18 +01:00
Erik Johnston	dcabef952c	Increase client_ip cache size	2017-05-08 15:09:19 +01:00
Erik Johnston	771c8a83c7	Bump version and changelog	2017-05-08 13:23:46 +01:00
Erik Johnston	6631985990	Merge pull request #2200 from matrix-org/erikj/revert_push ... Revert speed up push	2017-05-08 13:20:52 +01:00
Erik Johnston	e0f20e9425	Revert "Remove unused import" ... This reverts commit ab37bef83b.	2017-05-08 13:07:43 +01:00
Erik Johnston	fe7c1b969c	Revert "We don't care about forgotten rooms" ... This reverts commit ad8b316939.	2017-05-08 13:07:43 +01:00
Erik Johnston	78f306a6f7	Revert "Speed up filtering of a single event in push" ... This reverts commit 421fdf7460.	2017-05-08 13:07:41 +01:00
Erik Johnston	9ac98197bb	Bump version and changelog	2017-05-08 11:07:54 +01:00
Erik Johnston	27c28eaa27	Merge pull request #2190 from matrix-org/erikj/mark_remote_as_back_more ... Always mark remotes as up if we receive a signed request from them	2017-05-05 14:08:12 +01:00
Erik Johnston	be2672716d	Merge pull request #2189 from matrix-org/erikj/handle_remote_device_list ... Handle exceptions thrown in handling remote device list updates	2017-05-05 14:01:27 +01:00
Erik Johnston	653d90c1a5	Comment	2017-05-05 14:01:17 +01:00
Erik Johnston	310b1ccdc1	Use preserve_fn and add logs	2017-05-05 13:41:19 +01:00
Kegsay	a59b0ad1a1	Merge pull request #2192 from matrix-org/kegan/simple-http-client-timeouts ... Rewrite SimpleHttpClient.request to include timeouts	2017-05-05 11:52:43 +01:00
Erik Johnston	7b222fc56e	Remove redundant reset of destination timers	2017-05-05 11:14:09 +01:00
Kegan Dougal	d0debb2116	Remember how twisted works	2017-05-05 11:00:21 +01:00
Erik Johnston	66f371e8b8	Merge pull request #2176 from matrix-org/erikj/faster_get_joined ... Make get_joined_users faster	2017-05-05 10:59:55 +01:00
Erik Johnston	b843631d71	Add comment and TODO	2017-05-05 10:59:32 +01:00
Kegan Dougal	c2ddd773bc	Include the clock	2017-05-05 10:52:46 +01:00
Kegan Dougal	7dd3bf5e24	Rewrite SimpleHttpClient.request to include timeouts ... Fixes #2191	2017-05-05 10:49:19 +01:00
Erik Johnston	db7d0c3127	Always mark remotes as up if we receive a signed request from them	2017-05-05 10:34:53 +01:00
Erik Johnston	f346048a6e	Handle exceptions thrown in handling remote device list updates	2017-05-05 10:34:10 +01:00
Erik Johnston	e3aa8a7aa8	Merge pull request #2185 from matrix-org/erikj/smaller_caches ... Optimise caches for single key	2017-05-05 10:19:05 +01:00
Erik Johnston	cf589f2c1e	Fixes	2017-05-05 10:17:56 +01:00
Erik Johnston	8af4569583	Merge pull request #2174 from matrix-org/erikj/current_cache_hosts ... Add cache for get_current_hosts_in_room	2017-05-05 10:15:24 +01:00
Erik Johnston	587f07543f	Revert "Prefill state caches"	2017-05-04 15:07:27 +01:00
Erik Johnston	aa93cb9f44	Add comment	2017-05-04 14:59:28 +01:00
Erik Johnston	537dbadea0	Intern host strings	2017-05-04 14:55:28 +01:00
Erik Johnston	07a07588a0	Make caches bigger	2017-05-04 14:52:28 +01:00
Erik Johnston	dfaa58f72d	Fix comment and num args	2017-05-04 14:50:24 +01:00
Erik Johnston	9ac263ed1b	Add new storage functions to slave store	2017-05-04 14:29:03 +01:00
Erik Johnston	d2d8ed4884	Optimise caches with single key	2017-05-04 14:18:46 +01:00
Erik Johnston	5d8290429c	Reduce size of get_users_in_room	2017-05-04 13:43:19 +01:00
Luke Barnard	6aa423a1a8	Merge pull request #2183 from matrix-org/luke/username-availability ... Implement username availability checker	2017-05-04 09:58:40 +01:00
Luke Barnard	3669065466	Appease the flake8 gods	2017-05-03 18:05:49 +01:00
Erik Johnston	7ebf518c02	Make get_joined_users faster	2017-05-03 15:55:54 +01:00
Luke Barnard	34ed4f4206	Implement username availability checker ... Outlined here: https://github.com/vector-im/riot-web/issues/3605#issuecomment-298679388 ```HTTP GET /_matrix/.../register/available { "username": "desiredlocalpart123" } ``` If available, the response looks like ```HTTP HTTP/1.1 200 OK { "available": true } ``` Otherwise, ```HTTP HTTP/1.1 429 { "errcode": "M_LIMIT_EXCEEDED", "error": "Too Many Requests", "retry_after_ms": 2000 } ``` or ```HTTP HTTP/1.1 400 { "errcode": "M_USER_IN_USE", "error": "User ID already taken." } ``` or ```HTTP HTTP/1.1 400 { "errcode": "M_INVALID_USERNAME", "error": "Some reason for username being invalid" } ```	2017-05-03 12:04:12 +01:00
David Baker	60833c8978	Merge pull request #2147 from matrix-org/dbkr/http_request_propagate_error ... Propagate errors sensibly from proxied IS requests	2017-05-03 11:23:25 +01:00
David Baker	482a2ad122	No need for the exception variable	2017-05-03 11:02:59 +01:00
David Baker	c0380402bc	List caught expection types	2017-05-03 10:56:22 +01:00
Erik Johnston	cdbf38728d	Merge pull request #2175 from matrix-org/erikj/prefill_state ... Prefill state caches	2017-05-03 10:54:11 +01:00
Erik Johnston	0c27383dd7	Merge pull request #2170 from matrix-org/erikj/fed_hole_state ... Don't fetch state for missing events that we fetched	2017-05-03 10:49:37 +01:00
Erik Johnston	ef862186dd	Merge together redundant calculations/logging	2017-05-03 10:06:43 +01:00
Erik Johnston	2c2dcf81d0	Update comment	2017-05-03 10:00:29 +01:00
Erik Johnston	1827057acc	Comments	2017-05-03 09:56:05 +01:00
Erik Johnston	8346e6e696	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/prefill_state	2017-05-03 09:46:40 +01:00
Erik Johnston	e4c15fcb5c	Merge pull request #2178 from matrix-org/erikj/message_metrics ... Add more granular event send metrics	2017-05-02 17:57:34 +01:00
Erik Johnston	3e5a62ecd8	Add more granular event send metrics	2017-05-02 14:23:26 +01:00
Richard van der Hoff	82475a18d9	Merge pull request #2180 from matrix-org/rav/fix_timeout_on_timeout ... Instantiate DeferredTimedOutError correctly	2017-05-02 13:32:58 +01:00
Richard van der Hoff	2e996271fe	Instantiate DeferredTimedOutError correctly ... Call `super` correctly, so that we correctly initialise the `errcode` field. Fixes https://github.com/matrix-org/synapse/issues/2179.	2017-05-02 13:26:17 +01:00
Erik Johnston	a2c89a225c	Prefill state caches	2017-05-02 10:40:31 +01:00
Erik Johnston	7166854f41	Add cache for get_current_hosts_in_room	2017-05-02 10:36:35 +01:00
Erik Johnston	3033261891	Merge pull request #2080 from matrix-org/erikj/filter_speed ... Speed up filtering of a single event in push	2017-04-28 14:17:13 +01:00
Erik Johnston	2347efc065	Fixup	2017-04-28 12:46:53 +01:00
Erik Johnston	9b147cd730	Remove unncessary call in _get_missing_events_for_pdu	2017-04-28 11:55:25 +01:00
Erik Johnston	3a9f5bf6dd	Don't fetch state for missing events that we fetched	2017-04-28 11:26:46 +01:00
Erik Johnston	ab37bef83b	Remove unused import	2017-04-28 09:57:23 +01:00
Erik Johnston	ad8b316939	We don't care about forgotten rooms	2017-04-28 09:52:36 +01:00
Erik Johnston	421fdf7460	Speed up filtering of a single event in push	2017-04-28 09:52:36 +01:00
Erik Johnston	25a96e0c63	Merge pull request #2163 from matrix-org/erikj/fix_invite_state ... Fix invite state to always include all events	2017-04-27 17:36:30 +01:00
Erik Johnston	46826bb078	Comment and remove spurious logging	2017-04-27 17:25:44 +01:00
Erik Johnston	f87b287291	Merge pull request #2127 from APwhitehat/alreadystarted ... print something legible if synapse already running	2017-04-27 15:46:53 +01:00
Erik Johnston	bb9246e525	Merge pull request #2131 from matthewjwolff/develop ... web_client_location documentation fix	2017-04-27 15:46:40 +01:00
Richard van der Hoff	c84770b877	Fix bgupdate error if index already exists (#2167) ... When creating a new table index in the background, guard against it existing already. Fixes https://github.com/matrix-org/synapse/issues/2135. Also, make sure we restore the autocommit flag when we're done, otherwise we get more failures from other operations later on. Fixes https://github.com/matrix-org/synapse/issues/1890 (hopefully).	2017-04-27 15:27:48 +01:00
Erik Johnston	380fb87ecc	Merge pull request #2168 from matrix-org/erikj/federation_logging ... Add some extra logging for edge cases of federation	2017-04-27 15:19:12 +01:00
Erik Johnston	87ae59f5e9	Typo	2017-04-27 15:16:21 +01:00
Erik Johnston	e42b4ebf0f	Add some extra logging for edge cases of federation	2017-04-27 14:38:21 +01:00
Erik Johnston	d3c150411c	Merge pull request #2130 from APwhitehat/roomexists ... Check that requested room_id exists	2017-04-27 09:20:26 +01:00
Erik Johnston	1e166470ab	Fix tests	2017-04-26 16:23:30 +01:00
Erik Johnston	34e682d385	Fix invite state to always include all events	2017-04-26 16:18:08 +01:00
David Baker	5fd12dce01	Remove debugging	2017-04-26 12:36:26 +01:00
David Baker	82ae0238f9	Revert accidental commit	2017-04-26 11:43:16 +01:00
David Baker	81804909d3	Merge remote-tracking branch 'origin/develop' into dbkr/http_request_propagate_error	2017-04-26 11:31:55 +01:00
David Baker	c366276056	Fix get_json	2017-04-26 10:07:01 +01:00
David Baker	1a9255c12e	Use CodeMessageException subclass instead ... Parse json errors from get_json client methods and throw special errors.	2017-04-25 19:30:55 +01:00
Erik Johnston	f053a1409e	Make state caches cache in ascii	2017-04-25 17:22:55 +01:00
Erik Johnston	acb58bfb6a	fix up	2017-04-25 15:39:19 +01:00
Erik Johnston	f7181615f2	Don't specify default as dict	2017-04-25 15:22:59 +01:00
Erik Johnston	f144365281	Comment	2017-04-25 15:18:26 +01:00
Erik Johnston	d9aa645f86	Reduce size of joined_user cache ... The _get_joined_users_from_context cache stores a mapping from user_id to avatar_url and display_name. Instead of storing those in a dict, store them in a namedtuple as that uses much less memory. We also try converting the string to ascii to further reduce the size.	2017-04-25 14:38:51 +01:00
Erik Johnston	22f3d3ae76	Reduce _get_state_group_for_event cache size	2017-04-25 11:43:03 +01:00
Erik Johnston	b4da08cad8	Merge pull request #2158 from matrix-org/erikj/reduce_cache_size ... Reduce cache size by not storing deferreds	2017-04-25 11:18:07 +01:00
Erik Johnston	efab1dadde	Remove DEBUG_CACHES	2017-04-25 10:54:09 +01:00
Erik Johnston	119cb9bbcf	Reduce cache size by not storing deferreds ... Currently the cache descriptors store deferreds rather than raw values, this is a simple way of triggering only one database hit and sharing the result if two callers attempt to get the same value. However, there are a few caches that simply store a mapping from string to string (or int). These caches can have a large number of entries, under the assumption that each entry is small. However, the size of a deferred (specifically the size of ObservableDeferred) is signigicantly larger than that of the raw value, 2kb vs 32b. This PR therefore changes the cache descriptors to store the raw values rather than the deferreds. As a side effect cached storage function now either return a deferred or the actual value, as the cached list decriptor already does. This is fine as we always end up just yield'ing on the returned value eventually, which handles that case correctly.	2017-04-25 10:23:11 +01:00
Mark Haines	e6e2627636	Fix code for reporting old verify keys in synapse	2017-04-24 18:51:25 +01:00
Richard van der Hoff	30f7bfa121	Merge pull request #2145 from matrix-org/rav/reject_invite_to_unreachable_server ... Fix rejection of invites to unreachable servers	2017-04-24 15:20:52 +01:00
Erik Johnston	7af825bae4	Merge pull request #2155 from matrix-org/erikj/string_intern ... Only intern ascii strings	2017-04-24 14:28:41 +01:00
Erik Johnston	d134d0935e	Only intern ascii strings	2017-04-24 14:07:48 +01:00
Erik Johnston	e4f3431116	Remove unused cache	2017-04-24 13:27:38 +01:00
David Baker	a46982cee9	Need the HTTP status code	2017-04-21 16:20:12 +01:00
David Baker	70caf49914	Do the same for get_json	2017-04-21 16:09:03 +01:00
David Baker	a1595cec78	Don't error for 3xx responses	2017-04-21 11:51:17 +01:00
David Baker	2e165295b7	Merge remote-tracking branch 'origin/develop' into dbkr/http_request_propagate_error	2017-04-21 11:35:52 +01:00
David Baker	a90a0f5c8a	Propagate errors sensibly from proxied IS requests ... When we're proxying Matrix endpoints, parse out Matrix error responses and turn them into SynapseErrors so they can be propagated sensibly upstream.	2017-04-21 11:32:48 +01:00
Richard van der Hoff	91b3981800	Try harder when sending leave events ... When we're rejecting invites, ignore the backoff data, so that we have a better chance of not getting the room out of sync.	2017-04-21 01:50:36 +01:00
Richard van der Hoff	0cdb32fc43	Remove redundant try/except clauses ... The `except SynapseError` clauses were pointless because the wrapped functions would never throw a `SynapseError` (they either throw a `CodeMessageException` or a `RuntimeError`). The `except CodeMessageException` is now also pointless because the caller treats all exceptions equally, so we may as well just throw the `CodeMessageException`.	2017-04-21 01:32:01 +01:00
Richard van der Hoff	838810b76a	Broaden the conditions for locally_rejecting invites ... The logic for marking invites as locally rejected was all well and good, but didn't happen when the remote server returned a 500, or wasn't reachable, or had no DNS, or whatever. Just expand the except clause to catch everything. Fixes https://github.com/matrix-org/synapse/issues/761.	2017-04-21 01:31:37 +01:00
Richard van der Hoff	736b9a4784	Remove redundant function ... inline `reject_remote_invite`, which only existed to make tracing the callflow more difficult.	2017-04-21 01:31:09 +01:00
Richard van der Hoff	4903ccf159	Fix some lies, and other clarifications, in docstrings ... The documentation on get_json has been wrong ever since the very first commit to synapse...	2017-04-21 01:31:09 +01:00
Matthew Wolff	d4040e9e28	Queried CONDITIONAL_REQUIREMENTS	2017-04-18 16:19:48 -05:00
Luke Barnard	3fb8784c92	m.read_marker -> m.fully_read (#2128) ... Also: - change the REST endpoint to have a "S" on the end (so it's now /read_markers) - change the content of the m.read_up_to event to have the key "event_id" instead of "marker".	2017-04-18 17:46:15 +01:00
Matthew Wolff	54f9a4cb59	Fixed travis build failure ... Signed-off-by: Matthew Wolff <matthewjwolff@gmail.com>	2017-04-17 01:38:27 -05:00
Matthew Wolff	8e780b113d	web_server_root documentation fix ... Signed-off-by: Matthew Wolff <matthewjwolff@gmail.com>	2017-04-17 00:49:11 -05:00
Anant Prakash	574d573ac2	Check that requested room_id exists	2017-04-14 23:50:59 +05:30
Luke Barnard	78f0ddbfad	Merge pull request #2120 from matrix-org/luke/read-markers ... Implement Read Marker API	2017-04-13 14:21:31 +01:00
Luke Barnard	6a70647d45	Correct logic in is_event_after	2017-04-13 13:46:17 +01:00
Anant Prakash	c1f52a321d	synctl.py: Check if synapse is already running	2017-04-13 18:00:02 +05:30
Luke Barnard	b9557064bf	Simplify is_event_after logic	2017-04-12 14:36:20 +01:00
Luke Barnard	cf6121e3da	More null-guard changes	2017-04-12 14:02:03 +01:00
Erik Johnston	247c736b9b	Merge pull request #2115 from matrix-org/erikj/dedupe_federation_repl ... Reduce federation replication traffic	2017-04-12 11:07:13 +01:00
Paul Evans	8fbc0d29ee	Merge pull request #2121 from matrix-org/paul/sent-transactions-metric ... Add a counter metric for successfully-sent transactions	2017-04-12 11:04:31 +01:00
Erik Johnston	c06c00190f	Merge pull request #2116 from matrix-org/erikj/dedupe_federation_repl2 ... Dedupe KeyedEdu and Devices federation repl traffic	2017-04-12 10:57:24 +01:00
Luke Barnard	c0aba0a23e	Remove Unused ref to hs	2017-04-12 10:52:11 +01:00
Luke Barnard	b9676a75f6	Move a space	2017-04-12 10:51:17 +01:00
Luke Barnard	69a18514e9	Only notify user, not entire room	2017-04-12 10:50:37 +01:00
Luke Barnard	122cd52ce4	Remove comment, simplify null-guard	2017-04-12 10:48:32 +01:00
Erik Johnston	26ae5178a4	Add some comments	2017-04-12 10:36:29 +01:00
Erik Johnston	bf9060156a	Merge pull request #2117 from matrix-org/erikj/remove_http_replication ... Remove HTTP replication APIs	2017-04-12 10:21:42 +01:00
Erik Johnston	1745069543	Comment	2017-04-12 10:17:10 +01:00
Erik Johnston	c7ddb5ef7a	Reuse get_interested_parties	2017-04-12 10:16:26 +01:00
Erik Johnston	7b41013102	Merge pull request #2118 from matrix-org/erikj/no_devices ... Fix getting latest device IP for user with no devices	2017-04-12 10:14:32 +01:00
Luke Barnard	77fb2b72ae	Handle no previous RM	2017-04-12 09:47:29 +01:00
Luke Barnard	7f94709066	travis flake8..	2017-04-11 18:35:45 +01:00
Luke Barnard	867822fa1e	flake8	2017-04-11 17:36:04 +01:00
Luke Barnard	73880268ef	Refactor event ordering check to events store	2017-04-11 17:34:09 +01:00
Luke Barnard	131485ef66	Copyright	2017-04-11 17:33:51 +01:00
Paul "LeoNerd" Evans	11dbceb761	Add a counter metric for successfully-sent transactions	2017-04-11 17:16:12 +01:00
Luke Barnard	0127423027	flake8	2017-04-11 17:07:07 +01:00
Erik Johnston	85657eedf8	Bail on where clause instead	2017-04-11 16:24:31 +01:00
Erik Johnston	b48045a8f5	Don't bother with outer check for now	2017-04-11 16:23:24 +01:00
Erik Johnston	9c712a366f	Move get_presence_list_* to SlaveStore	2017-04-11 16:07:33 +01:00
Erik Johnston	a8c8e4efd4	Comment	2017-04-11 15:35:49 +01:00
Erik Johnston	414522aed5	Move get_interested_parties	2017-04-11 15:33:26 +01:00
Erik Johnston	2be8a281d2	Comments	2017-04-11 15:28:24 +01:00
Erik Johnston	6308ac45b0	Move get_interested_remotes back to presence handler	2017-04-11 15:19:26 +01:00
Erik Johnston	b9b72bc6e2	Comments	2017-04-11 15:15:34 +01:00
Luke Barnard	d892079844	Finish implementing RM endpoint ... - This change causes a 405 to be sent if "m.read_marker" is set via /account_data - This also fixes-up the RM endpoint so that it actually Works.	2017-04-11 15:01:39 +01:00
lukebarnard	e263c26690	Initial commit of RM server-side impl ... (See https://docs.google.com/document/d/1UWqdS-e1sdwkLDUY0wA4gZyIkRp-ekjsLZ8k6g_Zvso/edit#heading=h.lndohpg8at5u)	2017-04-11 11:55:30 +01:00
Erik Johnston	f3cf3ff8b6	Merge branch 'master' of github.com:matrix-org/synapse into develop	2017-04-11 11:13:32 +01:00
Erik Johnston	34840cdcef	Fix getting latest device IP for user with no devices	2017-04-11 09:56:54 +01:00
Erik Johnston	28a4649785	Remove HTTP replication APIs	2017-04-11 09:52:11 +01:00
Matthew Hodgson	7c551ec445	trust a hypothetical future riot.im IS	2017-04-10 17:58:36 +01:00
Erik Johnston	84fbb80c8f	Use generators	2017-04-10 16:55:56 +01:00
Erik Johnston	40453b3f84	Dedupe KeyedEdu and Devices federation repl traffic	2017-04-10 16:49:51 +01:00
Erik Johnston	29574fd5b3	Reduce federation presence replication traffic ... This is mainly done by moving the calculation of where to send presence updates from the presence handler to the transaction queue, so we only need to send the presence event (and not the destinations) across the replication connection. Before we were duplicating by sending the full state across once per destination.	2017-04-10 16:48:30 +01:00
Erik Johnston	2e6f5a4910	Typo	2017-04-10 16:17:40 +01:00
David Baker	405ba4178a	Merge pull request #2102 from DanielDent/add-auth-email ... Support authenticated SMTP	2017-04-10 15:42:16 +01:00
Erik Johnston	efcb6db688	Merge pull request #2109 from matrix-org/erikj/send_queue_fix ... Fix up federation SendQueue and document types	2017-04-10 13:09:25 +01:00
Erik Johnston	0018491af2	Rename variable	2017-04-10 12:44:43 +01:00
Erik Johnston	0364d23210	Up replication ping timeout	2017-04-10 11:32:05 +01:00
Erik Johnston	8c5f03cec7	Revert to sending the same data type as before	2017-04-10 10:07:18 +01:00
Erik Johnston	f8434db549	Change name	2017-04-10 10:03:07 +01:00
Erik Johnston	ab904caf33	Comments	2017-04-10 10:02:17 +01:00
Richard van der Hoff	64765e5199	When we do an invite rejection, save the signed leave event to the db ... During a rejection of an invite received over federation, we ask a remote server to make us a `leave` event, then sign it, then send that with `send_leave`. We were saving the *unsigned* version of the event (which has a different event id to the signed version) to our db (and sending it to the clients), whereas other servers in the room will have seen the *signed* version. We're not aware of any actual problems that caused, except that it makes the database confusing to look at and generally leaves the room in a weird state.	2017-04-07 14:39:32 +01:00
Erik Johnston	2a3e822f44	Comment	2017-04-07 13:47:04 +01:00
Erik Johnston	a828a64b75	Comment	2017-04-07 11:54:03 +01:00
Erik Johnston	d4d176e5d0	Add logging	2017-04-07 11:51:28 +01:00
Erik Johnston	449d1297ca	Fix up federation SendQueue and document types	2017-04-07 11:48:33 +01:00
Erik Johnston	d72667fcce	Speed up get_current_state_ids ... Using _simple_select_list is fairly expensive for functions that return a lot of rows and/or get called a lot. (This is because it carefully constructs a list of dicts). get_current_state_ids gets called a lot on startup and e.g. when the IRC bridge decided to send tonnes of joins/leaves (as it invalidates the cache). We therefore replace it with a custon txn function that builds up the final result dict without building up and intermediate representation.	2017-04-07 10:10:49 +01:00
Erik Johnston	a41fe500d6	Bump version and changelog	2017-04-07 10:03:48 +01:00
Erik Johnston	54f59bd7d4	Merge pull request #2107 from HarHarLinks/patch-1 ... fix typo in synctl help	2017-04-07 09:54:37 +01:00
Erik Johnston	98ce212093	Merge pull request #2103 from matrix-org/erikj/no-double-encode ... Don't double encode replication data	2017-04-07 09:39:52 +01:00
Kim Brose	8a1137ceab	fix typo in synctl help	2017-04-06 17:10:20 +02:00
Erik Johnston	877c029c16	Use iteritems	2017-04-06 15:51:22 +01:00
Erik Johnston	944692ef69	Merge pull request #2106 from matrix-org/erikj/reduce_user_sync ... Reduce rate of USER_SYNC repl commands	2017-04-06 13:35:31 +01:00
Erik Johnston	391712a4f9	Comment	2017-04-06 13:35:00 +01:00
Erik Johnston	ad544c803a	Document types of the replication streams	2017-04-06 13:28:52 +01:00
Erik Johnston	dbf87282d3	Docs	2017-04-06 13:11:21 +01:00
Erik Johnston	69b3fd485d	Fix incorrect type when using InvalidateCacheCommand	2017-04-06 09:36:38 +01:00
Daniel Dent	5058292537	Support authenticated SMTP ... Closes (SYN-714) #1385 Signed-off-by: Daniel Dent <matrixcontrib@contactdaniel.net>	2017-04-05 21:01:08 -07:00
Erik Johnston	fcc803b2bf	Add log lines	2017-04-05 17:13:44 +01:00
Erik Johnston	3f213d908d	Rearrange metrics	2017-04-05 14:15:09 +01:00
Erik Johnston	1ca0e78ca1	Fix typo	2017-04-05 13:43:39 +01:00
Erik Johnston	b43d3267e2	Fixup some metrics for tcp repl	2017-04-05 13:34:54 +01:00
Erik Johnston	b5cb6347a4	Don't immediately notify the master about users whose syncs have gone away	2017-04-05 13:25:40 +01:00
Erik Johnston	96b9b6c127	Don't double json encode typing replication data	2017-04-05 11:34:20 +01:00
Erik Johnston	f10ce8944b	Don't double json encode federation replication data	2017-04-05 11:10:28 +01:00
Erik Johnston	a5c401bd12	Merge pull request #2097 from matrix-org/erikj/repl_tcp_client ... Move to using TCP replication	2017-04-05 09:36:21 +01:00
Erik Johnston	d1d5362267	Add comment	2017-04-04 16:41:03 +01:00
Erik Johnston	9f26d3b75b	Deduplicate new deviceinbox rows for replication	2017-04-04 16:21:21 +01:00
Erik Johnston	a76886726b	Merge pull request #2098 from matrix-org/erikj/repl_tcp_fix ... Advance replication streams even if nothing is listening	2017-04-04 15:40:51 +01:00
Erik Johnston	ac66e11f2b	Add the appropriate amount of preserve_fn	2017-04-04 15:22:54 +01:00
Erik Johnston	4264ceb31c	Fiddle tcp replication logging	2017-04-04 14:14:03 +01:00
Erik Johnston	023ee197be	Advance replication streams even if nothing is listening ... Otherwise the streams don't advance and steadily fall behind, so when a worker does connect either a) they'll be streamed lots of old updates or b) the connection will fail as the streams are too far behind.	2017-04-04 13:19:26 +01:00
Erik Johnston	d1605794ad	Remove unused worker config option	2017-04-04 11:17:00 +01:00
Erik Johnston	3376f16012	Shuffle and comment synchrotron presence	2017-04-04 11:14:16 +01:00
Erik Johnston	6ce6bbedcb	Move where we ack federation	2017-04-04 11:02:44 +01:00
Erik Johnston	62b89daac6	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/repl_tcp_server	2017-04-04 09:46:16 +01:00
Richard van der Hoff	773e64cc1a	Merge pull request #2095 from matrix-org/rav/cull_log_preserves ... Cull spurious PreserveLoggingContexts	2017-04-03 17:02:25 +01:00
Richard van der Hoff	2d05eb3cf5	Merge remote-tracking branch 'origin/release-v0.20.0' into develop	2017-04-03 16:23:23 +01:00
Richard van der Hoff	30bcbf775a	Accept join events from all servers ... Make sure that we accept join events from any server, rather than just the origin server, to make the federation join dance work correctly. (Fixes #1893).	2017-04-03 15:58:07 +01:00
Richard van der Hoff	7eb9f34cc3	Remove spurious yield ... In `MessageHandler`, remove `yield` on call to `Notifier.on_new_room_event`: it doesn't return anything anyway.	2017-04-03 15:44:19 +01:00
Richard van der Hoff	0b08c48fc5	Remove more spurious PreserveLoggingContexts ... Remove `PreserveLoggingContext` around calls to `Notifier.on_new_room_event`; there is no problem if the logcontext is set when calling it.	2017-04-03 15:43:37 +01:00
Richard van der Hoff	65e1683680	Remove spurious PreserveLoggingContext ... In `on_new_room_event`, remove `PreserveLoggingContext` - we can call its subroutines with the logcontext set.	2017-04-03 15:42:38 +01:00
Richard van der Hoff	feb496056e	preserve_fn some deferred-returning things ... In `Notifier._on_new_room_event`, `preserve_fn` around its subroutines which return deferreds, so that it is safe to call it with an active logcontext.	2017-04-03 15:41:17 +01:00
Richard van der Hoff	e2eebf1696	Fix fixme in preserve_fn ... `preserve_fn` is no longer used as a decorator anywhere, so we can safely fix a fixme therein.	2017-04-03 15:38:02 +01:00
Erik Johnston	36c28bc467	Update all the workers and master to use TCP replication	2017-04-03 15:35:52 +01:00
Erik Johnston	3a1f3f8388	Change slave storage to use new replication interface ... As the TCP replication uses a slightly different API and streams than the HTTP replication. This breaks HTTP replication.	2017-04-03 15:34:19 +01:00
Erik Johnston	52bfa604e1	Add basic replication client handler and factory	2017-04-03 15:34:13 +01:00
Erik Johnston	0a6a966e2b	Always advance stream tokens	2017-04-03 15:22:56 +01:00
Richard van der Hoff	773e1c6d68	Remove spurious @preserve_fn decorators ... Remove `@preserve_fn` decorators on `on_new_room_event`, `_notify_pending_new_room_events`, `_on_new_room_event`, `on_new_event`, and `on_new_replication_data` - none of these functions return a deferred, and the decorator does nothing unless the wrapped function returns a deferred, so the decorator was a no-op.	2017-04-03 15:14:11 +01:00
Erik Johnston	0d1c85e643	Merge branch 'release-v0.20.0' of github.com:matrix-org/synapse into develop	2017-04-03 14:58:14 +01:00
Erik Johnston	1df7c28661	Use callbacks to notify tcp replication rather than deferreds	2017-03-31 15:42:51 +01:00
Erik Johnston	36d2b66f90	Add a timestamp to USER_SYNC command ... This timestamp is used to indicate when the user last sync'd	2017-03-31 15:42:22 +01:00
Erik Johnston	8a240e4f9c	Merge pull request #2078 from APwhitehat/assertuserfriendly ... add user friendly report of assertion error in synctl.py	2017-03-31 14:41:49 +01:00
Erik Johnston	142b6b4abf	Merge pull request #2011 from matrix-org/matthew/turn_allow_guests ... add setting (on by default) to support TURN for guests	2017-03-31 14:37:09 +01:00
Erik Johnston	2a06b44be2	Merge pull request #1986 from matrix-org/matthew/enable_guest_3p ... enable guest access for the 3pl/3pid APIs	2017-03-31 14:36:03 +01:00
Matthew Hodgson	9ee397b440	switch to allow_guest=True for authing 3Ps as per PR feedback	2017-03-31 13:54:26 +01:00
Erik Johnston	9d0170ac6c	Fix up presence	2017-03-31 11:36:32 +01:00
Erik Johnston	bfcf016714	Fix up docs	2017-03-31 11:19:24 +01:00
Erik Johnston	9cee0ce7db	Merge pull request #2075 from matrix-org/erikj/cache_speed ... Speed up cached function access	2017-03-31 10:10:56 +01:00
Erik Johnston	350333a09a	Merge pull request #2076 from matrix-org/erikj/as_perf ... Make AS's faster	2017-03-31 09:43:10 +01:00
Erik Johnston	4d17add8de	Remove unused instance variable	2017-03-31 09:38:27 +01:00
Erik Johnston	27b1b4a2c9	Speed up copy_and_replace	2017-03-30 17:50:31 +01:00
Erik Johnston	5b5b171f3e	Docs	2017-03-30 17:05:53 +01:00
Erik Johnston	b282fe7170	Revert log context change	2017-03-30 17:03:59 +01:00
Erik Johnston	9ff4e0e91b	Bump version and changelog	2017-03-30 16:37:40 +01:00
Erik Johnston	0834d1a70c	Merge pull request #2079 from matrix-org/erikj/push_regex_cache ... Cache glob to regex at a higher level for push	2017-03-30 16:28:18 +01:00
Erik Johnston	63fcc42990	Remove user from process_presence when stops syncing	2017-03-30 14:26:08 +01:00
Erik Johnston	6194a64ae9	Doc new instance variables	2017-03-30 14:19:10 +01:00
Erik Johnston	014fee93b3	Manually calculate cache key as getcallargs is expensive ... This is because getcallargs recomputes the getargspec, amongst other things, which we don't need to do as its already been done	2017-03-30 14:14:46 +01:00
Erik Johnston	86780a8bc3	Don't convert to deferreds when not necessary	2017-03-30 14:14:36 +01:00
Erik Johnston	3ba2859e0c	Add tcp replication listener type and hook it up	2017-03-30 13:31:10 +01:00
Erik Johnston	e9dd8370b0	Add functions to presence to support remote syncs ... The TCP replication protocol streams deltas of who has started or stopped syncing. This is different from the HTTP API which periodically sends the full list of users who are syncing. This commit adds support for the new TCP style of sending deltas.	2017-03-30 13:25:14 +01:00
Erik Johnston	4d7fc7f977	Add server side resource for tcp replication	2017-03-30 13:24:45 +01:00
Richard van der Hoff	f9b4bb05e0	Fix the logcontext handling in the cache wrappers (#2077) ... The cache wrappers had a habit of leaking the logcontext into the reactor while the lookup function was running, and then not restoring it correctly when the lookup function had completed. It's all the fault of `preserve_context_over_{fn,deferred}` which are basically a bit broken.	2017-03-30 13:22:24 +01:00
Erik Johnston	7450693435	Initial TCP protocol implementation ... This defines the low level TCP replication protocol	2017-03-30 12:54:46 +01:00
Erik Johnston	8da6f0be48	Define the various streams we will replicate	2017-03-30 12:54:46 +01:00
Erik Johnston	11880103b1	Make federation send queue take the current position	2017-03-30 12:54:36 +01:00
Erik Johnston	7984708a55	Add a simple hook to wait for replication traffic	2017-03-30 11:57:52 +01:00
Erik Johnston	24d35ab47b	Add new storage functions for new replication ... The new replication protocol will keep all the streams separate, rather than muxing multiple streams into one.	2017-03-30 11:48:35 +01:00
Erik Johnston	30348c924c	Use txn.fetchall() so we can reuse txn	2017-03-30 10:30:05 +01:00
Anant Prakash	6cdca71079	synctl.py: wait for synapse to stop before restarting (#2020)	2017-03-29 19:20:13 +01:00
Anant Prakash	305d16d612	add user friendly report of assertion error in synctl.py ... Signed-off-by: Anant Prakash <anantprakashjsr@gmail.com>	2017-03-29 20:41:39 +05:30
Erik Johnston	a3810136fe	Cache glob to regex at a higher level for push	2017-03-29 15:53:14 +01:00
Erik Johnston	3ce8d59176	Increase cache size for _get_state_group_for_event	2017-03-29 14:31:46 +01:00
Erik Johnston	b9c2ae6788	Merge pull request #1849 from matrix-org/erikj/state_typo ... Fix bug where current_state_events renamed to current_state_ids	2017-03-29 11:53:42 +01:00
Erik Johnston	85be3dde81	Bail early if remote wouldn't be retried (#2064) ... * Bail early if remote wouldn't be retried * Don't always return true * Just use get_retry_limiter * Spelling	2017-03-29 11:48:27 +01:00
Erik Johnston	2f8b580b64	Merge pull request #2053 from matrix-org/erikj/e2e_one_time_upsert ... Don't user upsert to persist new one time keys	2017-03-29 11:44:23 +01:00
Erik Johnston	c5b0bdd542	Merge pull request #2067 from matrix-org/erikj/notify_on_fed ... Notify on new federation traffic	2017-03-29 11:41:37 +01:00
Erik Johnston	e4df0e189d	Decrank last commit	2017-03-29 11:02:35 +01:00
Erik Johnston	4ad613f6be	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/e2e_one_time_upsert	2017-03-29 10:57:19 +01:00