7188 Commits

Author	SHA1	Message	Date
Luke Barnard	a34c586a89	Make the get_rooms_in_group API more sane ... Return entries with is_public = True when they're public and is_public = False otherwise.	2017-11-02 16:42:30 +00:00
Richard van der Hoff	6c3a02072b	support inhibit_login in /register ... Allow things to pass inhibit_login when registering to ... inhibit logins.	2017-11-02 16:31:07 +00:00
David Baker	4b36897cd9	Merge remote-tracking branch 'origin/develop' into rav/module_api_hooks	2017-11-02 15:19:17 +00:00
David Baker	d4553818a0	Merge pull request #2627 from matrix-org/rav/custom_rest_endpoints ... Add a hook for custom rest endpoints	2017-11-02 15:18:37 +00:00
David Baker	6b6f03ae05	Merge pull request #2626 from matrix-org/rav/refactor_module_api ... Factor _AccountHandler proxy out to ModuleApi	2017-11-02 15:15:30 +00:00
Richard van der Hoff	6b60f7dca0	Add more hooks to ModuleApi ... add `get_user_by_req` and `invalidate_access_token`	2017-11-02 14:37:39 +00:00
Richard van der Hoff	fcdfc911ee	Add a hook for custom rest endpoints ... Let the user specify custom modules which can be used for implementing extra endpoints.	2017-11-02 14:36:55 +00:00
Richard van der Hoff	1189be43a2	Factor _AccountHandler proxy out to ModuleApi ... We're going to need to use this from places that aren't password auth, so let's move it to a proper class.	2017-11-02 14:36:11 +00:00
Richard van der Hoff	6650a07ede	Factor out _configure_named_resource ... This was a bit of a code vomit, so let's factor it out to preserve some sanity	2017-11-02 14:33:37 +00:00
David Baker	b19d9e2174	Merge pull request #2624 from matrix-org/rav/password_provider_notify_logout ... Notify auth providers on logout	2017-11-02 10:55:17 +00:00
David Baker	1f080a6c97	Merge pull request #2623 from matrix-org/rav/callbacks_for_auth_providers ... Allow password_auth_providers to return a callback	2017-11-02 10:49:03 +00:00
David Baker	04897c9dc1	Merge pull request #2622 from matrix-org/rav/db_access_for_auth_providers ... Let auth providers get to the database	2017-11-02 10:41:25 +00:00
Richard van der Hoff	979eed4362	Fix user-interactive password auth ... this got broken in the previous commit	2017-11-01 17:03:20 +00:00
Richard van der Hoff	bc8a5c0330	Notify auth providers on logout ... Provide a hook by which auth providers can be notified of logouts.	2017-11-01 16:51:51 +00:00
Richard van der Hoff	4c8f94ac94	Allow password_auth_providers to return a callback ... ... so that they have a way to record access tokens.	2017-11-01 16:51:03 +00:00
Richard van der Hoff	846a94fbc9	Merge pull request #2620 from matrix-org/rav/auth_non_password ... Let password auth providers handle arbitrary login types	2017-11-01 16:45:33 +00:00
Richard van der Hoff	3cd6b22c7b	Let password auth providers handle arbitrary login types ... Provide a hook where password auth providers can say they know about other login types, and get passed the relevant parameters	2017-11-01 16:43:57 +00:00
David Baker	c9b9ef575b	Merge pull request #2621 from matrix-org/rav/refactor_accesstoken_delete ... Move access token deletion into auth handler	2017-11-01 16:26:06 +00:00
Matthew Hodgson	275826f234	Merge pull request #2617 from matrix-org/matthew/auto-displayname ... automatically set default displayname on register	2017-11-01 16:21:16 +00:00
David Baker	4f0488b307	Merge remote-tracking branch 'origin/develop' into rav/refactor_accesstoken_delete	2017-11-01 16:20:19 +00:00
David Baker	e5e930aec3	Merge pull request #2615 from matrix-org/rav/break_auth_device_dep ... Break dependency of auth_handler on device_handler	2017-11-01 16:06:31 +00:00
David Baker	fbbacb284e	Merge pull request #2613 from matrix-org/rav/kill_refresh_tokens ... Remove the last vestiges of refresh_tokens	2017-11-01 15:57:35 +00:00
Matthew Hodgson	9f7a555b4e	switch to setting default displayname in the storage layer ... to avoid clobbering guest user displaynames on registration	2017-11-01 15:51:30 +00:00
Richard van der Hoff	dd13310fb8	Move access token deletion into auth handler ... Also move duplicated deactivation code into the auth handler. I want to add some hooks when we deactivate an access token, so let's bring it all in here so that there's somewhere to put it.	2017-11-01 15:46:22 +00:00
David Baker	0bb253f37b	Apparently this is python	2017-11-01 14:02:52 +00:00
David Baker	59e7e62c4b	Log login requests ... Carefully though, to avoid logging passwords	2017-11-01 13:58:01 +00:00
Matthew Hodgson	f8420d6279	automatically set default displayname on register ... to avoid leaking ugly MXIDs and cluttering up the timeline with displayname changes as well as membership joins for autojoin rooms (e.g. the status autojoin rooms), automatically set the displayname to match the localpart of the mxid upon registration.	2017-11-01 13:15:41 +00:00
Luke Barnard	99354b430e	Merge pull request #2612 from matrix-org/luke/groups-room-relationship-is-public ... Modify group room association API to allow modification of is_public	2017-11-01 11:08:36 +00:00
Richard van der Hoff	74c56f794c	Break dependency of auth_handler on device_handler ... I'm going to need to make the device_handler depend on the auth_handler, so I need to break this dependency to avoid a cycle. It turns out that the auth_handler was only using the device_handler in one place which was an edge case which we can more elegantly handle by throwing an error rather than fixing it up.	2017-11-01 10:27:06 +00:00
Luke Barnard	318a249c8b	Leave is_public as required argument of update_room_group_association	2017-11-01 09:36:01 +00:00
Luke Barnard	207fabbc6a	Update docs for updating room group association	2017-11-01 09:35:15 +00:00
Richard van der Hoff	356bcafc44	Remove the last vestiges of refresh_tokens	2017-10-31 20:35:58 +00:00
Richard van der Hoff	3e0aaad190	Let auth providers get to the database ... Somewhat open to abuse, but also somewhat unavoidable :/	2017-10-31 17:22:29 +00:00
Luke Barnard	13b3d7b4a0	Flake8	2017-10-31 17:20:11 +00:00
Luke Barnard	20fe347906	Modify group room association API to allow modification of is_public ... also includes renamings to make things more consistent.	2017-10-31 17:04:28 +00:00
Richard van der Hoff	1650eb5847	DB schema interface for password auth providers ... Provide an interface by which password auth providers can register db schema files to be run at startup	2017-10-31 14:01:53 +00:00
David Baker	c31a7c3ff6	Merge pull request #2609 from matrix-org/rav/refactor_login ... Refactor some logic from LoginRestServlet into AuthHandler	2017-10-31 13:51:36 +00:00
Richard van der Hoff	1b65ae00ac	Refactor some logic from LoginRestServlet into AuthHandler ... I'm going to need some more flexibility in handling login types in password auth providers, so as a first step, move some stuff from LoginRestServlet into AuthHandler. In particular, we pass everything other than SAML, JWT and token logins down to the AuthHandler, which now has responsibility for checking the login type and fishing the password out of the login dictionary, as well as qualifying the user_id if need be. Ideally SAML, JWT and token would go that way too, but there's no real need for it right now and I'm trying to minimise impact. This commit *should* be non-functional.	2017-10-31 10:48:41 +00:00
Richard van der Hoff	ffc574a6f9	Clean up backwards-compat hacks for ldap ... try to make the backwards-compat flows follow the same code paths as the modern impl. This commit should be non-functional.	2017-10-31 10:47:02 +00:00
Luke Barnard	9bc17fc5fb	Fix wording on group creation error	2017-10-30 15:17:23 +00:00
Matthew Hodgson	208a6647f1	fix typo	2017-10-29 20:54:20 +00:00
Erik Johnston	d0abb4e8e6	Fix typo when checking if user is invited to group	2017-10-27 16:57:19 +01:00
Erik Johnston	977078f06d	Fix bad merge	2017-10-27 15:10:50 +01:00
Erik Johnston	6980c4557e	Merge branch 'erikj/attestation_jitter' of github.com:matrix-org/synapse into develop	2017-10-27 15:09:05 +01:00
Erik Johnston	af92f5b00f	Revert "Add jitter to validity period of attestations"	2017-10-27 15:07:21 +01:00
Erik Johnston	4ab8abbc2b	Merge branch 'erikj/attestation_local_fix' of github.com:matrix-org/synapse into develop	2017-10-27 15:07:08 +01:00
Erik Johnston	6af3656deb	Merge pull request #2595 from matrix-org/erikj/attestation_commnet ... Add comment about attestations	2017-10-27 14:20:19 +01:00
Richard van der Hoff	4d83632009	Merge pull request #2591 from matrix-org/rav/device_delete_auth ... Device deletion: check UI auth matches access token	2017-10-27 12:30:10 +01:00
Richard van der Hoff	110b373e9c	Merge pull request #2589 from matrix-org/rav/as_deactivate_account ... Allow ASes to deactivate their own users	2017-10-27 12:29:32 +01:00
Erik Johnston	ca571b0ec3	Add jitter to validity period of attestations ... This helps ensure that the renewals of attestations are spread out more evenly.	2017-10-27 11:57:27 +01:00
Luke Barnard	d8c26162a1	Merge pull request #2582 from matrix-org/luke/group-is-public ... Add is_public to groups table to allow for private groups	2017-10-27 11:41:13 +01:00
Erik Johnston	c067088747	Add comment about attestations	2017-10-27 11:35:41 +01:00
Luke Barnard	5451cc7792	Request is_public from database	2017-10-27 11:27:43 +01:00
Luke Barnard	124314672f	group is dict	2017-10-27 11:08:19 +01:00
Luke Barnard	6362298fa5	Create groups with is_public = True	2017-10-27 11:04:20 +01:00
Richard van der Hoff	8b56977b6f	Merge pull request #2586 from matrix-org/rav/frontend_proxy_auth_header ... Front-end proxy: pass through auth header	2017-10-27 11:01:50 +01:00
Richard van der Hoff	173567a7f2	Docstring for post_urlencoded_get_json	2017-10-27 10:59:50 +01:00
Luke Barnard	c7d9f25d22	Fix create_group to pass requester_user_id	2017-10-27 10:57:20 +01:00
Erik Johnston	e27b76d117	Import logger	2017-10-27 10:54:02 +01:00
Richard van der Hoff	8854c039f2	Merge pull request #2585 from matrix-org/rav/unstable_to_r0 ... Support /keys/upload on /r0 as well as /unstable	2017-10-27 10:53:48 +01:00
Richard van der Hoff	14f581abc2	Merge pull request #2584 from matrix-org/rav/fix_httpclient_logcontexts ... Fix logcontext leaks in httpclient	2017-10-27 10:53:29 +01:00
Luke Barnard	2ca46c7afc	Correct logic for checking private group membership	2017-10-27 10:48:01 +01:00
Erik Johnston	82d8c1bacb	Fixup	2017-10-27 10:30:21 +01:00
Erik Johnston	2fd9831f7c	Merge pull request #2574 from matrix-org/erikj/room_list_fixes ... Add logging and fix log contexts for publicRooms	2017-10-27 10:01:23 +01:00
Erik Johnston	195abfe7a5	Remove incorrect attestations	2017-10-27 09:58:13 +01:00
Erik Johnston	d8dde19f04	Log if we try to do attestations for our own user and group	2017-10-27 09:55:01 +01:00
Erik Johnston	585972b51a	Don't generate group attestations for local users	2017-10-27 09:46:56 +01:00
Richard van der Hoff	7a6546228b	Device deletion: check UI auth matches access token ... (otherwise there's no point in the UI auth)	2017-10-27 00:04:31 +01:00
Richard van der Hoff	785bd7fd75	Allow ASes to deactivate their own users	2017-10-27 00:01:00 +01:00
Richard van der Hoff	c89e6aadff	Merge pull request #2581 from matrix-org/rav/fix_init_with_no_logfile ... Fix error when running synapse with no logfile	2017-10-26 22:16:57 +01:00
Richard van der Hoff	54a2525133	Front-end proxy: pass through auth header ... So that access-token-in-an-auth-header works.	2017-10-26 18:19:01 +01:00
Richard van der Hoff	0a5866bec9	Support /keys/upload on /r0 as well as /unstable ... (So that we can stop riot relying on it in /unstable)	2017-10-26 18:18:23 +01:00
Richard van der Hoff	0d8e3ad48b	Fix logcontext leaks in httpclient ... `preserve_context_over_fn` is borked	2017-10-26 18:17:10 +01:00
Richard van der Hoff	12ef02dc3d	SimpleHTTPClient: add support for headers ... Sometimes we need to pass headers into these methods	2017-10-26 17:59:50 +01:00
Luke Barnard	69e8a05f35	Make it work	2017-10-26 17:55:58 +01:00
Luke Barnard	007cd48af6	Recreate groups table instead of adding column ... Adding a column with non-constant default not possible in sqlite3	2017-10-26 17:55:22 +01:00
Luke Barnard	713e60b9b6	Awful hack to get default true	2017-10-26 17:38:14 +01:00
Luke Barnard	cfa4e658e0	Bump schema version to 46	2017-10-26 17:23:49 +01:00
Luke Barnard	595fe67f01	delint	2017-10-26 17:20:24 +01:00
Luke Barnard	9b2feef9eb	Add is_public to groups table to allow for private groups ... Prevent group API access to non-members for private groups Also make all the group code paths consistent with `requester_user_id` always being the User ID of the requesting user.	2017-10-26 16:51:32 +01:00
Richard van der Hoff	f7f90e0c8d	Fix error when running synapse with no logfile ... Fixes 'UnboundLocalError: local variable 'sighup' referenced before assignment'	2017-10-26 16:45:20 +01:00
Krombel	8299b323ee	add release endpoints for /thirdparty	2017-10-26 16:58:20 +02:00
Krombel	9b436c8b4c	register some /unstable endpoints in /r0 as well	2017-10-26 15:22:50 +02:00
Erik Johnston	566e21eac8	Update room_list.py	2017-10-26 11:39:54 +01:00
Maxime Vaillancourt	5287e57c86	Ignore noscript tags when generating URL previews	2017-10-25 20:44:34 -04:00
Erik Johnston	2a7e9faeec	Do logcontexts outside ResponseCache	2017-10-25 15:21:08 +01:00
Erik Johnston	1ad1ba9e6a	Merge branch 'master' of github.com:matrix-org/synapse into develop	2017-10-25 10:27:23 +01:00
Erik Johnston	33a9026cdf	Add logging and fix log contexts for publicRooms	2017-10-25 10:26:06 +01:00
Matthew Hodgson	efd0f5a3c5	tip for generating tls_fingerprints	2017-10-24 18:49:49 +01:00
Erik Johnston	6ba4fabdb9	Bump version and changelog	2017-10-24 14:15:27 +01:00
Erik Johnston	39dc52157d	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/group_fed_update_profile	2017-10-24 09:16:20 +01:00
Richard van der Hoff	0be99858f3	fix vars named l ... E741 says "do not use variables named ‘l’, ‘O’, or ‘I’".	2017-10-23 15:56:38 +01:00
Richard van der Hoff	eaaabc6c4f	replace 'except:' with 'except Exception:' ... what could possibly go wrong	2017-10-23 15:52:32 +01:00
Erik Johnston	ce6d4914f4	Correctly wire in update group profile over federation	2017-10-23 15:21:24 +01:00
Richard van der Hoff	ecf198aab8	Merge pull request #2566 from matrix-org/rav/media_logcontext_leak ... Fix a logcontext leak in the media repo	2017-10-23 14:47:49 +01:00
Richard van der Hoff	3267b81b81	Merge pull request #2561 from matrix-org/rav/id_checking ... Updates to ID checking	2017-10-23 14:39:20 +01:00
Richard van der Hoff	d03cfc4258	Fix a logcontext leak in the media repo	2017-10-23 14:34:27 +01:00
Erik Johnston	1de557975f	Merge branch 'master' of github.com:matrix-org/synapse into develop	2017-10-23 13:18:12 +01:00
Erik Johnston	13e16cf302	Bump version and changelog	2017-10-23 13:13:31 +01:00
Richard van der Hoff	1135193dfd	Validate group ids when parsing ... May as well do it whenever we parse a Group ID. We check the sigil and basic structure here so it makes sense to check the grammar in the same place.	2017-10-21 00:30:39 +01:00
Richard van der Hoff	29812c628b	Allow = in mxids and groupids ... ... because the spec says we should.	2017-10-20 23:42:53 +01:00
Richard van der Hoff	58fbbe0f1d	Disallow capital letters in userids ... Factor out a common function for checking user ids and group ids, which forbids capitals.	2017-10-20 23:37:22 +01:00
Richard van der Hoff	631d7b87b5	Remove pointless create() method ... It just calls the constructor, so we may as well kill it rather than having random codepaths.	2017-10-20 22:14:55 +01:00
Erik Johnston	6070647774	Correctly bump version	2017-10-19 16:40:20 +01:00
Erik Johnston	0ef0aeceac	Bump version and changelog	2017-10-19 13:48:33 +01:00
Erik Johnston	c7d46510d7	Flake8	2017-10-19 13:36:06 +01:00
Erik Johnston	ffd3f1a783	Add missing file...	2017-10-19 12:17:30 +01:00
Erik Johnston	29bafe2f7e	Add config to enable group creation	2017-10-19 12:13:44 +01:00
Erik Johnston	513c23bfd9	Enforce sensible group IDs	2017-10-19 12:01:01 +01:00
Erik Johnston	011d03a0f6	Fix typo	2017-10-19 11:22:48 +01:00
Erik Johnston	9ab859f27b	Fix typo in group attestation handling	2017-10-19 10:55:52 +01:00
Erik Johnston	bd5718d0ad	Fix typo in thumbnail generation	2017-10-19 10:27:18 +01:00
Erik Johnston	161a862ffb	Fix typo	2017-10-19 10:17:43 +01:00
Richard van der Hoff	69994c385a	Merge pull request #2553 from matrix-org/rav/fix_500_on_event_send ... Fix 500 error when we get an error handling a PDU	2017-10-18 11:25:44 +01:00
Richard van der Hoff	b5dbbac308	Merge pull request #2552 from matrix-org/rav/fix_500_on_dodgy_powerlevels ... Fix 500 error when fields missing from power_levels event	2017-10-17 20:53:30 +01:00
Richard van der Hoff	582bd19ee9	Fix 500 error when we get an error handling a PDU ... FederationServer doesn't have a send_failure (and nor does its subclass, ReplicationLayer), so this was failing. I'm not really sure what the idea behind send_failure is, given (a) we don't do anything at the other end with it except log it, and (b) we also send back the failure via the transaction response. I suspect there's a whole lot of dead code around it, but for now I'm just removing the broken bit.	2017-10-17 20:52:40 +01:00
Richard van der Hoff	74f99f227c	Doc some more dynamic Homeserver methods	2017-10-17 20:51:29 +01:00
Richard van der Hoff	c2bd177ea0	Fix 500 error when fields missing from power_levels event ... If the users or events keys were missing from a power_levels event, then we would throw 500s when trying to auth them.	2017-10-17 17:05:42 +01:00
Erik Johnston	fe6e9f580b	Merge pull request #2550 from krombel/fix_thumbnail_2548 ... fix thumbnailing (#2548)	2017-10-17 15:35:18 +01:00
Richard van der Hoff	7216c76654	Improve error handling for missing files (#2551) ... `os.path.exists` doesn't allow us to distinguish between permissions errors and the path actually not existing, which repeatedly confuses people. It also means that we try to overwrite existing key files, which is super-confusing. (cf issues #2455, #2379). Use os.stat instead. Also, don't recomemnd the the use of --generate-config, which screws everything up if you're using debian (cf #2455).	2017-10-17 14:46:17 +01:00
Richard van der Hoff	4cc8bb0767	Merge pull request #2549 from matrix-org/rav/event_persist_logcontexts ... Fix logcontext handling for persist_events	2017-10-17 13:20:35 +01:00
David Baker	4e242b3e20	Merge pull request #2545 from matrix-org/dbkr/auto_join_rooms ... Add config option to auto-join new users to rooms	2017-10-17 11:45:49 +01:00
Krombel	a6245478c8	fix thumbnailing (#2548) ... in commit 0e28281a the code for thumbnailing got refactored and the renaming of this variables was not done correctly. Signed-Off-by: Matthias Kesler <krombel@krombel.de>	2017-10-17 12:45:33 +02:00
Richard van der Hoff	2e9f5ea31a	Fix logcontext handling for persist_events ... * don't use preserve_context_over_deferred, which is known broken. * remove a redundant preserve_fn. * add/improve some comments	2017-10-17 10:59:30 +01:00
David Baker	a9c2e930ac	pep8	2017-10-17 10:13:13 +01:00
David Baker	c05e6015cc	Add config option to auto-join new users to rooms ... New users who register on the server will be dumped into all rooms in auto_join_rooms in the config.	2017-10-16 17:57:27 +01:00
Luke Barnard	85f5674e44	Delint	2017-10-16 15:52:17 +01:00
Luke Barnard	c43e8a9736	Make it work. Warn about lack of user profile	2017-10-16 15:50:39 +01:00
Luke Barnard	a3ac4f6b0a	_create_rererouter for get_invited_users_in_group	2017-10-16 15:41:03 +01:00
Luke Barnard	5dfd0350c7	Merge branch 'develop' into luke/groups-invited-users	2017-10-16 15:34:36 +01:00
Luke Barnard	ca96d609e4	Merge pull request #2543 from matrix-org/luke/fix-on-group-invite-no-profile ... Log a warning when no profile for invited member	2017-10-16 16:33:53 +02:00
Luke Barnard	2c5972f87f	Implement GET /groups/$groupId/invited_users	2017-10-16 15:31:11 +01:00
Luke Barnard	6079d0027a	Log a warning when no profile for invited member ... And return empty profile	2017-10-16 14:20:45 +01:00
David Baker	9342bcfce0	Omit the *s for @room notifications ... They're just redundant	2017-10-16 13:38:10 +01:00
Erik Johnston	1b6b0b1e66	Add try/finally block to close t_byte_source	2017-10-13 15:34:08 +01:00
Erik Johnston	6b725cf56a	Remove old comment	2017-10-13 15:23:41 +01:00
Erik Johnston	2b24416e90	Don't reuse source but instead copy from primary media store to backup	2017-10-13 14:11:34 +01:00
Erik Johnston	b92a8e6e4a	PEP8	2017-10-13 13:58:57 +01:00
Erik Johnston	31aa7bd8d1	Move type into key	2017-10-13 13:47:38 +01:00
Erik Johnston	ad1911bbf4	Comment	2017-10-13 13:47:05 +01:00
Erik Johnston	c021c39cbd	Remove spurious addition	2017-10-13 13:46:53 +01:00
Erik Johnston	1f43d22397	Don't needlessly rename variable	2017-10-13 11:42:07 +01:00
Erik Johnston	a675bd08bd	Add paths back in...	2017-10-13 11:41:06 +01:00
Erik Johnston	4d7e1dde70	Remove unnecessary diff	2017-10-13 11:36:32 +01:00
Erik Johnston	ae5d18617a	Make things be absolute paths again	2017-10-13 11:35:44 +01:00
Erik Johnston	9732ec6797	s/write_to_file/write_to_file_and_backup/	2017-10-13 11:34:41 +01:00
Erik Johnston	0e28281a02	Fix up	2017-10-13 11:33:49 +01:00
Erik Johnston	505371414f	Fix up thumbnailing function	2017-10-13 11:23:53 +01:00
Erik Johnston	e3428d26ca	Fix typo	2017-10-13 10:39:59 +01:00
Erik Johnston	35332298ef	Fix up comments	2017-10-13 10:39:32 +01:00
Erik Johnston	64db043a71	Move makedirs to thread	2017-10-13 10:25:01 +01:00
Erik Johnston	b60859d6cc	Use make_deferred_yieldable	2017-10-13 10:24:19 +01:00
Erik Johnston	d76621a47b	Fix comments	2017-10-12 18:16:25 +01:00
Erik Johnston	4ae85ae121	Don't close prematurely..	2017-10-12 17:57:31 +01:00
Erik Johnston	cc505b4b5e	getvalue closes buffer	2017-10-12 17:52:30 +01:00
Erik Johnston	1259a76047	Get len before close	2017-10-12 17:39:23 +01:00
Erik Johnston	802ca12d05	Don't close file prematurely	2017-10-12 17:37:21 +01:00
Erik Johnston	e283b555b1	Copy everything to backup	2017-10-12 17:31:24 +01:00
Erik Johnston	b77a13812c	Typo	2017-10-12 15:32:32 +01:00
Erik Johnston	6dfde6d485	Remove dead code	2017-10-12 15:30:26 +01:00
Erik Johnston	c8eeef6947	Fix typos	2017-10-12 15:28:24 +01:00
Erik Johnston	67cb89fbdf	Fix typo	2017-10-12 15:23:41 +01:00
Erik Johnston	bf4fb1fb40	Basic implementation of backup media store	2017-10-12 15:20:59 +01:00
hera	f807f7f804	log when we get an exception handling replication updates	2017-10-12 11:51:24 +01:00
David Baker	b8d8ed1ba9	Merge pull request #2531 from matrix-org/dbkr/spamcheck_error_messages ... Allow error strings from spam checker	2017-10-12 10:31:03 +01:00
Richard van der Hoff	cc794d60e7	Merge pull request #2532 from matrix-org/rav/fix_linearizer ... Fix stackoverflow and logcontexts from linearizer	2017-10-11 17:29:32 +01:00
Richard van der Hoff	8dd0c85ac5	Merge pull request #2529 from matrix-org/rav/fix_transaction_failure_handling ... log pdu_failures from incoming transactions	2017-10-11 17:29:14 +01:00
Richard van der Hoff	76fa695241	Merge pull request #2515 from matrix-org/rav/fix_receipt_logcontext ... A logformatter which includes the stack when the exception was caught when logging exceptions.	2017-10-11 17:28:01 +01:00
Richard van der Hoff	f30c4ed2bc	logformatter: fix AttributeError ... make sure we have the relevant fields before we try to log them.	2017-10-11 17:26:17 +01:00
Erik Johnston	b752507b48	Fix fetching remote summaries	2017-10-11 16:59:18 +01:00
Erik Johnston	818b08d0e4	peeeeeeeeep8888888888888888888888888888	2017-10-11 15:54:00 +01:00
Erik Johnston	ea18996f54	Fix group stream replication ... The stream update functions expect the storage function to return a list of tuples.	2017-10-11 15:44:39 +01:00
Richard van der Hoff	68fd82e840	Merge pull request #2530 from matrix-org/rav/fix_receipt_logcontext ... fix a logcontext leak in read receipt handling	2017-10-11 15:08:53 +01:00
Richard van der Hoff	4fad8efbfb	Fix stackoverflow and logcontexts from linearizer ... 1. make it not blow out the stack when there are more than 50 things waiting for a lock. Fixes https://github.com/matrix-org/synapse/issues/2505. 2. Make it not mess up the log contexts.	2017-10-11 15:05:05 +01:00
David Baker	b78bae2d51	fix isinstance	2017-10-11 14:49:09 +01:00
Erik Johnston	271f5601f3	Fix typo in invite to group	2017-10-11 14:45:33 +01:00
David Baker	c3b7a45e84	Allow error strings from spam checker	2017-10-11 14:39:22 +01:00
Richard van der Hoff	c3e190ce67	fix a logcontext leak in read receipt handling	2017-10-11 14:37:20 +01:00
Richard van der Hoff	b75d443caf	log pdu_failures from incoming transactions ... ... even if we have no EDUs. This appears to have been introduced in 476899295f.	2017-10-11 14:36:13 +01:00
Erik Johnston	27e727a146	Fix typo	2017-10-11 14:32:40 +01:00
Erik Johnston	4ce4379235	Fix attestations to check correct server name	2017-10-11 14:11:43 +01:00
Erik Johnston	c2c47550f9	Fix schema delta versions	2017-10-11 13:23:15 +01:00
Erik Johnston	535cc49f27	Merge pull request #2466 from matrix-org/erikj/groups_merged ... Initial Group Implementation	2017-10-11 13:20:07 +01:00
Erik Johnston	dfbf73408c	Merge pull request #2501 from matrix-org/dbkr/channel_notifications ... Support for channel notifications	2017-10-11 13:19:29 +01:00
Erik Johnston	bc7f3eb32f	Merge pull request #2483 from jeremycline/unfreeze-ujson-dump ... Unfreeze event before serializing with ujson	2017-10-11 13:18:52 +01:00
Erik Johnston	ec954f47fb	Validate room ids	2017-10-11 13:15:44 +01:00
David Baker	81a5e0073c	pep8	2017-10-10 15:53:34 +01:00
David Baker	ab1bc9bf5f	Don't KeyError if no power_levels event	2017-10-10 15:34:05 +01:00
David Baker	0f1eb3e914	Use notification levels in power_levels ... Rather than making the condition directly require a specific power level. This way the level require to notify a room can be configured per room.	2017-10-10 15:23:00 +01:00
Erik Johnston	84e27a592d	Merge pull request #2490 from matrix-org/erikj/drop_left_room_events ... Ignore incoming events for rooms that we have left	2017-10-10 11:58:32 +01:00
David Baker	c9f034b4ac	There was already a constant for this ... also update copyright	2017-10-10 11:47:10 +01:00
David Baker	a9f9d68631	More optimisation	2017-10-10 11:38:31 +01:00
David Baker	707374d5dc	What year is it!? Who's the president!?	2017-10-10 11:21:41 +01:00
David Baker	89fa00ddff	Merge branch 'develop' into dbkr/channel_notifications	2017-10-10 11:20:17 +01:00
Richard van der Hoff	6a6cc27aee	fed server: process PDUs for different rooms in parallel ... With luck, this will give a real-time improvement when there are many rooms and the server ends up calling out to fetch missing events.	2017-10-09 18:30:31 +01:00
Richard van der Hoff	4c7c4d4061	Fed server: use a linearizer for ongoing transactions ... We don't want to process the same transaction multiple times concurrently, so use a linearizer.	2017-10-09 18:30:10 +01:00
Richard van der Hoff	ba5b9b80a5	fed server: refactor on_incoming_transaction ... Move as much as possible to after the have_responded check, and reduce the number of times we iterate over the pdu list.	2017-10-09 18:10:53 +01:00
Richard van der Hoff	a6e3222fe5	Fed server: Move origin-check code to _handle_received_pdu ... The response-building code expects there to be an entry in the `results` list for each entry in the pdu_list, so the early `continue` was messing this up. That doesn't really matter, because all that the federation client does is log any errors, but it's pretty poor form.	2017-10-09 17:53:32 +01:00
Richard van der Hoff	3cc852d339	Fancy logformatter to format exceptions better ... This is a bit of an experimental change at this point; the idea is to see if it helps us track down where our stack overflows are coming from by logging the stack when the exception was caught and turned into a Failure. (We'll also need edf2704420). If we deploy this, we'll be able to enable it via the log config yaml.	2017-10-09 17:44:42 +01:00
Richard van der Hoff	0eeaa25694	Merge pull request #2508 from matrix-org/rav/federation_queue_logcontexts ... Fix up logcontext handling in (federation) TransactionQueue	2017-10-09 17:43:48 +01:00
Richard van der Hoff	aa3fac8057	Merge pull request #2507 from matrix-org/rav/execute_concurrently_log_contexts ... Fix logcontext handling for concurrently_execute	2017-10-09 17:43:32 +01:00
Erik Johnston	e8496efe84	Fix up comment	2017-10-09 15:17:34 +01:00
Richard van der Hoff	01bbacf3c4	Fix up logcontext handling in (federation) TransactionQueue ... Avoid using preserve_context_over_function, which has problems with respect to logcontexts.	2017-10-06 22:39:25 +01:00
Richard van der Hoff	148428ce76	Fix logcontext handling for concurrently_execute ... Avoid preserve_context_over_deferred, which is broken.	2017-10-06 22:24:28 +01:00
Richard van der Hoff	c8f568ddf9	Fix up deferred handling in federation.py ... * Avoid preserve_context_over_deferred, which is broken * set consumeErrors=True on defer.gatherResults, to avoid spurious "unhandled failure" erros	2017-10-06 22:14:24 +01:00
Richard van der Hoff	3ddda939d3	some comments in the state res code	2017-10-05 14:58:17 +01:00
David Baker	f878e6f8af	Spam checking: add the invitee to user_may_invite	2017-10-05 14:02:28 +01:00
David Baker	269af961e9	Make be faster	2017-10-05 13:27:12 +01:00
David Baker	ed80c6b6cc	Add fastpath optimisation	2017-10-05 13:20:22 +01:00
David Baker	e433393c4f	pep8	2017-10-05 13:08:02 +01:00
David Baker	985ce80375	They're called rooms	2017-10-05 13:03:44 +01:00
David Baker	b9b9714fd5	Get rule type right	2017-10-05 13:02:19 +01:00
David Baker	fa969cfdde	Support for channel notifications ... Add condition type to check the sender's power level and add a base rule using it for @channel notifications.	2017-10-05 12:39:18 +01:00
David Baker	0c8da8b519	Use better method for word boundary searching ... From ebc95667b8	2017-10-05 11:57:43 +01:00
Erik Johnston	eaaa837e00	Don't corrupt cache	2017-10-05 11:43:22 +01:00
David Baker	cbe3c3fdd4	pep8	2017-10-05 11:43:10 +01:00
David Baker	6748f0a579	Fix notif kws that start/end with non-word chars ... Only prepend / append word bounary characters if the search expression starts or ends with a word character, otherwise they don't work because there's no word bounary between whitespace and a non-word char.	2017-10-05 11:33:30 +01:00
David Baker	d8ce68b09b	spam check room publishing	2017-10-04 14:29:33 +01:00
David Baker	78d4ced829	un-double indent	2017-10-04 12:44:27 +01:00
David Baker	197c14dbcf	Add room creation checks to spam checker ... Lets the spam checker deny attempts to create rooms and add aliases to them.	2017-10-04 10:47:54 +01:00
David Baker	1e2ac54351	s/roomid/room_id/	2017-10-03 17:41:38 +01:00
David Baker	1e375468de	pass room id too	2017-10-03 17:13:14 +01:00
David Baker	c2c188b699	Federation was passing strings anyway ... so pass string everywhere	2017-10-03 15:46:19 +01:00
David Baker	c46a0d7eb4	this shouldn't be debug	2017-10-03 15:20:14 +01:00
David Baker	bd769a81e1	better logging	2017-10-03 15:16:40 +01:00
David Baker	537088e7dc	Actually write warpper function	2017-10-03 14:28:12 +01:00
David Baker	41fd9989a2	Skip spam check for admin users	2017-10-03 14:17:44 +01:00
Erik Johnston	11d62f43c9	Invalidate cache	2017-10-03 14:12:28 +01:00
Erik Johnston	e4ab96021e	Update comments	2017-10-03 14:10:41 +01:00
David Baker	2a7ed700d5	Fix param name & lint	2017-10-03 14:04:10 +01:00
David Baker	84716d267c	Allow spam checker to reject invites too	2017-10-03 13:56:43 +01:00
Erik Johnston	f2da6df568	Remove spurious line feed	2017-10-03 11:31:06 +01:00
Erik Johnston	30848c0fcd	Ignore incoming events for rooms that we have left ... When synapse receives an event for a room its not in over federation, it double checks with the remote server to see if it is in fact in the room. This is done so that if the server has forgotten about the room (usually as a result of the database being dropped) it can recover from it. However, in the presence of state resets in large rooms, this can cause a lot of work for servers that have legitimately left. As a hacky solution that supports both cases we drop incoming events for rooms that we have explicitly left. This means that we no longer support the case of servers having forgotten that they've rejoined a room, but that is sufficiently rare that we're not going to support it for now.	2017-10-03 11:18:21 +01:00
Erik Johnston	e585c83209	Merge branch 'master' of github.com:matrix-org/synapse into develop	2017-10-02 18:11:24 +01:00
Erik Johnston	6c1bb1601e	Bump version and changelog	2017-10-02 18:05:17 +01:00
Erik Johnston	ea87cb1ba5	Make 'affinity' package optional	2017-10-02 18:03:59 +01:00
Erik Johnston	3fed5bb25f	Move quit_with_error	2017-10-02 17:59:34 +01:00
David Baker	27955056e0	Merge branch 'develop' into erikj/groups_merged	2017-10-02 16:20:41 +01:00
Erik Johnston	90d70af269	Merge branch 'master' of github.com:matrix-org/synapse into develop	2017-10-02 16:20:23 +01:00
Erik Johnston	e4a709eda3	Bump version and change log	2017-10-02 13:51:38 +01:00
Jeremy Cline	cafb8de132	Unfreeze event before serializing with ujson ... In newer versions of https://github.com/esnme/ultrajson, ujson does not serialize frozendicts (introduced in esnme/ultrajson@53f85b1). Although the PyPI version is still 1.35, Fedora ships with a build from commit esnme/ultrajson@2f1d487. This causes the serialization to fail if the distribution-provided package is used. This runs the event through the unfreeze utility before serializing it. Thanks to @ignatenkobrain for tracking down the root cause. fixes #2351 Signed-off-by: Jeremy Cline <jeremy@jcline.org>	2017-09-30 11:22:37 -04:00
Richard van der Hoff	d5325d7ef1	Merge pull request #2480 from matrix-org/rav/federation_client_logging ... Improve logging of failures in matrixfederationclient	2017-09-29 17:32:53 +01:00
Erik Johnston	d5694ac5fa	Only log if we've removed media	2017-09-28 16:08:08 +01:00
Richard van der Hoff	e43de3ae4b	Improve logging of failures in matrixfederationclient ... * don't log exception types twice * not all exceptions have a meaningful 'message'. Use the repr rather than attempting to build a string ourselves.	2017-09-28 15:38:09 +01:00
Richard van der Hoff	75e67b9ee4	Handle SERVFAILs when doing AAAA lookups for federation (#2477) ... ... to cope with people with broken dnssec setups, mostly	2017-09-28 15:24:00 +01:00
Erik Johnston	768f00dedb	Up the limits on number of url cache entries to delete at one time	2017-09-28 14:27:27 +01:00
Erik Johnston	4dc07e93a8	Add old indices	2017-09-28 14:10:33 +01:00
Erik Johnston	7cc483aa0e	Clear up expired url cache every 10s	2017-09-28 13:56:53 +01:00
Erik Johnston	e1e7d76cf1	Actually assign result to variable	2017-09-28 13:55:29 +01:00
Erik Johnston	93247a424a	Only pull out local media that were for url cache	2017-09-28 13:48:14 +01:00
Erik Johnston	5f501ec7e2	Fix typo in url cache expiry timer	2017-09-28 12:59:01 +01:00
Erik Johnston	761d255fdf	Merge pull request #2479 from matrix-org/erikj/expire_url_cache_thumbnails ... Support new and old style media id formats	2017-09-28 12:58:13 +01:00
Erik Johnston	ace8079086	Support new and old style media id formats	2017-09-28 12:52:51 +01:00
Erik Johnston	7a44c01d89	Fix typo	2017-09-28 12:46:04 +01:00
Erik Johnston	ae79764fe5	Change expires column to expires_ts	2017-09-28 12:37:53 +01:00
Erik Johnston	77f1d24de3	More brackets	2017-09-28 12:23:15 +01:00
Erik Johnston	9ccb4226ba	Delete expired url cache data	2017-09-28 12:18:06 +01:00
Erik Johnston	8090fd4664	Fix /joined_members to work with AS users	2017-09-28 10:09:32 +01:00
Erik Johnston	adec03395d	Fix bug where /joined_members didn't check user was in room	2017-09-27 15:14:39 +01:00
David Baker	ef3a5ae787	Don't test is spam_checker not None ... Sometimes it's a Mock object which is not none but is still not what we're after	2017-09-27 11:24:19 +01:00
David Baker	8c06dd6071	Remove unintentional debugging	2017-09-27 10:31:14 +01:00
David Baker	60c78666ab	pep8	2017-09-27 10:26:13 +01:00
David Baker	1786b0e768	Forgot the new file again :(	2017-09-27 10:22:54 +01:00
David Baker	6cd5fcd536	Make the spam checker a module	2017-09-26 19:20:23 +01:00
David Baker	9fd086e506	unnecessary parens	2017-09-26 17:59:46 +01:00
David Baker	0b03a97708	Add module_loader.py	2017-09-26 17:56:41 +01:00
David Baker	4824a33c31	Factor out module loading to a separate place ... So it can be reused	2017-09-26 17:51:26 +01:00
Erik Johnston	17b8e2bd02	Add remove room API	2017-09-26 15:52:41 +01:00
Erik Johnston	a8e2a3df32	Add unique index to group_rooms table	2017-09-26 15:39:21 +01:00
Erik Johnston	95298783bb	Add is_publicised to group summary	2017-09-26 11:04:37 +01:00
Erik Johnston	1a398b19fd	Merge branch 'develop' of github.com:matrix-org/synapse into release-v0.23.0	2017-09-26 10:08:59 +01:00
Erik Johnston	f4c8cd5e85	Bump changelog and version	2017-09-26 10:02:48 +01:00
Erik Johnston	e3edca3b5d	Refactor to speed up incremental syncs	2017-09-25 17:35:39 +01:00
Erik Johnston	44edac0497	Merge branch 'release-v0.23.0' of github.com:matrix-org/synapse into develop	2017-09-25 14:52:46 +01:00
Richard van der Hoff	94133d7ce8	Merge branch 'develop' into develop	2017-09-25 11:50:11 +01:00
Erik Johnston	ba8fdc925c	Bump version and changes	2017-09-25 11:01:31 +01:00
Richard van der Hoff	79b3cf3e02	Fix logcontxt leak in keyclient (#2465) ... preserve_context_over_function doesn't do what you want it to do.	2017-09-25 09:51:39 +01:00
Richard van der Hoff	f65e31d22f	Do an AAAA lookup on SRV record targets (#2462) ... Support SRV records which point at AAAA records, as well as A records. Fixes https://github.com/matrix-org/synapse/issues/2405	2017-09-22 20:26:47 +01:00
Matthew Hodgson	f496399ac4	fix thinko'd docstring	2017-09-22 15:34:14 +01:00
Erik Johnston	3166ed55b2	Fix device list when rejoining room (#2461)	2017-09-22 14:44:17 +01:00
Erik Johnston	e1dec2f1a7	Remove user from group summary when the leave the group	2017-09-21 16:09:57 +01:00
Erik Johnston	bb746a9de1	Revert: Keep room_id's in group summary	2017-09-21 15:57:22 +01:00
Erik Johnston	ae8d4bb0f0	Keep room_id's in group summary	2017-09-21 15:55:18 +01:00
Erik Johnston	197d82dc07	Correctly return next token	2017-09-20 11:12:11 +01:00
Erik Johnston	069ae2df12	Fix initial sync	2017-09-20 10:52:12 +01:00
Richard van der Hoff	6de74ea6d7	Fix logcontexts in _check_sigs_and_hashes	2017-09-20 01:32:42 +01:00
Richard van der Hoff	c5c24c239b	Fix logcontext handling in verify_json_objects_for_server ... preserve_context_over_fn is essentially broken, because (a) it pointlessly drops the current logcontext before calling its wrapped function, which means we don't get any useful logcontexts for _handle_key_deferred; (b) it wraps the resulting deferred in a _PreservingContextDeferred, which is very dangerous because you then can't yield on it without leaking context back into the reactor. Instead, let's specify that the resultant deferreds call their callbacks with no logcontext.	2017-09-20 01:32:42 +01:00
Richard van der Hoff	c5b0e9f485	Turn _start_key_lookups into an inlineCallbacks function ... ... which means that logcontexts can be correctly preserved for the stuff it does. get_server_verify_keys is now called with the logcontext, so needs to preserve_fn when it fires off its nested inlineCallbacks function. Also renames get_server_verify_keys to reflect the fact it's meant to be private.	2017-09-20 01:32:42 +01:00
Richard van der Hoff	abdefb8a01	Fix potential race in _start_key_lookups ... If the verify_request.deferred has already completed, then `remove_deferreds` will be called immediately. It therefore might resolve the server_to_deferred deferred while there are still other requests for that server in flight. To avoid that, we should build the complete list of requests, and *then* add the callbacks.	2017-09-20 01:32:42 +01:00
Richard van der Hoff	afbd773dc6	Add some comments to _start_key_lookups	2017-09-20 01:32:42 +01:00
Richard van der Hoff	2a4b9ea233	Consistency for how verify_request.deferred is called ... Define that it is run with no log context, and make sure that happens. If we aren't careful to reset the logcontext, we can't bung the deferreds into defer.gatherResults etc. We don't actually do that directly, but we *do* resolve other deferreds from affected callbacks (notably the server_to_deferred map in _start_key_lookups), and those *do* get passed into defer.gatherResults. It turns out that this way ends up being least confusing.	2017-09-20 01:32:42 +01:00
Richard van der Hoff	3b98439eca	Factor out _start_key_lookups ... ... to make it easier to see what's going on.	2017-09-20 01:32:42 +01:00
Richard van der Hoff	fde63b880d	Replace server_and_json with verify_requests ... This is a precursor to factoring some of this code out.	2017-09-20 01:32:42 +01:00
Richard van der Hoff	2d511defd9	pull out handle_key_deferred to top level ... There's no need for this to be a nested definition; pulling it out not only makes it more efficient, but makes it easier to check that it's not accessing any local variables it shouldn't be.	2017-09-20 01:32:42 +01:00
Richard van der Hoff	dd1ea9763a	Fix incorrect key_ids in error message	2017-09-20 01:32:42 +01:00
Richard van der Hoff	e76d1135dd	Invalidate signing key cache when we gat an update ... This might make the cache slightly more efficient.	2017-09-20 01:32:42 +01:00
Richard van der Hoff	fcf2c0fd1a	Remove redundant preserve_fn ... preserve_fn is a no-op unless the wrapped function returns a Deferred. verify_json_objects_for_server returns a list, so this is doing nothing.	2017-09-20 01:32:42 +01:00
Richard van der Hoff	9864efa532	Fix concurrent server_key requests (#2458) ... Fix a bug where we could end up firing off multiple requests for server_keys for the same server at the same time.	2017-09-19 23:25:44 +01:00
Richard van der Hoff	aa620d09a0	Add a config option to block all room invites (#2457) ... - allows sysadmins the ability to lock down their servers so that people can't send their users room invites.	2017-09-19 16:08:14 +01:00
Richard van der Hoff	2eabdf3f98	add some comments to on_exchange_third_party_invite_request	2017-09-19 12:20:36 +01:00