synapse-product

mirror of https://git.anonymousland.org/anonymousland/synapse-product.git synced 2024-12-21 22:35:00 -05:00

Author	SHA1	Message	Date
Quentin Gliech	616af44137	Implement OpenID Connect-based login (#7256 )	2020-05-08 08:30:40 -04:00
Manuel Stahl	a4a5ec4096	Add room details admin endpoint (#7317 )	2020-05-07 15:33:07 -04:00
Patrick Cloke	627b0f5f27	Persist user interactive authentication sessions (#7302 ) By persisting the user interactive authentication sessions to the database, this fixes situations where a user hits different works throughout their auth session and also allows sessions to persist through restarts of Synapse.	2020-04-30 13:47:49 -04:00
Manuel Stahl	04dd7d182d	Return total number of users and profile attributes in admin users endpoint (#6881 ) Signed-off-by: Manuel Stahl <manuel.stahl@awesome-technologies.de>	2020-04-28 18:19:36 +01:00
Brendan Abolivier	2e3b9a0fcb	Revert "Revert "Merge pull request #7315 from matrix-org/babolivier/request_token"" This reverts commit `1adf6a5587`.	2020-04-23 11:23:53 +02:00
Dirk Klimpel	6b6685db9f	Extend room admin api with additional attributes (#7225 )	2020-04-22 13:38:41 +01:00
Richard van der Hoff	2aa5bf13c8	Merge branch 'release-v1.12.4' into develop	2020-04-22 13:09:23 +01:00
Michael Kaye	5308239d5d	Reduce logging verbosity of URL cache cleanup. (#7295 )	2020-04-22 07:45:16 -04:00
Richard van der Hoff	974c0d726a	Support GET account_data requests on a worker (#7311 )	2020-04-21 10:46:30 +01:00
Patrick Cloke	054c231e58	Use a template for the SSO success page to allow for customization. (#7279 )	2020-04-17 13:34:55 -04:00
Andrew Morgan	a48138784e	Allow specifying the value of Accept-Language header for URL previews (#7265 )	2020-04-15 13:35:29 +01:00
Andrew Morgan	b21000a44f	Improve error responses when a remote server doesn't allow you to access its public rooms list (#6899 )	2020-04-06 12:35:30 +01:00
Patrick Cloke	694d8bed0e	Support CAS in UI Auth flows. (#7186 )	2020-04-03 15:35:05 -04:00
Patrick Cloke	b9930d24a0	Support SAML in the user interactive authentication workflow. (#7102 )	2020-04-01 08:48:00 -04:00
Dirk Klimpel	fb69690761	Admin API to join users to a room. (#7051 )	2020-03-27 19:16:43 +00:00
Dirk Klimpel	8327eb9280	Add options to prevent users from changing their profile. (#7096 )	2020-03-27 19:15:23 +00:00
Andrew Morgan	12aa5a7fa7	Ensure is_verified on /_matrix/client/r0/room_keys/keys is a boolean (#7150 )	2020-03-27 13:30:22 +00:00
Patrick Cloke	fa4f12102d	Refactor the CAS code (move the logic out of the REST layer to a handler) (#7136 )	2020-03-26 15:05:26 -04:00
Dirk Klimpel	e8e2ddb60a	Allow server admins to define and enforce a password policy (MSC2000). (#7118 )	2020-03-26 16:51:13 +00:00
Patrick Cloke	1c1242acba	Validate that the session is not modified during UI-Auth (#7068 )	2020-03-26 07:39:34 -04:00
Naugrimm	1fcf9c6f95	Fix CAS redirect url (#6634 ) Build the same service URL when requesting the CAS ticket and when calling the proxyValidate URL.	2020-03-24 11:59:04 +00:00
Dionysis Grigoropoulos	96071eea8f	Set Referrer-Policy to no-referrer for media (#7009 )	2020-03-23 09:48:28 +00:00
Patrick Cloke	477c4f5b1c	Clean-up some auth/login REST code (#7115 )	2020-03-20 16:22:47 -04:00
Patrick Cloke	caec7d4fa0	Convert some of the media REST code to async/await (#7110 )	2020-03-20 07:20:02 -04:00
Patrick Cloke	88b41986db	Add an option to the set password API to choose whether to logout other devices. (#7085 )	2020-03-18 07:50:00 -04:00
The Stranjer	5e477c1deb	Set charset to utf-8 when adding headers for certain text content types (#7044 ) Fixes #7043	2020-03-17 13:29:09 +00:00
Patrick Cloke	60724c46b7	Remove special casing of `m.room.aliases` events (#7034 )	2020-03-17 07:37:04 -04:00
Richard van der Hoff	6a35046363	Revert "Add options to disable setting profile info for prevent changes. (#7053 )" This reverts commit `54dd28621b`, reversing changes made to `6640460d05`.	2020-03-17 11:25:01 +00:00
Brendan Abolivier	54dd28621b	Add options to disable setting profile info for prevent changes. (#7053 )	2020-03-10 22:23:01 +00:00
Brendan Abolivier	6b0efe73e2	SAML2: render a comprehensible error page if something goes wrong If an error happened while processing a SAML AuthN response, or a client ends up doing a `GET` request to `/authn_response`, then render a customisable error page rather than a confusing error.	2020-03-10 13:59:22 +00:00
dklimpel	885134529f	updates after review	2020-03-09 22:09:29 +01:00
dklimpel	99bbe177b6	add disable_3pid_changes	2020-03-08 21:58:12 +01:00
Patrick Cloke	87972f07e5	Convert remote key resource REST layer to async/await. (#7020 )	2020-03-05 11:29:56 -05:00
Brendan Abolivier	43f874055d	Merge branch 'master' into develop	2020-03-03 15:20:49 +00:00
Brendan Abolivier	65c73cdfec	Factor out complete_sso_login and expose it to the Module API	2020-03-03 10:54:44 +00:00
Richard van der Hoff	b68041df3d	Add a whitelist for the SSO confirmation step.	2020-03-02 17:05:09 +00:00
Richard van der Hoff	27d099edd6	Merge remote-tracking branch 'origin/release-v1.11.1' into release-v1.11.1	2020-03-02 16:43:33 +00:00
Brendan Abolivier	b2bd54a2e3	Add a confirmation step to the SSO login flow	2020-03-02 16:36:32 +00:00
Dirk Klimpel	bbeee33d63	Fixed set a user as an admin with the new API (#6928 ) Fix #6910	2020-03-02 13:28:50 +00:00
Dirk Klimpel	9b06d8f8a6	Fixed set a user as an admin with the new API (#6928 ) Fix #6910	2020-02-28 09:58:05 +00:00
Andrew Morgan	8c75b621bf	Ensure 'deactivated' parameter is a boolean on user admin API, Fix error handling of call to deactivate user (#6990 )	2020-02-26 12:22:55 +00:00
Patrick Cloke	509e381afa	Clarify list/set/dict/tuple comprehensions and enforce via flake8 (#6957 ) Ensure good comprehension hygiene using flake8-comprehensions.	2020-02-21 07:15:07 -05:00
Richard van der Hoff	2fb7794e60	Merge pull request #6949 from matrix-org/rav/list_room_aliases_peekable Make room alias lists peekable	2020-02-19 11:19:11 +00:00
Richard van der Hoff	880aaac1d8	Move MSC2432 stuff onto unstable prefix (#6948 ) it's not in the spec yet, so needs to be unstable. Also add a feature flag for it. Also add a test for admin users.	2020-02-19 10:40:27 +00:00
Richard van der Hoff	a0a1fd0bec	Add `allow_departed_users` param to `check_in_room_or_world_readable` ... and set it everywhere it's called. while we're here, rename it for consistency with `check_user_in_room` (and to help check that I haven't missed any instances)	2020-02-19 08:52:51 +00:00
Richard van der Hoff	adfaea8c69	Implement GET /_matrix/client/r0/rooms/{roomId}/aliases (#6939 ) per matrix-org/matrix-doc#2432	2020-02-18 16:23:25 +00:00
Aaron Raimist	dc3f998706	Remove m.lazy_load_members from unstable features since it is in CS r0.5.0 (#6877 ) Fixes #5528	2020-02-13 12:02:32 +00:00
Andrew Morgan	d8994942f2	Return a 404 for admin api user lookup if user not found (#6901 )	2020-02-12 18:14:10 +00:00
Dirk Klimpel	56ca93ef59	Admin api to add an email address (#6789 )	2020-02-07 10:29:36 +00:00
Andrew Morgan	370080531e	Allow URL-encoded user IDs on user admin api paths (#6825 )	2020-02-03 13:18:42 +00:00
Richard van der Hoff	184303b865	MSC2260: Block direct sends of m.room.aliases events (#6794 ) as per MSC2260	2020-01-30 17:20:55 +00:00
Erik Johnston	8df862e45d	Add `rooms.room_version` column (#6729 ) This is so that we don't have to rely on pulling it out from `current_state_events` table.	2020-01-27 14:30:57 +00:00
Andrew Morgan	9f7aaf90b5	Validate client_secret parameter (#6767 )	2020-01-24 14:28:40 +00:00
Richard van der Hoff	6b7462a13f	a bit of debugging for media storage providers (#6757 ) * a bit of debugging for media storage providers * changelog	2020-01-23 12:11:44 +00:00
Richard van der Hoff	5bd3cb7260	Minor fixes to user admin api (#6761 ) * don't insist on a password (this is valid if you have an SSO login) * fix reference to undefined `requester`	2020-01-23 12:03:58 +00:00
Andrew Morgan	ce84dd9e20	Remove unnecessary abstractions in admin handler (#6751 )	2020-01-22 15:09:57 +00:00
Andrew Morgan	90a28fb475	Admin API to list, filter and sort rooms (#6720 )	2020-01-22 13:36:43 +00:00
Brendan Abolivier	ed83c3a018	Fix typo in _select_thumbnail	2020-01-22 12:27:42 +00:00
Andrew Morgan	aa9b00fb2f	Fix and add test to deprecated quarantine media admin api (#6756 )	2020-01-22 11:05:50 +00:00
Erik Johnston	b0a66ab83c	Fixup synapse.rest to pass mypy (#6732 )	2020-01-20 17:38:21 +00:00
Erik Johnston	74b74462f1	Fix `/events/:event_id` deprecated API. (#6731 )	2020-01-20 17:38:09 +00:00
Erik Johnston	ceecedc68b	Fix changing password via user admin API. (#6730 )	2020-01-20 17:23:59 +00:00
Brendan Abolivier	3b72bb780a	Merge pull request #6714 from matrix-org/babolivier/retention_select_event Fix instantiation of message retention purge jobs	2020-01-17 14:23:51 +00:00
Brendan Abolivier	842c2cfbf1	Remove get_room_event_after_stream_ordering entirely	2020-01-16 20:24:17 +00:00
Neil Johnson	38e0e59f42	Add org.matrix.e2e_cross_signing to unstable_features in /versions as per MSC1756 (#6712 )	2020-01-16 09:46:14 +00:00
Andrew Morgan	edc244eec4	Remove duplicate session check in web fallback servlet (#6702 )	2020-01-15 18:05:18 +00:00
Andrew Morgan	1177d3f3a3	Quarantine media by ID or user ID (#6681 )	2020-01-13 18:10:43 +00:00
Manuel Stahl	d2906fe666	Allow admin users to create or modify users without a shared secret (#6495 ) Signed-off-by: Manuel Stahl <manuel.stahl@awesome-technologies.de>	2020-01-09 13:31:00 +00:00
Richard van der Hoff	573fee759c	Back out ill-advised notary server hackery (#6657 ) This was ill-advised. We can't modify verify_keys here, because the response object has already been signed by the requested key. Furthermore, it's somewhat unnecessary because existing versions of Synapse (which get upset that the notary key isn't present in verify_keys) will fall back to a direct fetch via `/key/v2/server`. Also: more tests for fetching keys via perspectives: it would be nice if we actually tested when our fetcher can't talk to our notary impl.	2020-01-08 13:24:10 +00:00
Richard van der Hoff	18674eebb1	Workaround for error when fetching notary's own key (#6620 ) * Kill off redundant SynapseRequestFactory We already get the Site via the Channel, so there's no need for a dedicated RequestFactory: we can just use the right constructor. * Workaround for error when fetching notary's own key As a notary server, when we return our own keys, include all of our signing keys in verify_keys. This is a workaround for #6596.	2020-01-06 12:28:58 +00:00
Richard van der Hoff	98247c4a0e	Remove unused, undocumented "content repo" resource (#6628 ) This looks like it got half-killed back in #888. Fixes #6567.	2020-01-03 17:10:52 +00:00
Richard van der Hoff	d6752ce5da	Clean up startup for the pusher (#6558 ) * Remove redundant python2 support code `str.decode()` doesn't exist on python3, so presumably this code was doing nothing * Filter out pushers with corrupt data When we get a row with unparsable json, drop the row, rather than returning a row with null `data`, which will then cause an explosion later on. * Improve logging when we can't start a pusher Log the ID to help us understand the problem * Make email pusher setup more robust We know we'll have a `data` member, since that comes from the database. What we don't know is if that is a dict, and if that has a `brand` member, and if that member is a string.	2019-12-18 14:26:58 +00:00
Erik Johnston	72acca6a32	Back out change preventing setting null avatar URLs	2019-12-11 11:46:55 +00:00
Andrew Morgan	5e8abe9013	Better errors regarding changing avatar_url (#6497 )	2019-12-09 14:54:33 +00:00
Erik Johnston	2ace775d88	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/make_database_class	2019-12-06 11:33:34 +00:00
Manuel Stahl	649b6bc088	Replace /admin/v1/users_paginate endpoint with /admin/v2/users (#5925 )	2019-12-05 18:12:23 +00:00
Erik Johnston	9c41ba4c5f	Port rest.client.v2	2019-12-05 16:56:23 +00:00
Erik Johnston	1a0997bbd5	Port rest/v1 to async/await	2019-12-05 15:57:28 +00:00
Erik Johnston	4a33a6dd19	Move background update handling out of store	2019-12-05 11:11:26 +00:00
Brendan Abolivier	9dc84b7989	Merge branch 'develop' into babolivier/context_filters	2019-12-04 14:23:44 +00:00
Filip Štědronský	81731c6e75	Fix: Pillow error when uploading RGBA image (#3325 ) (#6241 ) Signed-Off-By: Filip Štědronský <g@regnarg.cz>	2019-12-02 12:12:55 +00:00
Andrew Morgan	23ea572125	Add User-Interactive Auth to /account/3pid/add (#6119 )	2019-11-29 13:51:14 +00:00
Erik Johnston	69d8fb83c6	MSC2367 Allow reason field on all member events	2019-11-28 11:02:04 +00:00
Hubert Chathi	0d27aba900	add etag and count to key backup endpoints (#5858 )	2019-11-27 16:14:44 -05:00
Richard van der Hoff	ef1a85e773	Fix startup error when http proxy is defined. (#6421 ) Guess I only tested this on python 2 :/ Fixes #6419.	2019-11-26 18:10:50 +00:00
Andrew Morgan	3916e1b97a	Clean up newline quote marks around the codebase (#6362 )	2019-11-21 12:00:14 +00:00
Manuel Stahl	4f5ca455bf	Move admin endpoints into separate files (#6308 )	2019-11-20 11:49:11 +00:00
Brendan Abolivier	83446a18fb	Merge pull request #6335 from matrix-org/erikj/rc_login_cleanups Only do `rc_login` ratelimiting on succesful login.	2019-11-20 09:52:38 +00:00
Brendan Abolivier	271c322d08	Lint	2019-11-20 09:29:48 +00:00
Erik Johnston	c7376cdfe3	Apply suggestions from code review Co-Authored-By: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> Co-Authored-By: Brendan Abolivier <babolivier@matrix.org>	2019-11-18 17:10:16 +00:00
Erik Johnston	541f1b92d9	Only do `rc_login` ratelimiting on succesful login. We were doing this in a number of places which meant that some login code paths incremented the counter multiple times. It was also applying ratelimiting to UIA endpoints, which was probably not intentional. In particular, some custom auth modules were calling `check_user_exists`, which incremented the counters, meaning that people would fail to login sometimes.	2019-11-06 11:08:58 +00:00
Richard van der Hoff	5570d1c93f	Merge pull request #6334 from matrix-org/rav/url_preview_limit_title_2 Fix exception when OpenGraph tag values are ints	2019-11-05 17:28:11 +00:00
Richard van der Hoff	81d49cbb07	Fix exception when OpenGraph tag values are ints	2019-11-05 17:22:58 +00:00
Richard van der Hoff	55a7da247a	Merge branch 'develop' into rav/url_preview_limit_title	2019-11-05 17:08:07 +00:00
Richard van der Hoff	e78167c94b	Apply suggestions from code review Co-Authored-By: Brendan Abolivier <babolivier@matrix.org> Co-Authored-By: Erik Johnston <erik@matrix.org>	2019-11-05 16:46:39 +00:00
Richard van der Hoff	e9bfe719ba	Strip overlong OpenGraph data from url preview ... to stop people causing DoSes with malicious web pages	2019-11-05 15:51:18 +00:00
Brendan Abolivier	8822b33111	Update copyrights	2019-11-05 14:46:35 +00:00
Brendan Abolivier	f496d25877	Merge pull request #6301 from matrix-org/babolivier/msc2326 Implement MSC2326 (label based filtering)	2019-11-01 17:04:45 +00:00
Richard van der Hoff	1cb84c6486	Support for routing outbound HTTP requests via a proxy (#6239 ) The `http_proxy` and `HTTPS_PROXY` env vars can be set to a `host[:port]` value which should point to a proxy. The address of the proxy should be excluded from IP blacklists such as the `url_preview_ip_range_blacklist`. The proxy will then be used for * push * url previews * phone-home stats * recaptcha validation * CAS auth validation It will not be used for: * Application Services * Identity servers * Outbound federation * In worker configurations, connections from workers to masters Fixes #4198.	2019-11-01 14:07:44 +00:00
Brendan Abolivier	e3689ac6f7	Add unstable feature flag	2019-11-01 10:41:23 +00:00
Amber Brown	020add5099	Update black to 19.10b0 (#6304 ) * update version of black and also fix the mypy config being overridden	2019-11-01 02:43:24 +11:00
Andrew Morgan	54fef094b3	Remove usage of deprecated logger.warn method from codebase (#6271 ) Replace every instance of `logger.warn` with `logger.warning` as the former is deprecated.	2019-10-31 10:23:24 +00:00
Erik Johnston	770d1ef673	Merge pull request #6280 from matrix-org/erikj/receipts_async_await Port receipt and read markers to async/wait	2019-10-30 11:44:18 +01:00
Erik Johnston	2c35ffead2	Port receipt and read markers to async/wait	2019-10-29 15:08:22 +00:00
Erik Johnston	9be41bc121	Port room rest handlers to async/await	2019-10-29 13:09:29 +00:00
Michael Kaye	e4d98188da	Address codestyle concerns	2019-10-24 18:43:13 +01:00
Michael Kaye	8f4a808d9d	Delay printf until logging is required. Using % will cause the string to be generated even if debugging is off.	2019-10-24 18:31:53 +01:00
Hubert Chathi	36adfaedab	Merge branch 'develop' into cross-signing_sig_upload	2019-10-18 18:34:42 +01:00
Hubert Chathi	cfc28325a6	Merge branch 'develop' into uhoreg/e2e_cross-signing_merged	2019-10-18 11:39:40 +01:00
Erik Johnston	d98029ea89	Merge pull request #6196 from matrix-org/erikj/await Move rest/admin to use async/await.	2019-10-18 11:53:02 +02:00
Erik Johnston	2e97a4c197	Port synapse/rest/client/_base.py to async/await	2019-10-11 15:29:26 +01:00
Erik Johnston	3c2d6c708c	Add maybe_awaitable and fix __init__ bugs	2019-10-11 15:26:09 +01:00
Hubert Chathi	691dd67fcd	Merge pull request #6189 from matrix-org/uhoreg/e2e_backup_optional_version make version optional in body of e2e backup version update	2019-10-11 10:11:59 -04:00
Valérian Rousset	be9b55e0d2	cas: support setting display name (#6114 ) Now, the CAS server can return an attribute stating what's the desired displayname, instead of using the username directly.	2019-10-11 12:33:12 +01:00
Erik Johnston	dfbb62c28d	Port synaps/rest/client/users.py to async/await	2019-10-11 12:21:40 +01:00
Erik Johnston	f95325e22a	Port synaps/rest/client/server_notice_servlet.py to async/await	2019-10-11 12:21:40 +01:00
Erik Johnston	281f887090	Port synaps/rest/client/media.py to async/await	2019-10-11 12:21:40 +01:00
Erik Johnston	fca3a541e7	Port rest/admin/__init__.py to async/await	2019-10-11 12:17:52 +01:00
Erik Johnston	ca3e01e50d	Fix store_url_cache using bytes	2019-10-10 14:52:29 +01:00
krombel	2efd050c9d	send 404 as http-status when filter-id is unknown to the server (#2380 ) This fixed the weirdness of 400 vs 404 as http status code in the case the filter id is not known by the server. As e.g. matrix-js-sdk expects 404 to catch this situation this leads to unwanted behaviour.	2019-10-10 12:59:55 +01:00
Hubert Chathi	4535a07f4a	make version optional in body of e2e backup version update to agree with latest version of the MSC	2019-10-09 17:54:03 -04:00
Anshul Angaria	474abf1eb6	add M_TOO_LARGE error code for uploading a too large file (#6151 ) Fixes #6109	2019-10-08 13:55:16 +01:00
Michael Kaye	dc795ba709	Log responder we are using. (#6139 ) This prevents us logging "Responding to media request with responder %s".	2019-10-07 15:41:25 +01:00
Robert Swain	39b40d6d99	media/thumbnailer: Better quality for 1-bit / 8-bit color palette images (#2142 ) Pillow will use nearest neighbour as the resampling algorithm if the source image is either 1-bit or a color palette using 8 bits. If we convert to RGB before scaling, we'll probably get a better result.	2019-10-04 09:34:52 +01:00
Erik Johnston	a5166e4d5f	Land improved room list based on room stats (#6019 ) Use room_stats and room_state for room directory search	2019-10-02 14:08:35 +01:00
Erik Johnston	67ee18daea	Merge pull request #6108 from matrix-org/erikj/remove_get_user_by_req-span Add some helpful opentracing tags and remove get_user_by_req span	2019-09-27 10:02:57 +01:00
Richard van der Hoff	54569c787b	Kill off half-implemented password-reset via sms (#6101 ) Doing a password reset via SMS has never worked, and in any case is a silly idea because msisdn recycling is a thing. See also matrix-org/matrix-doc#2303.	2019-09-26 15:38:25 +01:00
Richard van der Hoff	990928abde	Stop advertising unsupported flows for registration (#6107 ) If email or msisdn verification aren't supported, let's stop advertising them for registration. Fixes #6100.	2019-09-25 12:10:26 +01:00
Erik Johnston	dc2c97e1a3	isort	2019-09-25 11:59:05 +01:00
Erik Johnston	5d99713854	Add tags for event_id and txn_id in event sending This will make it easier to search for sending event requests.	2019-09-25 11:39:15 +01:00
Richard van der Hoff	8004d6ca2f	Refactor code for calculating registration flows (#6106 ) because, frankly, it looked like it was written by an axe-murderer. This should be a non-functional change, except that where `m.login.dummy` was previously advertised before `m.login.terms`, it will now be advertised afterwards. AFAICT that should have no effect, and will be more consistent with the flows that involve passing a 3pid.	2019-09-25 11:32:05 +01:00
Richard van der Hoff	ed8b92f0d2	Merge remote-tracking branch 'origin/develop' into rav/saml_mapping_work	2019-09-24 12:57:32 +01:00
Andrew Morgan	2b071a2ff1	Add an unstable feature flag for separate add/bind 3pid APIs (#6044 ) Add a m.separate_add_and_bind flag set to True. See MSC2290's Backward Compatibility section for details.	2019-09-23 20:46:34 +01:00
Andrew Morgan	1ea3ed7620	Add m.id_access_token to /versions unstable_features (MSC2264) (#5974 ) Adds a flag to /versions' unstable_features section indicating that this Synapse understands what an id_access_token is, as per MSC2264. Fixes #5927	2019-09-23 20:19:25 +01:00
Andrew Morgan	b38aa82b83	Add m.require_identity_server to /versions unstable_flags (#5972 ) As MSC2263 states, m.require_identity_server must be set to false when it does not require an identity server to be provided by the client for the purposes of email registration or password reset. Adds an m.require_identity_server flag to /versionss unstable_flags section. This will advertise that Synapse no longer needs id_server as a parameter.	2019-09-23 18:52:43 +01:00
Andrew Morgan	2c99c63453	Add POST submit_token endpoint for MSISDN (#6078 ) First part of solving #6076	2019-09-23 17:49:00 +01:00
Andrew Morgan	30af161af2	Implement MSC2290 (#6043 ) Implements MSC2290. This PR adds two new endpoints, /unstable/account/3pid/add and /unstable/account/3pid/bind. Depending on the progress of that MSC the unstable prefix may go away. This PR also removes the blacklist on some 3PID tests which occurs in #6042, as the corresponding Sytest PR changes them to use the new endpoints. Finally, it also modifies the account deactivation code such that it doesn't just try to deactivate 3PIDs that were bound to the user's account, but any 3PIDs that were bound through the homeserver on that user's account.	2019-09-23 16:50:27 +01:00
Andrew Morgan	1b519e0272	Disable /register/available if registration is disabled (#6082 ) Fixes #6066 This register endpoint should be disabled if registration is disabled, otherwise we're giving anyone the ability to check if a username exists on a server when we don't need to be. Error code is 403 (Forbidden) as that's the same returned by /register when registration is disabled.	2019-09-23 15:38:38 +01:00
Andrew Morgan	df3401a71d	Allow HS to send emails when adding an email to the HS (#6042 )	2019-09-20 15:21:30 +01:00
Andrew Morgan	7763dd3e95	Remove trailing slash ability from password reset's submit_token endpoint (#6074 ) Remove trailing slash ability from the password reset submit_token endpoint. Since we provide the link in an email, and have never sent it with a trailing slash, there's no point for us to accept them on the endpoint.	2019-09-20 14:58:37 +01:00
Andrew Morgan	3ac614eb6c	Drop support for bind param on POST /account/3pid (MSC2290) (#6067 ) As per [MSC2290](https://github.com/matrix-org/matrix-doc/pull/2290/files#diff-05cde9463e9209b701312b3baf2fb2ebR151), we're dropping the bind parameter from `/account/3pid`. This endpoint can now only be used for adding threepid's to the user's account on the homeserver.	2019-09-20 10:46:34 +01:00
J. Ryan Stinnett	36015d68ef	Use unstable prefix for 3PID unbind API (#6062 )	2019-09-19 22:28:29 +01:00
Richard van der Hoff	b65327ff66	Merge branch 'develop' into rav/saml_mapping_work	2019-09-19 18:13:31 +01:00
Andrew Morgan	62e3ff92fd	Remove POST method from password reset submit_token endpoint (#6056 ) Removes the POST method from `/password_reset/<medium>/submit_token/` as it's only used by phone number verification which Synapse does not support yet.	2019-09-19 10:53:14 +01:00
Richard van der Hoff	a8ac40445c	Record mappings from saml users in an external table We want to assign unique mxids to saml users based on an incrementing suffix. For that to work, we need to record the allocated mxid in a separate table.	2019-09-13 16:01:46 +01:00
Andrew Morgan	9fc71dc5ee	Use the v2 Identity Service API for lookups (MSC2134 + MSC2140) (#5976 ) This is a redo of https://github.com/matrix-org/synapse/pull/5897 but with `id_access_token` accepted. Implements [MSC2134](https://github.com/matrix-org/matrix-doc/pull/2134) plus Identity Service v2 authentication ala [MSC2140](https://github.com/matrix-org/matrix-doc/pull/2140). Identity lookup-related functions were also moved from `RoomMemberHandler` to `IdentityHandler`.	2019-09-11 16:02:42 +01:00
Andrew Morgan	b5833a2abf	Add changelog	2019-09-10 17:56:10 +01:00
Andrew Morgan	60d3c57bd0	Use account_threepid_delegate for 3pid validation	2019-09-10 17:56:10 +01:00
Hubert Chathi	8e86f5b65c	Merge branch 'develop' into uhoreg/e2e_cross-signing_merged	2019-09-07 13:20:34 -04:00
Hubert Chathi	0d61d1d735	Merge branch 'develop' into cross-signing_sig_upload	2019-09-07 13:14:45 -04:00
Andrew Morgan	cf5a420c8a	Apply suggestions from code review Co-Authored-By: Erik Johnston <erik@matrix.org>	2019-09-06 13:34:42 +01:00
Andrew Morgan	ca74b140f2	Fix destructuring assumption bug	2019-09-06 13:25:06 +01:00
Andrew Morgan	0c0b82b6d1	Allow Synapse to send registration emails + choose Synapse or an external server to handle 3pid validation (#5987 ) This is a combination of a few different PRs, finally all being merged into `develop`: * #5875 * #5876 * #5868 (This one added the `/versions` flag but the flag itself was actually [backed out](`891afb57cb (diff-e591d42d30690ffb79f63bb726200891)`) in #5969. What's left is just giving /versions access to the config file, which could be useful in the future) * #5835 * #5969 * #5940 Clients should not actually use the new registration functionality until https://github.com/matrix-org/synapse/pull/5972 is merged. UPGRADE.rst, changelog entries and config file changes should all be reviewed closely before this PR is merged.	2019-09-06 11:35:28 +01:00
Andrew Morgan	a0d294c306	Switch to using v2 Identity Service APIs other than lookup (MSC 2140) (#5892 )	2019-09-05 14:31:22 +01:00
Andrew Morgan	90d17a3d28	Add POST /_matrix/client/r0/account/3pid/unbind (MSC2140) (#5980 ) Implements `POST /_matrix/client/r0/account/3pid/unbind` from [MSC2140](https://github.com/matrix-org/matrix-doc/blob/dbkr/tos_2/proposals/2140-terms-of-service-2.md#post-_matrixclientr0account3pidunbind).	2019-09-05 14:00:30 +01:00
Hubert Chathi	c8dc740a94	update with newer coding style	2019-09-04 22:30:45 -04:00
Hubert Chathi	7d6c70fc7a	make black happy	2019-09-04 22:04:12 -04:00
Hubert Chathi	ac4746ac4b	allow uploading signatures of master key signed by devices	2019-09-04 22:03:41 -04:00
Hubert Chathi	4bb4544784	implement device signature uploading/fetching	2019-09-04 20:02:56 -04:00
Hubert Chathi	faf72a4c40	Merge branch 'develop' into cross-signing_keys	2019-09-04 19:12:29 -04:00
Andrew Morgan	b736c6cd3a	Remove bind_email and bind_msisdn (#5964 ) Removes the `bind_email` and `bind_msisdn` parameters from the `/register` C/S API endpoint as per [MSC2140: Terms of Service for ISes and IMs](https://github.com/matrix-org/matrix-doc/pull/2140/files#diff-c03a26de5ac40fb532de19cb7fc2aaf7R107).	2019-09-04 18:24:23 +01:00
Travis Ralston	2f416fc997	Ensure the list media admin API is always available (#5966 ) * Ensure the list media admin API is always available This API is required for some external media repo implementations to operate (mostly for doing quarantine operations on a room). * changelog	2019-09-03 13:35:20 -06:00
Travis Ralston	0eac7077c9	Ensure an auth instance is available to ListMediaInRoom (#5967 ) * Ensure an auth instance is available to ListMediaInRoom Fixes https://github.com/matrix-org/synapse/issues/5737 * Changelog	2019-09-03 09:01:30 -06:00
Andrew Morgan	2a44782666	Remove double return statements (#5962 ) Remove all the "double return" statements which were a result of us removing all the instances of ``` defer.returnValue(...) return ``` statements when we switched to python3 fully.	2019-09-03 11:42:45 +01:00
Jorik Schellekens	a90d16dabc	Opentrace device lists (#5853 ) Trace device list changes.	2019-09-03 10:21:30 +01:00
L0ric0	ce7803b8b0	fix thumbnail storage location (#5915 ) * fix thumbnail storage location Signed-off-by: Lorenz Steinert <lorenz@steinerts.de> * Add changelog file. Signed-off-by: Lorenz Steinert <lorenz@steinerts.de> * Update Changelog Signed-off-by: Lorenz Steinert <lorenz@steinerts.de>	2019-09-02 12:18:41 +01:00
Andrew Morgan	2a012e8a04	Revert "Add m.id_access_token flag (#5930 )" (#5945 ) This reverts commit `4765f0cfd9`.	2019-08-30 17:13:37 +01:00
Andrew Morgan	4548d1f87e	Remove unnecessary parentheses around return statements (#5931 ) Python will return a tuple whether there are parentheses around the returned values or not. I'm just sick of my editor complaining about this all over the place :)	2019-08-30 16:28:26 +01:00
Andrew Morgan	4765f0cfd9	Add m.id_access_token flag (#5930 ) Adds a flag to `/versions`' `unstable_features` section indicating that this Synapse understands what an `id_access_token` is, as per https://github.com/matrix-org/synapse/issues/5927#issuecomment-523566043 Fixes #5927	2019-08-30 15:22:51 +01:00
Hubert Chathi	e3d3fbf63f	Merge branch 'uhoreg/e2e_cross-signing_merged' into cross-signing_keys	2019-08-28 17:36:46 -07:00
reivilibre	a3f0635686	Merge pull request #5914 from matrix-org/rei/admin_getadmin Add GET method to admin API /users/@user:dom/admin	2019-08-28 09:44:22 +01:00
Olivier Wilkinson (reivilibre)	c88a119259	Add GET method to admin API /users/@user:dom/admin Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>	2019-08-27 13:12:27 +01:00
Erik Johnston	f5b50d0871	Merge pull request #5895 from matrix-org/erikj/notary_key Add config option to sign remote key query responses with a separate key.	2019-08-27 11:51:37 +01:00
Richard van der Hoff	7837a5f2ea	Merge pull request #5909 from aaronraimist/public_base_url public_base_url is actually public_baseurl	2019-08-27 11:49:59 +01:00
reivilibre	1a7e6eb633	Add Admin API capability to set adminship of a user (#5878 ) Admin API: Set adminship of a user	2019-08-27 10:14:00 +01:00
Aaron Raimist	e8e3e033ee	public_base_url is actually public_baseurl Signed-off-by: Aaron Raimist <aaron@raim.ist>	2019-08-26 21:01:56 -05:00
Erik Johnston	7af5a63063	Fixup review comments	2019-08-23 15:36:28 +01:00
Richard van der Hoff	0b39fa53b6	Merge pull request #5877 from Awesome-Technologies/remove_shared_secret_registration Remove shared secret registration	2019-08-22 18:12:25 +01:00
Manuel Stahl	0bab582fd6	Remove shared secret registration from client/r0/register endpoint This type of registration was probably never used. It only includes the user name in the HMAC but not the password. Shared secret registration is still available via client/r0/admin/register. Signed-off-by: Manuel Stahl <manuel.stahl@awesome-technologies.de>	2019-08-22 18:04:08 +02:00
Brendan Abolivier	1c5b8c6222	Revert "Add "require_consent" parameter for registration" This reverts commit `3320aaab3a`.	2019-08-22 14:47:34 +01:00
Half-Shot	3320aaab3a	Add "require_consent" parameter for registration	2019-08-22 14:21:54 +01:00
Jorik Schellekens	9a6f2be572	Opentrace e2e keys (#5855 ) Add opentracing tags and logs for e2e keys	2019-08-22 11:28:12 +01:00
Richard van der Hoff	119aa31b10	Servlet to purge old rooms (#5845 )	2019-08-22 10:42:59 +01:00
Hubert Chathi	7c3abc6572	apply PR review suggestions	2019-08-21 13:19:35 -07:00
Erik Johnston	97cbc96093	Only sign when we respond to remote key requests	2019-08-21 11:21:58 +01:00
Andrew Morgan	b13cac896d	Fix up password reset template config names (#5863 ) Fixes #5833 The emailconfig code was attempting to pull incorrect config file names. This corrects that, while also marking a difference between a config file variable that's a filepath versus a str containing HTML.	2019-08-15 16:27:11 +01:00
Amber Brown	0b6fbb28a8	Don't load the media repo when configured to use an external media repo (#5754 )	2019-08-13 21:49:28 +10:00
Andrew Morgan	edeae53221	Return 404 instead of 403 when retrieving an event without perms (#5798 ) Part of fixing matrix-org/sytest#652 Sytest PR: matrix-org/sytest#667	2019-08-06 13:33:55 +01:00
Hubert Chathi	336c546d6a	Merge branch 'cross-signing_hidden' into cross-signing_keys	2019-08-01 16:31:40 -04:00
Brendan Abolivier	f25f638c35	Lint	2019-08-01 12:19:08 +02:00
Brendan Abolivier	a4a9ded4d0	Allow defining HTML templates to serve the user on account renewal	2019-08-01 11:59:27 +02:00
Hubert Chathi	c659b9f94f	allow uploading keys for cross-signing	2019-07-25 11:08:24 -04:00
Andrew Morgan	baf081cd3b	Bugfixes -------- - Fix a regression introduced in v1.2.0rc1 which led to incorrect labels on some prometheus metrics. ([\#5734](https://github.com/matrix-org/synapse/issues/5734)) -----BEGIN PGP SIGNATURE----- iQJHBAABCgAxFiEEgQG31Z317NrSMt0QiISIDS7+X/QFAl04Ur0THGFuZHJld0Bh bW9yZ2FuLnh5egAKCRCIhIgNLv5f9F4oD/0TY6S/SEd2uAmzor64ojmbX5BOwPzf j/wzUTrfvuf40EvkNPDpnejNZSvy/ysbaGQaQusv0SQKlV3xrvdn4RuMvnOWVWck kBsO+lvzOaUTR0KHDxN4y9F5eI2NdPbub4847PPVzyqSIHAd+kolxXS8kSBBhwpL yfaICWV/AOy5L7xN+JZ9IQpnegVAvUj5DmgXzDHd6VdeiHDVJuARaBgrR5uCkwVS ZoLRqZ95XV/qiguMAUvPOwyEqht2mwO64989MswP16YYm8oMkB5QA6I5nYnACsTP qk9YcN/oNvEfQXUhttku6MxK1/4yUMPUhEoDBDH7ebc0440QDtWN+IHTdA6oPVZB IuStL9YGY16m7Ltx37ZUA4URfNMiSeLHo3zKc/mCAcwxN4HyOjJewtxbG5zKQAOZ SMs8UcDwGR4zL1hnt8ZDNYtWwfzJBQIdGjoHvjXJEY7/1csTv2lmAwewFTXiqSAr 30GW5ews94kotqBK53zZT6V0F5gHNqgGHniOz1ZpqLLxYLqO3LSAGe97CrqlWUdX GkhA9tZyweknociD9fyyBmKdcFJ4mL4a+oGI5CMnSMph8UvCY8Y5XMb1T+iYEABI tA9G3mBvgkLPj+5V+8QggNkBafSigW2Q4FX7enGsDmiiskZOtfeKrAcVkapD4ooi 3I7IW5aetZr2IQ== =+JBn -----END PGP SIGNATURE----- Merge tag 'v1.2.0rc2' into develop Bugfixes -------- - Fix a regression introduced in v1.2.0rc1 which led to incorrect labels on some prometheus metrics. ([\#5734](https://github.com/matrix-org/synapse/issues/5734))	2019-07-24 13:47:51 +01:00
Jorik Schellekens	cf2972c818	Fix servlet metric names (#5734 ) * Fix servlet metric names Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> * Remove redundant check * Cover all return paths	2019-07-24 13:07:35 +01:00
Amber Brown	4806651744	Replace returnValue with return (#5736 )	2019-07-23 23:00:55 +10:00
Andrew Morgan	b2a382efdb	Remove the ability to query relations when the original event was redacted. (#5629 ) Fixes #5594 Forbid viewing relations on an event once it has been redacted.	2019-07-18 14:41:42 +01:00
Andrew Morgan	24aa0e0a5b	fix typo: backgroud -> background	2019-07-12 15:29:40 +01:00
Richard van der Hoff	0a4001eba1	Clean up exception handling for access_tokens (#5656 ) First of all, let's get rid of `TOKEN_NOT_FOUND_HTTP_STATUS`. It was a hack we did at one point when it was possible to return either a 403 or a 401 if the creds were missing. We always return a 401 in these cases now (thankfully), so it's not needed. Let's also stop abusing `AuthError` for these cases. Honestly they have nothing that relates them to the other places that `AuthError` is used, other than the fact that they are loosely under the 'Auth' banner. It makes no sense for them to share exception classes. Instead, let's add a couple of new exception classes: `InvalidClientTokenError` and `MissingClientTokenError`, for the `M_UNKNOWN_TOKEN` and `M_MISSING_TOKEN` cases respectively - and an `InvalidClientCredentialsError` base class for the two of them.	2019-07-11 11:06:23 +01:00
Andrew Morgan	f281714583	Don't bundle aggregations when retrieving the original event (#5654 ) A fix for PR #5626, which returned the original event content as part of a call to /relations. Only problem was that we were attempting to aggregate the relations on top of it when we did so. We now set bundle_aggregations to False in the get_event call. We also do this when pulling the relation events as well, because edits of edits are not something we'd like to support here.	2019-07-10 14:43:11 +01:00
Andrew Morgan	d88421ab03	Include the original event in /relations (#5626 ) When asking for the relations of an event, include the original event in the response. This will mostly be used for efficiently showing edit history, but could be useful in other circumstances.	2019-07-09 13:43:08 +01:00
Richard van der Hoff	824707383b	Remove access-token support from RegistrationHandler.register (#5641 ) Nothing uses this now, so we can remove the dead code, and clean up the API. Since we're changing the shape of the return value anyway, we take the opportunity to give the method a better name.	2019-07-08 19:01:08 +01:00
Richard van der Hoff	f9e99f9534	Factor out some redundant code in the login impl (#5639 ) * Factor out some redundant code in the login impl Also fixes a redundant access_token which was generated during jwt login. * changelog	2019-07-08 23:54:22 +10:00
Amber Brown	463b072b12	Move logging utilities out of the side drawer of util/ and into logging/ (#5606 )	2019-07-04 00:07:04 +10:00
Amber Brown	0ee9076ffe	Fix media repo breaking (#5593 )	2019-07-02 19:01:28 +01:00
Richard van der Hoff	6eecb6e500	Complete the SAML2 implementation (#5422 ) * SAML2 Improvements and redirect stuff Signed-off-by: Alexander Trost <galexrt@googlemail.com> * Code cleanups and simplifications. Also: share the saml client between redirect and response handlers. * changelog * Revert redundant changes to static js * Move all the saml stuff out to a centralised handler * Add support for tracking SAML2 sessions. This allows us to correctly handle `allow_unsolicited: False`. * update sample config * cleanups * update sample config * rename BaseSSORedirectServlet for consistency * Address review comments	2019-07-02 11:18:11 +01:00
PauRE	948488e115	Fix JWT login with new users (#5586 ) Signed-off-by: Pau Rodriguez-Estivill <prodrigestivill@gmail.com>	2019-07-02 10:25:37 +01:00
Amber Brown	b4914681a5	fix async/await consentresource (#5585 ) Fixes #5582	2019-07-01 23:33:52 +10:00
Richard van der Hoff	b4fd86a9b4	Merge branch 'develop' into rav/saml2_client	2019-07-01 14:21:03 +01:00
Amber Brown	f40a7dc41f	Make the http server handle coroutine-making REST servlets (#5475 )	2019-06-29 17:06:55 +10:00
PauRE	856ea04eb3	Fix JWT login (#5555 ) * Fix JWT login with register Signed-off-by: Pau Rodriguez-Estivill <prodrigestivill@gmail.com> * Add pyjwt conditional dependency Signed-off-by: Pau Rodriguez-Estivill <prodrigestivill@gmail.com> * Added changelog file Signed-off-by: Pau Rodriguez-Estivill <prodrigestivill@gmail.com> * Improved changelog description Signed-off-by: Pau Rodriguez-Estivill <prodrigestivill@gmail.com>	2019-06-27 12:02:41 +01:00
Richard van der Hoff	79b9d9076d	rename BaseSSORedirectServlet for consistency	2019-06-27 00:46:57 +01:00
Richard van der Hoff	3705322103	Move all the saml stuff out to a centralised handler	2019-06-26 22:52:02 +01:00
Richard van der Hoff	a4daa899ec	Merge branch 'develop' into rav/saml2_client	2019-06-26 22:34:41 +01:00
Brendan Abolivier	bfe84e051e	Split public rooms directory auth config in two	2019-06-24 15:42:31 +01:00
Amber Brown	32e7c9e7f2	Run Black. (#5482 )	2019-06-20 19:32:02 +10:00
Erik Johnston	e9344e0dee	Merge pull request #5390 from matrix-org/erikj/dont_log_on_fail_to_get_file Don't log exception when failing to fetch remote content.	2019-06-14 14:25:14 +01:00
Erik Johnston	7e68691ce9	Merge branch 'master' of github.com:matrix-org/synapse into develop	2019-06-11 17:25:16 +01:00
Erik Johnston	10383e6e6f	Change password reset links to /_matrix.	2019-06-11 11:34:33 +01:00
Andrew Morgan	2ddc13577c	Don't warn user about password reset disabling through config code (#5387 ) Moves the warning about password resets being disabled to the point where a user actually tries to reset their password. Is this an appropriate place for it to happen? Also removed the disabling of msisdn password resets when you don't have an email config, as that just doesn't make sense. Also change the error a user receives upon disabled passwords to specify that only email-based password reset is disabled.	2019-06-11 00:25:07 +01:00
Richard van der Hoff	426049247b	Code cleanups and simplifications. Also: share the saml client between redirect and response handlers.	2019-06-11 00:03:57 +01:00
Richard van der Hoff	69a43d9974	Merge remote-tracking branch 'origin/develop' into rav/saml2_client	2019-06-10 20:28:08 +01:00
Brendan Abolivier	26b62796c2	Merge pull request #5363 from matrix-org/babolivier/account_validity_send_mail_auth Don't check whether the user's account is expired on /send_mail requests	2019-06-10 11:57:02 +01:00
Erik Johnston	95d38afe96	Don't log exception when failing to fetch remote content. In particular, let's not log stack traces when we stop processing becuase the response body was too large.	2019-06-07 12:39:10 +01:00
Andrew Morgan	3719680ee4	Add ability to perform password reset via email without trusting the identity server (#5377 ) Sends password reset emails from the homeserver instead of proxying to the identity server. This is now the default behaviour for security reasons. If you wish to continue proxying password reset requests to the identity server you must now enable the email.trust_identity_server_for_password_resets option. This PR is a culmination of 3 smaller PRs which have each been separately reviewed: * #5308 * #5345 * #5368	2019-06-06 17:34:07 +01:00
Brendan Abolivier	fe13bd52ac	Don't check whether the user's account is expired on /send_mail requests	2019-06-05 16:35:05 +01:00
Neil Johnson	94f6c674df	Neilj/add r0.5 to versions (#5360 ) * Update _matrix/client/versions to reference support for r0.5.0	2019-06-05 16:11:31 +01:00
Richard van der Hoff	cb683d3e3c	Merge pull request #5333 from matrix-org/rav/server_keys/09_improve_notary_server Fixes for the key-notary server	2019-06-04 11:48:18 +01:00
Amber Brown	b2b90b7d34	Hawkowl/fix missing auth (#5328 )	2019-06-04 15:54:27 +10:00
Richard van der Hoff	c5d60eadd5	Notary server: make requests to origins in parallel ... else we're guaranteed to time out.	2019-06-04 00:16:56 +01:00
Amber Brown	2889b05554	Unify v1 and v2 REST client APIs (#5226 )	2019-06-03 21:28:59 +10:00
Alexander Trost	dc3e586938	SAML2 Improvements and redirect stuff Signed-off-by: Alexander Trost <galexrt@googlemail.com>	2019-06-02 18:14:40 +02:00
Eisha Chen-yen-su	8824325b82	Fix ignored filter field in `/messages` endpoint This fixes a bug which were causing the "event_format" field to be ignored in the filter of requests to the `/messages` endpoint of the CS API. Signed-off-by: Eisha Chen-yen-su <chenyensu0@gmail.com>	2019-05-30 16:58:53 +02:00
Erik Johnston	45f5d8f3fd	Merge pull request #5256 from aaronraimist/logout-correct-error Show correct error when logging out and access token is missing	2019-05-30 13:33:44 +01:00
Aaron Raimist	123918b739	Lint Signed-off-by: Aaron Raimist <aaron@raim.ist>	2019-05-29 14:44:28 -05:00
Amber Brown	46c8f7a517	Implement the SHHS complexity API (#5216 )	2019-05-30 01:47:16 +10:00
Amber Brown	532b825ed9	Serve CAS login over r0 (#5286 )	2019-05-30 00:55:18 +10:00
Aaron Raimist	30858ff461	Fix error when downloading thumbnail with width/height param missing (#5258 ) Fix error when downloading thumbnail with width/height param missing Fixes #2748 Signed-off-by: Aaron Raimist <aaron@raim.ist>	2019-05-29 14:27:41 +01:00
Aaron Raimist	119c9c10b0	Get rid of try except Signed-off-by: Aaron Raimist <aaron@raim.ist>	2019-05-27 00:13:56 -05:00
Aaron Raimist	56f07d980a	Show correct error when logging out and access token is missing Signed-off-by: Aaron Raimist <aaron@raim.ist>	2019-05-24 16:32:27 -05:00
Erik Johnston	d16f5574b6	Merge pull request #5220 from matrix-org/erikj/dont_bundle_live_events Don't bundle aggregations with events in /sync or /events or state queries	2019-05-24 10:36:31 +01:00
Erik Johnston	8c41c04ee4	Merge pull request #5244 from matrix-org/rav/server_keys/00-factor-out-fetchers Factor out KeyFetchers from KeyRing	2019-05-23 16:28:51 +01:00
Andrew Morgan	6368150a74	Add config option for setting homeserver's default room version (#5223 ) Replaces DEFAULT_ROOM_VERSION constant with a method that first checks the config, then returns a hardcoded value if the option is not present. That hardcoded value is now located in the server.py config file.	2019-05-23 15:00:20 +01:00
Richard van der Hoff	ec24108cc2	Fix remote_key_resource	2019-05-23 14:52:13 +01:00
Erik Johnston	de7672b78f	Don't bundle events in /sync or /events As we'll send down the annotations too anyway, so this just ends up confusing clients.	2019-05-21 13:54:09 +01:00
Erik Johnston	57ba3451b6	Merge pull request #5209 from matrix-org/erikj/reactions_base Land basic reaction and edit support.	2019-05-20 14:06:40 +01:00
PauRE	f89f688a55	Fix image orientation when generating thumbnail (#5039 )	2019-05-16 19:04:26 +01:00
David Baker	07cff7b121	Merge pull request #5174 from matrix-org/dbkr/add_dummy_flow_to_recaptcha_only Re-order registration stages to do msisdn & email auth last	2019-05-16 17:27:39 +01:00
Erik Johnston	7a7eba8302	Move parsing of tokens out of storage layer	2019-05-16 14:26:23 +01:00
Erik Johnston	95f3fcda3c	Check that event is visible in new APIs	2019-05-16 14:19:06 +01:00
David Baker	fafb936de5	Merge pull request #5187 from matrix-org/dbkr/only_check_threepid_not_in_use_if_actually_registering Only check 3pids not in use when registering	2019-05-16 10:58:09 +01:00
Erik Johnston	a0603523d2	Add aggregations API	2019-05-16 09:37:20 +01:00
Amber Brown	f1e5b41388	Make all the rate limiting options more consistent (#5181 )	2019-05-15 12:06:04 -05:00
Richard van der Hoff	5f027a315f	Drop support for v2_alpha API prefix (#5190 )	2019-05-15 17:37:46 +01:00
Erik Johnston	e6459c26b4	Actually implement idempotency	2019-05-15 17:28:33 +01:00
Erik Johnston	b50641e357	Add simple pagination API	2019-05-15 13:36:51 +01:00
Erik Johnston	efe3c7977a	Add simple send_relation API and track in DB	2019-05-15 13:36:51 +01:00
David Baker	6ca88c4693	Only check 3pids not in use when registering We checked that 3pids were not already in use before we checked if we were going to return the account previously registered in the same UI auth session, in which case the 3pids will definitely be in use. https://github.com/vector-im/riot-web/issues/9586	2019-05-14 19:04:59 +01:00
Erik Johnston	a80e6b53f9	Newsfile	2019-05-14 13:12:23 +01:00
Erik Johnston	b54b03f9e1	Allow client event serialization to be async	2019-05-14 11:58:01 +01:00
Amber Brown	df2ebd75d3	Migrate all tests to use the dict-based config format instead of hanging items off HomeserverConfig (#5171 )	2019-05-13 15:01:14 -05:00
David Baker	9e99143c47	Merge remote-tracking branch 'origin/develop' into dbkr/add_dummy_flow_to_recaptcha_only	2019-05-13 15:37:03 +01:00
Andrew Morgan	2f48c4e1ae	URL preview blacklisting fixes (#5155 ) Prevents a SynapseError being raised inside of a IResolutionReceiver and instead opts to just return 0 results. This thus means that we have to lump a failed lookup and a blacklisted lookup together with the same error message, but the substitute should be generic enough to cover both cases.	2019-05-10 10:32:44 -07:00
David Baker	04299132af	Re-order flows so that email auth is done last It's more natural for the user if the bit that takes them away from the registration flow comes last. Adding the dummy stage allows us to do the stages in this order without the ambiguity.	2019-05-10 13:58:03 +01:00
David Baker	9c61dce3c8	Comment	2019-05-10 11:14:55 +01:00
David Baker	8714ff6d51	Add a DUMMY stage to captcha-only registration flow This allows the client to complete the email last which is more natual for the user. Without this stage, if the client would complete the recaptcha (and terms, if enabled) stages and then the registration request would complete because you've now completed a flow, even if you were intending to complete the flow that's the same except has email auth at the end. Adding a dummy auth stage to the recaptcha-only flow means it's always unambiguous which flow the client was trying to complete. Longer term we should think about changing the protocol so the client explicitly says which flow it's trying to complete. vector-im/riot-web#9586	2019-05-10 11:09:53 +01:00
David Baker	c2bb7476c9	Revert `085ae346ac` Accidentally went straight to develop	2019-05-10 11:08:01 +01:00
David Baker	085ae346ac	Add a DUMMY stage to captcha-only registration flow This allows the client to complete the email last which is more natual for the user. Without this stage, if the client would complete the recaptcha (and terms, if enabled) stages and then the registration request would complete because you've now completed a flow, even if you were intending to complete the flow that's the same except has email auth at the end. Adding a dummy auth stage to the recaptcha-only flow means it's always unambiguous which flow the client was trying to complete. Longer term we should think about changing the protocol so the client explicitly says which flow it's trying to complete. https://github.com/vector-im/riot-web/issues/9586	2019-05-10 10:52:24 +01:00
Matthew Hodgson	c0e0740bef	add options to require an access_token to GET /profile and /publicRooms on CS API (#5083 ) This commit adds two config options: * `restrict_public_rooms_to_local_users` Requires auth to fetch the public rooms directory through the CS API and disables fetching it through the federation API. * `require_auth_for_profile_requests` When set to `true`, requires that requests to `/profile` over the CS API are authenticated, and only returns the user's profile if the requester shares a room with the profile's owner, as per MSC1301. MSC1301 also specifies a behaviour for federation (only returning the profile if the server asking for it shares a room with the profile's owner), but that's currently really non-trivial to do in a not too expensive way. Next step is writing down a MSC that allows a HS to specify which user sent the profile query. In this implementation, Synapse won't send a profile query over federation if it doesn't believe it already shares a room with the profile's owner, though. Groups have been intentionally omitted from this commit.	2019-05-08 18:26:56 +01:00
Richard van der Hoff	59e2d2694d	Remove the requirement to authenticate for /admin/server_version. (#5122 ) This endpoint isn't much use for its intended purpose if you first need to get yourself an admin's auth token. I've restricted it to the `/_synapse/admin` path to make it a bit easier to lock down for those concerned about exposing this information. I don't imagine anyone is using it in anger currently.	2019-05-07 09:29:30 +01:00
Richard van der Hoff	12f9d51e82	Add admin api for sending server_notices (#5121 )	2019-05-02 11:59:16 +01:00
Richard van der Hoff	0836cbb9f5	Factor out an "assert_requester_is_admin" function (#5120 ) Rather than copying-and-pasting the same four lines hundreds of times	2019-05-02 10:45:52 +01:00
Richard van der Hoff	f203c98794	fix examples	2019-05-01 17:49:56 +01:00
Richard van der Hoff	40e576e29c	Move admin api impl to its own package It doesn't really belong under rest/client/v1 any more.	2019-05-01 15:44:30 +01:00
Richard van der Hoff	8e9ca83537	Move admin API to a new prefix	2019-05-01 15:44:30 +01:00
Richard van der Hoff	579b637b6c	Move admin API away from ClientV1RestServlet	2019-05-01 15:16:04 +01:00
Michael Kaye	788163e204	Remove log error for .well-known/matrix/client (#4972 )	2019-04-24 17:44:06 +01:00
Amber Brown	6b2b9a58c4	Prevent "producer not unregistered" message (#5009 )	2019-04-24 17:37:32 +01:00
Erik Johnston	ca90336a69	Merge branch 'develop' of github.com:matrix-org/synapse into babolivier/account_expiration	2019-04-17 19:44:40 +01:00
Brendan Abolivier	eaf41a943b	Add management endpoints for account validity	2019-04-17 19:34:45 +01:00
Brendan Abolivier	91934025b9	Merge pull request #5047 from matrix-org/babolivier/account_expiration Send out emails with links to extend an account's validity period	2019-04-17 14:57:39 +01:00
Brendan Abolivier	20f0617e87	Send out emails with links to extend an account's validity period	2019-04-17 14:42:20 +01:00
Brendan Abolivier	49ff74da9b	Merge pull request #5071 from matrix-org/babolivier/3pid-check Make sure we're not registering the same 3pid twice	2019-04-17 14:37:42 +01:00
Brendan Abolivier	600ec04739	Make sure we're not registering the same 3pid twice	2019-04-17 14:23:01 +01:00
Erik Johnston	fd2fcb817c	Merge pull request #5070 from matrix-org/erikj/postpath Remove usage of request.postpath	2019-04-17 09:03:26 +01:00
Erik Johnston	ad010f6306	Remove usage of request.postpath This is an undocumented variable in twisted, and relies on the servlet being mounted in the right way. This also breaks getting push rules on workers.	2019-04-16 17:42:50 +01:00
Erik Johnston	6e27a8620f	Merge pull request #5063 from matrix-org/erikj/move_endpoints Move some rest endpoints to client reader	2019-04-15 18:55:01 +01:00
Erik Johnston	ec638a1602	Only handle GET requests for /push_rules	2019-04-15 18:51:48 +01:00
Andrew Morgan	caa76e6021	Remove periods from copyright headers (#5046 )	2019-04-11 17:08:13 +01:00
Erik Johnston	c192bf8970	Add admin API for group deletion	2019-04-03 16:29:52 +01:00
Andrew Morgan	4a4d5c4fd6	Fix grammar and document get_current_users_in_room (#4998 )	2019-04-03 14:32:20 +01:00
Neil Johnson	e8419554ff	Remove presence lists (#4989 ) Remove presence list support as per MSC 1819	2019-04-03 11:11:15 +01:00
Erik Johnston	8f549c1177	Merge pull request #4982 from matrix-org/erikj/msc1915 Implement MSC1915 - 3PID unbind APIs	2019-04-03 11:07:09 +01:00
Erik Johnston	862d6e5ba5	Add unbind API to /r0 as it is now stabalised	2019-04-01 15:25:19 +01:00
Erik Johnston	057715aaa2	Allowing specifying IS to use in unbind API. By default the homeserver will use the identity server used during the binding of the 3PID to unbind the 3PID. However, we need to allow clients to explicitly ask the homeserver to unbind via a particular identity server, for the case where the 3PID was bound out of band from the homeserver. Implements MSC915.	2019-04-01 15:25:18 +01:00
Richard van der Hoff	54a87a7b08	Collect room-version variations into one place (#4969 ) Collect all the things that make room-versions different to one another into one place, so that it's easier to define new room versions.	2019-04-01 10:24:38 +01:00
Neil Johnson	c7296bcb98	remove log line for password (#4965 ) Remove log line for password.	2019-03-28 17:38:01 +00:00
Andrew Morgan	bbd244c7b2	Support 3PID login in password providers (#4931 ) Adds a new method, check_3pid_auth, which gives password providers the chance to allow authentication with third-party identifiers such as email or msisdn.	2019-03-26 17:48:30 +00:00
Erik Johnston	3959858eaa	Merge pull request #4904 from matrix-org/erikj/fix_shutdown Fixup shutdown room API	2019-03-21 11:24:42 +00:00
Erik Johnston	aa959a6c07	Use flags	2019-03-20 17:40:29 +00:00
Erik Johnston	72a14860ab	Gracefully handle failing to kick user	2019-03-20 16:54:21 +00:00
Erik Johnston	6b28890543	Log new room ID	2019-03-20 16:52:28 +00:00
Erik Johnston	320667a479	Add option to disable searching in the user dir We still populate it, as it can still be accessed via the admin API.	2019-03-19 16:40:19 +00:00
Brendan Abolivier	899e523d6d	Add ratelimiting on login (#4821 ) Add two ratelimiters on login (per-IP address and per-userID).	2019-03-15 17:46:16 +00:00
Matthew Hodgson	2326e00bc4	fix incorrect encoding of filenames with spaces in (#2090 ) fixes https://github.com/vector-im/riot-web/issues/3155	2019-03-11 09:53:45 +00:00
Erik Johnston	b879870b2d	Send message after room has been shutdown Currently the explanation message is sent to the abuse room before any users are forced joined, which means it tends to get lost in the backlog of joins. So instead we send the message after we've forced joined everyone.	2019-03-06 17:35:11 +00:00
Brendan Abolivier	6f3cde8b25	Make registration ratelimiter separate from the main events one	2019-03-06 11:02:42 +00:00
Brendan Abolivier	f4195f4118	Revert "Split ratelimiters in two (one for events, one for registration)" This reverts commit `d7dbad3526`.	2019-03-06 10:55:22 +00:00
Brendan Abolivier	d7dbad3526	Split ratelimiters in two (one for events, one for registration)	2019-03-05 18:41:27 +00:00
Erik Johnston	16c8b4ecbd	Merge pull request #4772 from jbweston/jbweston/server-version-api Add 'server_version' endpoint to admin API	2019-03-05 16:31:00 +00:00
Brendan Abolivier	a4c3a361b7	Add rate-limiting on registration (#4735 ) * Rate-limiting for registration * Add unit test for registration rate limiting * Add config parameters for rate limiting on auth endpoints * Doc * Fix doc of rate limiting function Co-Authored-By: babolivier <contact@brendanabolivier.com> * Incorporate review * Fix config parsing * Fix linting errors * Set default config for auth rate limiting * Fix tests * Add changelog * Advance reactor instead of mocked clock * Move parameters to registration specific config and give them more sensible default values * Remove unused config options * Don't mock the rate limiter un MAU tests * Rename _register_with_store into register_with_store * Make CI happy * Remove unused import * Update sample config * Fix ratelimiting test for py2 * Add non-guest test	2019-03-05 14:25:33 +00:00
Joseph Weston	1e8388b311	Add 'server_version' endpoint to admin API This is required because the 'Server' HTTP header is not always passed through proxies.	2019-03-01 09:56:58 +01:00
Richard van der Hoff	68f47d6744	Fix parsing of Content-Disposition headers (#4763 ) * Fix parsing of Content-Disposition headers TIL: filenames in content-dispostion headers can contain semicolons, and aren't %-encoded. * fix python2 incompatibility * Fix docstrings	2019-02-27 14:29:10 -08:00
Erik Johnston	899a119c2b	Don't log stack trace when client has gone away during media download (#4738 ) * Don't log stack trace when client has gone away during media download * Newsfile * Fixup newsfile	2019-02-25 11:17:22 -08:00
Matthew Hodgson	70ea2f4e1d	switch from google.com to recaptcha.net for reCAPTCHA (#4731 ) * add trivial clarification about jemalloc * switch from google.com to recaptcha.net because https://developers.google.com/recaptcha/docs/faq#can-i-use-recaptcha-globally	2019-02-25 11:15:36 -08:00
Erik Johnston	dbdc565dfd	Fix registration on workers (#4682 ) * Move RegistrationHandler init to HomeServer * Move post registration actions to RegistrationHandler * Add post regisration replication endpoint * Newsfile	2019-02-20 18:47:31 +11:00
Erik Johnston	fc2c245a1f	Merge pull request #4666 from matrix-org/erikj/register_login_split Split out registration to worker	2019-02-18 17:18:06 +00:00
Erik Johnston	af691e415c	Move register_device into handler	2019-02-18 16:49:38 +00:00
Erik Johnston	092b541401	Fixup	2019-02-18 15:52:26 +00:00
Erik Johnston	8b9ae6d3a6	Update docs	2019-02-18 15:26:13 +00:00
Matthew Hodgson	04dad5ac16	fix missig import	2019-02-18 15:59:23 +01:00
Erik Johnston	eb2b8523ae	Split out registration to worker This allows registration to be handled by a worker, though the actual write to the database still happens on master. Note: due to the in-memory session map all registration requests must be handled by the same worker.	2019-02-18 12:12:57 +00:00
Matthew Hodgson	bfcefbb230	tabs	2019-02-14 19:53:12 +00:00
Matthew Hodgson	6f47bc3fb2	set CORS on .well-known URI to unbreak modular otherwise a riot/web running on foo.riot.im can't query the .well-known on foo.modular.im...	2019-02-14 19:51:41 +00:00
Erik Johnston	719e073f00	Merge pull request #4580 from matrix-org/uhoreg/e2e_backup_add_updating add updating of backup versions	2019-02-11 13:45:49 +00:00
Valentin Anger	2dc2b6e9f1	Allow "unavailable" presence status for /sync (#4592 ) * Allow "unavailable" presence status for /sync Closes #3772, closes #3779 Signed-off-by: Valentin Anger <valentin.an.1999@gmail.com> * Add changelog for PR 4592	2019-02-08 21:09:56 +00:00
Hubert Chathi	afae8442b5	make sure version is in body and wrap in linearizer queue also add tests	2019-02-08 01:32:45 -05:00
Hubert Chathi	8248637173	add new endpoint to update backup versions	2019-02-06 17:57:10 -05:00
Neil Johnson	b37e8c9572	Merge branch 'neilj/room_capabilities' of github.com:matrix-org/synapse into neilj/room_capabilities	2019-01-30 10:56:47 +00:00
Neil Johnson	f834d98402	isort	2019-01-30 10:55:42 +00:00
Neil Johnson	ee4df7fd7a	Merge branch 'develop' into neilj/room_capabilities	2019-01-30 10:28:08 +00:00
Neil Johnson	9c850d9d5e	formatting and use constants where available	2019-01-30 10:23:26 +00:00
Neil Johnson	2f46804055	Populate default room version from Constants	2019-01-30 09:39:10 +00:00
Neil Johnson	c7837dce24	reflect that rooms v3 is a stable room version	2019-01-30 09:33:30 +00:00
Neil Johnson	e4bef9d470	rework format of change password capability	2019-01-29 18:04:56 +00:00
Neil Johnson	f03b3a7a3a	support change_password in capabilities end-point	2019-01-29 15:58:37 +00:00
Andrew Morgan	e65a17b26f	Check consent dir path on startup	2019-01-29 15:30:33 +00:00
Neil Johnson	a124025dab	enforce auth for capabilities endpoint	2019-01-29 11:37:56 +00:00
Neil Johnson	327b992e17	register capabilities servlet	2019-01-29 10:28:35 +00:00
Neil Johnson	893107be78	backout v3	2019-01-25 17:27:36 +00:00
Neil Johnson	95f871fc0d	Support room version capabilities in CS API (MSC1804)	2019-01-25 11:16:29 +00:00
Andrew Morgan	821b65aeb5	Merge branch 'develop' of github.com:matrix-org/synapse into anoa/dm_room_upgrade	2019-01-25 11:09:53 +00:00
Neil Johnson	c99c2d58d7	move guard out of is_threepid_reserved and into register.py	2019-01-22 17:47:00 +00:00
Neil Johnson	d619b113ed	Fix None guard in config.server.is_threepid_reserved	2019-01-22 16:52:29 +00:00
Andrew Morgan	25d64a846a	Fix typos	2019-01-22 11:00:04 +00:00
Neil Johnson	5349262302	Config option to disable requesting MSISDN on registration	2019-01-21 14:59:37 +00:00
Erik Johnston	b970cb0e96	Refactor request sending to have better excpetions (#4358 ) * Correctly retry and back off if we get a HTTPerror response * Refactor request sending to have better excpetions MatrixFederationHttpClient blindly reraised exceptions to the caller without differentiating "expected" failures (e.g. connection timeouts etc) versus more severe problems (e.g. programming errors). This commit adds a RequestSendFailed exception that is raised when "expected" failures happen, allowing the TransactionQueue to log them as warnings while allowing us to log other exceptions as actual exceptions.	2019-01-08 11:04:28 +00:00
Travis Ralston	bc1fa8cd01	Add GET account data routes (#4303 ) As per https://github.com/matrix-org/matrix-doc/issues/1339	2019-01-07 10:27:54 +00:00
Amber Brown	d7843f47b6	Remove v1 only REST APIs now we don't ship matrix console (#4334 )	2018-12-29 23:12:30 +11:00
Richard van der Hoff	f85676cc93	Return well_known in /login response (#4319 ) ... as per MSC1730.	2018-12-24 10:44:33 +01:00
Amber Brown	ea6abf6724	Fix IP URL previews on Python 3 (#4215 )	2018-12-22 01:56:13 +11:00
Neil Johnson	d2f7c4e6b1	create support user (#4141 ) Allow for the creation of a support user. A support user can access the server, join rooms, interact with other users, but does not appear in the user directory nor does it contribute to monthly active user limits.	2018-12-14 18:20:59 +00:00
David Baker	89ac2a5bdb	Add 'sandbox' to CSP for media repo (#4284 ) * Add 'sandbox' to the CSP for media repo * Changelog	2018-12-11 04:05:02 +11:00
Will Hunt	fee831c040	Move imports to one line	2018-12-10 13:52:33 +00:00
Will Hunt	466c1f3e01	Use `send_cors`	2018-12-10 13:11:37 +00:00
Will Hunt	91206e09f2	changelog & isort	2018-12-09 17:39:44 +00:00
Will Hunt	dbf736ba66	Make /config more CORS-y	2018-12-09 13:27:22 +00:00
Richard van der Hoff	30da50a5b8	Initialise user displayname from SAML2 data (#4272 ) When we register a new user from SAML2 data, initialise their displayname correctly.	2018-12-07 14:44:46 +01:00
Richard van der Hoff	c7401a697f	Implement SAML2 authentication (#4267 ) This implements both a SAML2 metadata endpoint (at `/_matrix/saml2/metadata.xml`), and a SAML2 response receiver (at `/_matrix/saml2/authn_response`). If the SAML2 response matches what's been configured, we complete the SSO login flow by redirecting to the client url (aka `RelayState` in SAML2 jargon) with a login token. What we don't yet have is anything to build a SAML2 request and redirect the user to the identity provider. That is left as an exercise for the reader.	2018-12-07 13:11:11 +01:00
Richard van der Hoff	c588b9b9e4	Factor SSO success handling out of CAS login (#4264 ) This is mostly factoring out the post-CAS-login code to somewhere we can reuse it for other SSO flows, but it also fixes the userid mapping while we're at it.	2018-12-07 13:10:07 +01:00
Richard van der Hoff	b0c24a66ec	Rip out half-implemented m.login.saml2 support (#4265 ) * Rip out half-implemented m.login.saml2 support This was implemented in an odd way that left most of the work to the client, in a way that I really didn't understand. It's going to be a pain to maintain, so let's start by ripping it out. * drop undocumented dependency on dateutil It turns out we were relying on dateutil being pulled in transitively by pysaml2. There's no need for that bloat.	2018-12-06 19:44:38 +11:00
Richard van der Hoff	e8d98466b0	Implement .well-known handling (#4262 ) Sometimes it's useful for synapse to generate its own .well-known file.	2018-12-05 14:38:58 +01:00
Amber Brown	fd96dd75a3	Fix non-ASCII pushrules (#4248 )	2018-12-04 12:44:02 +01:00
Amber Brown	d3c61ef906	fix type error	2018-12-03 22:27:41 +11:00
Neil Johnson	7039ece8fb	Neilj/fix autojoin (#4223 ) * Fix auto join failures for servers that require user consent * Fix auto join failures for servers that require user consent	2018-11-28 22:24:57 +11:00
Richard van der Hoff	944d524f18	Support m.login.sso (#4220 ) * Clean up the CSS for the fallback login form I was finding this hard to work with, so simplify a bunch of things. Each flow is now a form inside a div of class login_flow. The login_flow class now has a fixed width, as that looks much better than each flow having a differnt width. * Support m.login.sso MSC1721 renames m.login.cas to m.login.sso. This implements the change (retaining support for m.login.cas for older clients). * changelog	2018-11-27 18:51:52 +11:00
Amber Brown	80cac86b2c	Fix fallback auth on Python 3 (#4197 )	2018-11-19 12:27:33 -06:00
Travis Ralston	ab4526a153	Remove duplicate slashes in generated consent URLs	2018-11-15 20:41:53 -07:00
Amber Brown	8b1affe7d5	Fix Content-Disposition in media repository (#4176 )	2018-11-15 15:55:58 -06:00
Amber Brown	df758e155d	Use <meta> tags to discover the per-page encoding of html previews (#4183 )	2018-11-15 11:05:08 -06:00
Richard van der Hoff	c1efcd7c6a	Add a test for the public T&Cs form	2018-11-14 10:46:27 +00:00
Richard van der Hoff	83a5f459aa	Fix an internal server error when viewing the public privacy policy	2018-11-14 10:21:07 +00:00
hera	2b075fb03a	Fix encoding error for consent form on python3 The form was rendering this as "b'01234....'". -- richvdh	2018-11-08 11:05:39 +00:00
Amber Brown	b3708830b8	Fix URL preview bugs (type error when loading cache from db, content-type including quotes) (#4157 )	2018-11-08 01:37:43 +11:00
Travis Ralston	0f5e51f726	Add config variables for enabling terms auth and the policy name (#4142 ) So people can still collect consent the old way if they want to.	2018-11-06 10:32:34 +00:00
Hubert Chathi	f1087106cf	handle empty backups according to latest spec proposal (#4123 ) fixes #4056	2018-11-05 17:59:29 -05:00
Amber Brown	efdcbbe46b	Tests for user consent resource (#4140 )	2018-11-06 05:53:44 +11:00
Travis Ralston	642505abc3	Fix logic error that prevented guests from seeing the privacy policy	2018-11-01 16:48:32 -06:00
Travis Ralston	a8d41c6aff	Include a version query string arg for the consent route	2018-10-31 13:19:28 -06:00
Travis Ralston	d1e7b9c44c	Merge branch 'develop' into travis/login-terms	2018-10-31 13:15:14 -06:00
Amber Brown	f79f454485	Remove deprecated v1 key exchange endpoint (#4119 )	2018-10-31 22:29:02 +11:00
Amber Brown	4cd1c9f2ff	Delete the disused & unspecced identicon functionality (#4106 )	2018-10-29 23:57:24 +11:00
Richard van der Hoff	193cadc988	Address review comments Improve comments, get old room state from the context we already have	2018-10-26 17:10:30 +01:00
Richard van der Hoff	0f7d1c9906	Basic initial support for room upgrades Currently just creates a new, empty, room, and sends a tombstone in the old room.	2018-10-25 19:10:24 +01:00
Erik Johnston	cb53ce9d64	Refactor state group lookup to reduce DB hits (#4011 ) Currently when fetching state groups from the data store we make two hits two the database: once for members and once for non-members (unless request is filtered to one or the other). This adds needless load to the datbase, so this PR refactors the lookup to make only a single database hit.	2018-10-25 17:49:55 +01:00
Travis Ralston	54def42c19	Merge branch 'develop' into travis/login-terms	2018-10-24 13:22:59 -06:00
Richard van der Hoff	ef771cc4c2	Fix a number of flake8 errors Broadly three things here: * disable W504 which seems a bit whacko * remove a bunch of `as e` expressions from exception handlers that don't use them * use `r""` for strings which include backslashes Also, we don't use pep8 any more, so we can get rid of the duplicate config there.	2018-10-24 10:39:03 +01:00
Erik Johnston	cb23aa4c42	Merge pull request #4063 from matrix-org/erikj/cleaup_alias_creation Clean up room alias creation	2018-10-24 09:40:27 +01:00
Richard van der Hoff	5c445114d3	Correctly account for cpu usage by background threads (#4074 ) Wrap calls to deferToThread() in a thing which uses a child logcontext to attribute CPU usage to the right request. While we're in the area, remove the logcontext_tracer stuff, which is never used, and afaik doesn't work. Fixes #4064	2018-10-23 13:12:32 +01:00
Erik Johnston	74e7617083	Clean up room alias creation	2018-10-19 10:11:56 +01:00
Travis Ralston	49a044aa5f	Merge branch 'develop' into travis/login-terms	2018-10-18 09:57:58 -06:00
Erik Johnston	f6a0a02a62	Fix bug where we raised StopIteration in a generator This made python 3.7 unhappy	2018-10-17 16:10:52 +01:00
Travis Ralston	a8ed93a4b5	pep8	2018-10-15 16:10:29 -06:00
Travis Ralston	f293d124b6	Merge branch 'develop' into travis/login-terms	2018-10-15 14:44:32 -06:00
Travis Ralston	5119818e9d	Rely on the lack of ?u to represent public access also general cleanup	2018-10-12 18:03:17 -06:00
David Baker	a45f2c3a00	missed one	2018-10-12 14:33:55 +01:00
David Baker	bddfad253a	Don't mangle exceptions	2018-10-12 11:48:02 +01:00
David Baker	86ef9760a7	Split /room_keys/version into 2 servlets	2018-10-12 11:35:08 +01:00
David Baker	83e72bb2f0	PR feedback pt. 1	2018-10-12 11:26:18 +01:00
David Baker	d3464ce708	isort	2018-10-09 10:33:59 +01:00
David Baker	dc045ef202	Merge remote-tracking branch 'origin/develop' into dbkr/e2e_backups	2018-10-09 10:05:02 +01:00
Travis Ralston	537d0b7b36	Use a flag rather than a new route for the public policy This also means that the template now has optional parameters, which will need to be documented somehow.	2018-10-03 17:50:11 -06:00
Travis Ralston	f9d34a763c	Auto-consent to the privacy policy if the user registered with terms	2018-10-03 17:39:45 -06:00
Travis Ralston	dfcad5fad5	Make the terms flow requried	2018-10-03 17:39:00 -06:00
Travis Ralston	3099d96dba	Flesh out the fallback auth for terms	2018-10-03 17:39:00 -06:00
Travis Ralston	fd99787162	Incorporate Dave's work for GDPR login flows As per https://github.com/vector-im/riot-web/issues/7168#issuecomment-419996117	2018-10-03 15:57:42 -06:00
Richard van der Hoff	4c3e7eeec5	Merge pull request #3932 from matrix-org/erikj/auto_start_expiring_caches Fix some instances of ExpiringCache not expiring cache items	2018-09-25 12:02:57 +01:00
Jérémy Farnaud	6cf261930a	added "media-src: 'self'" to CSP for resources (#3578 ) Synapse doesn’t allow for media resources to be played directly from Chrome. It is a problem for users on other networks (e.g. IRC) communicating with Matrix users through a gateway. The gateway sends them the raw URL for the resource when a Matrix user uploads a video and the video cannot be played directly in Chrome using that URL. Chrome argues it is not authorized to play the video because of the Content Security Policy. Chrome checks for the "media-src" policy which is missing, and defauts to the "default-src" policy which is "none". As Synapse already sends "object-src: 'self'" I thought it wouldn’t be a problem to add "media-src: 'self'" to the CSP to fix this problem.	2018-09-25 11:55:02 +01:00
Erik Johnston	8601c24287	Fix some instances of ExpiringCache not expiring cache items ExpiringCache required that `start()` be called before it would actually start expiring entries. A number of places didn't do that. This PR removes `start` from ExpiringCache, and automatically starts backround reaping process on creation instead.	2018-09-21 14:19:46 +01:00
David Baker	bc74925c5b	WIP e2e key backups Continues from uhoreg's branch This just fixed the errcode on /room_keys/version if no backup and updates the schema delta to be on the latest so it gets run	2018-09-13 17:02:59 +01:00
Amber Brown	546aee7e52	Merge pull request #3835 from krombel/fix_3821 fix VOIP crashes under Python 3	2018-09-12 20:44:18 +10:00
Amber Brown	02aa41809b	Port rest/ to Python 3 (#3823 )	2018-09-12 20:41:31 +10:00
Krombel	7bc22539ff	fix VOIP crashes under Python 3 (#3821 )	2018-09-10 14:30:08 +02:00
Hubert Chathi	3801b8aa03	try to make flake8 and isort happy	2018-09-06 11:35:19 -04:00
Richard van der Hoff	dfba1d843d	Merge pull request #3790 from matrix-org/rav/respect_event_format_in_filter Implement 'event_format' filter param in /sync	2018-09-05 16:24:14 +01:00
Richard van der Hoff	87c18d12ee	Implement 'event_format' filter param in /sync This has been specced and part-implemented; let's implement it for /sync (but no other endpoints yet :/).	2018-09-04 15:20:09 +01:00
Neil Johnson	bae37cd811	improve human readable error message	2018-09-04 12:07:00 +01:00
Neil Johnson	c42f7fd7b9	improve human readable error messages	2018-09-04 12:03:17 +01:00
Neil Johnson	0b01281e77	move threepid checker to config, add missing yields	2018-08-31 17:11:11 +01:00
Neil Johnson	09f3cf1a7e	ensure post registration auth checks do not fail erroneously	2018-08-31 15:42:51 +01:00
Neil Johnson	ea068d6f3c	fix bug where preserved threepid user comes to sign up and server is mau blocked	2018-08-31 10:49:14 +01:00
Hubert Chathi	83caead95a	Merge branch 'develop' into e2e_backups	2018-08-24 11:44:26 -04:00
Erik Johnston	db10f553ba	Merge pull request #3724 from Half-Shot/hs/guest-fetch-event Allow guests to use /rooms/:roomId/event/:eventId	2018-08-22 13:41:08 +01:00
Richard van der Hoff	f7bf181a90	fix another consent encoding fail	2018-08-21 23:14:25 +01:00
Richard van der Hoff	f7baff6f7b	Fix 500 error from /consent form Fixes #3731	2018-08-21 22:47:07 +01:00
Will Hunt	f89f6b7c09	Allow guests to access /rooms/:roomId/event/:eventId	2018-08-20 18:25:54 +01:00
Amber Brown	324525f40c	Port over enough to get some sytests running on Python 3 (#3668 )	2018-08-20 23:54:49 +10:00
Amber Brown	c334ca67bb	Integrate presence from hotfixes (#3694 )	2018-08-18 01:08:45 +10:00
Amber Brown	04f5d2db62	Remove v1/register's broken shared secret functionality (#3703 )	2018-08-18 00:55:01 +10:00
Will Hunt	c151b32b1d	Add GET media/v1/config (#3184 )	2018-08-16 14:23:38 +01:00
Matthew Hodgson	762a758fea	lazyload aware /messages (#3589 )	2018-08-16 14:22:47 +01:00
Matthew Hodgson	3f543dc021	initial cut at a room summary API (#3574 )	2018-08-16 09:46:50 +01:00
Matthew Hodgson	2f78f432c4	speed up /members and add at= and membership params (#3568 )	2018-08-15 16:35:22 +01:00
Erik Johnston	fef2e65d12	Merge pull request #3667 from matrix-org/erikj/fixup_unbind Don't fail requests to unbind 3pids for non supporting ID servers	2018-08-15 10:32:12 +01:00
Matthew Hodgson	4f7064f6b5	missing import	2018-08-12 19:14:31 -04:00
Matthew Hodgson	54ac18e832	use parse_string	2018-08-12 19:14:31 -04:00
Matthew Hodgson	72788cf9c1	support DELETE /version with no args	2018-08-12 19:14:31 -04:00
Matthew Hodgson	fe87890b18	implement remaining tests and make them work	2018-08-12 19:14:31 -04:00
Matthew Hodgson	93d174bcc4	improve docstring	2018-08-12 19:14:31 -04:00
Matthew Hodgson	14b3da63a3	add a tonne of docstring; make upload_room_keys properly assert version	2018-08-12 19:14:31 -04:00
Matthew Hodgson	9f500cb39e	more docstring for the e2e_room_keys rest	2018-08-12 19:14:31 -04:00
Matthew Hodgson	0abb205b47	blindly incorporate PR review - needs testing & fixing	2018-08-12 19:14:31 -04:00
Matthew Hodgson	69e51c7ba4	make /room_keys/version work	2018-08-12 19:14:31 -04:00
Matthew Hodgson	8ae64b270f	implement /room_keys/version too (untested)	2018-08-12 19:14:31 -04:00
Matthew Hodgson	cf1e2000f6	document the API	2018-08-12 19:13:09 -04:00
Matthew Hodgson	6b8c07abc2	make it work and fix pep8	2018-08-12 19:13:09 -04:00
Matthew Hodgson	0bc4627a73	interim WIP checkin; doesn't build yet	2018-08-12 18:23:10 -04:00
Matthew Hodgson	53ace904b2	total WIP skeleton for /room_keys API	2018-08-12 18:23:10 -04:00
Amber Brown	b37c472419	Rename async to async_helpers because `async` is a keyword on Python 3.7 (#3678 )	2018-08-10 23:50:21 +10:00
Erik Johnston	360ba89c50	Don't fail requests to unbind 3pids for non supporting ID servers Older identity servers may not support the unbind 3pid request, so we shouldn't fail the requests if we received one of 400/404/501. The request still fails if we receive e.g. 500 responses, allowing clients to retry requests on transient identity server errors that otherwise do support the API. Fixes #3661	2018-08-08 12:06:18 +01:00
Richard van der Hoff	1fa98495d0	Merge pull request #3639 from matrix-org/rav/refactor_error_handling Clean up handling of errors from outbound requests	2018-08-02 17:38:24 +01:00
Richard van der Hoff	704c3e6239	Merge branch 'master' into develop	2018-08-02 15:43:30 +01:00
Richard van der Hoff	0bf5ec0db7	Check room visibility for /event/ requests Make sure that the user has permission to view the requeseted event for /event/{eventId} and /room/{roomId}/event/{eventId} requests. Also check that the event is in the given room for /room/{roomId}/event/{eventId}, for sanity.	2018-08-02 15:03:27 +01:00
Richard van der Hoff	b8d7d3996b	Merge pull request #3620 from fuzzmz/return-404-room-not-found return 404 if room not found	2018-08-01 16:34:32 +01:00
Richard van der Hoff	018d75a148	Refactor code for turning HttpResponseException into SynapseError This commit replaces SynapseError.from_http_response_exception with HttpResponseException.to_synapse_error. The new method actually returns a ProxiedRequestError, which allows us to pass through additional metadata from the API call.	2018-08-01 16:02:46 +01:00
Amber Brown	da7785147d	Python 3: Convert some unicode/bytes uses (#3569 )	2018-08-02 00:54:06 +10:00
Serban Constantin	70af98e361	return NotFoundError if room not found Per the Client-Server API[0] we should return `M_NOT_FOUND` if the room isn't found instead of generic SynapseError. This ensures that /directory/list API returns 404 for room not found instead of 400. [0]: https://matrix.org/docs/spec/client_server/unstable.html#get-matrix-client-r0-directory-list-room-roomid Signed-off-by: Serban Constantin <serban.constantin@gmail.com>	2018-07-31 21:47:23 +03:00
Matthew Hodgson	e9b2d047f6	make /context lazyload & filter aware (#3567 ) make /context lazyload & filter aware.	2018-07-27 15:12:50 +01:00
Richard van der Hoff	03751a6420	Fix some looping_call calls which were broken in #3604 It turns out that looping_call does check the deferred returned by its callback, and (at least in the case of client_ips), we were relying on this, and I broke it in #3604. Update run_as_background_process to return the deferred, and make sure we return it to clock.looping_call.	2018-07-26 11:48:08 +01:00
Richard van der Hoff	371da42ae4	Wrap a number of things that run in the background This will reduce the number of "Starting db connection from sentinel context" warnings, and will help with our metrics.	2018-07-25 09:41:12 +01:00
Erik Johnston	0b0b24cb82	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/client_apis_move	2018-07-23 13:21:15 +01:00
Amber Brown	e1a237eaab	Admin API for creating new users (#3415 )	2018-07-20 22:41:13 +10:00
Amber Brown	a97c845271	Move v1-only APIs into their own module & isolate deprecated ones (#3460 )	2018-07-19 20:03:33 +10:00
Erik Johnston	bacdf0cbf9	Move RoomContextHandler out of Handlers This is in preparation for moving GET /context/ to a worker	2018-07-18 15:33:03 +01:00
Erik Johnston	8cb8df55e9	Split MessageHandler into read only and writers This will let us call the read only parts from workers, and so be able to move some APIs off of master, e.g. the `/state` API.	2018-07-18 15:33:03 +01:00
Amber Brown	bc006b3c9d	Refactor REST API tests to use explicit reactors (#3351 )	2018-07-17 20:43:18 +10:00
Krombel	78a9ddcf9a	rerun isort with latest version	2018-07-16 14:23:25 +02:00
Krombel	4a27000548	check isort by travis	2018-07-16 13:57:33 +02:00
Amber Brown	8a4f05fefb	Fix develop because I broke it :( (#3535 )	2018-07-14 09:51:00 +10:00
Amber Brown	8532953c04	Merge pull request #3534 from krombel/use_parse_and_asserts_from_servlet Use parse and asserts from http.servlet	2018-07-14 09:09:19 +10:00
Amber Brown	a2374b2c7f	fix sytests	2018-07-14 07:52:58 +10:00
Amber Brown	33b60c01b5	Make auth & transactions more testable (#3499 )	2018-07-14 07:34:49 +10:00
Krombel	516f960ad8	add changelog	2018-07-13 22:19:19 +02:00
Krombel	3366b9c534	rename assert_params_in_request to assert_params_in_dict the method "assert_params_in_request" does handle dicts and not requests. A request body has to be parsed to json before this method can be used	2018-07-13 21:53:01 +02:00
Krombel	32fd6910d0	Use parse_{int,str} and assert from http.servlet parse_integer and parse_string can take a request and raise errors in case we have wrong or missing params. This PR tries to use them more to deduplicate some code and make it better readable	2018-07-13 21:40:14 +02:00
Richard van der Hoff	482d17b58b	Merge branch 'develop' into rav/enforce_report_api	2018-07-12 09:56:28 +01:00
Amber Brown	49af402019	run isort	2018-07-09 16:09:20 +10:00
Amber Brown	6350bf925e	Attempt to be more performant on PyPy (#3462 )	2018-06-28 14:49:57 +01:00
Matthew Hodgson	9570aa82eb	update doc for deactivate API	2018-06-26 10:42:50 +01:00
Matthew Hodgson	1e788db430	add GDPR erase param to deactivate API	2018-06-26 10:26:54 +01:00
Erik Johnston	244484bf3c	Revert "Revert "Merge pull request #3431 from matrix-org/rav/erasure_visibility"" This reverts commit `1d009013b3`.	2018-06-25 13:42:55 +01:00
Richard van der Hoff	1d009013b3	Revert "Merge pull request #3431 from matrix-org/rav/erasure_visibility" This reverts commit `ce0d911156`, reversing changes made to `b4a5d767a9`.	2018-06-22 16:35:10 +01:00
Erik Johnston	ce0d911156	Merge pull request #3431 from matrix-org/rav/erasure_visibility Support hiding events from deleted users	2018-06-22 15:06:44 +01:00
Amber Brown	77ac14b960	Pass around the reactor explicitly (#3385 )	2018-06-22 09:37:10 +01:00
Amber Brown	a61738b316	Remove run_on_reactor (#3395 )	2018-06-14 18:27:37 +10:00
Richard van der Hoff	f1023ebf4b	mark accounts as erased when requested	2018-06-12 09:53:18 +01:00
David Baker	187a546bff	Merge pull request #3276 from matrix-org/dbkr/unbind Remove email addresses / phone numbers from ID servers when they're removed from synapse	2018-06-11 16:02:00 +01:00
Richard van der Hoff	f4caf3f83d	fix log	2018-06-07 00:26:38 +01:00
Richard van der Hoff	0546715c18	Fix event-purge-by-ts admin API This got completely broken in 0.30. Fixes #3300.	2018-06-07 00:15:49 +01:00
Richard van der Hoff	7e15410f02	Enforce the specified API for report_event as per https://matrix.org/docs/spec/client_server/unstable.html#post-matrix-client-r0-rooms-roomid-report-eventid	2018-05-31 18:17:11 +01:00
Amber Brown	c936a52a9e	Consistently use six's iteritems and wrap lazy keys/values in list() if they're not meant to be lazy (#3307 )	2018-05-31 19:03:47 +10:00
Amber Brown	1f69693347	Merge pull request #3244 from NotAFile/py3-six-4 replace some iteritems with six	2018-05-24 13:04:07 -05:00
Amber Brown	36501068d8	Merge pull request #3247 from NotAFile/py3-misc Misc Python3 fixes	2018-05-24 12:58:37 -05:00
David Baker	77a23e2e05	Merge remote-tracking branch 'origin/develop' into dbkr/unbind	2018-05-24 16:20:53 +01:00
Erik Johnston	46345187cc	Merge pull request #3243 from NotAFile/py3-six-3 Replace some more comparisons with six	2018-05-24 16:08:57 +01:00
David Baker	9700d15611	pep8	2018-05-24 11:23:15 +01:00
David Baker	b3bff53178	Unbind 3pids when they're deleted too	2018-05-24 11:08:05 +01:00
Richard van der Hoff	7b36d06a69	Add a 'has_consented' template var to consent forms fixes #3260	2018-05-22 14:58:34 +01:00
Richard van der Hoff	669400e22f	Enable auto-escaping for the consent templates ... to reduce the risk of somebody introducing an html injection attack...	2018-05-22 14:58:34 +01:00
Richard van der Hoff	8810685df9	Stub out ServerNoticesSender on the workers ... and have the sync endpoints call it directly rather than obsure indirection via PresenceHandler	2018-05-22 11:54:51 +01:00
Richard van der Hoff	d5dca9a04f	Move consent config parsing into ConsentConfig turns out we need to reuse this, so it's better in the config class.	2018-05-22 11:54:51 +01:00
Richard van der Hoff	d14d7b8fdc	Rename 'version' param on user consent config we're going to use it for the version we require too.	2018-05-22 11:54:51 +01:00
Richard van der Hoff	6e1cb54a05	Fix logcontext leak in HttpTransactionCache ONE DAY I WILL PURGE THE WORLD OF THIS EVIL	2018-05-21 16:58:20 +01:00
Richard van der Hoff	6d6e7288fe	Stop the transaction cache caching failures The transaction cache has some code which tries to stop it caching failures, but if the callback function failed straight away, then things would happen backwards and we'd end up with the failure stuck in the cache.	2018-05-21 16:49:59 +01:00
Adrian Tschira	933bf2dd35	replace some iteritems with six Signed-off-by: Adrian Tschira <nota@notafile.com>	2018-05-19 17:59:26 +02:00
Adrian Tschira	d9fe2b2d9d	Replace some more comparisons with six plus a bonus b"" string I missed last time Signed-off-by: Adrian Tschira <nota@notafile.com>	2018-05-19 17:56:31 +02:00
Adrian Tschira	aafb0f6b0d	py3-ize url preview	2018-05-19 17:35:20 +02:00
Erik Johnston	fa30ac38cc	Merge pull request #3221 from matrix-org/erikj/purge_token Make purge_history operate on tokens	2018-05-18 10:35:23 +01:00
Richard van der Hoff	c46367d0d7	Move RoomCreationHandler out of synapse.handlers.Handlers Handlers is deprecated nowadays, so let's move this out before I add a new dependency on it. Also fix the docstrings on create_room.	2018-05-17 09:08:42 +01:00
Richard van der Hoff	8030a825c8	Merge pull request #3213 from matrix-org/rav/consent_handler ConsentResource to gather policy consent from users	2018-05-16 07:19:18 +01:00
Erik Johnston	5f27ed75ad	Make purge_history operate on tokens As we're soon going to change how topological_ordering works	2018-05-15 16:23:50 +01:00
Richard van der Hoff	47815edcfa	ConsentResource to gather policy consent from users Hopefully there are enough comments and docs in this that it makes sense on its own.	2018-05-15 15:11:59 +01:00
Richard van der Hoff	318711e139	Set Server header in SynapseRequest (instead of everywhere that writes a response. Or rather, the subset of places which write responses where we haven't forgotten it). This also means that we don't have to have the mysterious version_string attribute in anything with a request handler. Unfortunately it does mean that we have to pass the version string wherever we instantiate a SynapseSite, which has been c&ped 150 times, but that is code that ought to be cleaned up anyway really.	2018-05-10 18:50:27 +01:00
Richard van der Hoff	645cb4bf06	Remove redundant request_handler decorator This is needless complexity; we might as well use the wrapper directly. Also rename wrap_request_handler->wrap_json_request_handler.	2018-05-10 12:19:53 +01:00
Konstantinos Sideris	88868b2839	notifications: Convert next_token to string according to the spec Currently the parameter is serialized as an integer. Signed-off-by: Konstantinos Sideris <sideris.konstantin@gmail.com>	2018-05-05 12:55:02 +03:00
Richard van der Hoff	902673e356	Merge pull request #3161 from NotAFile/remove-v1auth Make Client-Server API return 403 for invalid token	2018-05-03 10:10:57 +01:00
Richard van der Hoff	be31adb036	Fix logcontext leak in media repo Make FileResponder.write_to_consumer uphold the logcontext contract	2018-05-02 16:14:50 +01:00
Adrian Tschira	6495dbb326	Burminate v1auth This closes #2602 v1auth was created to account for the differences in status code between the v1 and v2_alpha revisions of the protocol (401 vs 403 for invalid tokens). However since those protocols were merged, this makes the r0 version/endpoint internally inconsistent, and violates the specification for the r0 endpoint. This might break clients that rely on this inconsistency with the specification. This is said to affect the legacy angular reference client. However, I feel that restoring parity with the spec is more important. Either way, it is critical to inform developers about this change, in case they rely on the illegal behaviour. Signed-off-by: Adrian Tschira <nota@notafile.com>	2018-04-30 22:20:43 +02:00
Krombel	576b71dd3d	add guard for None on purge_history api	2018-04-30 14:29:48 +02:00
Richard van der Hoff	dbf6f28d64	Merge pull request #3155 from NotAFile/py3-bytes-1 more bytes strings	2018-04-30 00:38:21 +01:00
Richard van der Hoff	aab2e4da60	Merge pull request #3140 from matrix-org/rav/use_run_in_background Use run_in_background in preference to preserve_fn	2018-04-30 00:34:28 +01:00
Richard van der Hoff	1315d374cc	Merge pull request #3156 from NotAFile/py3-hmac-bytes Construct HMAC as bytes on py3	2018-04-30 00:33:20 +01:00
Richard van der Hoff	9e2601f830	Merge pull request #3108 from NotAFile/py3-six-urlparse Use six.moves.urlparse	2018-04-30 00:33:05 +01:00
Adrian Tschira	122593265b	Construct HMAC as bytes on py3 Signed-off-by: Adrian Tschira <nota@notafile.com>	2018-04-29 00:19:41 +02:00
Adrian Tschira	e9143b6593	more bytes strings Signed-off-by: Adrian Tschira <nota@notafile.com>	2018-04-29 00:13:57 +02:00
Richard van der Hoff	fc149b4eeb	Merge remote-tracking branch 'origin/develop' into rav/use_run_in_background	2018-04-27 14:31:23 +01:00
Richard van der Hoff	2a13af23bc	Use run_in_background in preference to preserve_fn While I was going through uses of preserve_fn for other PRs, I converted places which only use the wrapped function once to use run_in_background, to avoid creating the function object.	2018-04-27 12:55:51 +01:00
Richard van der Hoff	9255a6cb17	Improve exception handling for background processes There were a bunch of places where we fire off a process to happen in the background, but don't have any exception handling on it - instead relying on the unhandled error being logged when the relevent deferred gets garbage-collected. This is unsatisfactory for a number of reasons: - logging on garbage collection is best-effort and may happen some time after the error, if at all - it can be hard to figure out where the error actually happened. - it is logged as a scary CRITICAL error which (a) I always forget to grep for and (b) it's not really CRITICAL if a background process we don't care about fails. So this is an attempt to add exception handling to everything we fire off into the background.	2018-04-27 11:07:40 +01:00
Adrian Tschira	2a3c33ff03	Use six.moves.urlparse The imports were shuffled around a bunch in py3 Signed-off-by: Adrian Tschira <nota@notafile.com>	2018-04-15 21:22:43 +02:00
Erik Johnston	eaa2ebf20b	Merge pull request #3079 from matrix-org/erikj/limit_concurrent_sends Limit concurrent event sends for a room	2018-04-10 16:43:58 +01:00
Richard van der Hoff	a3599dda97	Merge pull request #2996 from krombel/allow_auto_join_rooms move handling of auto_join_rooms to RegisterHandler	2018-04-10 01:11:00 +01:00
Richard van der Hoff	87478c5a60	Merge pull request #3061 from NotAFile/add-some-byte-strings Add b prefixes to some strings that are bytes in py3	2018-04-09 23:54:05 +01:00
Richard van der Hoff	c508b2f2f0	Merge pull request #3073 from NotAFile/use-six-reraise Replace old-style raise with six.reraise	2018-04-09 23:53:40 +01:00
Richard van der Hoff	37354b55c9	Merge pull request #2938 from dklug/develop Return 401 for invalid access_token on logout	2018-04-09 23:52:56 +01:00
Erik Johnston	56b0589865	Use create_and_send_nonmember_event everywhere	2018-04-09 12:04:18 +01:00
Adrian Tschira	4f40d058cc	Replace old-style raise with six.reraise The old style raise is invalid syntax in python3. As noted in the docs, this adds one more frame in the traceback, but I think this is acceptable: <ipython-input-7-bcc5cba3de3f> in <module>() 16 except: 17 pass ---> 18 six.reraise(*x) /usr/lib/python3.6/site-packages/six.py in reraise(tp, value, tb) 691 if value.__traceback__ is not tb: 692 raise value.with_traceback(tb) --> 693 raise value 694 finally: 695 value = None <ipython-input-7-bcc5cba3de3f> in <module>() 9 10 try: ---> 11 x() 12 except: 13 x = sys.exc_info() Also note that this uses six, which is not formally a dependency yet, but is included indirectly since most packages depend on it. Signed-off-by: Adrian Tschira <nota@notafile.com>	2018-04-06 23:06:24 +02:00
Richard van der Hoff	c7f0969731	Merge pull request #2986 from jplatte/join_reponse_room_id Add room_id to the response of `rooms/{roomId}/join`	2018-04-05 17:29:06 +01:00
Luke Barnard	104c0bc1d5	Use "/settings/" (plural)	2018-04-05 14:07:16 +01:00
Adrian Tschira	6168351877	Add b prefixes to some strings that are bytes in py3 This has no effect on python2 Signed-off-by: Adrian Tschira <nota@notafile.com>	2018-04-04 13:48:51 +02:00
Luke Barnard	eb8d8d6f57	Use join_policy API instead of joinable The API is now under /groups/$group_id/setting/m.join_policy and expects a JSON blob of the shape ```json { "m.join_policy": { "type": "invite" } } ``` where "invite" could alternatively be "open".	2018-04-03 16:16:40 +01:00
David Baker	c5de6987c2	This should probably be a PUT	2018-03-28 16:44:11 +01:00
David Baker	79452edeee	Add joinability for groups Adds API to set the 'joinable' flag, and corresponding flag in the table.	2018-03-28 14:03:37 +01:00
Krombel	6152e253d8	Merge branch 'develop' of into allow_auto_join_rooms	2018-03-28 14:45:28 +02:00
Erik Johnston	fa72803490	Merge branch 'master' of github.com:matrix-org/synapse into develop	2018-03-19 11:41:01 +00:00
Erik Johnston	a8ce159be4	Replace some ujson with simplejson to make it work	2018-03-16 00:27:09 +00:00
Erik Johnston	926ba76e23	Replace ujson with simplejson	2018-03-15 23:43:31 +00:00
Krombel	91ea0202e6	move handling of auto_join_rooms to RegisterHandler Currently the handling of auto_join_rooms only works when a user registers itself via public register api. Registrations via registration_shared_secret and ModuleApi do not work This auto_joins the users in the registration handler which enables the auto join feature for all 3 registration paths. This is related to issue #2725 Signed-Off-by: Matthias Kesler <krombel@krombel.de>	2018-03-14 16:45:37 +01:00
Jonas Platte	47ce527f45	Add room_id to the response of `rooms/{roomId}/join` Fixes #2349	2018-03-13 14:48:12 +01:00
Erik Johnston	f5160d4a3e	RoomMembershipRestServlet doesn't handle /forget Due to the order we register the REST handlers `/forget` was handled by the correct handler.	2018-03-13 12:12:55 +00:00
Richard van der Hoff	e48c7aac4d	Add transactional API to history purge Make the purge request return quickly, and allow scripts to poll for updates.	2018-03-12 16:22:55 +00:00
Richard van der Hoff	f8bfcd7e0d	Provide a means to pass a timestamp to purge_history	2018-03-05 14:37:23 +00:00
dklug	af7ed8e1ef	Return 401 for invalid access_token on logout Signed-off-by: Duncan Klug <dklug@ucmerced.edu>	2018-03-02 22:01:27 -08:00
Erik Johnston	784f036306	Move RoomMemberHandler out of Handlers	2018-03-01 14:36:50 +00:00
Erik Johnston	92c52df702	Make store_file use store_into_file	2018-02-14 17:55:18 +00:00
Erik Johnston	c0c9327fe0	Merge pull request #2854 from matrix-org/erikj/event_create_worker Create a worker for event creation	2018-02-13 18:07:10 +00:00
Richard van der Hoff	8fd1a32456	Fix typos in purge api & doc * It's supposed to be purge_local_events, not ..._history * Fix the doc to have valid json	2018-02-13 13:09:39 +00:00
Erik Johnston	1026690cd2	Merge pull request #2857 from matrix-org/erikj/upload_store Tell storage providers about new file so they can upload	2018-02-12 13:52:58 +00:00
Richard van der Hoff	10b34dbb9a	Merge pull request #2858 from matrix-org/rav/purge_updates delete_local_events for purge_room_history	2018-02-09 14:11:00 +00:00
Richard van der Hoff	74fcbf741b	delete_local_events for purge_history Add a flag which makes the purger delete local events	2018-02-09 13:07:41 +00:00
Erik Johnston	5fa571a91b	Tell storage providers about new file so they can upload	2018-02-07 13:35:08 +00:00
Erik Johnston	8ec2e638be	Add event_creator worker	2018-02-07 10:32:32 +00:00
Erik Johnston	3e1e69ccaf	Update copyright	2018-02-06 16:40:38 +00:00
Erik Johnston	3fa362502c	Update places where we create events	2018-02-05 16:01:48 +00:00
Erik Johnston	1f881e0746	Merge pull request #2791 from matrix-org/erikj/media_storage_refactor Ensure media is in local cache before thumbnailing	2018-02-05 11:28:52 +00:00
Richard van der Hoff	9c9356512e	Merge pull request #2845 from matrix-org/rav/urlcache_error_handling Handle url_previews with no content-type	2018-02-02 15:27:52 +01:00
Travis Ralston	6e87b34f7b	Merge branch 'develop' into travis/admin-list-media	2018-02-01 18:05:47 -07:00
Richard van der Hoff	d5352cbba8	Handle url_previews with no content-type avoid failing with an exception if the remote server doesn't give us a Content-Type header. Also, clean up the exception handling a bit.	2018-02-02 00:53:46 +00:00
Matthew Hodgson	9a72b70630	fix thinko on 3pid whitelisting	2018-01-24 11:07:47 +01:00
Matthew Hodgson	d32385336f	add ?ts massaging for ASes (#2754 ) blindly implement ?ts for AS. untested	2018-01-23 09:59:06 +01:00
Matthew Hodgson	ab9f844aaf	Add federation_domain_whitelist option (#2820 ) Add federation_domain_whitelist gives a way to restrict which domains your HS is allowed to federate with. useful mainly for gracefully preventing a private but internet-connected HS from trying to federate to the wider public Matrix network	2018-01-22 19:11:18 +01:00
Matthew Hodgson	d84f65255e	Merge pull request #2813 from matrix-org/matthew/registrations_require_3pid add registrations_require_3pid and allow_local_3pids	2018-01-22 13:57:22 +00:00
Travis Ralston	5552ed9a7f	Add an admin route to get all the media in a room This is intended to be used by administrators to monitor the media that is passing through their server, if they wish. Signed-off-by: Travis Ralston <travpc@gmail.com>	2018-01-20 22:37:53 -07:00
Matthew Hodgson	49fce04624	fix typo (thanks sytest)	2018-01-19 19:55:38 +00:00
Richard van der Hoff	b0d9e633ee	Merge pull request #2814 from matrix-org/rav/fix_urlcache_thumbs Use the right path for url_preview thumbnails	2018-01-19 18:57:15 +00:00
Richard van der Hoff	ad7ec63d08	Use the right path for url_preview thumbnails This was introduced by #2627: we were overwriting the original media for url previews with the thumbnails :/ (fixes https://github.com/vector-im/riot-web/issues/6012, hopefully)	2018-01-19 18:29:39 +00:00
Matthew Hodgson	62d7d66ae5	oops, check all login types	2018-01-19 18:23:56 +00:00
Matthew Hodgson	293380bef7	trailing commas	2018-01-19 15:38:53 +00:00
Matthew Hodgson	447f4f0d5f	rewrite based on PR feedback: * [ ] split config options into allowed_local_3pids and registrations_require_3pid * [ ] simplify and comment logic for picking registration flows * [ ] fix docstring and move check_3pid_allowed into a new util module * [ ] use check_3pid_allowed everywhere @erikjohnston PTAL	2018-01-19 15:33:55 +00:00
Matthew Hodgson	9d332e0f79	fix up v1, and improve errors	2018-01-19 00:53:58 +00:00
Matthew Hodgson	0af58f14ee	fix pep8	2018-01-19 00:33:51 +00:00
Matthew Hodgson	28a6ccb49c	add registrations_require_3pid lets homeservers specify a whitelist for 3PIDs that users are allowed to associate with. Typically useful for stopping people from registering with non-work emails	2018-01-19 00:19:58 +00:00
Erik Johnston	cd871a3057	Fix storage provider bug introduced when renamed to store_local	2018-01-18 18:37:59 +00:00
Erik Johnston	8ff6726c0d	Merge pull request #2812 from matrix-org/erikj/media_storage_provider_config Make storage providers configurable	2018-01-18 18:33:57 +00:00
Erik Johnston	3fe2bae857	Missing staticmethod	2018-01-18 17:11:45 +00:00
Erik Johnston	aae77da73f	Fixup comments	2018-01-18 17:11:29 +00:00
Erik Johnston	9a89dae8c5	Fix typo in thumbnail resource causing access times to be incorrect	2018-01-18 15:06:24 +00:00
Erik Johnston	0af5dc63a8	Make storage providers more configurable	2018-01-18 14:07:21 +00:00
Erik Johnston	2cf6a7bc20	Use better file consumer	2018-01-18 12:00:46 +00:00
Erik Johnston	4a53f3a3e8	Ensure media is in local cache before thumbnailing	2018-01-18 12:00:46 +00:00
Erik Johnston	300edc2348	Update last access time when thumbnails are viewed	2018-01-17 10:24:43 +00:00
Erik Johnston	05f98a2224	Keep track of last access time for local media	2018-01-17 10:24:43 +00:00
Erik Johnston	d728c47142	Add docstring	2018-01-17 10:06:14 +00:00
Erik Johnston	d863f68cab	Use local vars	2018-01-16 16:24:15 +00:00
Erik Johnston	6368e5c0ab	Change _generate_thumbnails to take media_type	2018-01-16 16:17:38 +00:00
Erik Johnston	0a90d9ede4	Move setting of file_id up to caller	2018-01-16 16:03:05 +00:00
Erik Johnston	5dfc83704b	Fix typo	2018-01-16 14:32:56 +00:00
Erik Johnston	307f88dfb6	Fix up log lines	2018-01-16 13:53:52 +00:00
Erik Johnston	9795b9ebb1	Correctly use server_name/file_id when generating/fetching remote thumbnails	2018-01-16 12:02:06 +00:00
Erik Johnston	c5b589f2e8	Log when we respond with 404	2018-01-16 12:01:40 +00:00
Erik Johnston	a4c5e4a645	Fix thumbnailing remote files	2018-01-16 11:37:50 +00:00
Erik Johnston	1159abbdd2	Merge pull request #2767 from matrix-org/erikj/media_storage_refactor Refactor MediaRepository to separate out storage	2018-01-16 10:23:50 +00:00
Richard van der Hoff	21bf87a146	Reinstate media download on thumbnail request We need to actually download the remote media when we get a request for a thumbnail.	2018-01-12 15:38:06 +00:00
Erik Johnston	694f1c1b18	Fix up comments	2018-01-12 15:02:46 +00:00
Erik Johnston	e21370ba54	Correctly reraise exception	2018-01-12 14:44:02 +00:00
Erik Johnston	85a4d78213	Make Responder a context manager	2018-01-12 13:32:03 +00:00
Erik Johnston	dcc8eded41	Add missing class var	2018-01-12 13:16:27 +00:00
Erik Johnston	81391fa162	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/media_storage_refactor	2018-01-12 11:28:49 +00:00
Erik Johnston	1e4edd1717	Remove unnecessary condition	2018-01-12 11:28:32 +00:00
Erik Johnston	c6c009603c	Remove unused variables	2018-01-12 11:24:05 +00:00
Erik Johnston	4d88958cf6	Make class var local	2018-01-12 11:23:54 +00:00
Erik Johnston	227c491510	Comments	2018-01-12 11:22:41 +00:00
Richard van der Hoff	5f23b6d5ea	Merge pull request #2766 from matrix-org/rav/room_event Add /room/{id}/event/{id} to synapse	2018-01-11 14:47:18 +00:00
Erik Johnston	8f03aa9f61	Add StorageProvider concept	2018-01-09 16:16:12 +00:00
Erik Johnston	2442e9876c	Make PreviewUrlResource use MediaStorage	2018-01-09 16:15:07 +00:00
Erik Johnston	9d30a7691c	Make ThumbnailResource use MediaStorage	2018-01-09 16:15:07 +00:00
Erik Johnston	9e20840e02	Use MediaStorage for remote media	2018-01-09 16:15:07 +00:00
Erik Johnston	dd3092c3a3	Use MediaStorage for local files	2018-01-09 16:15:07 +00:00
Erik Johnston	ada470bccb	Add MediaStorage class	2018-01-09 16:15:07 +00:00
Erik Johnston	1ee787912b	Add some helper classes	2018-01-09 16:15:07 +00:00
Erik Johnston	47ca5eb882	Split out add_file_headers	2018-01-09 16:15:07 +00:00
Erik Johnston	b6c9deffda	Remove dead TODO	2018-01-09 15:53:23 +00:00
Richard van der Hoff	51c9d9ed65	Add /room/{id}/event/{id} to synapse Turns out that there is a valid usecase for retrieving event by id (notably having received a push), but event ids should be scoped to room, so /event/{id} is wrong.	2018-01-09 14:39:12 +00:00
Erik Johnston	b30cd5b107	Remove dead code related to default thumbnails	2018-01-09 14:38:33 +00:00
Richard van der Hoff	75c1b8df01	Better logging when login can't find a 3pid	2017-12-20 19:31:00 +00:00
Richard van der Hoff	16ec3805e5	Fix error when deleting devices This was introduced in `d7ea8c4` / PR #2728	2017-12-05 09:49:22 +00:00
Richard van der Hoff	d7ea8c4800	Factor out a validate_user_via_ui_auth method Collect together all the places that validate a logged-in user via UI auth.	2017-12-05 09:42:30 +00:00
Richard van der Hoff	d5f9fb06b0	Refactor UI auth implementation Instead of returning False when auth is incomplete, throw an exception which can be caught with a wrapper.	2017-12-05 09:40:05 +00:00
Richard van der Hoff	ad7e570d07	Delete devices in various logout situations Make sure that we delete devices whenever a user is logged out due to any of the following situations: * /logout * /logout_all * change password * deactivate account (by the user or by an admin) * invalidate access token from a dynamic module Fixes #2672.	2017-11-29 16:44:35 +00:00
Richard van der Hoff	ae31f8ce45	Move set_password into its own handler Non-functional refactoring to move set_password. This means that we'll be able to properly deactivate devices and access tokens without introducing a dependency loop.	2017-11-29 16:44:35 +00:00
Richard van der Hoff	7ca5c68233	Move deactivate_account into its own handler Non-functional refactoring to move deactivate_account. This means that we'll be able to properly deactivate devices and access tokens without introducing a dependency loop.	2017-11-29 16:44:35 +00:00
Luke Barnard	ab1b2d0ff2	Allow guest access to group APIs for reading	2017-11-28 11:23:00 +00:00
Richard van der Hoff	5a4da5bf78	Merge pull request #2697 from matrix-org/rav/fix_urlcache_index_error Fix error on sqlite 3.7	2017-11-27 12:25:48 +00:00
Richard van der Hoff	8132a6b7ac	Fix OPTIONS on preview_url Fixes #2706	2017-11-23 17:52:31 +00:00
Richard van der Hoff	2908f955d1	Check database in has_completed_background_updates so that the right thing happens on workers.	2017-11-22 18:02:15 +00:00
Richard van der Hoff	7098b65cb8	Fix error on sqlite 3.7 Create the url_cache index on local_media_repository as a background update, so that we can detect whether we are on sqlite or not and create a partial or complete index accordingly. To avoid running the cleanup job before we have built the index, add a bailout which will defer the cleanup if the bg updates are still running. Fixes https://github.com/matrix-org/synapse/issues/2572.	2017-11-21 11:14:17 +00:00
Richard van der Hoff	30d2730ee2	Declare support for r0.3.0	2017-11-15 16:24:22 +00:00
Richard van der Hoff	7bd6c87eca	Merge pull request #2668 from turt2live/travis/whoami Add a route for determining who you are	2017-11-14 09:54:21 +00:00
Travis Ralston	812c191939	Remove redundent call Signed-off-by: Travis Ralston <travpc@gmail.com>	2017-11-13 12:44:21 -07:00
Travis Ralston	2d314b771f	Add a route for determining who you are Useful for applications which may have an access token, but no idea as to who owns it. Signed-off-by: Travis Ralston <travpc@gmail.com>	2017-11-12 23:39:38 -07:00
Richard van der Hoff	5d15abb120	Bit more logging	2017-11-10 16:58:04 +00:00
Richard van der Hoff	46790f50cf	Cache failures in url_preview handler Reshuffle the caching logic in the url_preview handler so that failures are cached (and to generally simplify things and fix the logcontext leaks).	2017-11-10 16:50:50 +00:00
Richard van der Hoff	e508145c9b	Add some more comments appservice user registration Explain why we don't validate userids registered via app services	2017-11-10 12:39:45 +00:00
Richard van der Hoff	e0ebd1e4bd	Downcase userids for shared-secret registration	2017-11-10 12:39:05 +00:00
Richard van der Hoff	9b599bc18d	Downcase userid on registration Force username to lowercase before attempting to register https://github.com/matrix-org/synapse/issues/2660	2017-11-09 22:20:01 +00:00
Erik Johnston	4e2b2508af	Register group servlet	2017-11-09 15:49:42 +00:00
Erik Johnston	e8814410ef	Have an explicit API to update room config	2017-11-08 16:13:27 +00:00
Erik Johnston	94ff2cda73	Revert "Modify group room association API to allow modification of is_public"	2017-11-08 15:43:34 +00:00
Matthew Hodgson	3db5ff69b2	Merge pull request #2576 from maximevaillancourt/exclude-noscript-url-preview Ignore <noscript> tags when generating URL preview descriptions	2017-11-07 11:09:22 +00:00
Richard van der Hoff	6c3a02072b	support inhibit_login in /register Allow things to pass inhibit_login when registering to ... inhibit logins.	2017-11-02 16:31:07 +00:00
Richard van der Hoff	4c8f94ac94	Allow password_auth_providers to return a callback ... so that they have a way to record access tokens.	2017-11-01 16:51:03 +00:00
David Baker	4f0488b307	Merge remote-tracking branch 'origin/develop' into rav/refactor_accesstoken_delete	2017-11-01 16:20:19 +00:00

... 11 12 13 14 15 ...

2357 Commits