1482 Commits

Author	SHA1	Message	Date
Richard van der Hoff	eaada74075	JWT OIDC secrets for Sign in with Apple (#9549) ... Apple had to be special. They want a client secret which is generated from an EC key. Fixes #9220. Also fixes #9212 while I'm here.	2021-03-09 15:03:37 +00:00
Erik Johnston	576c91c7c1	Fixup sample config ... After 0764d0c6e5	2021-03-09 11:40:45 +00:00
Leo Bärring	0fc4eb103a	Update reverse proxy to add OpenBSD relayd example configuration. (#9508) ... Update reverse proxy to add OpenBSD relayd example configuration. Signed-off-by: Leo Bärring <leo.barring@protonmail.com>	2021-03-06 11:49:19 +00:00
Richard van der Hoff	a5daae2a5f	Update nginx reverse-proxy docs (#9512) ... Turns out nginx overwrites the Host header by default.	2021-03-03 11:08:11 +00:00
Richard van der Hoff	d8e95e5452	Add support for X-Forwarded-Proto (#9472) ... rewrite XForwardedForRequest to set `isSecure()` based on `X-Forwarded-Proto`. Also implement `getClientAddress()` while we're here.	2021-02-24 18:11:33 +00:00
Andrew Morgan	00bf80cb8e	Fix typo in spam checker documentation	2021-02-24 17:51:52 +00:00
Erik Johnston	66f4949e7f	Fix deleting pushers when using sharded pushers. (#9465)	2021-02-22 21:14:42 +00:00
Dirk Klimpel	71c9f8de6d	Add an order_by field to list users' media admin API. (#8978)	2021-02-22 14:38:51 -05:00
Richard van der Hoff	70ea9593ff	example systemd config: propagate reloads to units (#9463) ... It should be possible to reload `synapse.target` to have the reload propagate to all the synapse units.	2021-02-22 18:43:17 +00:00
Andrew Morgan	e22b71810e	Clean up the user directory sample config section (#9385) ... The user directory sample config section was a little messy, and didn't adhere to our [recommended config format guidelines](https://github.com/matrix-org/synapse/blob/develop/docs/code_style.md#configuration-file-format). This PR cleans that up a bit.	2021-02-22 11:44:31 +00:00
Andrew Morgan	13e9029f44	Add a config option to prioritise local users in user directory search results (#9383) ... This PR adds a homeserver config option, `user_directory.prefer_local_users`, that when enabled will show local users higher in user directory search results than remote users. This option is off by default. Note that turning this on doesn't necessarily mean that remote users will always be put below local users, but they should be assuming all other ranking factors (search query match, profile information present etc) are identical. This is useful for, say, University networks that are openly federating, but want to prioritise local students and staff in the user directory over other random users.	2021-02-19 11:02:03 +00:00
AndrewFerr	9bc74743d5	Add configs to make profile data more private (#9203) ... Add off-by-default configuration settings to: - disable putting an invitee's profile info in invite events - disable profile lookup via federation Signed-off-by: Andrew Ferrazzutti <fair@miscworks.net>	2021-02-19 09:50:41 +00:00
Rishabh Arya	e17553e185	Parse ui_auth.session_timeout as a duration (instead of treating it as ms) (#9426)	2021-02-18 09:18:14 -05:00
David Vo	bb2577f6b7	Add http2 to the nginx example config (#9390)	2021-02-18 08:46:16 -05:00
Richard van der Hoff	626afd7e89	Revert "Update workers.md" ... This reverts commit a8878960c0.	2021-02-18 11:56:25 +00:00
Dirk Klimpel	c8d9383cfb	Add the shadow-banning status to the display user admin API. (#9400)	2021-02-17 15:19:23 -05:00
Richard van der Hoff	53f1c4da81	Update workers.md	2021-02-17 17:14:23 +00:00
Richard van der Hoff	a8878960c0	Update workers.md ... tiny typo in sso paths	2021-02-17 17:11:24 +00:00
Eric Eastwood	0a00b7ff14	Update black, and run auto formatting over the codebase (#9381) ... - Update black version to the latest - Run black auto formatting over the codebase - Run autoformatting according to [`docs/code_style.md `](80d6dc9783/docs/code_style.md) - Update `code_style.md` docs around installing black to use the correct version	2021-02-16 22:32:34 +00:00
Marcus	5636e597c3	Fix OIDC gitiea redirect URL. (#9404) ... Fixes a "conflict" from 846b9d3df0 and d1f13c7485.	2021-02-16 14:06:55 -05:00
Erik Johnston	ddfdf94506	Document that pusher instances are shardable (#9407)	2021-02-16 13:27:49 +00:00
Andrew Morgan	ff40c8099d	Fix sample config ... Just a small change missed in 7950aa8a27.	2021-02-12 22:18:40 +00:00
Patrick Cloke	2c9b4a5f16	Synapse 1.27.0rc2 (2021-02-11) ... ============================== Features -------- - Further improvements to the user experience of registration via single sign-on. ([\#9297](https://github.com/matrix-org/synapse/issues/9297)) Bugfixes -------- - Fix ratelimiting introduced in v1.27.0rc1 for invites to respect the `ratelimit` flag on application services. ([\#9302](https://github.com/matrix-org/synapse/issues/9302)) - Do not automatically calculate `public_baseurl` since it can be wrong in some situations. Reverts behaviour introduced in v1.26.0. ([\#9313](https://github.com/matrix-org/synapse/issues/9313)) Improved Documentation ---------------------- - Clarify the sample configuration for changes made to the template loading code. ([\#9310](https://github.com/matrix-org/synapse/issues/9310)) -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEF3tZXk38tRDFVnUIM/xY9qcRMEgFAmAlX4AACgkQM/xY9qcR MEjxhg/+JzOGDmgxy4Vm9oU84htkkJsflU755ykxhwbIjVy2j+07pMgtWMmebXIS /tWjinwRB9OgKIK+j5RWdYzz/GRTkLCbafTLhhNOWsnaCO7nnAfZFozLOMd9g2qN 309cS8efZkxGCISGSRm1QQKjQDx6HHx7nZVAwpqb778Q9TMrBYRAQeCk9yws7FfL GsP5YrBI42n84YhvflWA8J8QRYkeJJaggVWzE0XvwdI6raY1BmxMKcDvyl9iKpGb /QXsdketD3eJxdOlU92O5ZxRXmL00bdyAFdw2+J3Y+wjoGBnC5njKSFG3j8Z2UcP xIdB6w/zyGoPLKd4s7tkDI3axE7qrwFetA4NytannWGEHZ9q5tgOd5PA1kTeKYvn ao2os4pKMjWQOHiWLskqZVXgmoW5Kb2zgyZU/vhFAz8ImuidFULOjpEytAWqtt1B iuL/fRlM3z0BpwQNd3NBsblYTpZri8gdTp7ULJYtaKpT2MeG0sg3swJAptUjw9L4 awbQuBFZ8o/E/0xwiGieSxbR6b6Nz5WiCyLaeMz6b3Y4YdomttOAZ6pL6E9y8ygU 2wKYleSmg+tbaBKKZR5CSCwXOduSYPhWw7oqsfEfvT0NCQLuItfGNFZqxRzsf73H lCV0wwxNZBv8kRyMs6KivMvSI6SKinAjXS8AjzHE402ozdwOCv0= =NSyp -----END PGP SIGNATURE----- Merge tag 'v1.27.0rc2' into develop Synapse 1.27.0rc2 (2021-02-11) ============================== Features -------- - Further improvements to the user experience of registration via single sign-on. ([\#9297](https://github.com/matrix-org/synapse/issues/9297)) Bugfixes -------- - Fix ratelimiting introduced in v1.27.0rc1 for invites to respect the `ratelimit` flag on application services. ([\#9302](https://github.com/matrix-org/synapse/issues/9302)) - Do not automatically calculate `public_baseurl` since it can be wrong in some situations. Reverts behaviour introduced in v1.26.0. ([\#9313](https://github.com/matrix-org/synapse/issues/9313)) Improved Documentation ---------------------- - Clarify the sample configuration for changes made to the template loading code. ([\#9310](https://github.com/matrix-org/synapse/issues/9310))	2021-02-11 11:56:03 -05:00
Patrick Cloke	e40d88cff3	Backout changes for automatically calculating the public baseurl. (#9313) ... This breaks some people's configurations (if their Client-Server API is not accessed via port 443).	2021-02-11 11:16:54 -05:00
Thomas Mortagne	5f716fa777	Add XWiki OIDC provider example. (#9324)	2021-02-09 11:54:52 -05:00
David Teller	b0b2cac057	Merge pull request #9150 from Yoric/develop-context ... New API /_synapse/admin/rooms/{roomId}/context/{eventId}	2021-02-08 15:53:44 +01:00
Erik Johnston	adc96d4236	Merge branch 'erikj/media_spam_checker' into develop	2021-02-04 17:01:59 +00:00
Erik Johnston	7e8083eb48	Add check_media_file_for_spam spam checker hook	2021-02-04 17:01:30 +00:00
dykstranet	982d9eb211	Correct matrix-synapse.service reference in TURN howto docs. (#9308)	2021-02-04 11:22:44 -05:00
Patrick Cloke	2ab6e67ab7	Fix escaping of braces in OIDC sample config. (#9317) ... This fixes the Jinja2 templates for the mapping provider.	2021-02-04 09:06:20 -05:00
Richard van der Hoff	3f534d3fdf	Merge branch 'social_login_hotfixes' into develop	2021-02-03 20:34:27 +00:00
Patrick Cloke	afa18f1baa	Clarify documentation about escaping URLs in templates. (#9310)	2021-02-03 14:51:38 -05:00
dykstranet	e4cdecb310	config: Add detail to auto_join_rooms comment (#9291) ... config: Add detail to auto_join_rooms comment Signed-off-by: Gary Dykstra <gary@dykstranet.com>	2021-02-03 15:21:30 +00:00
Patrick Cloke	4ca054a4ea	Convert blacklisted IPv4 addresses to compatible IPv6 addresses. (#9240) ... Also add a few more IP ranges to the default blacklist.	2021-02-03 07:13:46 -05:00
Travis Ralston	b60bb28bbc	Add an admin API to get the current room state (#9168) ... This could arguably replace the existing admin API for `/members`, however that is out of scope of this change. This sort of endpoint is ideal for moderation use cases as well as other applications, such as needing to retrieve various bits of information about a room to perform a task (like syncing power levels between two places). This endpoint exposes nothing more than an admin would be able to access with a `select *` query on their database.	2021-02-02 11:16:29 +00:00
Richard van der Hoff	8f75bf1df7	Put SAML callback URI under /_synapse/client. (#9289)	2021-02-02 09:43:50 +00:00
Richard van der Hoff	846b9d3df0	Put OIDC callback URI under /_synapse/client. (#9288)	2021-02-01 22:56:01 +00:00
Oliver Hanikel	d1f13c7485	Add an OpenID example config for Gitea. (#9134)	2021-02-01 16:21:09 -05:00
Richard van der Hoff	5963426b95	Merge branch 'social_login' into develop	2021-02-01 18:46:12 +00:00
Richard van der Hoff	c543bf87ec	Collect terms consent from the user during SSO registration (#9276)	2021-02-01 18:37:41 +00:00
Richard van der Hoff	e5d70c8a82	Improve styling and wording of SSO UIA templates (#9286) ... fixes #9171	2021-02-01 18:36:04 +00:00
Richard van der Hoff	18ab35284a	Merge branch 'social_login' into develop	2021-02-01 17:28:37 +00:00
Richard van der Hoff	4167494c90	Replace username picker with a template (#9275) ... There's some prelimiary work here to pull out the construction of a jinja environment to a separate function. I wanted to load the template at display time rather than load time, so that it's easy to update on the fly. Honestly, I think we should do this with all our templates: the risk of ending up with malformed templates is far outweighed by the improved turnaround time for an admin trying to update them.	2021-02-01 15:52:50 +00:00
Richard van der Hoff	8aed29dc61	Improve styling and wording of SSO redirect confirm template (#9272)	2021-02-01 15:50:56 +00:00
Richard van der Hoff	9c715a5f19	Fix SSO on workers (#9271) ... Fixes #8966. * Factor out build_synapse_client_resource_tree Start a function which will mount resources common to all workers. * Move sso init into build_synapse_client_resource_tree ... so that we don't have to do it for each worker * Fix SSO-login-via-a-worker Expose the SSO login endpoints on workers, like the documentation says. * Update workers config for new endpoints Add documentation for endpoints recently added (#8942, #9017, #9262) * remove submit_token from workers endpoints list this *doesn't* work on workers (yet). * changelog * Add a comment about the odd path for SAML2Resource	2021-02-01 15:47:59 +00:00
Erik Johnston	f2c1560eca	Ratelimit invites by room and target user (#9258)	2021-01-29 16:38:29 +00:00
Richard van der Hoff	0d81a6fa3e	Merge branch 'social_login' into develop	2021-01-28 22:08:11 +00:00
Erik Johnston	4b73488e81	Ratelimit 3PID /requestToken API (#9238)	2021-01-28 17:39:21 +00:00
David Teller	fe52dae6bd	FIXUP: Documenting /_synapse/admin/v1/rooms/<room_id>/context/<event_id>	2021-01-28 12:30:21 +01:00
Richard van der Hoff	34efb4c604	Add notes on integrating with Facebook for SSO login. (#9244)	2021-01-27 22:57:16 +00:00
Richard van der Hoff	a083aea396	Add 'brand' field to MSC2858 response (#9242) ... We've decided to add a 'brand' field to help clients decide how to style the buttons. Also, fix up the allowed characters for idp_id, while I'm in the area.	2021-01-27 21:31:45 +00:00
Richard van der Hoff	869667760f	Support for scraping email addresses from OIDC providers (#9245)	2021-01-27 21:28:59 +00:00
Erik Johnston	93b61589b0	Add a note to changelog about redis usage (#9227)	2021-01-27 14:06:27 +00:00
Patrick Cloke	26837d5dbe	Do not require the CAS service URL setting (use public_baseurl instead). (#9199) ... The current configuration is handled for backwards compatibility, but is considered deprecated.	2021-01-26 10:49:25 -05:00
Jason Robinson	cee4010f94	Merge branch 'develop' into jaywink/admin-forward-extremities ... # Conflicts: # synapse/rest/admin/__init__.py	2021-01-26 10:15:32 +02:00
Patrick Cloke	4a55d267ee	Add an admin API for shadow-banning users. (#9209) ... This expands the current shadow-banning feature to be usable via the admin API and adds documentation for it. A shadow-banned users receives successful responses to their client-server API requests, but the events are not propagated into rooms. Shadow-banning a user should be used as a tool of last resort and may lead to confusing or broken behaviour for the client.	2021-01-25 14:49:39 -05:00
Jason Robinson	8965b6cfec	Merge branch 'develop' into jaywink/admin-forward-extremities	2021-01-23 21:41:35 +02:00
Jason Robinson	930ba00971	Add depth and received_ts to forward_extremities admin API response ... Also add a warning on the admin API documentation. Signed-off-by: Jason Robinson <jasonr@matrix.org>	2021-01-23 21:34:32 +02:00
Richard van der Hoff	b5120f09f1	Merge remote-tracking branch 'origin/release-v1.26.0' into develop	2021-01-21 13:17:07 +00:00
Richard van der Hoff	7447f19702	Prefix idp_id with "oidc-" (#9189) ... ... to avoid clashes with other SSO mechanisms	2021-01-21 12:25:02 +00:00
Patrick Cloke	f81d02d75b	Synapse 1.26.0rc1 (2021-01-20) ... ============================== This release brings a new schema version for Synapse and rolling back to a previous verious is not trivial. Please review [UPGRADE.rst](UPGRADE.rst) for more details on these changes and for general upgrade guidance. Features -------- - Add support for multiple SSO Identity Providers. ([\#9015](https://github.com/matrix-org/synapse/issues/9015), [\#9017](https://github.com/matrix-org/synapse/issues/9017), [\#9036](https://github.com/matrix-org/synapse/issues/9036), [\#9067](https://github.com/matrix-org/synapse/issues/9067), [\#9081](https://github.com/matrix-org/synapse/issues/9081), [\#9082](https://github.com/matrix-org/synapse/issues/9082), [\#9105](https://github.com/matrix-org/synapse/issues/9105), [\#9107](https://github.com/matrix-org/synapse/issues/9107), [\#9109](https://github.com/matrix-org/synapse/issues/9109), [\#9110](https://github.com/matrix-org/synapse/issues/9110), [\#9127](https://github.com/matrix-org/synapse/issues/9127), [\#9153](https://github.com/matrix-org/synapse/issues/9153), [\#9154](https://github.com/matrix-org/synapse/issues/9154), [\#9177](https://github.com/matrix-org/synapse/issues/9177)) - During user-interactive authentication via single-sign-on, give a better error if the user uses the wrong account on the SSO IdP. ([\#9091](https://github.com/matrix-org/synapse/issues/9091)) - Give the `public_baseurl` a default value, if it is not explicitly set in the configuration file. ([\#9159](https://github.com/matrix-org/synapse/issues/9159)) - Improve performance when calculating ignored users in large rooms. ([\#9024](https://github.com/matrix-org/synapse/issues/9024)) - Implement [MSC2176](https://github.com/matrix-org/matrix-doc/pull/2176) in an experimental room version. ([\#8984](https://github.com/matrix-org/synapse/issues/8984)) - Add an admin API for protecting local media from quarantine. ([\#9086](https://github.com/matrix-org/synapse/issues/9086)) - Remove a user's avatar URL and display name when deactivated with the Admin API. ([\#8932](https://github.com/matrix-org/synapse/issues/8932)) - Update `/_synapse/admin/v1/users/<user_id>/joined_rooms` to work for both local and remote users. ([\#8948](https://github.com/matrix-org/synapse/issues/8948)) - Add experimental support for handling to-device messages on worker processes. ([\#9042](https://github.com/matrix-org/synapse/issues/9042), [\#9043](https://github.com/matrix-org/synapse/issues/9043), [\#9044](https://github.com/matrix-org/synapse/issues/9044), [\#9130](https://github.com/matrix-org/synapse/issues/9130)) - Add experimental support for handling `/keys/claim` and `/room_keys` APIs on worker processes. ([\#9068](https://github.com/matrix-org/synapse/issues/9068)) - Add experimental support for handling `/devices` API on worker processes. ([\#9092](https://github.com/matrix-org/synapse/issues/9092)) - Add experimental support for moving off receipts and account data persistence off master. ([\#9104](https://github.com/matrix-org/synapse/issues/9104), [\#9166](https://github.com/matrix-org/synapse/issues/9166)) Bugfixes -------- - Fix a long-standing issue where an internal server error would occur when requesting a profile over federation that did not include a display name / avatar URL. ([\#9023](https://github.com/matrix-org/synapse/issues/9023)) - Fix a long-standing bug where some caches could grow larger than configured. ([\#9028](https://github.com/matrix-org/synapse/issues/9028)) - Fix error handling during insertion of client IPs into the database. ([\#9051](https://github.com/matrix-org/synapse/issues/9051)) - Fix bug where we didn't correctly record CPU time spent in `on_new_event` block. ([\#9053](https://github.com/matrix-org/synapse/issues/9053)) - Fix a minor bug which could cause confusing error messages from invalid configurations. ([\#9054](https://github.com/matrix-org/synapse/issues/9054)) - Fix incorrect exit code when there is an error at startup. ([\#9059](https://github.com/matrix-org/synapse/issues/9059)) - Fix `JSONDecodeError` spamming the logs when sending transactions to remote servers. ([\#9070](https://github.com/matrix-org/synapse/issues/9070)) - Fix "Failed to send request" errors when a client provides an invalid room alias. ([\#9071](https://github.com/matrix-org/synapse/issues/9071)) - Fix bugs in federation catchup logic that caused outbound federation to be delayed for large servers after start up. Introduced in v1.8.0 and v1.21.0. ([\#9114](https://github.com/matrix-org/synapse/issues/9114), [\#9116](https://github.com/matrix-org/synapse/issues/9116)) - Fix corruption of `pushers` data when a postgres bouncer is used. ([\#9117](https://github.com/matrix-org/synapse/issues/9117)) - Fix minor bugs in handling the `clientRedirectUrl` parameter for SSO login. ([\#9128](https://github.com/matrix-org/synapse/issues/9128)) - Fix "Unhandled error in Deferred: BodyExceededMaxSize" errors when .well-known files that are too large. ([\#9108](https://github.com/matrix-org/synapse/issues/9108)) - Fix "UnboundLocalError: local variable 'length' referenced before assignment" errors when the response body exceeds the expected size. This bug was introduced in v1.25.0. ([\#9145](https://github.com/matrix-org/synapse/issues/9145)) - Fix a long-standing bug "ValueError: invalid literal for int() with base 10" when `/publicRooms` is requested with an invalid `server` parameter. ([\#9161](https://github.com/matrix-org/synapse/issues/9161)) Improved Documentation ---------------------- - Add some extra docs for getting Synapse running on macOS. ([\#8997](https://github.com/matrix-org/synapse/issues/8997)) - Correct a typo in the `systemd-with-workers` documentation. ([\#9035](https://github.com/matrix-org/synapse/issues/9035)) - Correct a typo in `INSTALL.md`. ([\#9040](https://github.com/matrix-org/synapse/issues/9040)) - Add missing `user_mapping_provider` configuration to the Keycloak OIDC example. Contributed by @chris-ruecker. ([\#9057](https://github.com/matrix-org/synapse/issues/9057)) - Quote `pip install` packages when extras are used to avoid shells interpreting bracket characters. ([\#9151](https://github.com/matrix-org/synapse/issues/9151)) Deprecations and Removals ------------------------- - Remove broken and unmaintained `demo/webserver.py` script. ([\#9039](https://github.com/matrix-org/synapse/issues/9039)) Internal Changes ---------------- - Improve efficiency of large state resolutions. ([\#8868](https://github.com/matrix-org/synapse/issues/8868), [\#9029](https://github.com/matrix-org/synapse/issues/9029), [\#9115](https://github.com/matrix-org/synapse/issues/9115), [\#9118](https://github.com/matrix-org/synapse/issues/9118), [\#9124](https://github.com/matrix-org/synapse/issues/9124)) - Various clean-ups to the structured logging and logging context code. ([\#8939](https://github.com/matrix-org/synapse/issues/8939)) - Ensure rejected events get added to some metadata tables. ([\#9016](https://github.com/matrix-org/synapse/issues/9016)) - Ignore date-rotated homeserver logs saved to disk. ([\#9018](https://github.com/matrix-org/synapse/issues/9018)) - Remove an unused column from `access_tokens` table. ([\#9025](https://github.com/matrix-org/synapse/issues/9025)) - Add a `-noextras` factor to `tox.ini`, to support running the tests with no optional dependencies. ([\#9030](https://github.com/matrix-org/synapse/issues/9030)) - Fix running unit tests when optional dependencies are not installed. ([\#9031](https://github.com/matrix-org/synapse/issues/9031)) - Allow bumping schema version when using split out state database. ([\#9033](https://github.com/matrix-org/synapse/issues/9033)) - Configure the linters to run on a consistent set of files. ([\#9038](https://github.com/matrix-org/synapse/issues/9038)) - Various cleanups to device inbox store. ([\#9041](https://github.com/matrix-org/synapse/issues/9041)) - Drop unused database tables. ([\#9055](https://github.com/matrix-org/synapse/issues/9055)) - Remove unused `SynapseService` class. ([\#9058](https://github.com/matrix-org/synapse/issues/9058)) - Remove unnecessary declarations in the tests for the admin API. ([\#9063](https://github.com/matrix-org/synapse/issues/9063)) - Remove `SynapseRequest.get_user_agent`. ([\#9069](https://github.com/matrix-org/synapse/issues/9069)) - Remove redundant `Homeserver.get_ip_from_request` method. ([\#9080](https://github.com/matrix-org/synapse/issues/9080)) - Add type hints to media repository. ([\#9093](https://github.com/matrix-org/synapse/issues/9093)) - Fix the wrong arguments being passed to `BlacklistingAgentWrapper` from `MatrixFederationAgent`. Contributed by Timothy Leung. ([\#9098](https://github.com/matrix-org/synapse/issues/9098)) - Reduce the scope of caught exceptions in `BlacklistingAgentWrapper`. ([\#9106](https://github.com/matrix-org/synapse/issues/9106)) - Improve `UsernamePickerTestCase`. ([\#9112](https://github.com/matrix-org/synapse/issues/9112)) - Remove dependency on `distutils`. ([\#9125](https://github.com/matrix-org/synapse/issues/9125)) - Enforce that replication HTTP clients are called with keyword arguments only. ([\#9144](https://github.com/matrix-org/synapse/issues/9144)) - Fix the Python 3.5 / old dependencies build in CI. ([\#9146](https://github.com/matrix-org/synapse/issues/9146)) - Replace the old `perspectives` option in the Synapse docker config file template with `trusted_key_servers`. ([\#9157](https://github.com/matrix-org/synapse/issues/9157)) -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEF3tZXk38tRDFVnUIM/xY9qcRMEgFAmAIVFkACgkQM/xY9qcR MEhLfQ//Z4xnP5Icx732k2xpdR+JaXX6heNVFm/UuLNd6wq+uXjO/Dgc9IKQ5LXD kfG+/OnNGDr/eUk4mQ5s/ccBvaTrOid2IIemJd4lUBy/Q5LvPvfKIp08QFr72WJT 7Y4Ma54nNSSMUAdESgj/aFAHjATbfHhxPOZ5OdQHGLAuJ/OUwR2ksasVNCuINbgh 8vrxrzjeYn1Zl3UTBrsdCcat7AFqYjxK/Y0i6JxfSgRwDZjUbW4M2C4OGWZJPUrU 2YTPHangxd+22+HObE2KDEzaHDdCR/Kj4dhCiJBynZ5iucuHSS+mgf39RtbjqWML 01cmXXUV+FH7rnCilmF9rYdmHZ6L9BJxtrQDQU4dB106BX9G7hKR6rYHc9Z6tjOh 3tg0cKWYMXOsTNnvRqiFNh2q0yBKS228HRAwyGosW/6NiYj43ArtYP6knX7dQyBX 0mYNkfry2dyX6Kj5el0i9MOTHYQxfc8apsBm2M3OTNYukgQKA0NhbAquibaAxEEW 2qIqbSp4CUlGPvM7+u+ZGTu0hEbVKKNjqMfXhuY8A4/BWiV2mSqKcEirR5cE/LqY mq5Ac7vbO4Uh+1xiRw/G9ITi1dqAjYIzhBawlhdUPOh+aINWTUikYYzjUSmm4PbN H9BTPLA9iAdVRfuWJ9um2G0DdS8Qpx/aIRh3MScXVly6cGCC6Do= =kgqn -----END PGP SIGNATURE----- Merge tag 'v1.26.0rc1' into develop Synapse 1.26.0rc1 (2021-01-20) ============================== This release brings a new schema version for Synapse and rolling back to a previous verious is not trivial. Please review [UPGRADE.rst](UPGRADE.rst) for more details on these changes and for general upgrade guidance. Features -------- - Add support for multiple SSO Identity Providers. ([\#9015](https://github.com/matrix-org/synapse/issues/9015), [\#9017](https://github.com/matrix-org/synapse/issues/9017), [\#9036](https://github.com/matrix-org/synapse/issues/9036), [\#9067](https://github.com/matrix-org/synapse/issues/9067), [\#9081](https://github.com/matrix-org/synapse/issues/9081), [\#9082](https://github.com/matrix-org/synapse/issues/9082), [\#9105](https://github.com/matrix-org/synapse/issues/9105), [\#9107](https://github.com/matrix-org/synapse/issues/9107), [\#9109](https://github.com/matrix-org/synapse/issues/9109), [\#9110](https://github.com/matrix-org/synapse/issues/9110), [\#9127](https://github.com/matrix-org/synapse/issues/9127), [\#9153](https://github.com/matrix-org/synapse/issues/9153), [\#9154](https://github.com/matrix-org/synapse/issues/9154), [\#9177](https://github.com/matrix-org/synapse/issues/9177)) - During user-interactive authentication via single-sign-on, give a better error if the user uses the wrong account on the SSO IdP. ([\#9091](https://github.com/matrix-org/synapse/issues/9091)) - Give the `public_baseurl` a default value, if it is not explicitly set in the configuration file. ([\#9159](https://github.com/matrix-org/synapse/issues/9159)) - Improve performance when calculating ignored users in large rooms. ([\#9024](https://github.com/matrix-org/synapse/issues/9024)) - Implement [MSC2176](https://github.com/matrix-org/matrix-doc/pull/2176) in an experimental room version. ([\#8984](https://github.com/matrix-org/synapse/issues/8984)) - Add an admin API for protecting local media from quarantine. ([\#9086](https://github.com/matrix-org/synapse/issues/9086)) - Remove a user's avatar URL and display name when deactivated with the Admin API. ([\#8932](https://github.com/matrix-org/synapse/issues/8932)) - Update `/_synapse/admin/v1/users/<user_id>/joined_rooms` to work for both local and remote users. ([\#8948](https://github.com/matrix-org/synapse/issues/8948)) - Add experimental support for handling to-device messages on worker processes. ([\#9042](https://github.com/matrix-org/synapse/issues/9042), [\#9043](https://github.com/matrix-org/synapse/issues/9043), [\#9044](https://github.com/matrix-org/synapse/issues/9044), [\#9130](https://github.com/matrix-org/synapse/issues/9130)) - Add experimental support for handling `/keys/claim` and `/room_keys` APIs on worker processes. ([\#9068](https://github.com/matrix-org/synapse/issues/9068)) - Add experimental support for handling `/devices` API on worker processes. ([\#9092](https://github.com/matrix-org/synapse/issues/9092)) - Add experimental support for moving off receipts and account data persistence off master. ([\#9104](https://github.com/matrix-org/synapse/issues/9104), [\#9166](https://github.com/matrix-org/synapse/issues/9166)) Bugfixes -------- - Fix a long-standing issue where an internal server error would occur when requesting a profile over federation that did not include a display name / avatar URL. ([\#9023](https://github.com/matrix-org/synapse/issues/9023)) - Fix a long-standing bug where some caches could grow larger than configured. ([\#9028](https://github.com/matrix-org/synapse/issues/9028)) - Fix error handling during insertion of client IPs into the database. ([\#9051](https://github.com/matrix-org/synapse/issues/9051)) - Fix bug where we didn't correctly record CPU time spent in `on_new_event` block. ([\#9053](https://github.com/matrix-org/synapse/issues/9053)) - Fix a minor bug which could cause confusing error messages from invalid configurations. ([\#9054](https://github.com/matrix-org/synapse/issues/9054)) - Fix incorrect exit code when there is an error at startup. ([\#9059](https://github.com/matrix-org/synapse/issues/9059)) - Fix `JSONDecodeError` spamming the logs when sending transactions to remote servers. ([\#9070](https://github.com/matrix-org/synapse/issues/9070)) - Fix "Failed to send request" errors when a client provides an invalid room alias. ([\#9071](https://github.com/matrix-org/synapse/issues/9071)) - Fix bugs in federation catchup logic that caused outbound federation to be delayed for large servers after start up. Introduced in v1.8.0 and v1.21.0. ([\#9114](https://github.com/matrix-org/synapse/issues/9114), [\#9116](https://github.com/matrix-org/synapse/issues/9116)) - Fix corruption of `pushers` data when a postgres bouncer is used. ([\#9117](https://github.com/matrix-org/synapse/issues/9117)) - Fix minor bugs in handling the `clientRedirectUrl` parameter for SSO login. ([\#9128](https://github.com/matrix-org/synapse/issues/9128)) - Fix "Unhandled error in Deferred: BodyExceededMaxSize" errors when .well-known files that are too large. ([\#9108](https://github.com/matrix-org/synapse/issues/9108)) - Fix "UnboundLocalError: local variable 'length' referenced before assignment" errors when the response body exceeds the expected size. This bug was introduced in v1.25.0. ([\#9145](https://github.com/matrix-org/synapse/issues/9145)) - Fix a long-standing bug "ValueError: invalid literal for int() with base 10" when `/publicRooms` is requested with an invalid `server` parameter. ([\#9161](https://github.com/matrix-org/synapse/issues/9161)) Improved Documentation ---------------------- - Add some extra docs for getting Synapse running on macOS. ([\#8997](https://github.com/matrix-org/synapse/issues/8997)) - Correct a typo in the `systemd-with-workers` documentation. ([\#9035](https://github.com/matrix-org/synapse/issues/9035)) - Correct a typo in `INSTALL.md`. ([\#9040](https://github.com/matrix-org/synapse/issues/9040)) - Add missing `user_mapping_provider` configuration to the Keycloak OIDC example. Contributed by @chris-ruecker. ([\#9057](https://github.com/matrix-org/synapse/issues/9057)) - Quote `pip install` packages when extras are used to avoid shells interpreting bracket characters. ([\#9151](https://github.com/matrix-org/synapse/issues/9151)) Deprecations and Removals ------------------------- - Remove broken and unmaintained `demo/webserver.py` script. ([\#9039](https://github.com/matrix-org/synapse/issues/9039)) Internal Changes ---------------- - Improve efficiency of large state resolutions. ([\#8868](https://github.com/matrix-org/synapse/issues/8868), [\#9029](https://github.com/matrix-org/synapse/issues/9029), [\#9115](https://github.com/matrix-org/synapse/issues/9115), [\#9118](https://github.com/matrix-org/synapse/issues/9118), [\#9124](https://github.com/matrix-org/synapse/issues/9124)) - Various clean-ups to the structured logging and logging context code. ([\#8939](https://github.com/matrix-org/synapse/issues/8939)) - Ensure rejected events get added to some metadata tables. ([\#9016](https://github.com/matrix-org/synapse/issues/9016)) - Ignore date-rotated homeserver logs saved to disk. ([\#9018](https://github.com/matrix-org/synapse/issues/9018)) - Remove an unused column from `access_tokens` table. ([\#9025](https://github.com/matrix-org/synapse/issues/9025)) - Add a `-noextras` factor to `tox.ini`, to support running the tests with no optional dependencies. ([\#9030](https://github.com/matrix-org/synapse/issues/9030)) - Fix running unit tests when optional dependencies are not installed. ([\#9031](https://github.com/matrix-org/synapse/issues/9031)) - Allow bumping schema version when using split out state database. ([\#9033](https://github.com/matrix-org/synapse/issues/9033)) - Configure the linters to run on a consistent set of files. ([\#9038](https://github.com/matrix-org/synapse/issues/9038)) - Various cleanups to device inbox store. ([\#9041](https://github.com/matrix-org/synapse/issues/9041)) - Drop unused database tables. ([\#9055](https://github.com/matrix-org/synapse/issues/9055)) - Remove unused `SynapseService` class. ([\#9058](https://github.com/matrix-org/synapse/issues/9058)) - Remove unnecessary declarations in the tests for the admin API. ([\#9063](https://github.com/matrix-org/synapse/issues/9063)) - Remove `SynapseRequest.get_user_agent`. ([\#9069](https://github.com/matrix-org/synapse/issues/9069)) - Remove redundant `Homeserver.get_ip_from_request` method. ([\#9080](https://github.com/matrix-org/synapse/issues/9080)) - Add type hints to media repository. ([\#9093](https://github.com/matrix-org/synapse/issues/9093)) - Fix the wrong arguments being passed to `BlacklistingAgentWrapper` from `MatrixFederationAgent`. Contributed by Timothy Leung. ([\#9098](https://github.com/matrix-org/synapse/issues/9098)) - Reduce the scope of caught exceptions in `BlacklistingAgentWrapper`. ([\#9106](https://github.com/matrix-org/synapse/issues/9106)) - Improve `UsernamePickerTestCase`. ([\#9112](https://github.com/matrix-org/synapse/issues/9112)) - Remove dependency on `distutils`. ([\#9125](https://github.com/matrix-org/synapse/issues/9125)) - Enforce that replication HTTP clients are called with keyword arguments only. ([\#9144](https://github.com/matrix-org/synapse/issues/9144)) - Fix the Python 3.5 / old dependencies build in CI. ([\#9146](https://github.com/matrix-org/synapse/issues/9146)) - Replace the old `perspectives` option in the Synapse docker config file template with `trusted_key_servers`. ([\#9157](https://github.com/matrix-org/synapse/issues/9157))	2021-01-20 11:27:39 -05:00
Richard van der Hoff	0cd2938bc8	Support icons for Identity Providers (#9154)	2021-01-20 08:15:14 -05:00
rht	a5b9c87ac6	docs: Add link to Matrix VoIP tester for turn-howto (#9135) ... Signed-off-by: rht <rhtbot@protonmail.com>	2021-01-20 12:41:57 +00:00
Richard van der Hoff	fa50e4bf4d	Give public_baseurl a default value (#9159)	2021-01-20 12:30:41 +00:00
Patrick Cloke	de45bf5b5b	Quote pip install with brackets to avoid shell interpretation. (#9151)	2021-01-18 11:12:20 -05:00
Matthew Hodgson	883d4e6f2b	link to the scalability blog post from workers.md	2021-01-18 00:27:27 +00:00
Richard van der Hoff	9de6b94117	Land support for multiple OIDC providers (#9110) ... This is the final step for supporting multiple OIDC providers concurrently. First of all, we reorganise the config so that you can specify a list of OIDC providers, instead of a single one. Before: oidc_config: enabled: true issuer: "https://oidc_provider" # etc After: oidc_providers: - idp_id: prov1 issuer: "https://oidc_provider" - idp_id: prov2 issuer: "https://another_oidc_provider" The old format is still grandfathered in. With that done, it's then simply a matter of having OidcHandler instantiate a new OidcProvider for each configured provider.	2021-01-15 16:55:29 +00:00
Patrick Cloke	3e4cdfe5d9	Add an admin API endpoint to protect media. (#9086) ... Protecting media stops it from being quarantined when e.g. all media in a room is quarantined. This is useful for sticker packs and other media that is uploaded by server administrators, but used by many people.	2021-01-15 11:18:09 -05:00
Richard van der Hoff	5310808d3b	Give the user a better error when they present bad SSO creds ... If a user tries to do UI Auth via SSO, but uses the wrong account on the SSO IdP, try to give them a better error. Previously, the UIA would claim to be successful, but then the operation in question would simply fail with "auth fail". Instead, serve up an error page which explains the failure.	2021-01-13 20:22:41 +00:00
Patrick Cloke	d1eb1b96e8	Register the /devices endpoint on workers. (#9092)	2021-01-13 12:35:40 -05:00
Dirk Klimpel	7a2e9b549d	Remove user's avatar URL and displayname when deactivated. (#8932) ... This only applies if the user's data is to be erased.	2021-01-12 16:30:15 -05:00
Jason Robinson	da16d06301	Address pr feedback ... * docs updates * prettify SQL * add missing copyright * cursor_to_dict * update touched files copyright years Signed-off-by: Jason Robinson <jasonr@matrix.org>	2021-01-11 23:43:58 +02:00
Jason Robinson	0b77329fe2	Clarify rooms.md ... Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>	2021-01-11 23:05:36 +02:00
David Teller	b161528fcc	Also support remote users on the joined_rooms admin API. (#8948) ... For remote users, only the rooms which the server knows about are returned. Local users have all of their joined rooms returned.	2021-01-11 14:32:17 -05:00
Erik Johnston	1315a2e8be	Use a chain cover index to efficiently calculate auth chain difference (#8868)	2021-01-11 16:09:22 +00:00
Jason Robinson	e2c16edc78	Add changelog and admin API docs ... Signed-off-by: Jason Robinson <jasonr@matrix.org>	2021-01-09 22:58:29 +02:00
Christopher Rücker	bce0c91d9a	Keycloak mapping_provider example (#9037) (#9057) ... This PR adds the missing user_mapping_provider section in oidc.md Signed-off-by: Christopher Rücker chris-ruecker@protonmail.com	2021-01-08 18:29:30 +00:00
Emelie	9066c2fd7f	Fix typo in docs/systemd-with-workers/README.md (#9035) ... Signed-off-by: Emelie em@nao.sh	2021-01-07 15:31:01 +00:00
Richard van der Hoff	111b673fc1	Add initial support for a "pick your IdP" page (#9017) ... During login, if there are multiple IdPs enabled, offer the user a choice of IdPs.	2021-01-05 11:25:28 +00:00
Jerin J Titus	cfcf5541b4	Update the value of group_creation_prefix in sample config. (#8992) ... Removes the trailing slash with causes issues with matrix.to/Element.	2020-12-29 09:30:48 -05:00
Patrick Cloke	68bb26da69	Allow redacting events on workers (#8994) ... Adds the redacts endpoint to workers that have the client listener.	2020-12-29 07:40:12 -05:00
Patrick Cloke	4218473f9e	Refactor the CAS handler in prep for using the abstracted SSO code. (#8958) ... This makes the CAS handler look more like the SAML/OIDC handlers: * Render errors to users instead of throwing JSON errors. * Internal reorganization.	2020-12-18 13:09:45 -05:00
Patrick Cloke	56e00ca85e	Send the location of the web client to the IS when inviting via 3PIDs. (#8930) ... Adds a new setting `email.invite_client_location` which, if defined, is passed to the identity server during invites.	2020-12-18 11:01:57 -05:00
Erik Johnston	d781a81e69	Allow server admin to get admin bit in rooms where local user is an admin (#8756) ... This adds an admin API that allows a server admin to get power in a room if a local user has power in a room. Will also invite the user if they're not in the room and its a private room. Can specify another user (rather than the admin user) to be granted power. Co-authored-by: Matthew Hodgson <matthew@matrix.org>	2020-12-18 15:37:19 +00:00
Richard van der Hoff	28877fade9	Implement a username picker for synapse (#8942) ... The final part (for now) of my work to implement a username picker in synapse itself. The idea is that we allow `UsernameMappingProvider`s to return `localpart=None`, in which case, rather than redirecting the browser back to the client, we redirect to a username-picker resource, which allows the user to enter a username. We *then* complete the SSO flow (including doing the client permission checks). The static resources for the username picker itself (in https://github.com/matrix-org/synapse/tree/rav/username_picker/synapse/res/username_picker) are essentially lifted wholesale from https://github.com/matrix-org/matrix-synapse-saml-mozilla/tree/master/matrix_synapse_saml_mozilla/res. As the comment says, we might want to think about making them customisable, but that can be a follow-up. Fixes #8876.	2020-12-18 14:19:46 +00:00
Patrick Cloke	5d4c330ed9	Allow re-using a UI auth validation for a period of time (#8970)	2020-12-18 07:33:57 -05:00
Dirk Klimpel	06006058d7	Make search statement in List Room and User Admin API case-insensitive (#8931)	2020-12-17 10:43:37 +00:00
Patrick Cloke	44b7d4c6d6	Fix the sample config location for the ip_range_whitelist setting. (#8954) ... Move it from the federation section to the server section to match ip_range_blacklist.	2020-12-16 14:40:47 -05:00
David Teller	f14428b25c	Allow spam-checker modules to be provide async methods. (#8890) ... Spam checker modules can now provide async methods. This is implemented in a backwards-compatible manner.	2020-12-11 14:05:15 -05:00
Dirk Klimpel	0a34cdfc66	Add number of local devices to Room Details Admin API (#8886)	2020-12-11 10:42:47 +00:00
Dirk Klimpel	a5f7aff5e5	Deprecate Shutdown Room and Purge Room Admin API (#8829) ... Deprecate both APIs in favour of the Delete Room API. Related: #8663 and #8810	2020-12-10 11:42:48 +00:00
Patrick Cloke	344ab0b53a	Default to blacklisting reserved IP ranges and add a whitelist. (#8870) ... This defaults `ip_range_blacklist` to reserved IP ranges and also adds an `ip_range_whitelist` setting to override it.	2020-12-09 13:56:06 -05:00
Dirk Klimpel	43bf3c5178	Combine related media admin API docs (#8839) ... Related: #8810 Also a few small improvements. Signed-off-by: Dirk Klimpel dirk@klimpel.org	2020-12-09 16:19:57 +00:00
Richard van der Hoff	025fa06fc7	Clarify config template comments (#8891)	2020-12-08 14:03:08 +00:00
Patrick Cloke	96358cb424	Add authentication to replication endpoints. (#8853) ... Authentication is done by checking a shared secret provided in the Synapse configuration file.	2020-12-04 10:56:28 -05:00
Patrick Cloke	112f6bd49e	Synapse 1.24.0rc2 (2020-12-04) ... ============================== Bugfixes -------- - Fix a regression in v1.24.0rc1 which failed to allow SAML mapping providers which were unable to redirect users to an additional page. ([\#8878](https://github.com/matrix-org/synapse/issues/8878)) Internal Changes ---------------- - Add support for the `prometheus_client` newer than 0.9.0. Contributed by Jordan Bancino. ([\#8875](https://github.com/matrix-org/synapse/issues/8875)) -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEF3tZXk38tRDFVnUIM/xY9qcRMEgFAl/KQ6MACgkQM/xY9qcR MEjqOg/9Fd4Ow0QdnK3O6J0bmXP6+g2GfjJCAKlYlF/nwX1ez9jZVed00D2h9/hD YCUJCEv+XZpbv5F4usiYwoLGithbueo7AKEOQJQMEuQBHU1/E6TJ9iiKR3lTBNNw ccuyMMqqEatYxoa4DIO/lrwm2mhwRNdMPt0r1DPlML13bb/TVbeXXykbOCyZSKm1 YuXpxYDngRPL9o46I77l7/KNTSpYYeZi2qwp2orl6GBfa0KSvR2Oi6uohKYzuoHC +XLvHSFMpHAZkaZsIMxU5hRVH6jtkb/D37CABuKQsdlvHcooGK4Sdt6aoUyhPtoh 6YwajSazWKEJ/9YO5JV22qkWFPi9T6s/lPZfjOSu/euwMw7DDQJ/8t8Sm6N7sJDS A/5FctOAd1jd9BiAaqkHB2zLHZSomIHQpJJfDyE3UgAGQQ6fm2Wg7SkLByDx/MYC 4D74dBuVBPENM2VxNoyjHbpVSeQi8t0RgWb44/PUAOtpwm3f3eVRuS8zY6uR2bZr YZUbJylqKhRW6dBdZH+EzVolRGvBgLkc27IzRCQ3kGUEo2Wem05s6MuKMvLxoIvN NHLWNClUWITT8FgkfFP3c6CK40dwIGJzYryI+71qPW/R7snf2fazuoHlH1pAmFh0 E0/nBMTw1deixOYXbG1ah63AhJ1NjdTfimDsKmWZLqVc6g7g4d4= =is1Y -----END PGP SIGNATURE----- Merge tag 'v1.24.0rc2' into develop Synapse 1.24.0rc2 (2020-12-04) ============================== Bugfixes -------- - Fix a regression in v1.24.0rc1 which failed to allow SAML mapping providers which were unable to redirect users to an additional page. ([\#8878](https://github.com/matrix-org/synapse/issues/8878)) Internal Changes ---------------- - Add support for the `prometheus_client` newer than 0.9.0. Contributed by Jordan Bancino. ([\#8875](https://github.com/matrix-org/synapse/issues/8875))	2020-12-04 09:14:31 -05:00
Patrick Cloke	22c6c19f91	Fix a regression that mapping providers should be able to redirect users. (#8878) ... This was broken in #8801.	2020-12-04 08:25:15 -05:00
Richard van der Hoff	6e4f71c057	Fix a buglet in the SAML username mapping provider doc (#8873) ... the constructor is called with a `module_api`.	2020-12-04 10:14:15 +00:00
Patrick Cloke	30fba62108	Apply an IP range blacklist to push and key revocation requests. (#8821) ... Replaces the `federation_ip_range_blacklist` configuration setting with an `ip_range_blacklist` setting with wider scope. It now applies to: * Federation * Identity servers * Push notifications * Checking key validitity for third-party invite events The old `federation_ip_range_blacklist` setting is still honored if present, but with reduced scope (it only applies to federation and identity servers).	2020-12-02 11:09:24 -05:00
Andrew Morgan	d1be293f00	Fix typo in password_auth_providers doc ... A word got removed accidentally in 83434df381.	2020-12-01 10:34:52 +00:00
Andrew Morgan	17fa58bdd1	Add a config option to change whether unread push notification counts are per-message or per-room (#8820) ... This PR adds a new config option to the `push` section of the homeserver config, `group_unread_count_by_room`. By default Synapse will group push notifications by room (so if you have 1000 unread messages, if they lie in 55 rooms, you'll see an unread count on your phone of 55). However, it is also useful to be able to send out the true count of unread messages if desired. If `group_unread_count_by_room` is set to `false`, then with the above example, one would see an unread count of 1000 (email anyone?).	2020-11-30 18:43:54 +00:00
Richard van der Hoff	a090b86209	Add force_purge option to delete-room admin api. (#8843)	2020-11-30 16:48:12 +00:00
Dirk Klimpel	14f81a6d24	Improve documentation how to configure prometheus for workers (#8822)	2020-11-26 10:42:55 +00:00
Dirk Klimpel	3f0ff53158	Remove deprecated /_matrix/client/*/admin endpoints (#8785) ... These are now only available via `/_synapse/admin/v1`.	2020-11-25 16:26:11 -05:00
Andrew Morgan	2b110dda2a	Fix the formatting of push config section (#8818) ... This PR updates the push config's formatting to better align with our [code style guidelines](https://github.com/matrix-org/synapse/blob/develop/docs/code_style.md#configuration-file-format).	2020-11-25 21:02:53 +00:00
Patrick Cloke	4fd222ad70	Support trying multiple localparts for OpenID Connect. (#8801) ... Abstracts the SAML and OpenID Connect code which attempts to regenerate the localpart of a matrix ID if it is already in use.	2020-11-25 10:04:22 -05:00
Dirk Klimpel	b08dc7effe	Clarify documentation of the admin list media API (#8795) ... Clarify that the list media API only shows media from unencrypted events.	2020-11-24 09:04:51 -05:00
Richard van der Hoff	e3d7806704	Update turn-howto (#8779) ... Some hopefully-useful notes on setting up a turnserver.	2020-11-24 12:52:22 +00:00
Patrick Cloke	79bfe966e0	Improve error checking for OIDC/SAML mapping providers (#8774) ... Checks that the localpart returned by mapping providers for SAML and OIDC are valid before registering new users. Extends the OIDC tests for existing users and invalid data.	2020-11-19 14:25:17 -05:00
Ben Banfield-Zanin	53a6f5ddf0	SAML: Allow specifying the IdP entityid to use. (#8630) ... If the SAML metadata includes multiple IdPs it is necessary to specify which IdP to redirect users to for authentication.	2020-11-19 09:57:13 -05:00
Marcus Schopen	d356588339	SAML: Document allowing a clock/time difference from IdP (#8731) ... Updates the sample configuration with the pysaml2 configuration for accepting clock skew/drift between the homeserver and IdP.	2020-11-18 07:36:28 -05:00
chagai95	e487d9fabc	a comma too much (#8771) ... Signed-off-by: Chagai Friedlander chagai95@gmail.com	2020-11-17 14:13:56 +00:00
Erik Johnston	f737368a26	Add admin API for logging in as a user (#8617)	2020-11-17 10:51:25 +00:00
Adrian Wannenmacher	f1de4bb58b	Clarify the usecase for an msisdn delegate (#8734) ... Signed-off-by: Adrian Wannenmacher <tfld@tfld.dev>	2020-11-14 23:09:36 +00:00
Dirk Klimpel	023f791143	Migrate documentation docs/admin_api/event_reports to markdown (#8742) ... Related to #8714. `event_reports.rst` was introduced in Synapse 1.21.0.	2020-11-13 13:57:55 +00:00
Marcus Schopen	68fc0dcb5a	SAML: add <mdui:UIInfo> element examples (#8718) ... add some mdui:UIInfo element examples for saml2_config in homeserver.yaml	2020-11-13 12:07:50 +00:00
Marcus Schopen	c059413001	Notes on SSO logins and media_repository worker (#8701) ... If SSO login is used (e.g. SAML) in a multi worker setup, it should be mentioned that currently all SAML logins must run on the same worker, see https://github.com/matrix-org/synapse/issues/7530 Also, if you are using different ports (for example 443 and 8448) in a reverse proxy for client and federation, the path `/_matrix/media` on the client and federation port must point to the listener of the `media_repository` worker, otherwise you'll get a 404 on the federation port for the path `/_matrix/media`, if a remote server is trying to get the media object on federation port, see https://github.com/matrix-org/synapse/issues/8695	2020-11-06 14:33:07 +00:00
Dirk Klimpel	c3119d1536	Add an admin API for users' media statistics (#8700) ... Add `GET /_synapse/admin/v1/statistics/users/media` to get statisics about local media usage by users. Related to #6094 It is the first API for statistics. Goal is to avoid/reduce usage of sql queries like [Wiki analyzing Synapse](https://github.com/matrix-org/synapse/wiki/SQL-for-analyzing-Synapse-PostgreSQL-database-stats) Signed-off-by: Dirk Klimpel dirk@klimpel.org	2020-11-05 18:59:12 +00:00
Dirk Klimpel	e4676bd877	Add displayname to Shared-Secret Registration for admins (#8722) ... Add `displayname` to Shared-Secret Registration for admins to `POST /_synapse/admin/v1/register`	2020-11-05 13:55:45 +00:00
Dirk Klimpel	4fda58ddd2	Remove the "draft" status of the Room Details Admin API (#8702) ... Fixes #8550	2020-11-03 12:48:25 +00:00
Erik Johnston	4b09b7438e	Document how to set up multiple event persisters (#8706)	2020-11-03 10:27:11 +00:00
Matthew Hodgson	d04c2d19b3	grammar	2020-11-02 21:22:36 +00:00
Matthew Hodgson	11fd90a2b7	typo	2020-11-02 13:33:56 +00:00
Andrew Morgan	26b46796ea	Fix typos in systemd-with-workers doc	2020-11-02 12:56:16 +00:00
Andrew Morgan	305545682d	Fix typo in workers doc	2020-11-02 12:36:18 +00:00
Patrick Cloke	8f1aefa694	Improve the sample config for SSO (OIDC, SAML, and CAS). (#8635)	2020-10-30 10:01:59 -04:00
Patrick Cloke	00b24aa545	Support generating structured logs in addition to standard logs. (#8607) ... This modifies the configuration of structured logging to be usable from the standard Python logging configuration. This also separates the formatting of logs from the transport allowing JSON logs to files or standard logs to sockets.	2020-10-29 07:27:37 -04:00
Dirk Klimpel	2239813278	Add an admin APIs to allow server admins to list users' pushers (#8610) ... Add an admin API `GET /_synapse/admin/v1/users/<user_id>/pushers` like https://matrix.org/docs/spec/client_server/latest#get-matrix-client-r0-pushers	2020-10-28 15:02:42 +00:00
Michael Kaye	f49c2093b5	Cross-link documentation to the prometheus recording rules. (#8667)	2020-10-27 15:29:50 -04:00
Dirk Klimpel	9b7c28283a	Add admin API to list users' local media (#8647) ... Add admin API `GET /_synapse/admin/v1/users/<user_id>/media` to get information of users' uploaded files.	2020-10-27 14:12:31 +00:00
Dirk Klimpel	66e6801c3e	Split admin API for reported events into a detail and a list view (#8539) ... Split admin API for reported events in detail und list view. API was introduced with #8217 in synapse v.1.21.0. It makes the list (`GET /_synapse/admin/v1/event_reports`) less complex and provides a better overview. The details can be queried with: `GET /_synapse/admin/v1/event_reports/<report_id>`. It is similar to room and users API. It is a kind of regression in `GET /_synapse/admin/v1/event_reports`. `event_json` was removed. But the api was introduced one version before and it is an admin API (not under spec). Signed-off-by: Dirk Klimpel dirk@klimpel.org	2020-10-26 18:16:37 +00:00
Peter Krantz	6c9ab61df5	Added basic instructions for Azure AD to OpenId documentation (#8582) ... Signed-off-by: Peter Krantz peter.krantz@gmail.com	2020-10-26 17:49:55 +00:00
Dirk Klimpel	49d72dea2a	Add an admin api to delete local media. (#8519) ... Related to: #6459, #3479 Add `DELETE /_synapse/admin/v1/media/<server_name>/<media_id>` to delete a single file from server.	2020-10-26 17:02:28 +00:00
Andrew Morgan	f6a3859a73	Fix filepath of Dex example config (#8657)	2020-10-26 16:53:11 +00:00
Andrew Morgan	73d8209694	Correct the package name in OpenID Connect install instructions (#8634) ... The OpenID Connect install instructions suggested installing `synapse[oidc]`, but our PyPI package is called `matrix-synapse`.	2020-10-26 14:45:33 +00:00
Dirk Klimpel	913f8a06e4	Add field total to device list in admin API (#8644)	2020-10-26 14:07:51 +00:00
Patrick Cloke	34a5696f93	Fix typos and spelling errors. (#8639)	2020-10-23 12:38:40 -04:00
Andrew Morgan	4fb7a68a65	Correct the package name in authlib install instructions	2020-10-22 18:25:58 +01:00
Christopher May-Townsend	1cf4a68108	Add note to manhole.md about bind_address when using with docker (#8526) ... Signed-off-by: Christopher May-Townsend <chris@maytownsend.co.uk>	2020-10-14 15:28:59 +01:00
Brendan Abolivier	9e66f3761c	Update documentation on retention policies limits (#8529) ... * Update documentation on retention policies limits Document the changes from https://github.com/matrix-org/synapse/pull/8104	2020-10-14 15:00:49 +01:00
Erik Johnston	8de3703d21	Make event persisters periodically announce position over replication. (#8499) ... Currently background proccesses stream the events stream use the "minimum persisted position" (i.e. `get_current_token()`) rather than the vector clock style tokens. This is broadly fine as it doesn't matter if the background processes lag a small amount. However, in extreme cases (i.e. SyTests) where we only write to one event persister the background processes will never make progress. This PR changes it so that the `MultiWriterIDGenerator` keeps the current position of a given instance as up to date as possible (i.e using the latest token it sees if its not in the process of persisting anything), and then periodically announces that over replication. This then allows the "minimum persisted position" to advance, albeit with a small lag.	2020-10-12 15:51:41 +01:00
Mateusz Przybyłowicz	ca2db5dd0c	Increase default max_upload_size from 10M to 50M (#8502) ... Signed-off-by: Mateusz Przybyłowicz <uamfhq@gmail.com>	2020-10-09 16:58:23 +01:00
Richard van der Hoff	4f0637346a	Combine SpamCheckerApi with the more generic ModuleApi. (#8464) ... Lots of different module apis is not easy to maintain. Rather than adding yet another ModuleApi(hs, hs.get_auth_handler()) incantation, first add an hs.get_module_api() method and use it where possible.	2020-10-07 12:03:26 +01:00
Andrew Morgan	01f82bfe32	Remove docs/sphinx and related references (#8480) ... https://github.com/matrix-org/synapse/tree/develop/docs/sphinx doesn't seem to really be utilised or changed recently since the initial commit. I like the idea of exportable documentation of the codebase, but at the moment after running through the build instructions the generated website wasn't very useful...	2020-10-07 11:45:31 +01:00
Richard van der Hoff	785437dc0d	Update default room version to 6 (#8461) ... Per https://github.com/matrix-org/matrix-doc/pull/2788	2020-10-05 21:40:51 +01:00
Patrick Cloke	f64c6aae68	Update manhole documentation for async/await. (#8462)	2020-10-05 09:40:19 -04:00
Patrick Cloke	62894673e6	Allow background tasks to be run on a separate worker. (#8369)	2020-10-02 08:23:15 -04:00
BBBSnowball	05ee048f2c	Add config option for always using "userinfo endpoint" for OIDC (#7658) ... This allows for connecting to certain IdPs, e.g. GitLab.	2020-10-01 13:54:35 -04:00
Richard van der Hoff	c1ef579b63	Add prometheus metrics to track federation delays (#8430) ... Add a pair of federation metrics to track the delays in sending PDUs to/from particular servers.	2020-10-01 11:09:12 +01:00
Patrick Cloke	8b40843392	Allow additional SSO properties to be passed to the client (#8413)	2020-09-30 13:02:43 -04:00
Aaron Raimist	8238b55e08	Update description of server_name config option (#8415)	2020-09-29 13:50:25 -04:00
Erik Johnston	bd380d942f	Add checks for postgres sequence consistency (#8402)	2020-09-28 18:00:30 +01:00
Tdxdxoz	abd04b6af0	Allow existing users to login via OpenID Connect. (#8345) ... Co-authored-by: Benjamin Koch <bbbsnowball@gmail.com> This adds configuration flags that will match a user to pre-existing users when logging in via OpenID Connect. This is useful when switching to an existing SSO system.	2020-09-25 07:01:45 -04:00
Julian Fietkau	a4e63e5a47	Add note to reverse_proxy.md about disabling Apache's mod_security2 (#8375) ... This change adds a note and a few lines of configuration settings for Apache users to disable ModSecurity for Synapse's virtual hosts. With ModSecurity enabled and running with its default settings, Matrix clients are unable to send chat messages through the Synapse installation. With this change, ModSecurity can be disabled only for the Synapse virtual hosts.	2020-09-23 11:14:08 +01:00
Dirk Klimpel	4da01f9c61	Admin API for reported events (#8217) ... Add an admin API to read entries of table `event_reports`. API: `GET /_synapse/admin/v1/event_reports`	2020-09-22 18:15:04 +01:00
Dirk Klimpel	d688b4bafc	Admin API for querying rooms where a user is a member (#8306) ... Add a new admin API `GET /_synapse/admin/v1/users/<user_id>/joined_rooms` to list all rooms where a user is a member.	2020-09-18 15:26:36 +01:00
Tulir Asokan	b82d68c0bd	Add the topic and avatar to the room details admin API (#8305)	2020-09-14 10:07:04 -04:00
Patrick Cloke	6605470bfb	Improve SAML error messages (#8248)	2020-09-14 09:05:36 -04:00
Patrick Cloke	a9dbe98ef9	Synapse 1.20.0rc3 (2020-09-11) ... ============================== Bugfixes -------- - Fix a bug introduced in v1.20.0rc1 where the wrong exception was raised when invalid JSON data is encountered. ([\#8291](https://github.com/matrix-org/synapse/issues/8291)) -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEF3tZXk38tRDFVnUIM/xY9qcRMEgFAl9bbA8ACgkQM/xY9qcR MEjjJg//ZGcvPLr8y3F5JXptcjWA3AFja3DgztBA3uNFoDrwVhR9m3+F4rL/Fbat gnaccWsKIXOjZw5WOThQEUIfYMKmHEsaDl7xtlUPC4ylmSV+31ypu1KiRSxND2kT TmOupvqw+4/E4RajpYX6WT3e2oQUpIcBAKsUyZL3ZXECR/yNYrA6w2w7i4wrv3cU QDzBFrCcr5aJql7VUc88BlaZUG6xom2/kkPtjmO6imPnPGBHLN22uPU7zQ4n7Vgi Nrg45v5WHVCx57WoXqAEap1zdKgoRCna1x0NkqYh0OAXq1l6aVwlf/Pdynt91f2x yyaMYjMskjyatHlzONMV4kz0w03dUrGJXiAx2ldEDd32SKBUxLJ/CYtNEdoXZ4Mm o0OJfDbaSJwlqK7FhdZJE3kCSkUvlmVasDErXfQRDVHy5sx9Ufr4P0kkaZyXDys1 UAWFcCJwy9dgFR2679PoXk1s/gHf10wuk6FOs8ESbcJJOvCljxWrOIqKNzSUL3HQ Hj5V1zHYXB6HtgiI7tbKxSn/d2uajHI9b8LVOkyV9RKsb1DGGRgNQQ5Kz4Zud7dC vuQ2jVw7JIo41W3QIXvcL1KdQeje2nwYwuwNREZaEDZJsGLjxaJ6dNyBVgkmYYFP gxhplSOSsFjQRGEsBrc2utWxDnLVGPHEAt7iHVSqjwVUyARvxjo= =e2T1 -----END PGP SIGNATURE----- Merge tag 'v1.20.0rc3' into develop Synapse 1.20.0rc3 (2020-09-11) ============================== Bugfixes -------- - Fix a bug introduced in v1.20.0rc1 where the wrong exception was raised when invalid JSON data is encountered. ([\#8291](https://github.com/matrix-org/synapse/issues/8291))	2020-09-11 08:30:36 -04:00
Andrew Morgan	95d869c357	Add /_synapse/client to the reverse proxy docs (#8227) ... This PR adds a information about forwarding `/_synapse/client` endpoints through your reverse proxy. The first of these endpoints are introduced in https://github.com/matrix-org/synapse/pull/8004.	2020-09-10 13:26:34 +01:00
Andrew Morgan	192e98111d	Remove shared rooms info from upgrade/workers doc as it's still experimental (#8290)	2020-09-10 13:08:08 +01:00
Andrew Morgan	a3a90ee031	Show a confirmation page during user password reset (#8004) ... This PR adds a confirmation step to resetting your user password between clicking the link in your email and your password actually being reset. This is to better align our password reset flow with the industry standard of requiring a confirmation from the user after email validation.	2020-09-10 11:45:12 +01:00
Andrew Morgan	094896a69d	Add a config option for validating 'next_link' parameters against a domain whitelist (#8275) ... This is a config option ported over from DINUM's Sydent: https://github.com/matrix-org/sydent/pull/285 They've switched to validating 3PIDs via Synapse rather than Sydent, and would like to retain this functionality. This original purpose for this change is phishing prevention. This solution could also potentially be replaced by a similar one to https://github.com/matrix-org/synapse/pull/8004, but across all `*/submit_token` endpoint. This option may still be useful to enterprise even with that safeguard in place though, if they want to be absolutely sure that their employees don't follow links to other domains.	2020-09-08 16:03:09 +01:00
Richard van der Hoff	ad28030c12	Systemd docs: configure workers to start after main process. (#8276)	2020-09-08 10:57:43 +01:00
Will Hunt	b257c788c0	Add /user/{user_id}/shared_rooms/ api (#7785) ... * Add shared_rooms api * Add changelog * Add . * Wrap response in {"rooms": } * linting * Add unstable_features key * Remove options from isort that aren't part of 5.x `-y` and `-rc` are now default behaviour and no longer exist. `dont-skip` is no longer required https://timothycrosley.github.io/isort/CHANGELOG/#500-penny-july-4-2020 * Update imports to make isort happy * Add changelog * Update tox.ini file with correct invocation * fix linting again for isort * Vendor prefix unstable API * Fix to match spec * import Codes * import Codes * Use FORBIDDEN * Update changelog.d/7785.feature Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> * Implement get_shared_rooms_for_users * a comma * trailing whitespace * Handle the easy feedback * Switch to using runInteraction * Add tests * Feedback * Seperate unstable endpoint from v2 * Add upgrade node * a line * Fix style by adding a blank line at EOF. * Update synapse/storage/databases/main/user_directory.py Co-authored-by: Tulir Asokan <tulir@maunium.net> * Update synapse/storage/databases/main/user_directory.py Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> * Update UPGRADE.rst Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> * Fix UPGRADE/CHANGELOG unstable paths unstable unstable unstable Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> Co-authored-by: Tulir Asokan <tulir@maunium.net> Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com> Co-authored-by: Tulir Asokan <tulir@maunium.net>	2020-09-02 13:18:40 +01:00
Brendan Abolivier	b5133dd97f	Explain better what GDPR-erased means (#8189) ... Fixes https://github.com/matrix-org/synapse/issues/8185	2020-09-01 16:31:59 +01:00
Andrew Morgan	74bf8d4d06	Wording fixes to 'name' user admin api filter (#8163) ... Some fixes to wording I noticed after merging #7377.	2020-08-25 15:03:24 +01:00
Andrew Morgan	79ac619403	Fix missing double-backtick in RST document	2020-08-25 14:24:06 +01:00
Manuel Stahl	97962ad17b	Search in columns 'name' and 'displayname' in the admin users endpoint (#7377) ... * Search in columns 'name' and 'displayname' in the admin users endpoint Signed-off-by: Manuel Stahl <manuel.stahl@awesome-technologies.de>	2020-08-25 14:18:14 +01:00
Brendan Abolivier	420484a334	Allow capping a room's retention policy (#8104)	2020-08-24 18:21:04 +01:00
Ryan Cole	cfeb37f039	Updated docs: Added note about missing 308 redirect support. (#8120) ... * Updated docs: Added note about missing 308 redirect support. * Added changelog	2020-08-19 12:26:50 +01:00
Patrick Cloke	acfb7c3b5d	Add a link to the matrix-synapse-rest-password-provider. (#8111)	2020-08-18 09:54:35 -04:00
Andrew Morgan	e04e465b4d	Use the default templates when a custom template file cannot be found (#8037) ... Fixes https://github.com/matrix-org/synapse/issues/6583	2020-08-17 17:05:00 +01:00
Erik Johnston	0304ad0c3d	Move setting of Filter into code. ... We do this to prevent foot guns. The default config uses a MemoryFilter, but users are free to change to logging to files directly. If they do then they have to ensure to set the `filters: [context]` on the right handler, otherwise records get written with the wrong context. Instead we move the logic to happen when we generate a record, which is when we *log* rather than *handle*. (It's possible to add filters to loggers in the config, however they don't apply to descendant loggers and so they have to be manually set on *every* logger used in the code base)	2020-08-11 18:10:46 +01:00
Erik Johnston	db131b6b22	Change the default log config to reduce disk I/O and storage (#8040) ... * Change default log config to buffer by default. This batches up writes to the filesystem, which is more efficient for disk I/O. This means that it can take some time for logs to get written to disk. Note that ERROR logs (and above) immediately flush the buffer. This only effects new installs, as we only write the log config if started with `--generate-config` (in the same way we do for generating signing keys). * Default to keeping last 4 days of logs. This hopefully reduces the amount of logs kept for new servers. Keeping the last 1GB of logs is likely overkill for new servers, but equally may not be enough for busy ones. Instead, we keep the last four days worth of logs, enough so that admins can investigate any problems that happened over e.g. a long weekend.	2020-08-11 18:09:46 +01:00
Richard van der Hoff	0cb169900e	Implement login blocking based on SAML attributes (#8052) ... Hopefully this mostly speaks for itself. I also did a bit of cleaning up of the error handling. Fixes #8047	2020-08-11 16:08:10 +01:00
Travis Ralston	1048ed2afa	Clarify that undoing a shutdown might not be possible (#8010)	2020-08-07 17:16:24 +01:00
Erik Johnston	7620912d84	Add health check endpoint (#8048)	2020-08-07 14:21:24 +01:00
Erik Johnston	079bc3c8e3	Fixup worker doc (again) (#8000)	2020-08-06 10:35:59 +01:00
Erik Johnston	a7bdf98d01	Rename database classes to make some sense (#8033)	2020-08-05 21:38:57 +01:00
Erik Johnston	faba873d4b	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/add_rate_limiting_to_joins	2020-07-31 15:07:01 +01:00
Erik Johnston	18de00adb4	Add ratelimiting on joins	2020-07-31 15:06:56 +01:00
Travis Ralston	e2a4ba6f9b	Add docs for undoing room shutdowns (#7998) ... These docs were tested successfully in production by a customer, so it's probably fine.	2020-07-31 04:41:44 +01:00
Stuart Mumford	6d4b790021	Update workers docs (#7990)	2020-07-30 17:30:11 +01:00
Erik Johnston	606805bf06	Fix typo in docs/workers.md (#7992)	2020-07-30 16:28:36 +01:00
Olivier Wilkinson (reivilibre)	3aa36b782c	Merge branch 'master' into develop	2020-07-30 15:18:36 +01:00
Erik Johnston	2c1b9d6763	Update worker docs with recent enhancements (#7969)	2020-07-29 23:22:13 +01:00
Aaron Raimist	2184f61fae	Various improvements to the docs (#7899)	2020-07-29 10:35:44 -04:00
Dirk Klimpel	e866e3b896	Add an option to disable purge in delete room admin API (#7964) ... Add option ```purge``` to ```POST /_synapse/admin/v1/rooms/<room_id>/delete``` Fixes: #3761 Signed-off-by: Dirk Klimpel dirk@klimpel.org	2020-07-28 20:08:23 +01:00
lugino-emeritus	3857de2194	Option to allow server admins to join complex rooms (#7902) ... Fixes #7901. Signed-off-by: Niklas Tittjung <nik_t.01@web.de>	2020-07-28 13:41:44 +01:00
Erik Johnston	aaf9ce72a0	Fix typo in metrics docs (#7966)	2020-07-28 10:03:18 +01:00
Patrick Cloke	83434df381	Update the auth providers to be async. (#7935)	2020-07-23 15:45:39 -04:00
Brendan Abolivier	55f2617f8c	Update the dates for ACME v1 EOL ... As per https://community.letsencrypt.org/t/end-of-life-plan-for-acmev1/88430	2020-07-22 16:18:40 +01:00
Adrian	64d2280299	Fix a typo in the sample config. (#7890)	2020-07-20 13:42:52 -04:00
Andrew Morgan	5ecf98f59e	Change sample config's postgres user to synapse_user (#7889) ... The [postgres setup docs](https://github.com/matrix-org/synapse/blob/develop/docs/postgres.md#set-up-database) recommend setting up your database with user `synapse_user`. However, uncommenting the postgres defaults in the sample config leave you with user `synapse`. This PR switches the sample config to recommend `synapse_user`. Took a me a second to figure this out, so assume this will beneficial to others.	2020-07-20 18:29:25 +01:00
Patrick Cloke	852930add7	Add a default limit (of 100) to get/sync operations. (#7858)	2020-07-17 07:59:23 -04:00
Michael Albert	fff483ea96	Add admin endpoint to get members in a room. (#7842)	2020-07-16 16:43:23 -04:00
Luke Faraone	b0f031f92a	Combine nginx federation server blocks (#7823) ... I'm pretty sure there's no technical reason these have to be distinct server blocks, so collapse into one and go with the more terse location block. Signed-off-by: Luke W Faraone <luke@faraone.cc>	2020-07-16 16:01:45 +01:00
Patrick Cloke	8c7d0f163d	Allow accounts to be re-activated from the admin APIs. (#7847)	2020-07-15 11:00:21 -04:00
Patrick Cloke	111e70d75c	Return the proper 403 Forbidden error during errors with JWT logins. (#7844)	2020-07-15 07:10:21 -04:00
Brendan Abolivier	85223106f3	Allow email subjects to be customised through Synapse's configuration (#7846)	2020-07-14 19:10:42 +01:00
Dirk Klimpel	491f0dab1b	Add delete room admin endpoint (#7613) ... The Delete Room admin API allows server admins to remove rooms from server and block these rooms. `DELETE /_synapse/admin/v1/rooms/<room_id>` It is a combination and improvement of "[Shutdown room](https://github.com/matrix-org/synapse/blob/develop/docs/admin_api/shutdown_room.md)" and "[Purge room](https://github.com/matrix-org/synapse/blob/develop/docs/admin_api/purge_room.md)" API. Fixes: #6425 It also fixes a bug in [synapse/storage/data_stores/main/room.py](synapse/storage/data_stores/main/room.py) in ` get_room_with_stats`. It should return `None` if the room is unknown. But it returns an `IndexError`. 901b1fa561/synapse/storage/data_stores/main/room.py (L99-L105) Related to: - #5575 - https://github.com/Awesome-Technologies/synapse-admin/issues/17 Signed-off-by: Dirk Klimpel dirk@klimpel.org	2020-07-14 12:36:23 +01:00
Patrick Cloke	77d2c05410	Add the option to validate the iss and aud claims for JWT logins. (#7827)	2020-07-14 07:16:43 -04:00
Erik Johnston	f299441cc6	Add ability to shard the federation sender (#7798)	2020-07-10 18:26:36 +01:00
Nicolai Søborg	96bb01d8ec	Change Caddy links (old is deprecated) (#7789) ... * Change Caddy links Current links points to Caddy v1 which is deprecated. Signed-off-by: Nicolai Søborg <git@xn--sb-lka.org>	2020-07-08 10:09:16 +01:00
Patrick Cloke	2a266f4511	Add documentation for JWT login type and improve sample config. (#7776)	2020-07-06 08:31:51 -04:00
Patrick Cloke	71cccf1593	Additional configuration options for auto-join rooms (#7763)	2020-06-30 15:41:36 -04:00
Erik Johnston	b44bdd7f7b	Support running multiple media repos. (#7706) ... This requires a new config option to specify which media repo should be responsible for running background jobs to e.g. clear out expired URL preview caches.	2020-06-17 14:13:30 +01:00
Richard van der Hoff	e452973fd2	fix broken link in sample config (#7712)	2020-06-16 19:50:16 +01:00
hungrymonkey	5c5516f80e	Add instructions for authing with Keycloak via OpenID (#7659)	2020-06-16 11:28:21 -04:00
Patrick Cloke	b9df7f70bb	Increase the default SAML session expirary time to 15 minutes. (#7664)	2020-06-11 07:55:45 -04:00
wondratsch	c746889bb0	fix typo in sample_config.yaml (#7652) ... Just a simple typo fix. Signed-off-by: wondratsch 28294257+wondratsch@users.noreply.github.com	2020-06-11 11:51:10 +01:00
Andrew Morgan	fcd6961441	Add option to enable encryption by default for new rooms (#7639) ... Fixes https://github.com/matrix-org/synapse/issues/2431 Adds config option `encryption_enabled_by_default_for_room_type`, which determines whether encryption should be enabled with the default encryption algorithm in private or public rooms upon creation. Whether the room is private or public is decided based upon the room creation preset that is used. Part of this PR is also pulling out all of the individual instances of `m.megolm.v1.aes-sha2` into a constant variable to eliminate typos ala https://github.com/matrix-org/synapse/pull/7637 Based on #7637	2020-06-10 17:44:34 +01:00
Travis Ralston	09099313e6	Add an option to disable autojoin for guest accounts (#6637) ... Fixes https://github.com/matrix-org/synapse/issues/3177	2020-06-05 18:18:15 +01:00
Richard van der Hoff	1bc00fd76d	Clarifications to the admin api documentation (#7647) ... * Clarify how to authenticate * path params are not the same thing as query params * Fix documentation for `/_synapse/admin/v2/users/<user_id>`	2020-06-05 17:31:05 +01:00
Dirk Klimpel	2970ce8367	Add device management to admin API (#7481) ... - Admin is able to - change displaynames - delete devices - list devices - get device informations Fixes #7330	2020-06-05 13:07:22 +01:00
Richard van der Hoff	11de843626	Cleanups to the OpenID Connect integration (#7628) ... docs, default configs, comments. Nothing very significant.	2020-06-03 21:13:17 +01:00
Richard van der Hoff	1bbc9e2df6	Clean up exception handling in SAML2ResponseResource (#7614) ... * Expose `return_html_error`, and allow it to take a Jinja2 template instead of a raw string * Clean up exception handling in SAML2ResponseResource * use the existing code in `return_html_error` instead of re-implementing it (giving it a jinja2 template rather than inventing a new form of template) * do the exception-catching in the REST layer rather than in the handler layer, to make sure we catch all exceptions.	2020-06-03 10:41:12 +01:00
Christopher Cooper	c4a820b32a	allow emails to be passed through SAML (#7385) ... Signed-off-by: Christopher Cooper <cooperc@ocf.berkeley.edu>	2020-05-27 17:40:08 +01:00
Jason Robinson	4be968d05d	Fix sample config docs error (#7581) ... 'client_auth_method' commented out value was erronously 'client_auth_basic', when code and docstring says it should be 'client_secret_basic'. Signed-off-by: Jason Robinson <jasonr@matrix.org>	2020-05-27 13:52:18 +01:00
Erik Johnston	d7d8a2e7ee	Fix up comments	2020-05-27 13:34:46 +01:00
Erik Johnston	4ba55559ac	Fix specifying cache factors via env vars with * in name. (#7580) ... This mostly applise to `*stateGroupCache*` and co. Broke in #6391.	2020-05-27 13:17:01 +01:00
Richard van der Hoff	66a564c859	Fix some DETECTED VIOLATIONS in the config file (#7550) ... consistency ftw	2020-05-22 10:11:50 +01:00
David Vo	d74cdc1a42	Ensure worker config exists in systemd service (#7528)	2020-05-21 13:47:23 +01:00
Richard van der Hoff	4fa74c7606	Minor clarifications to the TURN docs (#7533)	2020-05-20 11:04:34 +01:00
Paul Tötterman	ab3e19d814	Improve API doc readability (#7527)	2020-05-19 11:20:23 +01:00
Richard van der Hoff	24d9151a08	Formatting for reverse-proxy docs (#7514) ... also a small clarification to nginx	2020-05-15 15:13:39 +01:00
Jeff Peeler	572b444dab	Add Caddy 2 example (#7463) ... The specific headers that are passed using this new configuration format are Host and X-Forwarded-For, which should be all that's required. Note that for production another matcher should be added in the first section to properly handle the base_url lookup: reverse_proxy /.well-known/matrix/* http://localhost:8008 Signed-off-by: Jeff Peeler <jpeeler@gmail.com>	2020-05-15 14:36:01 +01:00
Richard van der Hoff	ec0b72bc4e	Merge branch 'master' into develop	2020-05-14 18:12:00 +01:00
Richard van der Hoff	66d03639dc	Notes on using git (#7496) ... * general updates to CONTRIBUTING.md * notes on updating your PR * Notes on squash-merging or otherwise * document git branching model	2020-05-14 18:03:10 +01:00
Patrick Cloke	51fb0fc2e5	Update documentation about SSO mapping providers (#7458)	2020-05-12 10:51:07 -04:00
Amber Brown	7cb8b4bc67	Allow configuration of Synapse's cache without using synctl or environment variables (#6391)	2020-05-11 18:45:23 +01:00
Andrew Morgan	5cf758cdd6	Merge branch 'release-v1.13.0' into develop ... * release-v1.13.0: Don't UPGRADE database rows RST indenting Put rollback instructions in upgrade notes Fix changelog typo Oh yeah, RST Absolute URL it is then Fix upgrade notes link Provide summary of upgrade issues in changelog. Fix ) Move next version notes from changelog to upgrade notes Changelog fixes 1.13.0rc1 Documentation on setting up redis (#7446) Rework UI Auth session validation for registration (#7455) Fix errors from malformed log line (#7454) Drop support for redis.dbid (#7450)	2020-05-11 16:46:33 +01:00
Neil Johnson	85155654c5	Documentation on setting up redis (#7446)	2020-05-11 13:21:15 +01:00
Andrew Morgan	67feea8044	Extend spam checker to allow for multiple modules (#7435)	2020-05-08 19:25:48 +01:00
Quentin Gliech	616af44137	Implement OpenID Connect-based login (#7256)	2020-05-08 08:30:40 -04:00
Manuel Stahl	a4a5ec4096	Add room details admin endpoint (#7317)	2020-05-07 15:33:07 -04:00
Brendan Abolivier	5bb26b7c4f	Merge branch 'release-v1.13.0' into develop	2020-05-07 17:31:19 +02:00
Erik Johnston	d7983b63a6	Support any process writing to cache invalidation stream. (#7436)	2020-05-07 13:51:08 +01:00
Brendan Abolivier	d9b8d27494	Add a configuration setting for the dummy event threshold (#7422) ... Add dummy_events_threshold which allows configuring the number of forward extremities a room needs for Synapse to send forward extremities in it.	2020-05-07 10:35:23 +01:00
Richard van der Hoff	207b1737ee	Update reverse_proxy.md ... a couple of cleanups	2020-05-05 11:29:29 +01:00
Erik Johnston	37f6823f5b	Add instance name to RDATA/POSITION commands (#7364) ... This is primarily for allowing us to send those commands from workers, but for now simply allows us to ignore echoed RDATA/POSITION commands that we sent (we get echoes of sent commands when using redis). Currently we log a WARNING on the master process every time we receive an echoed RDATA.	2020-04-29 16:23:08 +01:00
Andrew Morgan	c58ae367d8	Clean up admin api docs (#7361)	2020-04-28 20:06:03 +01:00
Manuel Stahl	04dd7d182d	Return total number of users and profile attributes in admin users endpoint (#6881) ... Signed-off-by: Manuel Stahl <manuel.stahl@awesome-technologies.de>	2020-04-28 18:19:36 +01:00
Brendan Abolivier	036fab5d8a	Document monitoring workers (#7357) ... It doesn't seem to be documented anywhere and means that you suddenly start losing metrics without any obvious reason when you go from monolith to workers (e.g. #7312).	2020-04-27 21:36:47 +02:00
lub	aa2492907f	Add some explanation to application_services.md (#7091) ... Signed-off-by: Simon Körner <git@lubiland.de>	2020-04-27 15:03:09 +01:00
Patrick Cloke	7bfe0902ce	Add documentation to the sample config about the templates for SSO. (#7343)	2020-04-24 15:03:49 -04:00
Patrick Cloke	204664d1ad	Synapse v1.12.4 ... Features: * Always send users their own device updates. (#7160) * Add support for handling GET requests for account_data on a worker. (#7311) Bugfixes: * Fix a bug that prevented cross-signing with users on worker-mode synapses. (#7255) * Do not treat display names as globs in push rules. (#7271) * Fix a bug with cross-signing devices belonging to remote users who did not share a room with any user on the local homeserver. (#7289) -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEF3tZXk38tRDFVnUIM/xY9qcRMEgFAl6hs9EACgkQM/xY9qcR MEjq2w/9EC6m4Bx/2bDlohUy3GKfe9BUpZnMDGSgQR1v+J+OIA4yZ1s16hmY95fr rGjNaZlcUbLUFj9GfaLOOZjvuXQ0KslO2ojP1lcSaKCfx9WK1CH0DtCNVB0lAYQv sfu2C2I91XetkiMHo5GqYBP1d6IK3OTI7YRz72sHOQTVPFcSB1/wCUPuan7P90HE gBdfxtl7Xl1TU0tfvJoeRLVarktVnE4LiXREfm1eGpRECPclewu7sbx5p1ESN5mg /RBIh0FP4rOL4cP4Qt5qiCo68woJmBc4CwiV6pvcOyUMtMGqmwO9K/wJxqY+pxnh JKBNqv90BCSxD55Nywslx9yA52kshuRdcdqHuOVYOHo4Dcwu+ecyySpmzuZ2EJPc hDEkf3dnpfaVoMn3uesbwGNvHEr69QXxR8yE1XZvlAdKzvy4Po/0qRAZuD+NG9qf YjN94bDbkASlFQgPn3wF7R+4RA/HdOf/1Ns6YKt2dFRikHb18WCBPWxHcSwscrgE W8xma8rVTlqfRBuAcZ6y4K3KSHFX6dxLUNxqUngyAhjTInSPuqt2EzlAdqRpnJBS t+hhj+zOh2ixxbz4ZrGdMx0jPF2XARr38x3F/LIjhHLcgOlHdCl4WWkxi9BSXOTd cscvQ8GJBH9c0ANy7PGBntXlohQiW3eVVqblWRbV95TpHVoimME= =Hc/7 -----END PGP SIGNATURE----- Merge tag 'v1.12.4' Synapse v1.12.4 Features: * Always send users their own device updates. (#7160) * Add support for handling GET requests for account_data on a worker. (#7311) Bugfixes: * Fix a bug that prevented cross-signing with users on worker-mode synapses. (#7255) * Do not treat display names as globs in push rules. (#7271) * Fix a bug with cross-signing devices belonging to remote users who did not share a room with any user on the local homeserver. (#7289)	2020-04-23 12:03:33 -04:00
Brendan Abolivier	2e3b9a0fcb	Revert "Revert "Merge pull request #7315 from matrix-org/babolivier/request_token"" ... This reverts commit 1adf6a5587.	2020-04-23 11:23:53 +02:00
Brendan Abolivier	fb825759e3	Merge branch 'master' into develop	2020-04-23 11:23:33 +02:00
Brendan Abolivier	1adf6a5587	Revert "Merge pull request #7315 from matrix-org/babolivier/request_token" ... This reverts commit 6f4319368b, reversing changes made to 0d775fcc2d.	2020-04-23 11:23:10 +02:00
Brendan Abolivier	6f4319368b	Merge pull request #7315 from matrix-org/babolivier/request_token ... Config option to inhibit 3PID errors on /requestToken	2020-04-23 10:38:57 +02:00
Richard van der Hoff	71a1abb8a1	Stop the master relaying USER_SYNC for other workers (#7318) ... Long story short: if we're handling presence on the current worker, we shouldn't be sending USER_SYNC commands over replication. In an attempt to figure out what is going on here, I ended up refactoring some bits of the presencehandler code, so the first 4 commits here are non-functional refactors to move this code slightly closer to sanity. (There's still plenty to do here :/). Suggest reviewing individual commits. Fixes (I hope) #7257.	2020-04-22 22:39:04 +01:00
Brendan Abolivier	69ad7cc13b	Config option to inhibit 3PID errors on /requestToken ... Adds a request_token_inhibit_errors configuration flag (disabled by default) which, if enabled, change the behaviour of all /requestToken endpoints so that they return a 200 and a fake sid if the 3PID was/was not found associated with an account (depending on the endpoint), instead of an error. Co-Authored-By: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>	2020-04-22 23:38:42 +02:00
Dirk Klimpel	6b6685db9f	Extend room admin api with additional attributes (#7225)	2020-04-22 13:38:41 +01:00
Richard van der Hoff	2aa5bf13c8	Merge branch 'release-v1.12.4' into develop	2020-04-22 13:09:23 +01:00
Richard van der Hoff	974c0d726a	Support GET account_data requests on a worker (#7311)	2020-04-21 10:46:30 +01:00
Lars Franke	13917232d5	Fix indention in generated config file (#7300) ... Also adjust sample_config.yaml Signed-off-by: Lars Franke <frcl@mailbox.org>	2020-04-20 16:51:27 +01:00
nataraj-hates-MS-for-stealing-github	0d775fcc2d	Improve example TURN configuration in documentation (#7284)	2020-04-17 08:04:23 -04:00
Tristan Lins	c07fca9e2f	Clarify the comments for media_storage_providers options (#7272)	2020-04-17 07:09:33 -04:00
Andrew Morgan	a48138784e	Allow specifying the value of Accept-Language header for URL previews (#7265)	2020-04-15 13:35:29 +01:00
Ryan Hovland	4a0dadafbe	Add setting to nginx configuration to allow larger file uploads (#7251)	2020-04-13 17:23:36 +01:00
Andrew Morgan	a026bdaab7	Add matrix-synapse-shared-secret-auth as an example password provider (#7248)	2020-04-09 12:49:05 +01:00
Richard van der Hoff	cae4121484	Make systemd-with-workers doc official (#7234) ... Simplify and update this documentation, and make it part of the core dist.	2020-04-08 11:59:26 +01:00
Andrew Morgan	29b7e22b93	Add documentation to password_providers config option (#7238)	2020-04-08 00:46:50 +01:00
Martin Milata	b0db928c63	Extend web_client_location to handle absolute URLs (#7006) ... Log warning when filesystem path is used. Signed-off-by: Martin Milata <martin@martinmilata.cz>	2020-04-03 11:57:34 -04:00
Richard van der Hoff	0122ef1037	Revert "Merge pull request #7153 from matrix-org/babolivier/sso_whitelist_login_fallback" ... This was incorrectly merged to master. This reverts commit 319c41f573, reversing changes made to 229eb81498.	2020-04-03 11:17:39 +01:00
siroccal	250f87d0de	Update postgres.md (#7119)	2020-04-01 12:44:51 +01:00
Jostein Kjønigsen	2e826cd80c	Improve TURN documentation. (#7167)	2020-03-31 15:50:48 +01:00
Andrew Morgan	d9f29f8dae	Fix a small typo in the metrics_flags config option. (#7171)	2020-03-30 17:38:21 +01:00
Erik Johnston	4f21c33be3	Remove usage of "conn_id" for presence. (#7128) ... * Remove `conn_id` usage for UserSyncCommand. Each tcp replication connection is assigned a "conn_id", which is used to give an ID to a remotely connected worker. In a redis world, there will no longer be a one to one mapping between connection and instance, so instead we need to replace such usages with an ID generated by the remote instances and included in the replicaiton commands. This really only effects UserSyncCommand. * Add CLEAR_USER_SYNCS command that is sent on shutdown. This should help with the case where a synchrotron gets restarted gracefully, rather than rely on 5 minute timeout.	2020-03-30 16:37:24 +01:00
Patrick Cloke	c5f89fba55	Add developer documentation for running a local CAS server (#7147)	2020-03-30 07:28:42 -04:00
Richard van der Hoff	b7da598a61	Always whitelist the login fallback for SSO (#7153) ... That fallback sets the redirect URL to itself (so it can process the login token then return gracefully to the client). This would make it pointless to ask the user for confirmation, since the URL the confirmation page would be showing wouldn't be the client's.	2020-03-27 20:24:52 +00:00
Dirk Klimpel	fb69690761	Admin API to join users to a room. (#7051)	2020-03-27 19:16:43 +00:00
Dirk Klimpel	8327eb9280	Add options to prevent users from changing their profile. (#7096)	2020-03-27 19:15:23 +00:00
Brendan Abolivier	319c41f573	Merge pull request #7153 from matrix-org/babolivier/sso_whitelist_login_fallback ... Always whitelist the login fallback for SSO	2020-03-27 15:34:41 +01:00
Brendan Abolivier	63aea691a7	Update the wording of the config comment	2020-03-27 15:09:12 +01:00
Brendan Abolivier	7083147961	Regenerate sample config	2020-03-26 19:01:54 +01:00
Dirk Klimpel	e8e2ddb60a	Allow server admins to define and enforce a password policy (MSC2000). (#7118)	2020-03-26 16:51:13 +00:00
Aaron Raimist	6ca5e56fd1	Remove unused captcha_bypass_secret option (#7137) ... Signed-off-by: Aaron Raimist <aaron@raim.ist>	2020-03-25 17:49:34 +00:00
Erik Johnston	4cff617df1	Move catchup of replication streams to worker. (#7024) ... This changes the replication protocol so that the server does not send down `RDATA` for rows that happened before the client connected. Instead, the server will send a `POSITION` and clients then query the database (or master out of band) to get up to date.	2020-03-25 14:54:01 +00:00
Richard van der Hoff	39230d2171	Clean up some LoggingContext stuff (#7120) ... * Pull Sentinel out of LoggingContext ... and drop a few unnecessary references to it * Factor out LoggingContext.current_context move `current_context` and `set_context` out to top-level functions. Mostly this means that I can more easily trace what's actually referring to LoggingContext, but I think it's generally neater. * move copy-to-parent into `stop` this really just makes `start` and `stop` more symetric. It also means that it behaves correctly if you manually `set_log_context` rather than using the context manager. * Replace `LoggingContext.alive` with `finished` Turn `alive` into `finished` and make it a bit better defined.	2020-03-24 14:45:33 +00:00
Richard van der Hoff	5126cb1253	Merge branch 'master' into develop	2020-03-23 13:54:29 +00:00
Richard van der Hoff	229eb81498	Synapse 1.12.0 (2020-03-23) ... =========================== No significant changes since 1.12.0rc1. Debian packages and Docker images are rebuilt using the latest versions of dependency libraries, including Twisted 20.3.0. **Please see security advisory below**. Security advisory ----------------- Synapse may be vulnerable to request-smuggling attacks when it is used with a reverse-proxy. The vulnerabilties are fixed in Twisted 20.3.0, and are described in [CVE-2020-10108](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-10108) and [CVE-2020-10109](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-10109). For a good introduction to this class of request-smuggling attacks, see https://portswigger.net/research/http-desync-attacks-request-smuggling-reborn. We are not aware of these vulnerabilities being exploited in the wild, and do not believe that they are exploitable with current versions of any reverse proxies. Nevertheless, we recommend that all Synapse administrators ensure that they have the latest versions of the Twisted library to ensure that their installation remains secure. * Administrators using the [`matrix.org` Docker image](https://hub.docker.com/r/matrixdotorg/synapse/) or the [Debian/Ubuntu packages from `matrix.org`](https://github.com/matrix-org/synapse/blob/master/INSTALL.md#matrixorg-packages) should ensure that they have version 1.12.0 installed: these images include Twisted 20.3.0. * Administrators who have [installed Synapse from source](https://github.com/matrix-org/synapse/blob/master/INSTALL.md#installing-from-source) should upgrade Twisted within their virtualenv by running: ```sh <path_to_virtualenv>/bin/pip install 'Twisted>=20.3.0' ``` * Administrators who have installed Synapse from distribution packages should consult the information from their distributions. The `matrix.org` Synapse instance was not vulnerable to these vulnerabilities. Advance notice of change to the default `git` branch for Synapse ---------------------------------------------------------------- Currently, the default `git` branch for Synapse is `master`, which tracks the latest release. After the release of Synapse 1.13.0, we intend to change this default to `develop`, which is the development tip. This is more consistent with common practice and modern `git` usage. Although we try to keep `develop` in a stable state, there may be occasions where regressions creep in. Developers and distributors who have scripts which run builds using the default branch of `Synapse` should therefore consider pinning their scripts to `master`. Synapse 1.12.0rc1 (2020-03-19) ============================== Features -------- - Changes related to room alias management ([MSC2432](https://github.com/matrix-org/matrix-doc/pull/2432)): - Publishing/removing a room from the room directory now requires the user to have a power level capable of modifying the canonical alias, instead of the room aliases. ([\#6965](https://github.com/matrix-org/synapse/issues/6965)) - Validate the `alt_aliases` property of canonical alias events. ([\#6971](https://github.com/matrix-org/synapse/issues/6971)) - Users with a power level sufficient to modify the canonical alias of a room can now delete room aliases. ([\#6986](https://github.com/matrix-org/synapse/issues/6986)) - Implement updated authorization rules and redaction rules for aliases events, from [MSC2261](https://github.com/matrix-org/matrix-doc/pull/2261) and [MSC2432](https://github.com/matrix-org/matrix-doc/pull/2432). ([\#7037](https://github.com/matrix-org/synapse/issues/7037)) - Stop sending m.room.aliases events during room creation and upgrade. ([\#6941](https://github.com/matrix-org/synapse/issues/6941)) - Synapse no longer uses room alias events to calculate room names for push notifications. ([\#6966](https://github.com/matrix-org/synapse/issues/6966)) - The room list endpoint no longer returns a list of aliases. ([\#6970](https://github.com/matrix-org/synapse/issues/6970)) - Remove special handling of aliases events from [MSC2260](https://github.com/matrix-org/matrix-doc/pull/2260) added in v1.10.0rc1. ([\#7034](https://github.com/matrix-org/synapse/issues/7034)) - Expose the `synctl`, `hash_password` and `generate_config` commands in the snapcraft package. Contributed by @devec0. ([\#6315](https://github.com/matrix-org/synapse/issues/6315)) - Check that server_name is correctly set before running database updates. ([\#6982](https://github.com/matrix-org/synapse/issues/6982)) - Break down monthly active users by `appservice_id` and emit via Prometheus. ([\#7030](https://github.com/matrix-org/synapse/issues/7030)) - Render a configurable and comprehensible error page if something goes wrong during the SAML2 authentication process. ([\#7058](https://github.com/matrix-org/synapse/issues/7058), [\#7067](https://github.com/matrix-org/synapse/issues/7067)) - Add an optional parameter to control whether other sessions are logged out when a user's password is modified. ([\#7085](https://github.com/matrix-org/synapse/issues/7085)) - Add prometheus metrics for the number of active pushers. ([\#7103](https://github.com/matrix-org/synapse/issues/7103), [\#7106](https://github.com/matrix-org/synapse/issues/7106)) - Improve performance when making HTTPS requests to sygnal, sydent, etc, by sharing the SSL context object between connections. ([\#7094](https://github.com/matrix-org/synapse/issues/7094)) Bugfixes -------- - When a user's profile is updated via the admin API, also generate a displayname/avatar update for that user in each room. ([\#6572](https://github.com/matrix-org/synapse/issues/6572)) - Fix a couple of bugs in email configuration handling. ([\#6962](https://github.com/matrix-org/synapse/issues/6962)) - Fix an issue affecting worker-based deployments where replication would stop working, necessitating a full restart, after joining a large room. ([\#6967](https://github.com/matrix-org/synapse/issues/6967)) - Fix `duplicate key` error which was logged when rejoining a room over federation. ([\#6968](https://github.com/matrix-org/synapse/issues/6968)) - Prevent user from setting 'deactivated' to anything other than a bool on the v2 PUT /users Admin API. ([\#6990](https://github.com/matrix-org/synapse/issues/6990)) - Fix py35-old CI by using native tox package. ([\#7018](https://github.com/matrix-org/synapse/issues/7018)) - Fix a bug causing `org.matrix.dummy_event` to be included in responses from `/sync`. ([\#7035](https://github.com/matrix-org/synapse/issues/7035)) - Fix a bug that renders UTF-8 text files incorrectly when loaded from media. Contributed by @TheStranjer. ([\#7044](https://github.com/matrix-org/synapse/issues/7044)) - Fix a bug that would cause Synapse to respond with an error about event visibility if a client tried to request the state of a room at a given token. ([\#7066](https://github.com/matrix-org/synapse/issues/7066)) - Repair a data-corruption issue which was introduced in Synapse 1.10, and fixed in Synapse 1.11, and which could cause `/sync` to return with 404 errors about missing events and unknown rooms. ([\#7070](https://github.com/matrix-org/synapse/issues/7070)) - Fix a bug causing account validity renewal emails to be sent even if the feature is turned off in some cases. ([\#7074](https://github.com/matrix-org/synapse/issues/7074)) Improved Documentation ---------------------- - Updated CentOS8 install instructions. Contributed by Richard Kellner. ([\#6925](https://github.com/matrix-org/synapse/issues/6925)) - Fix `POSTGRES_INITDB_ARGS` in the `contrib/docker/docker-compose.yml` example docker-compose configuration. ([\#6984](https://github.com/matrix-org/synapse/issues/6984)) - Change date in [INSTALL.md](./INSTALL.md#tls-certificates) for last date of getting TLS certificates to November 2019. ([\#7015](https://github.com/matrix-org/synapse/issues/7015)) - Document that the fallback auth endpoints must be routed to the same worker node as the register endpoints. ([\#7048](https://github.com/matrix-org/synapse/issues/7048)) Deprecations and Removals ------------------------- - Remove the unused query_auth federation endpoint per [MSC2451](https://github.com/matrix-org/matrix-doc/pull/2451). ([\#7026](https://github.com/matrix-org/synapse/issues/7026)) Internal Changes ---------------- - Add type hints to `logging/context.py`. ([\#6309](https://github.com/matrix-org/synapse/issues/6309)) - Add some clarifications to `README.md` in the database schema directory. ([\#6615](https://github.com/matrix-org/synapse/issues/6615)) - Refactoring work in preparation for changing the event redaction algorithm. ([\#6874](https://github.com/matrix-org/synapse/issues/6874), [\#6875](https://github.com/matrix-org/synapse/issues/6875), [\#6983](https://github.com/matrix-org/synapse/issues/6983), [\#7003](https://github.com/matrix-org/synapse/issues/7003)) - Improve performance of v2 state resolution for large rooms. ([\#6952](https://github.com/matrix-org/synapse/issues/6952), [\#7095](https://github.com/matrix-org/synapse/issues/7095)) - Reduce time spent doing GC, by freezing objects on startup. ([\#6953](https://github.com/matrix-org/synapse/issues/6953)) - Minor perfermance fixes to `get_auth_chain_ids`. ([\#6954](https://github.com/matrix-org/synapse/issues/6954)) - Don't record remote cross-signing keys in the `devices` table. ([\#6956](https://github.com/matrix-org/synapse/issues/6956)) - Use flake8-comprehensions to enforce good hygiene of list/set/dict comprehensions. ([\#6957](https://github.com/matrix-org/synapse/issues/6957)) - Merge worker apps together. ([\#6964](https://github.com/matrix-org/synapse/issues/6964), [\#7002](https://github.com/matrix-org/synapse/issues/7002), [\#7055](https://github.com/matrix-org/synapse/issues/7055), [\#7104](https://github.com/matrix-org/synapse/issues/7104)) - Remove redundant `store_room` call from `FederationHandler._process_received_pdu`. ([\#6979](https://github.com/matrix-org/synapse/issues/6979)) - Update warning for incorrect database collation/ctype to include link to documentation. ([\#6985](https://github.com/matrix-org/synapse/issues/6985)) - Add some type annotations to the database storage classes. ([\#6987](https://github.com/matrix-org/synapse/issues/6987)) - Port `synapse.handlers.presence` to async/await. ([\#6991](https://github.com/matrix-org/synapse/issues/6991), [\#7019](https://github.com/matrix-org/synapse/issues/7019)) - Add some type annotations to the federation base & client classes. ([\#6995](https://github.com/matrix-org/synapse/issues/6995)) - Port `synapse.rest.keys` to async/await. ([\#7020](https://github.com/matrix-org/synapse/issues/7020)) - Add a type check to `is_verified` when processing room keys. ([\#7045](https://github.com/matrix-org/synapse/issues/7045)) - Add type annotations and comments to the auth handler. ([\#7063](https://github.com/matrix-org/synapse/issues/7063)) -----BEGIN PGP SIGNATURE----- iQEzBAABCAAdFiEEv27Axt/F4vrTL/8QOSor00I9eP8FAl54vN0ACgkQOSor00I9 eP/LOgf/U22VypPu2Cl4vofxigeeUL+ph0gEQDCsF7i3EPi9ObeTi4aUggC95dBs MY4JQENmVrk1NhVWQpC08mjehsr4xUzJuwnPPIeGQ6X5U/2BS4YUasxOt9A+iLUz 8IxTzUgx4T+CTfibZvn6xdE/tZK/b2N3BoOikesutD2aQGC5Fm6w2HReoY4Qrdgw AwHlsIV22PYgEn0RL5y6DJ2NUU9SdeSmPjKNe+R1rHDlTpvH7LSyINhOFkYDgRPY xmlH+Ek5+7vLi3AlWg6pA001mMWGADlC4T84URcf1fQv6hXT1iM+A9CqC57jGlfT nHDphCtz0Uk9kmgT4To+hclLDWecYw== =iSEk -----END PGP SIGNATURE----- Merge tag 'v1.12.0' Synapse 1.12.0 (2020-03-23) =========================== No significant changes since 1.12.0rc1. Debian packages and Docker images are rebuilt using the latest versions of dependency libraries, including Twisted 20.3.0. **Please see security advisory below**. Security advisory ----------------- Synapse may be vulnerable to request-smuggling attacks when it is used with a reverse-proxy. The vulnerabilties are fixed in Twisted 20.3.0, and are described in [CVE-2020-10108](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-10108) and [CVE-2020-10109](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-10109). For a good introduction to this class of request-smuggling attacks, see https://portswigger.net/research/http-desync-attacks-request-smuggling-reborn. We are not aware of these vulnerabilities being exploited in the wild, and do not believe that they are exploitable with current versions of any reverse proxies. Nevertheless, we recommend that all Synapse administrators ensure that they have the latest versions of the Twisted library to ensure that their installation remains secure. * Administrators using the [`matrix.org` Docker image](https://hub.docker.com/r/matrixdotorg/synapse/) or the [Debian/Ubuntu packages from `matrix.org`](https://github.com/matrix-org/synapse/blob/master/INSTALL.md#matrixorg-packages) should ensure that they have version 1.12.0 installed: these images include Twisted 20.3.0. * Administrators who have [installed Synapse from source](https://github.com/matrix-org/synapse/blob/master/INSTALL.md#installing-from-source) should upgrade Twisted within their virtualenv by running: ```sh <path_to_virtualenv>/bin/pip install 'Twisted>=20.3.0' ``` * Administrators who have installed Synapse from distribution packages should consult the information from their distributions. The `matrix.org` Synapse instance was not vulnerable to these vulnerabilities. Advance notice of change to the default `git` branch for Synapse ---------------------------------------------------------------- Currently, the default `git` branch for Synapse is `master`, which tracks the latest release. After the release of Synapse 1.13.0, we intend to change this default to `develop`, which is the development tip. This is more consistent with common practice and modern `git` usage. Although we try to keep `develop` in a stable state, there may be occasions where regressions creep in. Developers and distributors who have scripts which run builds using the default branch of `Synapse` should therefore consider pinning their scripts to `master`. Synapse 1.12.0rc1 (2020-03-19) ============================== Features -------- - Changes related to room alias management ([MSC2432](https://github.com/matrix-org/matrix-doc/pull/2432)): - Publishing/removing a room from the room directory now requires the user to have a power level capable of modifying the canonical alias, instead of the room aliases. ([\#6965](https://github.com/matrix-org/synapse/issues/6965)) - Validate the `alt_aliases` property of canonical alias events. ([\#6971](https://github.com/matrix-org/synapse/issues/6971)) - Users with a power level sufficient to modify the canonical alias of a room can now delete room aliases. ([\#6986](https://github.com/matrix-org/synapse/issues/6986)) - Implement updated authorization rules and redaction rules for aliases events, from [MSC2261](https://github.com/matrix-org/matrix-doc/pull/2261) and [MSC2432](https://github.com/matrix-org/matrix-doc/pull/2432). ([\#7037](https://github.com/matrix-org/synapse/issues/7037)) - Stop sending m.room.aliases events during room creation and upgrade. ([\#6941](https://github.com/matrix-org/synapse/issues/6941)) - Synapse no longer uses room alias events to calculate room names for push notifications. ([\#6966](https://github.com/matrix-org/synapse/issues/6966)) - The room list endpoint no longer returns a list of aliases. ([\#6970](https://github.com/matrix-org/synapse/issues/6970)) - Remove special handling of aliases events from [MSC2260](https://github.com/matrix-org/matrix-doc/pull/2260) added in v1.10.0rc1. ([\#7034](https://github.com/matrix-org/synapse/issues/7034)) - Expose the `synctl`, `hash_password` and `generate_config` commands in the snapcraft package. Contributed by @devec0. ([\#6315](https://github.com/matrix-org/synapse/issues/6315)) - Check that server_name is correctly set before running database updates. ([\#6982](https://github.com/matrix-org/synapse/issues/6982)) - Break down monthly active users by `appservice_id` and emit via Prometheus. ([\#7030](https://github.com/matrix-org/synapse/issues/7030)) - Render a configurable and comprehensible error page if something goes wrong during the SAML2 authentication process. ([\#7058](https://github.com/matrix-org/synapse/issues/7058), [\#7067](https://github.com/matrix-org/synapse/issues/7067)) - Add an optional parameter to control whether other sessions are logged out when a user's password is modified. ([\#7085](https://github.com/matrix-org/synapse/issues/7085)) - Add prometheus metrics for the number of active pushers. ([\#7103](https://github.com/matrix-org/synapse/issues/7103), [\#7106](https://github.com/matrix-org/synapse/issues/7106)) - Improve performance when making HTTPS requests to sygnal, sydent, etc, by sharing the SSL context object between connections. ([\#7094](https://github.com/matrix-org/synapse/issues/7094)) Bugfixes -------- - When a user's profile is updated via the admin API, also generate a displayname/avatar update for that user in each room. ([\#6572](https://github.com/matrix-org/synapse/issues/6572)) - Fix a couple of bugs in email configuration handling. ([\#6962](https://github.com/matrix-org/synapse/issues/6962)) - Fix an issue affecting worker-based deployments where replication would stop working, necessitating a full restart, after joining a large room. ([\#6967](https://github.com/matrix-org/synapse/issues/6967)) - Fix `duplicate key` error which was logged when rejoining a room over federation. ([\#6968](https://github.com/matrix-org/synapse/issues/6968)) - Prevent user from setting 'deactivated' to anything other than a bool on the v2 PUT /users Admin API. ([\#6990](https://github.com/matrix-org/synapse/issues/6990)) - Fix py35-old CI by using native tox package. ([\#7018](https://github.com/matrix-org/synapse/issues/7018)) - Fix a bug causing `org.matrix.dummy_event` to be included in responses from `/sync`. ([\#7035](https://github.com/matrix-org/synapse/issues/7035)) - Fix a bug that renders UTF-8 text files incorrectly when loaded from media. Contributed by @TheStranjer. ([\#7044](https://github.com/matrix-org/synapse/issues/7044)) - Fix a bug that would cause Synapse to respond with an error about event visibility if a client tried to request the state of a room at a given token. ([\#7066](https://github.com/matrix-org/synapse/issues/7066)) - Repair a data-corruption issue which was introduced in Synapse 1.10, and fixed in Synapse 1.11, and which could cause `/sync` to return with 404 errors about missing events and unknown rooms. ([\#7070](https://github.com/matrix-org/synapse/issues/7070)) - Fix a bug causing account validity renewal emails to be sent even if the feature is turned off in some cases. ([\#7074](https://github.com/matrix-org/synapse/issues/7074)) Improved Documentation ---------------------- - Updated CentOS8 install instructions. Contributed by Richard Kellner. ([\#6925](https://github.com/matrix-org/synapse/issues/6925)) - Fix `POSTGRES_INITDB_ARGS` in the `contrib/docker/docker-compose.yml` example docker-compose configuration. ([\#6984](https://github.com/matrix-org/synapse/issues/6984)) - Change date in [INSTALL.md](./INSTALL.md#tls-certificates) for last date of getting TLS certificates to November 2019. ([\#7015](https://github.com/matrix-org/synapse/issues/7015)) - Document that the fallback auth endpoints must be routed to the same worker node as the register endpoints. ([\#7048](https://github.com/matrix-org/synapse/issues/7048)) Deprecations and Removals ------------------------- - Remove the unused query_auth federation endpoint per [MSC2451](https://github.com/matrix-org/matrix-doc/pull/2451). ([\#7026](https://github.com/matrix-org/synapse/issues/7026)) Internal Changes ---------------- - Add type hints to `logging/context.py`. ([\#6309](https://github.com/matrix-org/synapse/issues/6309)) - Add some clarifications to `README.md` in the database schema directory. ([\#6615](https://github.com/matrix-org/synapse/issues/6615)) - Refactoring work in preparation for changing the event redaction algorithm. ([\#6874](https://github.com/matrix-org/synapse/issues/6874), [\#6875](https://github.com/matrix-org/synapse/issues/6875), [\#6983](https://github.com/matrix-org/synapse/issues/6983), [\#7003](https://github.com/matrix-org/synapse/issues/7003)) - Improve performance of v2 state resolution for large rooms. ([\#6952](https://github.com/matrix-org/synapse/issues/6952), [\#7095](https://github.com/matrix-org/synapse/issues/7095)) - Reduce time spent doing GC, by freezing objects on startup. ([\#6953](https://github.com/matrix-org/synapse/issues/6953)) - Minor perfermance fixes to `get_auth_chain_ids`. ([\#6954](https://github.com/matrix-org/synapse/issues/6954)) - Don't record remote cross-signing keys in the `devices` table. ([\#6956](https://github.com/matrix-org/synapse/issues/6956)) - Use flake8-comprehensions to enforce good hygiene of list/set/dict comprehensions. ([\#6957](https://github.com/matrix-org/synapse/issues/6957)) - Merge worker apps together. ([\#6964](https://github.com/matrix-org/synapse/issues/6964), [\#7002](https://github.com/matrix-org/synapse/issues/7002), [\#7055](https://github.com/matrix-org/synapse/issues/7055), [\#7104](https://github.com/matrix-org/synapse/issues/7104)) - Remove redundant `store_room` call from `FederationHandler._process_received_pdu`. ([\#6979](https://github.com/matrix-org/synapse/issues/6979)) - Update warning for incorrect database collation/ctype to include link to documentation. ([\#6985](https://github.com/matrix-org/synapse/issues/6985)) - Add some type annotations to the database storage classes. ([\#6987](https://github.com/matrix-org/synapse/issues/6987)) - Port `synapse.handlers.presence` to async/await. ([\#6991](https://github.com/matrix-org/synapse/issues/6991), [\#7019](https://github.com/matrix-org/synapse/issues/7019)) - Add some type annotations to the federation base & client classes. ([\#6995](https://github.com/matrix-org/synapse/issues/6995)) - Port `synapse.rest.keys` to async/await. ([\#7020](https://github.com/matrix-org/synapse/issues/7020)) - Add a type check to `is_verified` when processing room keys. ([\#7045](https://github.com/matrix-org/synapse/issues/7045)) - Add type annotations and comments to the auth handler. ([\#7063](https://github.com/matrix-org/synapse/issues/7063))	2020-03-23 13:54:17 +00:00
Richard van der Hoff	c165c1233b	Improve database configuration docs (#6988) ... Attempts to clarify the sample config for databases, and add some stuff about tcp keepalives to `postgres.md`.	2020-03-20 15:24:22 +00:00
Patrick Cloke	88b41986db	Add an option to the set password API to choose whether to logout other devices. (#7085)	2020-03-18 07:50:00 -04:00
Richard van der Hoff	4ce50519cd	Update postgres.md ... fix broken link	2020-03-17 18:08:43 +00:00
Richard van der Hoff	6a35046363	Revert "Add options to disable setting profile info for prevent changes. (#7053)" ... This reverts commit 54dd28621b, reversing changes made to 6640460d05.	2020-03-17 11:25:01 +00:00
Brendan Abolivier	f9e98176bf	Put the file in the templates directory	2020-03-11 20:31:42 +00:00
Brendan Abolivier	900bca9707	Update wording and config	2020-03-11 19:40:30 +00:00
Brendan Abolivier	54dd28621b	Add options to disable setting profile info for prevent changes. (#7053)	2020-03-10 22:23:01 +00:00
Dirk Klimpel	751d51dd12	Update sample_config.yaml	2020-03-10 21:41:25 +01:00
Brendan Abolivier	51c094c4ac	Update sample config	2020-03-10 14:00:29 +00:00
dklimpel	885134529f	updates after review	2020-03-09 22:09:29 +01:00
Patrick Cloke	66315d862f	Update routing of fallback auth in the worker docs. (#7048)	2020-03-09 07:19:24 -04:00
dklimpel	99bbe177b6	add disable_3pid_changes	2020-03-08 21:58:12 +01:00
dklimpel	20545a2199	lint2	2020-03-08 15:28:00 +01:00
dklimpel	fb078f921b	changelog	2020-03-08 15:19:07 +01:00
Brendan Abolivier	43f874055d	Merge branch 'master' into develop	2020-03-03 15:20:49 +00:00
Richard van der Hoff	b68041df3d	Add a whitelist for the SSO confirmation step.	2020-03-02 17:05:09 +00:00
Brendan Abolivier	b2bd54a2e3	Add a confirmation step to the SSO login flow	2020-03-02 16:36:32 +00:00
Richard van der Hoff	4c2ed3f20e	Fix minor issues with email config (#6962) ... * Give `notif_template_html`, `notif_template_text` default values (fixes #6960) * Don't complain if `smtp_host` and `smtp_port` are unset, since they have sensible defaults (fixes #6961) * Set the example for `enable_notifs` to `True`, for consistency and because it's more useful * Raise errors as ConfigError rather than RuntimeError for nicer formatting	2020-02-24 15:18:38 +00:00
Patrick Cloke	509e381afa	Clarify list/set/dict/tuple comprehensions and enforce via flake8 (#6957) ... Ensure good comprehension hygiene using flake8-comprehensions.	2020-02-21 07:15:07 -05:00
Brendan Abolivier	bbe39f808c	Merge pull request #6940 from matrix-org/babolivier/federate.md ... Clean up and update federation docs	2020-02-19 10:58:59 +00:00
Brendan Abolivier	71d65407e7	Incorporate review	2020-02-19 10:03:19 +00:00
Brendan Abolivier	61b457e3ec	Incorporate review	2020-02-18 17:20:03 +00:00
Brendan Abolivier	a0d2f9d089	Phrasing	2020-02-18 16:16:49 +00:00
Brendan Abolivier	d484126bf7	Merge pull request #6907 from matrix-org/babolivier/acme-config ... Add mention and warning about ACME v1 deprecation to the TLS config	2020-02-18 16:11:31 +00:00
Erik Johnston	818def8248	Fix worker docs to point /publicised_groups API correctly. (#6938)	2020-02-18 15:27:45 +00:00
Brendan Abolivier	d009535639	Add mention of SRV records as an advanced topic	2020-02-18 14:07:41 +00:00
Brendan Abolivier	ba7a523854	Argh trailing spaces	2020-02-18 13:57:15 +00:00
Brendan Abolivier	e837be5b5c	Fix links in the reverse proxy doc	2020-02-18 13:53:58 +00:00
Brendan Abolivier	3c67eee6dc	Make federate.md more of a sumary of the steps to follow to set up replication	2020-02-18 13:51:03 +00:00
Brendan Abolivier	8ee0d74516	Split the delegating documentation out of federate.md and trim it down	2020-02-18 12:05:45 +00:00
Richard van der Hoff	97a42bbc3a	Add a warning about indentation to generated config (#6920) ... Fixes #6916.	2020-02-14 16:22:30 +00:00
Patrick Cloke	49f877d32e	Filter the results of user directory searching via the spam checker (#6888) ... Add a method to the spam checker to filter the user directory results.	2020-02-14 07:17:54 -05:00
Brendan Abolivier	5820ed905f	Add mention and warning about ACME v1 deprecation to the Synapse config	2020-02-13 14:20:08 +00:00
Patrick Cloke	361de49c90	Add documentation for the spam checker module (#6906) ... Add documentation for the spam checker.	2020-02-13 07:40:57 -05:00
Brendan Abolivier	862669d6cc	Update docs/ACME.md	2020-02-13 11:29:08 +00:00
Brendan Abolivier	459d089af7	Mention that using Synapse to serve certificates requires restarts	2020-02-12 21:05:30 +00:00
Brendan Abolivier	e45a7c0939	Remove duplicated info about certbot et al	2020-02-12 20:14:59 +00:00
Brendan Abolivier	f092029d2d	Update ACME.md to mention ACME v1 deprecation	2020-02-12 20:14:16 +00:00
Brendan Abolivier	6cd34da8b1	Merge pull request #6891 from matrix-org/babolivier/retention-doc-amend ... Spell out that the last event sent to a room won't be deleted by a purge	2020-02-12 20:12:20 +00:00
Brendan Abolivier	08e050c3fd	Rephrase	2020-02-12 15:39:40 +00:00
Brendan Abolivier	47acbc519f	Merge branch 'master' into develop	2020-02-12 13:24:09 +00:00
Brendan Abolivier	d9239b5257	Synapse 1.10.0 (2020-02-12) ... =========================== **WARNING to client developers**: As of this release Synapse validates `client_secret` parameters in the Client-Server API as per the spec. See [\#6766](https://github.com/matrix-org/synapse/issues/6766) for details. Updates to the Docker image --------------------------- - Update the docker images to Alpine Linux 3.11. ([\#6897](https://github.com/matrix-org/synapse/issues/6897)) Synapse 1.10.0rc5 (2020-02-11) ============================== Bugfixes -------- - Fix the filtering introduced in 1.10.0rc3 to also apply to the state blocks returned by `/sync`. ([\#6884](https://github.com/matrix-org/synapse/issues/6884)) Synapse 1.10.0rc4 (2020-02-11) ============================== This release candidate was built incorrectly and is superceded by 1.10.0rc5. Synapse 1.10.0rc3 (2020-02-10) ============================== Features -------- - Filter out `m.room.aliases` from the CS API to mitigate abuse while a better solution is specced. ([\#6878](https://github.com/matrix-org/synapse/issues/6878)) Internal Changes ---------------- - Fix continuous integration failures with old versions of `pip`, which were introduced by a release of the `zipp` library. ([\#6880](https://github.com/matrix-org/synapse/issues/6880)) Synapse 1.10.0rc2 (2020-02-06) ============================== Bugfixes -------- - Fix an issue with cross-signing where device signatures were not sent to remote servers. ([\#6844](https://github.com/matrix-org/synapse/issues/6844)) - Fix to the unknown remote device detection which was introduced in 1.10.rc1. ([\#6848](https://github.com/matrix-org/synapse/issues/6848)) Internal Changes ---------------- - Detect unexpected sender keys on remote encrypted events and resync device lists. ([\#6850](https://github.com/matrix-org/synapse/issues/6850)) Synapse 1.10.0rc1 (2020-01-31) ============================== Features -------- - Add experimental support for updated authorization rules for aliases events, from [MSC2260](https://github.com/matrix-org/matrix-doc/pull/2260). ([\#6787](https://github.com/matrix-org/synapse/issues/6787), [\#6790](https://github.com/matrix-org/synapse/issues/6790), [\#6794](https://github.com/matrix-org/synapse/issues/6794)) Bugfixes -------- - Warn if postgres database has a non-C locale, as that can cause issues when upgrading locales (e.g. due to upgrading OS). ([\#6734](https://github.com/matrix-org/synapse/issues/6734)) - Minor fixes to `PUT /_synapse/admin/v2/users` admin api. ([\#6761](https://github.com/matrix-org/synapse/issues/6761)) - Validate `client_secret` parameter using the regex provided by the Client-Server API, temporarily allowing `:` characters for older clients. The `:` character will be removed in a future release. ([\#6767](https://github.com/matrix-org/synapse/issues/6767)) - Fix persisting redaction events that have been redacted (or otherwise don't have a redacts key). ([\#6771](https://github.com/matrix-org/synapse/issues/6771)) - Fix outbound federation request metrics. ([\#6795](https://github.com/matrix-org/synapse/issues/6795)) - Fix bug where querying a remote user's device keys that weren't cached resulted in only returning a single device. ([\#6796](https://github.com/matrix-org/synapse/issues/6796)) - Fix race in federation sender worker that delayed sending of device updates. ([\#6799](https://github.com/matrix-org/synapse/issues/6799), [\#6800](https://github.com/matrix-org/synapse/issues/6800)) - Fix bug where Synapse didn't invalidate cache of remote users' devices when Synapse left a room. ([\#6801](https://github.com/matrix-org/synapse/issues/6801)) - Fix waking up other workers when remote server is detected to have come back online. ([\#6811](https://github.com/matrix-org/synapse/issues/6811)) Improved Documentation ---------------------- - Clarify documentation related to `user_dir` and `federation_reader` workers. ([\#6775](https://github.com/matrix-org/synapse/issues/6775)) Internal Changes ---------------- - Record room versions in the `rooms` table. ([\#6729](https://github.com/matrix-org/synapse/issues/6729), [\#6788](https://github.com/matrix-org/synapse/issues/6788), [\#6810](https://github.com/matrix-org/synapse/issues/6810)) - Propagate cache invalidates from workers to other workers. ([\#6748](https://github.com/matrix-org/synapse/issues/6748)) - Remove some unnecessary admin handler abstraction methods. ([\#6751](https://github.com/matrix-org/synapse/issues/6751)) - Add some debugging for media storage providers. ([\#6757](https://github.com/matrix-org/synapse/issues/6757)) - Detect unknown remote devices and mark cache as stale. ([\#6776](https://github.com/matrix-org/synapse/issues/6776), [\#6819](https://github.com/matrix-org/synapse/issues/6819)) - Attempt to resync remote users' devices when detected as stale. ([\#6786](https://github.com/matrix-org/synapse/issues/6786)) - Delete current state from the database when server leaves a room. ([\#6792](https://github.com/matrix-org/synapse/issues/6792)) - When a client asks for a remote user's device keys check if the local cache for that user has been marked as potentially stale. ([\#6797](https://github.com/matrix-org/synapse/issues/6797)) - Add background update to clean out left rooms from current state. ([\#6802](https://github.com/matrix-org/synapse/issues/6802), [\#6816](https://github.com/matrix-org/synapse/issues/6816)) - Refactoring work in preparation for changing the event redaction algorithm. ([\#6803](https://github.com/matrix-org/synapse/issues/6803), [\#6805](https://github.com/matrix-org/synapse/issues/6805), [\#6806](https://github.com/matrix-org/synapse/issues/6806), [\#6807](https://github.com/matrix-org/synapse/issues/6807), [\#6820](https://github.com/matrix-org/synapse/issues/6820)) -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEdVkXOgzrGzds0jtrHgFcFF8ZFs0FAl5D780ACgkQHgFcFF8Z Fs32MxAApvBbTGMhC5E2SLt8Yh4DSfDmg8yCPLMRVTdJuUQyUVgKYFSl9LqzqCGe oLpzwbXXDpN1FYGuO3daFBSN3RaU1XCQPMayekTaF6Wvq9N9rNW1tAi6cQTjUf20 bTAxZljXAIkfXOMbahlFIqAtL+ItTbECKPW4zbndIbC4TiWw/bJPFHeSuoQbHbsI TOgPjM4q2aydHFeTDL2Wqqm99pobPvnEji5sBzHOK+IyIaN0HZI7BZAsm38SDSEA HQTW342RNl7Vm0t2KPdhBq/43r4ENKSUbZjCctFZA+iqbV2mmE96zd2pLOIh9OLP 7KoCDKlplQjInJjN3MQvPzn/cxHnZOmt3AwhQjfhOMcJWcwz00ouWp38oszgZi1c 8y1htf1/Tr26veYxF7umlPHT0h8LNj4Wi7ceQjoIDPu4X70X6vXOk5sJmeJ+KqI5 PJ4yUgw0erWX0I7jybavIvruyA9ZqCwk5FIIyG54IBIZLfughi/fcce49gfvJTa6 Rdjvp0g28qxynkRKq9hVOBSX/TRXTedHe8tHBvSzoDRScvI2aDWQYjJkmGzzLF6L +eWiZ2ICJ2fs7XYIxPhEr/vSpmc/1R+k6vZIPeBTAs0muH8WoK2en6KZPmJ7C25L 2FHdA+1S1Ybqsvyue5S5Sl6yGcvpzGzc1R4mHoB0jZki2v3Cwcs= =j25Z -----END PGP SIGNATURE----- Merge tag 'v1.10.0' Synapse 1.10.0 (2020-02-12) =========================== **WARNING to client developers**: As of this release Synapse validates `client_secret` parameters in the Client-Server API as per the spec. See [\#6766](https://github.com/matrix-org/synapse/issues/6766) for details. Updates to the Docker image --------------------------- - Update the docker images to Alpine Linux 3.11. ([\#6897](https://github.com/matrix-org/synapse/issues/6897)) Synapse 1.10.0rc5 (2020-02-11) ============================== Bugfixes -------- - Fix the filtering introduced in 1.10.0rc3 to also apply to the state blocks returned by `/sync`. ([\#6884](https://github.com/matrix-org/synapse/issues/6884)) Synapse 1.10.0rc4 (2020-02-11) ============================== This release candidate was built incorrectly and is superceded by 1.10.0rc5. Synapse 1.10.0rc3 (2020-02-10) ============================== Features -------- - Filter out `m.room.aliases` from the CS API to mitigate abuse while a better solution is specced. ([\#6878](https://github.com/matrix-org/synapse/issues/6878)) Internal Changes ---------------- - Fix continuous integration failures with old versions of `pip`, which were introduced by a release of the `zipp` library. ([\#6880](https://github.com/matrix-org/synapse/issues/6880)) Synapse 1.10.0rc2 (2020-02-06) ============================== Bugfixes -------- - Fix an issue with cross-signing where device signatures were not sent to remote servers. ([\#6844](https://github.com/matrix-org/synapse/issues/6844)) - Fix to the unknown remote device detection which was introduced in 1.10.rc1. ([\#6848](https://github.com/matrix-org/synapse/issues/6848)) Internal Changes ---------------- - Detect unexpected sender keys on remote encrypted events and resync device lists. ([\#6850](https://github.com/matrix-org/synapse/issues/6850)) Synapse 1.10.0rc1 (2020-01-31) ============================== Features -------- - Add experimental support for updated authorization rules for aliases events, from [MSC2260](https://github.com/matrix-org/matrix-doc/pull/2260). ([\#6787](https://github.com/matrix-org/synapse/issues/6787), [\#6790](https://github.com/matrix-org/synapse/issues/6790), [\#6794](https://github.com/matrix-org/synapse/issues/6794)) Bugfixes -------- - Warn if postgres database has a non-C locale, as that can cause issues when upgrading locales (e.g. due to upgrading OS). ([\#6734](https://github.com/matrix-org/synapse/issues/6734)) - Minor fixes to `PUT /_synapse/admin/v2/users` admin api. ([\#6761](https://github.com/matrix-org/synapse/issues/6761)) - Validate `client_secret` parameter using the regex provided by the Client-Server API, temporarily allowing `:` characters for older clients. The `:` character will be removed in a future release. ([\#6767](https://github.com/matrix-org/synapse/issues/6767)) - Fix persisting redaction events that have been redacted (or otherwise don't have a redacts key). ([\#6771](https://github.com/matrix-org/synapse/issues/6771)) - Fix outbound federation request metrics. ([\#6795](https://github.com/matrix-org/synapse/issues/6795)) - Fix bug where querying a remote user's device keys that weren't cached resulted in only returning a single device. ([\#6796](https://github.com/matrix-org/synapse/issues/6796)) - Fix race in federation sender worker that delayed sending of device updates. ([\#6799](https://github.com/matrix-org/synapse/issues/6799), [\#6800](https://github.com/matrix-org/synapse/issues/6800)) - Fix bug where Synapse didn't invalidate cache of remote users' devices when Synapse left a room. ([\#6801](https://github.com/matrix-org/synapse/issues/6801)) - Fix waking up other workers when remote server is detected to have come back online. ([\#6811](https://github.com/matrix-org/synapse/issues/6811)) Improved Documentation ---------------------- - Clarify documentation related to `user_dir` and `federation_reader` workers. ([\#6775](https://github.com/matrix-org/synapse/issues/6775)) Internal Changes ---------------- - Record room versions in the `rooms` table. ([\#6729](https://github.com/matrix-org/synapse/issues/6729), [\#6788](https://github.com/matrix-org/synapse/issues/6788), [\#6810](https://github.com/matrix-org/synapse/issues/6810)) - Propagate cache invalidates from workers to other workers. ([\#6748](https://github.com/matrix-org/synapse/issues/6748)) - Remove some unnecessary admin handler abstraction methods. ([\#6751](https://github.com/matrix-org/synapse/issues/6751)) - Add some debugging for media storage providers. ([\#6757](https://github.com/matrix-org/synapse/issues/6757)) - Detect unknown remote devices and mark cache as stale. ([\#6776](https://github.com/matrix-org/synapse/issues/6776), [\#6819](https://github.com/matrix-org/synapse/issues/6819)) - Attempt to resync remote users' devices when detected as stale. ([\#6786](https://github.com/matrix-org/synapse/issues/6786)) - Delete current state from the database when server leaves a room. ([\#6792](https://github.com/matrix-org/synapse/issues/6792)) - When a client asks for a remote user's device keys check if the local cache for that user has been marked as potentially stale. ([\#6797](https://github.com/matrix-org/synapse/issues/6797)) - Add background update to clean out left rooms from current state. ([\#6802](https://github.com/matrix-org/synapse/issues/6802), [\#6816](https://github.com/matrix-org/synapse/issues/6816)) - Refactoring work in preparation for changing the event redaction algorithm. ([\#6803](https://github.com/matrix-org/synapse/issues/6803), [\#6805](https://github.com/matrix-org/synapse/issues/6805), [\#6806](https://github.com/matrix-org/synapse/issues/6806), [\#6807](https://github.com/matrix-org/synapse/issues/6807), [\#6820](https://github.com/matrix-org/synapse/issues/6820))	2020-02-12 13:23:22 +00:00
Brendan Abolivier	6b21986e4e	Also spell it out in the purge history API doc	2020-02-11 17:56:04 +00:00
Brendan Abolivier	a443d2a25d	Spell out that Synapse never purges the last event sent in a room	2020-02-11 17:37:09 +00:00
Erik Johnston	21db35f77e	Add support for putting fed user query API on workers (#6873)	2020-02-07 15:45:39 +00:00
Erik Johnston	de2d267375	Allow moving group read APIs to workers (#6866)	2020-02-07 11:14:19 +00:00
Dirk Klimpel	56ca93ef59	Admin api to add an email address (#6789)	2020-02-07 10:29:36 +00:00
Robin Vleij	f0561fcffd	Update documentation (#6859) ... Update documentation to reflect the correct format of user_id (fully qualified).	2020-02-05 21:27:38 +00:00
Erik Johnston	02b44db922	Warn if postgres database has non-C locale. (#6734) ... As using non-C locale can cause issues on upgrading OS.	2020-01-28 13:44:21 +00:00
Jason Robinson	1fe5001369	Fix federation_reader listeners doc as per PR review ... Signed-off-by: Jason Robinson <jasonr@matrix.org>	2020-01-27 10:21:25 +02:00
Jason Robinson	aa6ad288f1	Clarifications to the workers documentation ... * Add note that user_dir requires disabling user dir updates from the main synapse process. * Add note that federation_reader should have the federation listener resource. Signed-off-by: Jason Robinson <jasonr@matrix.org>	2020-01-24 11:08:50 +02:00
Andrew Morgan	d31f5f4d89	Update admin room docs with correct endpoints (#6770)	2020-01-23 11:37:26 +00:00
Andrew Morgan	90a28fb475	Admin API to list, filter and sort rooms (#6720)	2020-01-22 13:36:43 +00:00
Erik Johnston	5d7a6ad223	Allow streaming cache invalidate all to workers. (#6749)	2020-01-22 10:37:00 +00:00
Erik Johnston	a8a50f5b57	Wake up transaction queue when remote server comes back online (#6706) ... This will be used to retry outbound transactions to a remote server if we think it might have come back up.	2020-01-17 10:27:19 +00:00
Richard van der Hoff	5ce0b17e38	Clarify the account_validity and email sections of the sample configuration. (#6685) ... Generally try to make this more comprehensible, and make it match the conventions. I've removed the documentation for all the settings which allow you to change the names of the template files, because I can't really see why they are useful.	2020-01-17 10:04:15 +00:00
Andrew Morgan	1177d3f3a3	Quarantine media by ID or user ID (#6681)	2020-01-13 18:10:43 +00:00
Richard van der Hoff	47f4f493f0	Document more supported endpoints for workers (#6698)	2020-01-13 15:32:02 +00:00
Manuel Stahl	d2906fe666	Allow admin users to create or modify users without a shared secret (#6495) ... Signed-off-by: Manuel Stahl <manuel.stahl@awesome-technologies.de>	2020-01-09 13:31:00 +00:00
Brendan Abolivier	3889fcd9d7	Fix typo in message retention policies doc	2020-01-08 13:27:29 +00:00
Brendan Abolivier	cff1cb8685	Merge pull request #6624 from matrix-org/babolivier/retention_doc ... Add complete documentation of the message retention policies support	2020-01-08 11:24:47 +00:00
Brendan Abolivier	2b6b7f482a	Merge pull request #6621 from matrix-org/babolivier/purge_job_config_typo ... Fix a typo in the purge jobs configuration example	2020-01-07 16:17:40 +01:00
Brendan Abolivier	3675fb9bc6	Fix reference	2020-01-07 15:15:16 +00:00
Brendan Abolivier	7ba98a2874	Incorporate review	2020-01-07 15:14:33 +00:00
Brendan Abolivier	4be582d7c8	Merge branch 'develop' into babolivier/retention_doc	2020-01-07 15:07:19 +00:00
Brendan Abolivier	01fbd95736	Apply suggestions from code review ... Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>	2020-01-07 15:59:38 +01:00
Brendan Abolivier	391fb47791	Reword	2020-01-07 14:54:32 +00:00
Brendan Abolivier	3a86477162	Change the example from 5min to 12h ... Have a purge job running every 5min is probably not something we want to advise admins to do as a sort-of default.	2020-01-07 14:53:07 +00:00
Richard van der Hoff	08815566bc	Automate generation of the sample and debian log configs (#6627)	2020-01-03 17:14:00 +00:00
Richard van der Hoff	98247c4a0e	Remove unused, undocumented "content repo" resource (#6628) ... This looks like it got half-killed back in #888. Fixes #6567.	2020-01-03 17:10:52 +00:00
Brendan Abolivier	b7dec300b7	Fix vacuum instructions for sqlite	2020-01-03 13:51:59 +01:00
Brendan Abolivier	9279a2c4e4	Add a complete documentation of the message retention policies support	2020-01-03 13:45:03 +01:00
Brendan Abolivier	dd2954f78d	Update sample config	2020-01-03 12:58:12 +01:00
ewaf1	0495097a7f	Added the section 'Configuration' in /docs/turn-howto.md (#6614) ... put the 2nd part of the "source installation"-section into a new section, because it also applies to Debian packages	2020-01-02 10:41:30 +00:00
Aaron Raimist	32779b59fa	Reword sections of federate.md that explained delegation at time of Synapse 1.0 transition (#6601) ... * Remove sections of federate.md explaining delegation at time of Synapse 1.0 transition Signed-off-by: Aaron Raimist <aaron@raim.ist> * Add changelog Signed-off-by: Aaron Raimist <aaron@raim.ist>	2020-01-02 10:28:20 +00:00
Richard van der Hoff	dc96943d51	Merge branch 'master' into develop	2019-12-31 11:01:06 +00:00
Richard van der Hoff	f03c877b32	sample log config ... TODO: automate generation of this	2019-12-24 17:39:58 +00:00
Richard van der Hoff	b95b762560	Add an export_signing_key script (#6546) ... I want to do some key rotation, and it is silly that we don't have a way to do this.	2019-12-19 11:11:14 +00:00
Will Hunt	bfb95654c9	Add option to allow profile queries without sharing a room (#6523)	2019-12-16 16:11:55 +00:00
Andrew Morgan	0b90fc6ed2	Document Shutdown Room admin API (#6541)	2019-12-13 15:28:48 +00:00
Erik Johnston	f46e05d053	Merge branch 'master' into develop	2019-12-13 10:55:47 +00:00
Erik Johnston	bee1982d17	Synapse 1.7.0 (2019-12-13) ... ========================== This release changes the default settings so that only local authenticated users can query the server's room directory. See the [upgrade notes](UPGRADE.rst#upgrading-to-v170) for details. Support for SQLite versions before 3.11 is now deprecated. A future release will refuse to start if used with an SQLite version before 3.11. Administrators are reminded that SQLite should not be used for production instances. Instructions for migrating to Postgres are available [here](docs/postgres.md). A future release of synapse will, by default, disable federation for servers using SQLite. No significant changes since 1.7.0rc2. Synapse 1.7.0rc2 (2019-12-11) ============================= Bugfixes -------- - Fix incorrect error message for invalid requests when setting user's avatar URL. ([\#6497](https://github.com/matrix-org/synapse/issues/6497)) - Fix support for SQLite 3.7. ([\#6499](https://github.com/matrix-org/synapse/issues/6499)) - Fix regression where sending email push would not work when using a pusher worker. ([\#6507](https://github.com/matrix-org/synapse/issues/6507), [\#6509](https://github.com/matrix-org/synapse/issues/6509)) Synapse 1.7.0rc1 (2019-12-09) ============================= Features -------- - Implement per-room message retention policies. ([\#5815](https://github.com/matrix-org/synapse/issues/5815), [\#6436](https://github.com/matrix-org/synapse/issues/6436)) - Add etag and count fields to key backup endpoints to help clients guess if there are new keys. ([\#5858](https://github.com/matrix-org/synapse/issues/5858)) - Add `/admin/v2/users` endpoint with pagination. Contributed by Awesome Technologies Innovationslabor GmbH. ([\#5925](https://github.com/matrix-org/synapse/issues/5925)) - Require User-Interactive Authentication for `/account/3pid/add`, meaning the user's password will be required to add a third-party ID to their account. ([\#6119](https://github.com/matrix-org/synapse/issues/6119)) - Implement the `/_matrix/federation/unstable/net.atleastfornow/state/<context>` API as drafted in MSC2314. ([\#6176](https://github.com/matrix-org/synapse/issues/6176)) - Configure privacy-preserving settings by default for the room directory. ([\#6355](https://github.com/matrix-org/synapse/issues/6355)) - Add ephemeral messages support by partially implementing [MSC2228](https://github.com/matrix-org/matrix-doc/pull/2228). ([\#6409](https://github.com/matrix-org/synapse/issues/6409)) - Add support for [MSC 2367](https://github.com/matrix-org/matrix-doc/pull/2367), which allows specifying a reason on all membership events. ([\#6434](https://github.com/matrix-org/synapse/issues/6434)) Bugfixes -------- - Transfer non-standard power levels on room upgrade. ([\#6237](https://github.com/matrix-org/synapse/issues/6237)) - Fix error from the Pillow library when uploading RGBA images. ([\#6241](https://github.com/matrix-org/synapse/issues/6241)) - Correctly apply the event filter to the `state`, `events_before` and `events_after` fields in the response to `/context` requests. ([\#6329](https://github.com/matrix-org/synapse/issues/6329)) - Fix caching devices for remote users when using workers, so that we don't attempt to refetch (and potentially fail) each time a user requests devices. ([\#6332](https://github.com/matrix-org/synapse/issues/6332)) - Prevent account data syncs getting lost across TCP replication. ([\#6333](https://github.com/matrix-org/synapse/issues/6333)) - Fix bug: TypeError in `register_user()` while using LDAP auth module. ([\#6406](https://github.com/matrix-org/synapse/issues/6406)) - Fix an intermittent exception when handling read-receipts. ([\#6408](https://github.com/matrix-org/synapse/issues/6408)) - Fix broken guest registration when there are existing blocks of numeric user IDs. ([\#6420](https://github.com/matrix-org/synapse/issues/6420)) - Fix startup error when http proxy is defined. ([\#6421](https://github.com/matrix-org/synapse/issues/6421)) - Fix error when using synapse_port_db on a vanilla synapse db. ([\#6449](https://github.com/matrix-org/synapse/issues/6449)) - Fix uploading multiple cross signing signatures for the same user. ([\#6451](https://github.com/matrix-org/synapse/issues/6451)) - Fix bug which lead to exceptions being thrown in a loop when a cross-signed device is deleted. ([\#6462](https://github.com/matrix-org/synapse/issues/6462)) - Fix `synapse_port_db` not exiting with a 0 code if something went wrong during the port process. ([\#6470](https://github.com/matrix-org/synapse/issues/6470)) - Improve sanity-checking when receiving events over federation. ([\#6472](https://github.com/matrix-org/synapse/issues/6472)) - Fix inaccurate per-block Prometheus metrics. ([\#6491](https://github.com/matrix-org/synapse/issues/6491)) - Fix small performance regression for sending invites. ([\#6493](https://github.com/matrix-org/synapse/issues/6493)) - Back out cross-signing code added in Synapse 1.5.0, which caused a performance regression. ([\#6494](https://github.com/matrix-org/synapse/issues/6494)) Improved Documentation ---------------------- - Update documentation and variables in user contributed systemd reference file. ([\#6369](https://github.com/matrix-org/synapse/issues/6369), [\#6490](https://github.com/matrix-org/synapse/issues/6490)) - Fix link in the user directory documentation. ([\#6388](https://github.com/matrix-org/synapse/issues/6388)) - Add build instructions to the docker readme. ([\#6390](https://github.com/matrix-org/synapse/issues/6390)) - Switch Ubuntu package install recommendation to use python3 packages in INSTALL.md. ([\#6443](https://github.com/matrix-org/synapse/issues/6443)) - Write some docs for the quarantine_media api. ([\#6458](https://github.com/matrix-org/synapse/issues/6458)) - Convert CONTRIBUTING.rst to markdown (among other small fixes). ([\#6461](https://github.com/matrix-org/synapse/issues/6461)) Deprecations and Removals ------------------------- - Remove admin/v1/users_paginate endpoint. Contributed by Awesome Technologies Innovationslabor GmbH. ([\#5925](https://github.com/matrix-org/synapse/issues/5925)) - Remove fallback for federation with old servers which lack the /federation/v1/state_ids API. ([\#6488](https://github.com/matrix-org/synapse/issues/6488)) Internal Changes ---------------- - Add benchmarks for structured logging and improve output performance. ([\#6266](https://github.com/matrix-org/synapse/issues/6266)) - Improve the performance of outputting structured logging. ([\#6322](https://github.com/matrix-org/synapse/issues/6322)) - Refactor some code in the event authentication path for clarity. ([\#6343](https://github.com/matrix-org/synapse/issues/6343), [\#6468](https://github.com/matrix-org/synapse/issues/6468), [\#6480](https://github.com/matrix-org/synapse/issues/6480)) - Clean up some unnecessary quotation marks around the codebase. ([\#6362](https://github.com/matrix-org/synapse/issues/6362)) - Complain on startup instead of 500'ing during runtime when `public_baseurl` isn't set when necessary. ([\#6379](https://github.com/matrix-org/synapse/issues/6379)) - Add a test scenario to make sure room history purges don't break `/messages` in the future. ([\#6392](https://github.com/matrix-org/synapse/issues/6392)) - Clarifications for the email configuration settings. ([\#6423](https://github.com/matrix-org/synapse/issues/6423)) - Add more tests to the blacklist when running in worker mode. ([\#6429](https://github.com/matrix-org/synapse/issues/6429)) - Refactor data store layer to support multiple databases in the future. ([\#6454](https://github.com/matrix-org/synapse/issues/6454), [\#6464](https://github.com/matrix-org/synapse/issues/6464), [\#6469](https://github.com/matrix-org/synapse/issues/6469), [\#6487](https://github.com/matrix-org/synapse/issues/6487)) - Port synapse.rest.client.v1 to async/await. ([\#6482](https://github.com/matrix-org/synapse/issues/6482)) - Port synapse.rest.client.v2_alpha to async/await. ([\#6483](https://github.com/matrix-org/synapse/issues/6483)) - Port SyncHandler to async/await. ([\#6484](https://github.com/matrix-org/synapse/issues/6484)) -----BEGIN PGP SIGNATURE----- iQJEBAABCgAuFiEEumuwyPtYLL2OMhYdOtoG7cdT0R4FAl3zbaUQHGVyaWtAbWF0 cml4Lm9yZwAKCRA62gbtx1PRHqxED/9Ldjx5pxRm99Rkfhpv6qaw79GaQsAERW+i rPzdC0u59PRDLyfB2MRGU+q4oTTmBN3APPuW79mhY4CRisKfUvw6tuaJM5avvRsF vlY6m4ObewPXvK9dumlKD4ORqks45Sv0VGLHoZH4MLvWu/3Z+ycUn0qf+KNy0JrV TNNnNfsEeo3FXrkLU/12TiEBqNTVCU2vi/nZaHoGlHtZGVoykyFk4z2i9hLKENH4 ScwaN4zeFRTIWYMiEb+kI/6QQLTASvar5FstwXPpM2F+aZhrMc5M7YsEe16jx+bQ jOM1L67hMetcWWmE9QrgP030clIDg30xhNbVxFSD4WoGgEPFkf/zE91v0CF5ZMRJ u83hlhe8tHoO8DOfVl2DNw3x6Evx4Vb5P8bvszNwKAyVDanvkSe0GvYm84B0Tat6 2UCyPR5mbU3fyiwpouaNYun8MppyMRQ/fCnNElvZwwoj9EQQcJ8QOnMK0pFv1AXV n4wwXqRb+WvvbhnKPxTsiTK0DPBbIiGqo1TIONSbhPF+wySR3YV80HCWofd+qlRT V+GbZihk4SCcmamgqYLeQYneOrIu6UWGP348t3scCz/RCUp7hmxaJhJ3GAC3hOmU alN6tDJn7Lo0CwY2frX8xPS6S31sq9IZyDhHkaL0chsEa1rgWJMOK3FRp/b+c8yU b3CEJObIjw== =RQVG -----END PGP SIGNATURE----- Merge tag 'v1.7.0' Synapse 1.7.0 (2019-12-13) ========================== This release changes the default settings so that only local authenticated users can query the server's room directory. See the [upgrade notes](UPGRADE.rst#upgrading-to-v170) for details. Support for SQLite versions before 3.11 is now deprecated. A future release will refuse to start if used with an SQLite version before 3.11. Administrators are reminded that SQLite should not be used for production instances. Instructions for migrating to Postgres are available [here](docs/postgres.md). A future release of synapse will, by default, disable federation for servers using SQLite. No significant changes since 1.7.0rc2. Synapse 1.7.0rc2 (2019-12-11) ============================= Bugfixes -------- - Fix incorrect error message for invalid requests when setting user's avatar URL. ([\#6497](https://github.com/matrix-org/synapse/issues/6497)) - Fix support for SQLite 3.7. ([\#6499](https://github.com/matrix-org/synapse/issues/6499)) - Fix regression where sending email push would not work when using a pusher worker. ([\#6507](https://github.com/matrix-org/synapse/issues/6507), [\#6509](https://github.com/matrix-org/synapse/issues/6509)) Synapse 1.7.0rc1 (2019-12-09) ============================= Features -------- - Implement per-room message retention policies. ([\#5815](https://github.com/matrix-org/synapse/issues/5815), [\#6436](https://github.com/matrix-org/synapse/issues/6436)) - Add etag and count fields to key backup endpoints to help clients guess if there are new keys. ([\#5858](https://github.com/matrix-org/synapse/issues/5858)) - Add `/admin/v2/users` endpoint with pagination. Contributed by Awesome Technologies Innovationslabor GmbH. ([\#5925](https://github.com/matrix-org/synapse/issues/5925)) - Require User-Interactive Authentication for `/account/3pid/add`, meaning the user's password will be required to add a third-party ID to their account. ([\#6119](https://github.com/matrix-org/synapse/issues/6119)) - Implement the `/_matrix/federation/unstable/net.atleastfornow/state/<context>` API as drafted in MSC2314. ([\#6176](https://github.com/matrix-org/synapse/issues/6176)) - Configure privacy-preserving settings by default for the room directory. ([\#6355](https://github.com/matrix-org/synapse/issues/6355)) - Add ephemeral messages support by partially implementing [MSC2228](https://github.com/matrix-org/matrix-doc/pull/2228). ([\#6409](https://github.com/matrix-org/synapse/issues/6409)) - Add support for [MSC 2367](https://github.com/matrix-org/matrix-doc/pull/2367), which allows specifying a reason on all membership events. ([\#6434](https://github.com/matrix-org/synapse/issues/6434)) Bugfixes -------- - Transfer non-standard power levels on room upgrade. ([\#6237](https://github.com/matrix-org/synapse/issues/6237)) - Fix error from the Pillow library when uploading RGBA images. ([\#6241](https://github.com/matrix-org/synapse/issues/6241)) - Correctly apply the event filter to the `state`, `events_before` and `events_after` fields in the response to `/context` requests. ([\#6329](https://github.com/matrix-org/synapse/issues/6329)) - Fix caching devices for remote users when using workers, so that we don't attempt to refetch (and potentially fail) each time a user requests devices. ([\#6332](https://github.com/matrix-org/synapse/issues/6332)) - Prevent account data syncs getting lost across TCP replication. ([\#6333](https://github.com/matrix-org/synapse/issues/6333)) - Fix bug: TypeError in `register_user()` while using LDAP auth module. ([\#6406](https://github.com/matrix-org/synapse/issues/6406)) - Fix an intermittent exception when handling read-receipts. ([\#6408](https://github.com/matrix-org/synapse/issues/6408)) - Fix broken guest registration when there are existing blocks of numeric user IDs. ([\#6420](https://github.com/matrix-org/synapse/issues/6420)) - Fix startup error when http proxy is defined. ([\#6421](https://github.com/matrix-org/synapse/issues/6421)) - Fix error when using synapse_port_db on a vanilla synapse db. ([\#6449](https://github.com/matrix-org/synapse/issues/6449)) - Fix uploading multiple cross signing signatures for the same user. ([\#6451](https://github.com/matrix-org/synapse/issues/6451)) - Fix bug which lead to exceptions being thrown in a loop when a cross-signed device is deleted. ([\#6462](https://github.com/matrix-org/synapse/issues/6462)) - Fix `synapse_port_db` not exiting with a 0 code if something went wrong during the port process. ([\#6470](https://github.com/matrix-org/synapse/issues/6470)) - Improve sanity-checking when receiving events over federation. ([\#6472](https://github.com/matrix-org/synapse/issues/6472)) - Fix inaccurate per-block Prometheus metrics. ([\#6491](https://github.com/matrix-org/synapse/issues/6491)) - Fix small performance regression for sending invites. ([\#6493](https://github.com/matrix-org/synapse/issues/6493)) - Back out cross-signing code added in Synapse 1.5.0, which caused a performance regression. ([\#6494](https://github.com/matrix-org/synapse/issues/6494)) Improved Documentation ---------------------- - Update documentation and variables in user contributed systemd reference file. ([\#6369](https://github.com/matrix-org/synapse/issues/6369), [\#6490](https://github.com/matrix-org/synapse/issues/6490)) - Fix link in the user directory documentation. ([\#6388](https://github.com/matrix-org/synapse/issues/6388)) - Add build instructions to the docker readme. ([\#6390](https://github.com/matrix-org/synapse/issues/6390)) - Switch Ubuntu package install recommendation to use python3 packages in INSTALL.md. ([\#6443](https://github.com/matrix-org/synapse/issues/6443)) - Write some docs for the quarantine_media api. ([\#6458](https://github.com/matrix-org/synapse/issues/6458)) - Convert CONTRIBUTING.rst to markdown (among other small fixes). ([\#6461](https://github.com/matrix-org/synapse/issues/6461)) Deprecations and Removals ------------------------- - Remove admin/v1/users_paginate endpoint. Contributed by Awesome Technologies Innovationslabor GmbH. ([\#5925](https://github.com/matrix-org/synapse/issues/5925)) - Remove fallback for federation with old servers which lack the /federation/v1/state_ids API. ([\#6488](https://github.com/matrix-org/synapse/issues/6488)) Internal Changes ---------------- - Add benchmarks for structured logging and improve output performance. ([\#6266](https://github.com/matrix-org/synapse/issues/6266)) - Improve the performance of outputting structured logging. ([\#6322](https://github.com/matrix-org/synapse/issues/6322)) - Refactor some code in the event authentication path for clarity. ([\#6343](https://github.com/matrix-org/synapse/issues/6343), [\#6468](https://github.com/matrix-org/synapse/issues/6468), [\#6480](https://github.com/matrix-org/synapse/issues/6480)) - Clean up some unnecessary quotation marks around the codebase. ([\#6362](https://github.com/matrix-org/synapse/issues/6362)) - Complain on startup instead of 500'ing during runtime when `public_baseurl` isn't set when necessary. ([\#6379](https://github.com/matrix-org/synapse/issues/6379)) - Add a test scenario to make sure room history purges don't break `/messages` in the future. ([\#6392](https://github.com/matrix-org/synapse/issues/6392)) - Clarifications for the email configuration settings. ([\#6423](https://github.com/matrix-org/synapse/issues/6423)) - Add more tests to the blacklist when running in worker mode. ([\#6429](https://github.com/matrix-org/synapse/issues/6429)) - Refactor data store layer to support multiple databases in the future. ([\#6454](https://github.com/matrix-org/synapse/issues/6454), [\#6464](https://github.com/matrix-org/synapse/issues/6464), [\#6469](https://github.com/matrix-org/synapse/issues/6469), [\#6487](https://github.com/matrix-org/synapse/issues/6487)) - Port synapse.rest.client.v1 to async/await. ([\#6482](https://github.com/matrix-org/synapse/issues/6482)) - Port synapse.rest.client.v2_alpha to async/await. ([\#6483](https://github.com/matrix-org/synapse/issues/6483)) - Port SyncHandler to async/await. ([\#6484](https://github.com/matrix-org/synapse/issues/6484))	2019-12-13 10:55:33 +00:00
Mark Nowiasz	58fdcbdfe7	Update workers.md to make media_repository work (again) (#6519)	2019-12-11 16:23:38 +00:00
Andrew Morgan	4947de5a14	Allow SAML username provider plugins (#6411)	2019-12-10 17:30:16 +00:00
Manuel Stahl	649b6bc088	Replace /admin/v1/users_paginate endpoint with /admin/v2/users (#5925)	2019-12-05 18:12:23 +00:00
Neil Johnson	cb0aeb147e	privacy by default for room dir (#6355) ... Ensure that the the default settings for the room directory are that the it is hidden from public view by default.	2019-12-04 09:46:16 +00:00
Richard van der Hoff	620f98b65b	write some docs for the quarantine_media api (#6458)	2019-12-03 18:20:39 +00:00
Richard van der Hoff	c48ea98007	Clarifications for the email configuration settings. (#6423) ... Cf #6422	2019-11-28 09:29:18 +00:00
Brendan Abolivier	9e937c28ee	Merge branch 'develop' into babolivier/message_retention	2019-11-26 17:53:57 +00:00
Aaron Raimist	24cc31ee96	Fix link to user_dir_populate.sql in the user directory docs (#6388)	2019-11-21 17:38:14 +00:00
Andrew Morgan	473acedcdd	Merge branch 'develop' of github.com:matrix-org/synapse into anoa/homeserver_copy ... * 'develop' of github.com:matrix-org/synapse: Blacklist PurgeRoomTestCase (#6361) Set room version default to 5	2019-11-14 10:26:27 +00:00
Brendan Abolivier	a42567e4a8	Merge pull request #6220 from matrix-org/neilj/set_room_version_default_to_5 ... Set room version default to 5	2019-11-14 10:21:00 +00:00
Andrew Morgan	e1648dc576	sample config	2019-11-12 13:15:59 +00:00
Brendan Abolivier	09957ce0e4	Implement per-room message retention policies	2019-11-04 17:09:22 +00:00
Richard van der Hoff	cc6243b4c0	document the REPLICATE command a bit better (#6305) ... since I found myself wonder how it works	2019-11-04 12:40:18 +00:00
Yash Jipkate	9677613e9c	Modify doc to update Google ReCaptcha terms (#6257)	2019-10-30 12:30:20 +00:00
Andrew Morgan	46c12918ad	Fix typo in domain name in account_threepid_delegates config option (#6273)	2019-10-30 11:07:42 +00:00
Neil Johnson	2794b79052	Option to suppress resource exceeded alerting (#6173) ... The expected use case is to suppress MAU limiting on small instances	2019-10-24 11:48:46 +01:00
Andrew Morgan	409c62b27b	Add config linting script that checks for bool casing (#6203) ... Add a linting script that enforces all boolean values in the default config be lowercase. This has annoyed me for a while so I decided to fix it.	2019-10-23 13:22:54 +01:00
Adrien Luxey	0327a00a37	Update postgres.md (#6234) ... Added database owner authentication with `sudo` when `su` does not work	2019-10-22 13:48:02 +02:00
Neil Johnson	82c8799ec7	Set room version default to 5	2019-10-19 09:06:15 +01:00
Valérian Rousset	be9b55e0d2	cas: support setting display name (#6114) ... Now, the CAS server can return an attribute stating what's the desired displayname, instead of using the username directly.	2019-10-11 12:33:12 +01:00
Brendan Abolivier	baf12bc02a	Merge branch 'master' into develop	2019-10-02 14:41:02 +01:00
Erik Johnston	3423633d50	Fix 'redaction_retention_period' sampel config to match guidelines	2019-09-26 16:43:52 +01:00
Neil Johnson	8b8f8c7b3c	Explicitly log when a homeserver does not have a trusted key server configured (#6090)	2019-09-26 12:57:01 +01:00
Erik Johnston	4fb3c129aa	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/cleanup_user_ips_2	2019-09-25 17:53:13 +01:00
Erik Johnston	39b50ad42a	Review comments	2019-09-25 17:22:33 +01:00
Erik Johnston	242017db8b	Prune rows in user_ips older than configured period ... Defaults to pruning everything older than 28d.	2019-09-24 15:53:17 +01:00
Richard van der Hoff	ed8b92f0d2	Merge remote-tracking branch 'origin/develop' into rav/saml_mapping_work	2019-09-24 12:57:32 +01:00
Andrew Morgan	50776261e1	Add submit_url response parameter to msisdn /requestToken (#6079) ... Second part of solving #6076 Fixes #6076 We return a submit_url parameter on calls to POST */msisdn/requestToken so that clients know where to submit token information to.	2019-09-23 21:21:03 +01:00
Richard van der Hoff	78e8ec368e	Merge pull request #6064 from matrix-org/rav/saml_config_cleanup ... Make the sample saml config closer to our standards	2019-09-23 20:36:51 +01:00
Andrew Morgan	e08ea43463	Use the federation blacklist for requests to untrusted Identity Servers (#6000) ... Uses a SimpleHttpClient instance equipped with the federation_ip_range_blacklist list for requests to identity servers provided by user input. Does not use a blacklist when contacting identity servers specified by account_threepid_delegates. The homeserver trusts the latter and we don't want to prevent homeserver admins from specifying delegates that are on internal IP addresses. Fixes #5935	2019-09-23 20:23:20 +01:00
Andrew Morgan	df3401a71d	Allow HS to send emails when adding an email to the HS (#6042)	2019-09-20 15:21:30 +01:00
Richard van der Hoff	b65327ff66	Merge branch 'develop' into rav/saml_mapping_work	2019-09-19 18:13:31 +01:00
Richard van der Hoff	b789c7eb03	Merge branch 'develop' into rav/saml_config_cleanup	2019-09-19 15:05:31 +01:00
Jorik Schellekens	38fd1f8e3f	Fix typo in account_threepid_delegates config (#6028)	2019-09-18 22:30:44 +01:00
Richard van der Hoff	7100b5cc9d	fix sample config ... this was apparently broken by #6040.	2019-09-18 10:16:00 +01:00
dstipp	379d2a8c39	(#5849) Convert rst to markdown (#6040) ... Converting some of the rst documentation to markdown. Attempted to preserve whitespace and line breaks to minimize cosmetic change.	2019-09-17 12:55:29 +01:00
Richard van der Hoff	a8ac40445c	Record mappings from saml users in an external table ... We want to assign unique mxids to saml users based on an incrementing suffix. For that to work, we need to record the allocated mxid in a separate table.	2019-09-13 16:01:46 +01:00
Richard van der Hoff	785cbd3999	Make the sample saml config closer to our standards ... It' still not great, thanks to the nested dictionaries, but it's better.	2019-09-13 12:07:03 +01:00
Travis Ralston	c755955f33	Add developer docs for using SAML without a server (#6032)	2019-09-13 08:58:18 +01:00
axel simon	1c7df13e7b	add explanations on how to actually include an access_token (#6031)	2019-09-13 08:50:17 +01:00
Sorunome	dd2e5b0038	add report_stats_endpoint config option (#6012) ... This PR adds the optional `report_stats_endpoint` to configure where stats are reported to, if enabled.	2019-09-12 11:24:57 +01:00
Erik Johnston	66ace43546	Update sample config	2019-09-11 14:50:40 +01:00
Erik Johnston	8df88b5ff3	Update sample config	2019-09-11 10:58:26 +01:00
Erik Johnston	470dc621ae	Merge pull request #5934 from matrix-org/erikj/censor_redactions ... Censor redactions in DB after a month	2019-09-09 15:29:39 +01:00
Erik Johnston	8b9ade8c78	Default to censoring redactions after seven days	2019-09-09 13:55:28 +01:00
Amber Brown	55d5b3af88	Servers-known-about statistic (#5981)	2019-09-07 01:45:51 +10:00
Andrew Morgan	0c0b82b6d1	Allow Synapse to send registration emails + choose Synapse or an external server to handle 3pid validation (#5987) ... This is a combination of a few different PRs, finally all being merged into `develop`: * #5875 * #5876 * #5868 (This one added the `/versions` flag but the flag itself was actually [backed out](891afb57cb (diff-e591d42d30690ffb79f63bb726200891)) in #5969. What's left is just giving /versions access to the config file, which could be useful in the future) * #5835 * #5969 * #5940 Clients should not actually use the new registration functionality until https://github.com/matrix-org/synapse/pull/5972 is merged. UPGRADE.rst, changelog entries and config file changes should all be reviewed closely before this PR is merged.	2019-09-06 11:35:28 +01:00
Erik Johnston	591d82f06b	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/censor_redactions	2019-09-05 17:27:46 +01:00
Erik Johnston	3ff0422d2d	Make redaction retention period configurable	2019-09-05 17:16:45 +01:00
Erik Johnston	6e834e94fc	Fix and refactor room and user stats (#5971) ... Previously the stats were not being correctly populated.	2019-09-04 13:04:27 +01:00
Jorik Schellekens	92c1550f4a	Add a link to python's logging config schema (#5926)	2019-08-28 19:08:32 +01:00
Jorik Schellekens	6d97843793	Config templating (#5900) ... Template config files * Imagine a system composed entirely of x, y, z etc and the basic operations.. Wait George, why XOR? Why not just neq? George: Eh, I didn't think of that.. Co-Authored-By: Erik Johnston <erik@matrix.org>	2019-08-28 13:12:22 +01:00
Amber Brown	7dc398586c	Implement a structured logging output system. (#5680)	2019-08-28 21:18:53 +10:00
reivilibre	a3f0635686	Merge pull request #5914 from matrix-org/rei/admin_getadmin ... Add GET method to admin API /users/@user:dom/admin	2019-08-28 09:44:22 +01:00
Olivier Wilkinson (reivilibre)	1b959b6977	Document GET method for retrieving admin bit of user in admin API ... Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>	2019-08-27 13:19:19 +01:00
Erik Johnston	f5b50d0871	Merge pull request #5895 from matrix-org/erikj/notary_key ... Add config option to sign remote key query responses with a separate key.	2019-08-27 11:51:37 +01:00
reivilibre	1a7e6eb633	Add Admin API capability to set adminship of a user (#5878) ... Admin API: Set adminship of a user	2019-08-27 10:14:00 +01:00
Erik Johnston	7af5a63063	Fixup review comments	2019-08-23 15:36:28 +01:00
Jorik Schellekens	8767b63a82	Propagate opentracing contexts through EDUs (#5852) ... Propagate opentracing contexts through EDUs Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>	2019-08-22 18:21:10 +01:00
Richard van der Hoff	119aa31b10	Servlet to purge old rooms (#5845)	2019-08-22 10:42:59 +01:00
Erik Johnston	5906be8589	Add config option for keys to use to sign keys ... This allows servers to separate keys that are used to sign remote keys when acting as a notary server.	2019-08-21 10:44:58 +01:00
Amber Brown	0b6fbb28a8	Don't load the media repo when configured to use an external media repo (#5754)	2019-08-13 21:49:28 +10:00
Brendan Abolivier	f25f638c35	Lint	2019-08-01 12:19:08 +02:00
Brendan Abolivier	3ff3dfe5a3	Sample config	2019-08-01 12:08:25 +02:00
Richard van der Hoff	8c97f6414c	Remove non-functional 'expire_access_token' setting (#5782) ... The `expire_access_token` didn't do what it sounded like it should do. What it actually did was make Synapse enforce the 'time' caveat on macaroons used as access tokens, but since our access token macaroons never contained such a caveat, it was always a no-op. (The code to add 'time' caveats was removed back in v0.18.5, in #1656)	2019-07-30 08:25:02 +01:00
Amber Brown	865077f1d1	Room Complexity Client Implementation (#5783)	2019-07-30 02:47:27 +10:00
Jorik Schellekens	3641784e8c	Make Jaeger fully configurable (#5694) ... * Allow Jaeger to be configured * Update sample config	2019-07-23 15:46:04 +01:00
Amber Brown	4806651744	Replace returnValue with return (#5736)	2019-07-23 23:00:55 +10:00
Jorik Schellekens	0fd171770a	Merge branch 'release-v1.2.0' into develop	2019-07-22 11:18:50 +01:00
Jorik Schellekens	826e6ec3bd	Opentracing Documentation (#5703) ... * Opentracing survival guide * Update decorator names in doc * Doc cleanup These are all alterations as a result of comments in #5703, it includes mostly typos and clarifications. The most interesting changes are: - Split developer and user docs into two sections - Add a high level description of OpenTracing * newsfile * Move contributer specific info to docstring. * Sample config. * Trailing whitespace. * Update 5703.misc * Apply suggestions from code review Mostly just rewording parts of the docs for clarity. Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>	2019-07-22 11:15:21 +01:00
Richard van der Hoff	b73ce4ba81	Update the coding style doc (#5719) ... A few fixes and removal of duplicated stuff, but mostly a bunch of the words on the config file.	2019-07-19 11:55:14 +01:00
Richard van der Hoff	82345bc09a	Clean up opentracing configuration options (#5712) ... Clean up config settings and dead code. This is mostly about cleaning up the config format, to bring it into line with our conventions. In particular: * There should be a blank line after `## Section ##' headings * There should be a blank line between each config setting * There should be a `#`-only line between a comment and the setting it describes * We don't really do the `# #` style commenting-out of whole sections if we can help it * rename `tracer_enabled` to `enabled` While we're here, do more config parsing upfront, which makes it easier to use later on. Also removes redundant code from LogContextScopeManager. Also changes the changelog fragment to a `feature` - it's exciting!	2019-07-18 15:06:54 +01:00
Amber Brown	7ad1d76356	Support Prometheus_client 0.4.0+ (#5636)	2019-07-18 23:57:15 +10:00
Richard van der Hoff	1def298119	Improve Depends specs in debian package. (#5675) ... This is basically a contrived way of adding a `Recommends` on `libpq5`, to fix #5653. The way this is supposed to happen in debhelper is to run `dh_shlibdeps`, which in turn runs `dpkg-shlibdeps`, which spits things out into `debian/<package>.substvars` whence they can later be included by `control`. Previously, we had disabled `dh_shlibdeps`, mostly because `dpkg-shlibdeps` gets confused about PIL's interdependent objects, but that's not really the right thing to do and there is another way to work around that. Since we don't always use postgres, we don't necessarily want a hard Depends on libpq5, so I've actually ended up adding an explicit invocation of `dpkg-shlibdeps` for `psycopg2`. I've also updated the build-depends list for the package, which was missing a couple of entries.	2019-07-17 17:47:07 +01:00
Richard van der Hoff	5f158ec039	Implement access token expiry (#5660) ... Record how long an access token is valid for, and raise a soft-logout once it expires.	2019-07-12 17:26:02 +01:00
Ulrik Günther	d445b3ae57	Update reverse_proxy.rst (#5397) ... Updates reverse_proxy.rst with information about nginx' URI normalisation.	2019-07-12 11:46:18 +01:00
Lrizika	39e9839a04	Improved docs on setting up Postgresql (#5661) ... Added that synapse_user needs a database to access before it can auth Noted you'll need to enable password auth, linked to pg_hba.conf docs	2019-07-11 14:31:36 +01:00
Jorik Schellekens	38a6d3eea7	Add basic opentracing support (#5544) ... * Configure and initialise tracer Includes config options for the tracer and sets up JaegerClient. * Scope manager using LogContexts We piggy-back our tracer scopes by using log context. The current log context gives us the current scope. If new scope is created we create a stack of scopes in the context. * jaeger is a dependency now * Carrier inject and extraction for Twisted Headers * Trace federation requests on the way in and out. The span is created in _started_processing and closed in _finished_processing because we need a meaningful log context. * Create logcontext for new scope. Instead of having a stack of scopes in a logcontext we create a new context for a new scope if the current logcontext already has a scope. * Remove scope from logcontext if logcontext is top level * Disable tracer if not configured * typo * Remove dependence on jaeger internals * bools * Set service name * :Explicitely state that the tracer is disabled * Black is the new black * Newsfile * Code style * Use the new config setup. * Generate config. * Copyright * Rename config to opentracing * Remove user whitelisting * Empty whitelist by default * User ConfigError instead of RuntimeError * Use isinstance * Use tag constants for opentracing. * Remove debug comment and no need to explicitely record error * Two errors a "s(c)entry" * Docstrings! * Remove debugging brainslip * Homeserver Whitlisting * Better opentracing config comment * linting * Inclue worker name in service_name * Make opentracing an optional dependency * Neater config retreival * Clean up dummy tags * Instantiate tracing as object instead of global class * Inlcude opentracing as a homeserver member. * Thread opentracing to the request level * Reference opetnracing through hs * Instantiate dummy opentracin g for tests. * About to revert, just keeping the unfinished changes just in case * Revert back to global state, commit number: 9ce4a3d9067bf9889b86c360c05ac88618b85c4f * Use class level methods in tracerutils * Start and stop requests spans in a place where we have access to the authenticated entity * Seen it, isort it * Make sure to close the active span. * I'm getting black and blue from this. * Logger formatting Co-Authored-By: Erik Johnston <erik@matrix.org> * Outdated comment * Import opentracing at the top * Return a contextmanager * Start tracing client requests from the servlet * Return noop context manager if not tracing * Explicitely say that these are federation requests * Include servlet name in client requests * Use context manager * Move opentracing to logging/ * Seen it, isort it again! * Ignore twisted return exceptions on context exit * Escape the scope * Scopes should be entered to make them useful. * Nicer decorator names * Just one init, init? * Don't need to close something that isn't open * Docs make you smarter	2019-07-11 10:36:03 +01:00
Amber Brown	463b072b12	Move logging utilities out of the side drawer of util/ and into logging/ (#5606)	2019-07-04 00:07:04 +10:00
Richard van der Hoff	b4fd86a9b4	Merge branch 'develop' into rav/saml2_client	2019-07-01 14:21:03 +01:00
Amber Brown	be3b901ccd	Update the TLS cipher string and provide configurability for TLS on outgoing federation (#5550)	2019-06-28 18:19:09 +10:00
Daniel Hoffend	9646a593ac	Added possibilty to disable local password authentication (#5092) ... Signed-off-by: Daniel Hoffend <dh@dotlan.net>	2019-06-27 18:37:29 +01:00
Andrew Morgan	c548dbc4b1	Make it clearer that the template dir is relative to synapse's root dir (#5543) ... Helps address #5444	2019-06-27 18:20:17 +01:00
Richard van der Hoff	47fa836abb	Merge pull request #5313 from twrist/patch-1 ... Update HAProxy example rules	2019-06-27 00:53:48 +01:00
Richard van der Hoff	dde4118341	update sample config	2019-06-27 00:41:04 +01:00
Richard van der Hoff	a0acfcc73e	update sample config	2019-06-26 23:56:28 +01:00
Andrew Morgan	3eb8c7b0eb	Merge branch 'master' into develop ... * master: Fix broken link in MSC1711 FAQ Update changelog to better expain password reset change (#5545)	2019-06-25 18:08:56 +01:00
Richard van der Hoff	c8cb186260	Fix broken link in MSC1711 FAQ	2019-06-25 12:27:56 +01:00
Andrew Morgan	28604ab03d	Add info about black to code_style.rst (#5537) ... Fixes #5533 Adds information about how to install and run black on the codebase.	2019-06-24 17:48:05 +01:00
Richard van der Hoff	4ac7ef4b67	Merge pull request #5524 from matrix-org/rav/new_cmdline_options ... Add --data-dir and --open-private-ports options.	2019-06-24 17:25:57 +01:00
Brendan Abolivier	deb4fe6ef3	Merge pull request #5534 from matrix-org/babolivier/federation-publicrooms ... Split public rooms directory auth config in two	2019-06-24 16:08:02 +01:00
Brendan Abolivier	bfe84e051e	Split public rooms directory auth config in two	2019-06-24 15:42:31 +01:00
Richard van der Hoff	3f8a252dd8	Add "--open-private-ports" cmdline option ... This is helpful when generating a config file for running synapse under docker.	2019-06-24 14:15:34 +01:00
Richard van der Hoff	edea4bb5be	Allow configuration of the path used for ACME account keys. ... Because sticking it in the same place as the config isn't necessarily the right thing to do.	2019-06-24 13:51:22 +01:00
Richard van der Hoff	6cda36777b	Drop support for cpu_affinity (#5525) ... This has no useful purpose on python3, and is generally a source of confusion.	2019-06-22 11:01:55 +10:00
Erik Johnston	60b912cf0d	Update docs/workers.rst ... E_TOO_MANY_NEGATIVES Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>	2019-06-21 11:54:03 +01:00
Erik Johnston	f3ab533374	Support pagination API in client_reader worker	2019-06-21 10:43:12 +01:00
Amber Brown	32e7c9e7f2	Run Black. (#5482)	2019-06-20 19:32:02 +10:00
Amber Brown	eba7caf09f	Remove Postgres 9.4 support (#5448)	2019-06-18 00:59:00 +10:00
Brendan Abolivier	f12e1f029c	Merge pull request #5440 from matrix-org/babolivier/third_party_event_rules ... Allow server admins to define implementations of extra rules for allowing or denying incoming events	2019-06-14 19:37:59 +01:00
Brendan Abolivier	f874b16b2e	Add plugin APIs for implementations of custom event rules.	2019-06-14 18:16:03 +01:00
Neil Johnson	426218323b	Neilj/improve federation docs (#5419) ... Add FAQ questions to federate.md. Add a health warning making it clear that the 1711 upgrade FAQ is now out of date.	2019-06-11 12:17:43 +01:00
Neil Johnson	a11865016e	Set default room version to v4. (#5379) ... Set default room version to v4.	2019-06-06 20:13:47 +01:00
Andrew Morgan	3719680ee4	Add ability to perform password reset via email without trusting the identity server (#5377) ... Sends password reset emails from the homeserver instead of proxying to the identity server. This is now the default behaviour for security reasons. If you wish to continue proxying password reset requests to the identity server you must now enable the email.trust_identity_server_for_password_resets option. This PR is a culmination of 3 smaller PRs which have each been separately reviewed: * #5308 * #5345 * #5368	2019-06-06 17:34:07 +01:00
Richard van der Hoff	9fbb20a531	Stop hardcoding trust of old matrix.org key (#5374) ... There are a few changes going on here: * We make checking the signature on a key server response optional: if no verify_keys are specified, we trust to TLS to validate the connection. * We change the default config so that it does not require responses to be signed by the old key. * We replace the old 'perspectives' config with 'trusted_key_servers', which is also formatted slightly differently. * We emit a warning to the logs every time we trust a key server response signed by the old key.	2019-06-06 17:33:11 +01:00
Neil Johnson	833c406b9b	Neilj/1.0 upgrade notes (#5371) ... 1.0 upgrade/install notes	2019-06-06 17:23:02 +01:00
Richard van der Hoff	7603a706eb	Merge branch 'rav/fix_custom_ca' into rav/enable_tls_verification	2019-06-05 16:32:35 +01:00
Richard van der Hoff	b4f1cd31f4	Update sample config	2019-06-05 15:30:10 +01:00
Richard van der Hoff	95ab2eb4a1	Fix notes about well-known and acme (#5357) ... fixes #4951	2019-06-05 15:12:33 +01:00
Neil Johnson	26713515de	Neilj/mau tracking config explainer (#5284) ... Improve documentation of monthly active user blocking and mau_trial_days	2019-06-05 13:16:23 +01:00
Ike Johnson	145f57897d	Update HAProxy example rules ... These new rules allow a user to instead route only matrix traffic, allowing them to run matrix on the domain without affecting their existing websites	2019-06-02 23:10:27 +08:00
Erik Johnston	58cce39f3a	Merge pull request #5276 from matrix-org/babolivier/account_validity_job_delta ... Allow configuring a range for the account validity startup job	2019-05-31 12:11:56 +01:00
Brendan Abolivier	e975b15101	Sample config	2019-05-31 11:14:21 +01:00
Brendan Abolivier	6bfc5ad3a1	Sample config	2019-05-31 09:56:57 +01:00
Travis Ralston	3e1af5109c	Clarify that the admin change password endpoint logs them out (#5303)	2019-05-31 09:45:46 +01:00
Erik Johnston	8541db741a	Merge pull request #5283 from aaronraimist/captcha-docs ... Specify the type of reCAPTCHA key to use (#5013)	2019-05-29 19:02:27 +01:00
Amber Brown	0729ef01f8	regenerate sample config	2019-05-29 16:41:25 +10:00
Aaron Raimist	f795595e95	Specify the type of reCAPTCHA key to use (#5013) ... Signed-off-by: Aaron Raimist <aaron@raim.ist>	2019-05-28 22:04:27 -05:00
Aaron Raimist	9b6f72663e	Fix docs on resetting the user directory (#5036) ... Signed-off-by: Aaron Raimist <aaron@raim.ist>	2019-05-28 20:54:01 -05:00
Brendan Abolivier	4aba561c65	Config and changelog	2019-05-28 16:55:10 +01:00
Richard van der Hoff	dba9152d15	Add missing blank line in config (#5249)	2019-05-24 14:12:38 +01:00
Andrew Morgan	6368150a74	Add config option for setting homeserver's default room version (#5223) ... Replaces DEFAULT_ROOM_VERSION constant with a method that first checks the config, then returns a hardcoded value if the option is not present. That hardcoded value is now located in the server.py config file.	2019-05-23 15:00:20 +01:00
Amber Brown	4a30e4acb4	Room Statistics (#4338)	2019-05-21 11:36:50 -05:00
Brendan Abolivier	6a5a70edf0	Merge pull request #5204 from matrix-org/babolivier/account_validity_expiration_date ... Add startup background job for account validity	2019-05-21 14:55:15 +01:00
Brendan Abolivier	384122efa8	Doc	2019-05-21 14:39:36 +01:00
Richard van der Hoff	da5ef0bb42	Merge remote-tracking branch 'origin/master' into develop	2019-05-17 12:39:48 +01:00
Richard van der Hoff	7ce1f97a13	Stop telling people to install the optional dependencies. (#5197) ... * Stop telling people to install the optional dependencies. They're optional. Also update the postgres docs a bit for clarity(?)	2019-05-17 12:38:03 +01:00
Brendan Abolivier	a5fe16c5a7	Changelog + sample config	2019-05-16 15:11:37 +01:00
Amber Brown	f1e5b41388	Make all the rate limiting options more consistent (#5181)	2019-05-15 12:06:04 -05:00
Andrew Morgan	5a4b328f52	Add ability to blacklist ip ranges for federation traffic (#5043)	2019-05-13 19:05:06 +01:00
Gergely Polonkai	cd3f30014a	Make Prometheus snippet less confusing on the metrics collection doc (#4288) ... Signed-off-by: Gergely Polonkai <gergely@polonkai.eu>	2019-05-10 09:15:08 +01:00
colonelkrud	d9a02d1201	Add AllowEncodedSlashes to apache (#5068) ... * Add AllowEncodedSlashes to apache Add `AllowEncodedSlashes On` to apache config to support encoding for v3 rooms. "The AllowEncodedSlashes setting is not inherited by virtual hosts, and virtual hosts are used in many default Apache configurations, such as the one in Ubuntu. The workaround is to add the AllowEncodedSlashes setting inside a <VirtualHost> container (/etc/apache2/sites-available/default in Ubuntu)." Source: https://stackoverflow.com/questions/4390436/need-to-allow-encoded-slashes-on-apache * change allowencodedslashes to nodecode	2019-05-09 23:27:04 +01:00
Matthew Hodgson	c0e0740bef	add options to require an access_token to GET /profile and /publicRooms on CS API (#5083) ... This commit adds two config options: * `restrict_public_rooms_to_local_users` Requires auth to fetch the public rooms directory through the CS API and disables fetching it through the federation API. * `require_auth_for_profile_requests` When set to `true`, requires that requests to `/profile` over the CS API are authenticated, and only returns the user's profile if the requester shares a room with the profile's owner, as per MSC1301. MSC1301 also specifies a behaviour for federation (only returning the profile if the server asking for it shares a room with the profile's owner), but that's currently really non-trivial to do in a not too expensive way. Next step is writing down a MSC that allows a HS to specify which user sent the profile query. In this implementation, Synapse won't send a profile query over federation if it doesn't believe it already shares a room with the profile's owner, though. Groups have been intentionally omitted from this commit.	2019-05-08 18:26:56 +01:00
Richard van der Hoff	59e2d2694d	Remove the requirement to authenticate for /admin/server_version. (#5122) ... This endpoint isn't much use for its intended purpose if you first need to get yourself an admin's auth token. I've restricted it to the `/_synapse/admin` path to make it a bit easier to lock down for those concerned about exposing this information. I don't imagine anyone is using it in anger currently.	2019-05-07 09:29:30 +01:00
Travis Ralston	3fdff14207	Fix spelling in server notices admin API docs (#5142)	2019-05-06 22:15:02 +01:00
Richard van der Hoff	4804206dbe	Fix sample config ... ... after it got broken in 1565ebec2c.	2019-05-06 22:13:35 +01:00
Richard van der Hoff	836d3adcce	Merge branch 'master' into develop	2019-05-03 19:25:01 +01:00
Richard van der Hoff	1565ebec2c	more config comment updates	2019-05-03 15:50:59 +01:00
Richard van der Hoff	1a7104fde3	Blacklist 0.0.0.0 and :: by default for URL previews	2019-05-03 15:35:49 +01:00
Richard van der Hoff	12f9d51e82	Add admin api for sending server_notices (#5121)	2019-05-02 11:59:16 +01:00
Brendan Abolivier	c193b39134	Merge pull request #5124 from matrix-org/babolivier/aliases ... Add some limitations to alias creation	2019-05-02 11:22:40 +01:00
Brendan Abolivier	84196cb231	Add some limitations to alias creation	2019-05-02 11:05:11 +01:00
Richard van der Hoff	cc4bd762df	Fix sample config	2019-05-01 16:48:23 +01:00
Richard van der Hoff	8e9ca83537	Move admin API to a new prefix	2019-05-01 15:44:30 +01:00
Brendan Abolivier	c1799b0f85	Merge pull request #5116 from matrix-org/babolivier/account_expiration ... Fix path in account validity admin route's doc	2019-05-01 11:59:56 +01:00
Brendan Abolivier	031919dafb	Fix whole path for admin route	2019-05-01 11:38:27 +01:00
Brendan Abolivier	d8e357b7cf	Fix typo in account validity admin route	2019-05-01 11:34:22 +01:00
Andrew Morgan	6824ddd93d	Config option for verifying federation certificates (MSC 1711) (#4967)	2019-04-25 14:22:49 +01:00
Erik Johnston	ca90336a69	Merge branch 'develop' of github.com:matrix-org/synapse into babolivier/account_expiration	2019-04-17 19:44:40 +01:00
Brendan Abolivier	eaf41a943b	Add management endpoints for account validity	2019-04-17 19:34:45 +01:00
Brendan Abolivier	91934025b9	Merge pull request #5047 from matrix-org/babolivier/account_expiration ... Send out emails with links to extend an account's validity period	2019-04-17 14:57:39 +01:00
Brendan Abolivier	20f0617e87	Send out emails with links to extend an account's validity period	2019-04-17 14:42:20 +01:00
Erik Johnston	6e27a8620f	Merge pull request #5063 from matrix-org/erikj/move_endpoints ... Move some rest endpoints to client reader	2019-04-15 18:55:01 +01:00
Erik Johnston	ec638a1602	Only handle GET requests for /push_rules	2019-04-15 18:51:48 +01:00
Erik Johnston	d5adf297e6	Move some rest endpoints to client reader	2019-04-15 17:21:03 +01:00
Brendan Abolivier	bfc8fdf1fc	Merge pull request #5027 from matrix-org/babolivier/account_expiration ... Add time-based account expiration	2019-04-09 17:02:41 +01:00
Brendan Abolivier	747aa9f8ca	Add account expiration feature	2019-04-09 16:46:04 +01:00
Neil Johnson	b25e387c0d	add context to phonehome stats (#5020) ... add context to phonehome stats	2019-04-08 15:47:39 +01:00
Brendan Abolivier	8e85493b0c	Add config option to block users from looking up 3PIDs (#5010)	2019-04-04 17:25:47 +01:00
Erik Johnston	c192bf8970	Add admin API for group deletion	2019-04-03 16:29:52 +01:00
Andrew Morgan	bbd244c7b2	Support 3PID login in password providers (#4931) ... Adds a new method, check_3pid_auth, which gives password providers the chance to allow authentication with third-party identifiers such as email or msisdn.	2019-03-26 17:48:30 +00:00
Richard van der Hoff	7105057cf2	Fix nginx example in ACME doc. (#4923)	2019-03-25 09:59:36 +00:00
Colin W	ab4e4c6c2f	Update Apache Setup To Remove Location Syntax (#4870) ... This one should close #4841. Many thanks to @dev4223 for bringing it up and finding a solution. Signed-off-by: Colin White	2019-03-21 14:05:56 +00:00
Erik Johnston	09f991a63d	Merge pull request #4896 from matrix-org/erikj/disable_room_directory ... Add option to disable search room lists	2019-03-21 10:16:54 +00:00
Erik Johnston	263f2c9ce1	Merge pull request #4895 from matrix-org/erikj/disable_user_search ... Add option to disable searching in the user dir	2019-03-20 16:47:15 +00:00
Richard van der Hoff	a902d13180	Batch up outgoing read-receipts to reduce federation traffic. (#4890) ... Rate-limit outgoing read-receipts as per #4730.	2019-03-20 16:02:25 +00:00
Erik Johnston	cd8c5b91ad	Fix up sample config	2019-03-20 14:35:41 +00:00
Erik Johnston	926f29ea6d	Fix up config comments	2019-03-20 14:24:53 +00:00
Erik Johnston	213c98c00a	Add option to disable search room lists ... This disables both local and remote room list searching.	2019-03-19 17:10:52 +00:00
Erik Johnston	855bf4658d	Update sample config	2019-03-19 16:47:04 +00:00
Erik Johnston	b616a8717b	Add note on tuning postgres	2019-03-19 16:05:32 +00:00
Richard van der Hoff	d2a537ea60	Merge remote-tracking branch 'origin/master' into develop	2019-03-19 10:37:50 +00:00
Michael Kaye	9482a84c0a	Repoint docs for federation (#4881)	2019-03-19 10:37:18 +00:00
Richard van der Hoff	fd463b4f5d	Comment out most options in the generated config. (#4863) ... Make it so that most options in the config are optional, and commented out in the generated config. The reasons this is a good thing are as follows: * If we decide that we should change the default for an option, we can do so, and only those admins that have deliberately chosen to override that option will be stuck on the old setting. * It moves us towards a point where we can get rid of the super-surprising feature of synapse where the default settings for the config come from the generated yaml. * It makes setting up a test config for unit testing an order of magnitude easier (see forthcoming PR). * It makes the generated config more consistent, and hopefully easier for users to understand.	2019-03-19 10:06:40 +00:00
Brendan Abolivier	651ad8bc96	Add ratelimiting on failed login attempts (#4865)	2019-03-18 12:57:20 +00:00
Brendan Abolivier	899e523d6d	Add ratelimiting on login (#4821) ... Add two ratelimiters on login (per-IP address and per-userID).	2019-03-15 17:46:16 +00:00
Richard van der Hoff	9ffadcdbad	fix some typos in federate.md	2019-03-15 09:43:24 +00:00
Andrew Morgan	7998ca3a66	Document using a certificate with a full chain (#4849)	2019-03-13 15:26:29 +00:00
Neil Johnson	332b60ec68	Merge branch 'master' of github.com:matrix-org/synapse into develop	2019-03-12 17:15:21 +00:00
Neil Johnson	83193a9362	fix orphaned sentence	2019-03-12 16:57:17 +00:00
Neil Johnson	8b692bf7c2	Neilj/improved delegation doc 2 (#4832) ... Improved federation configuration docs. Specifically detailing .well-known and SRV based delegation methods. Inspiration Valentin Lab <valentin.lab@kalysto.org> for https://github.com/matrix-org/synapse/pull/4781	2019-03-12 14:23:28 +00:00
Aaron Raimist	8ea1b41a0e	Clarify what registration_shared_secret allows for (#2885) (#4844) ... * Clarify what registration_shared_secret allows for (#2885) Signed-off-by: Aaron Raimist <aaron@raim.ist> * Add changelog Signed-off-by: Aaron Raimist <aaron@raim.ist>	2019-03-11 18:21:52 +00:00
Matthew Hodgson	8f4b9f5210	Reword the sample config header to be less scary (#4801)	2019-03-07 07:09:01 +00:00
Brendan Abolivier	c23e8c3333	Update sample config	2019-03-05 18:03:48 +00:00
Erik Johnston	16c8b4ecbd	Merge pull request #4772 from jbweston/jbweston/server-version-api ... Add 'server_version' endpoint to admin API	2019-03-05 16:31:00 +00:00
Brendan Abolivier	a4c3a361b7	Add rate-limiting on registration (#4735) ... * Rate-limiting for registration * Add unit test for registration rate limiting * Add config parameters for rate limiting on auth endpoints * Doc * Fix doc of rate limiting function Co-Authored-By: babolivier <contact@brendanabolivier.com> * Incorporate review * Fix config parsing * Fix linting errors * Set default config for auth rate limiting * Fix tests * Add changelog * Advance reactor instead of mocked clock * Move parameters to registration specific config and give them more sensible default values * Remove unused config options * Don't mock the rate limiter un MAU tests * Rename _register_with_store into register_with_store * Make CI happy * Remove unused import * Update sample config * Fix ratelimiting test for py2 * Add non-guest test	2019-03-05 14:25:33 +00:00
Erik Johnston	c3c542bb4a	Merge pull request #4796 from matrix-org/erikj/factor_out_e2e_keys ... Allow /keys/{changes,query} API to run on worker	2019-03-05 09:06:25 +00:00
Erik Johnston	bfa7d46a10	Allow /keys/{changes,query} API to run on worker	2019-03-04 18:30:01 +00:00
Richard van der Hoff	8e28bc5eee	Include a default configuration file in the 'docs' directory. (#4791)	2019-03-04 17:14:58 +00:00
Seebi	aba5eeabd5	Fix v4v6 option in HAProxy example config (#4790) ... The v4v6 option only has a usage one ipv6 socket: https://serverfault.com/q/747895 Signed-off-by: Flakebi <flakebi@t-online.de>	2019-03-04 13:19:41 +00:00
Joseph Weston	144cbfd650	add API documentation ... Signed-off-by: Joseph Weston <joseph@weston.cloud>	2019-03-02 03:07:04 +01:00
Erik Johnston	76550c58d2	Merge pull request #4759 from matrix-org/erikj/3pid_client_reader ... Move /account/3pid to client_reader	2019-02-27 16:11:21 +00:00
Erik Johnston	54f9ce11a7	Move /account/3pid to client_reader	2019-02-27 14:26:08 +00:00
Erik Johnston	4cff9376f7	Move server key queries to federation reader	2019-02-27 13:43:53 +00:00
Erik Johnston	7590e9fa28	Merge pull request #4749 from matrix-org/erikj/replication_connection_backoff ... Fix tightloop over connecting to replication server	2019-02-27 11:00:59 +00:00
Paul Tötterman	4bc7483518	Fix apache reverse proxy example (#4742) ... So that it actually works. See https://httpd.apache.org/docs/2.4/mod/mod_proxy.html#proxypass Signed-off-by: Paul Tötterman <paul.totterman@iki.fi>	2019-02-26 18:01:45 +00:00
Erik Johnston	313987187e	Fix tightloop over connecting to replication server ... If the client failed to process incoming commands during the initial set up of the replication connection it would immediately disconnect and reconnect, resulting in a tightloop. This can happen, for example, when subscribing to a stream that has a row that is too long in the backlog. The fix here is to not consider the connection successfully set up until the client has succesfully subscribed and caught up with the streams. This ensures that the retry logic timers aren't reset until then, meaning that if an error does happen during start up the client will continue backing off before retrying again.	2019-02-26 15:05:41 +00:00
Richard van der Hoff	fcd6f01dc7	Minor tweaks to acme docs (#4689)	2019-02-22 10:56:42 +00:00
Benoît S	16e0680498	Added HAProxy example (#4660) ... * Added HAProxy example Proposal of an example with HAProxy. Asked by #4541. Signed-off-by: Benoît S. (“Benpro”) <gitlab@benpro.fr> * Following suggestions of @richvdh	2019-02-21 17:44:10 +00:00
Erik Johnston	c003450057	Merge pull request #4671 from matrix-org/erikj/state_cache_invalidation ... Batch cache invalidation over replication	2019-02-19 13:14:30 +00:00
Erik Johnston	62175a20e5	Docs	2019-02-19 11:38:40 +00:00
Erik Johnston	bc8fa1509d	Documentation	2019-02-19 11:24:59 +00:00
Erik Johnston	128902d60a	Update worker docs	2019-02-18 17:21:51 +00:00
Erik Johnston	8b9ae6d3a6	Update docs	2019-02-18 15:26:13 +00:00
Richard van der Hoff	00cf679bf2	Synapse 0.99.1 (2019-02-14) ... =========================== Features -------- - Include m.room.encryption on invites by default ([\#3902](https://github.com/matrix-org/synapse/issues/3902)) - Federation OpenID listener resource can now be activated even if federation is disabled ([\#4420](https://github.com/matrix-org/synapse/issues/4420)) - Synapse's ACME support will now correctly reprovision a certificate that approaches its expiry while Synapse is running. ([\#4522](https://github.com/matrix-org/synapse/issues/4522)) - Add ability to update backup versions ([\#4580](https://github.com/matrix-org/synapse/issues/4580)) - Allow the "unavailable" presence status for /sync. This change makes Synapse compliant with r0.4.0 of the Client-Server specification. ([\#4592](https://github.com/matrix-org/synapse/issues/4592)) - There is no longer any need to specify `no_tls`: it is inferred from the absence of TLS listeners ([\#4613](https://github.com/matrix-org/synapse/issues/4613), [\#4615](https://github.com/matrix-org/synapse/issues/4615), [\#4617](https://github.com/matrix-org/synapse/issues/4617), [\#4636](https://github.com/matrix-org/synapse/issues/4636)) - The default configuration no longer requires TLS certificates. ([\#4614](https://github.com/matrix-org/synapse/issues/4614)) Bugfixes -------- - Copy over room federation ability on room upgrade. ([\#4530](https://github.com/matrix-org/synapse/issues/4530)) - Fix noisy "twisted.internet.task.TaskStopped" errors in logs ([\#4546](https://github.com/matrix-org/synapse/issues/4546)) - Synapse is now tolerant of the `tls_fingerprints` option being None or not specified. ([\#4589](https://github.com/matrix-org/synapse/issues/4589)) - Fix 'no unique or exclusion constraint' error ([\#4591](https://github.com/matrix-org/synapse/issues/4591)) - Transfer Server ACLs on room upgrade. ([\#4608](https://github.com/matrix-org/synapse/issues/4608)) - Fix failure to start when not TLS certificate was given even if TLS was disabled. ([\#4618](https://github.com/matrix-org/synapse/issues/4618)) - Fix self-signed cert notice from generate-config. ([\#4625](https://github.com/matrix-org/synapse/issues/4625)) - Fix performance of `user_ips` table deduplication background update ([\#4626](https://github.com/matrix-org/synapse/issues/4626), [\#4627](https://github.com/matrix-org/synapse/issues/4627)) Internal Changes ---------------- - Change the user directory state query to use a filtered call to the db instead of a generic one. ([\#4462](https://github.com/matrix-org/synapse/issues/4462)) - Reject federation transactions if they include more than 50 PDUs or 100 EDUs. ([\#4513](https://github.com/matrix-org/synapse/issues/4513)) - Reduce duplication of ``synapse.app`` code. ([\#4567](https://github.com/matrix-org/synapse/issues/4567)) - Fix docker upload job to push -py2 images. ([\#4576](https://github.com/matrix-org/synapse/issues/4576)) - Add port configuration information to ACME instructions. ([\#4578](https://github.com/matrix-org/synapse/issues/4578)) - Update MSC1711 FAQ to calrify .well-known usage ([\#4584](https://github.com/matrix-org/synapse/issues/4584)) - Clean up default listener configuration ([\#4586](https://github.com/matrix-org/synapse/issues/4586)) - Clarifications for reverse proxy docs ([\#4607](https://github.com/matrix-org/synapse/issues/4607)) - Move ClientTLSOptionsFactory init out of `refresh_certificates` ([\#4611](https://github.com/matrix-org/synapse/issues/4611)) - Fail cleanly if listener config lacks a 'port' ([\#4616](https://github.com/matrix-org/synapse/issues/4616)) - Remove redundant entries from docker config ([\#4619](https://github.com/matrix-org/synapse/issues/4619)) - README updates ([\#4621](https://github.com/matrix-org/synapse/issues/4621)) -----BEGIN PGP SIGNATURE----- iQFHBAABCgAxFiEEQlNDQm4FMsm53u1sih+T1XW16NUFAlxlemgTHHJpY2hhcmRA bWF0cml4Lm9yZwAKCRCKH5PVdbXo1eKYCACR9TcOvMver/YyD2qP+dY6Lt24f8zG zYYzHGAHin+p204q8Pp6o0XLe4UuLDuhAyNVPZyj1wzwHYdubRvdah1uFwPdxmCY tGbJG5p37ykSEfEwcxdXEjYfPqflOwQL5aCeXyCCLWSdVVFKkWCXGgw8F6WPkgrI QwWKTfsM3wCnfa8ryKAXHxcmX2G1JncZ0ouUZTVNz5vokBsA19IaLvfJ5Rv3Kk59 eXsBB/yE+9Dat4A439AGfVDQDKiGYvuhppJmUdYRMqxulzakd8diyZqBDAHZafqt QdjxnDx2e0OtSxI3RSevABnDnNyJ4NsUEtrny1Lh/MV72T9K3yEbHuwH =UCD1 -----END PGP SIGNATURE----- Merge tag 'v0.99.1' Synapse 0.99.1 (2019-02-14) =========================== Features -------- - Include m.room.encryption on invites by default ([\#3902](https://github.com/matrix-org/synapse/issues/3902)) - Federation OpenID listener resource can now be activated even if federation is disabled ([\#4420](https://github.com/matrix-org/synapse/issues/4420)) - Synapse's ACME support will now correctly reprovision a certificate that approaches its expiry while Synapse is running. ([\#4522](https://github.com/matrix-org/synapse/issues/4522)) - Add ability to update backup versions ([\#4580](https://github.com/matrix-org/synapse/issues/4580)) - Allow the "unavailable" presence status for /sync. This change makes Synapse compliant with r0.4.0 of the Client-Server specification. ([\#4592](https://github.com/matrix-org/synapse/issues/4592)) - There is no longer any need to specify `no_tls`: it is inferred from the absence of TLS listeners ([\#4613](https://github.com/matrix-org/synapse/issues/4613), [\#4615](https://github.com/matrix-org/synapse/issues/4615), [\#4617](https://github.com/matrix-org/synapse/issues/4617), [\#4636](https://github.com/matrix-org/synapse/issues/4636)) - The default configuration no longer requires TLS certificates. ([\#4614](https://github.com/matrix-org/synapse/issues/4614)) Bugfixes -------- - Copy over room federation ability on room upgrade. ([\#4530](https://github.com/matrix-org/synapse/issues/4530)) - Fix noisy "twisted.internet.task.TaskStopped" errors in logs ([\#4546](https://github.com/matrix-org/synapse/issues/4546)) - Synapse is now tolerant of the `tls_fingerprints` option being None or not specified. ([\#4589](https://github.com/matrix-org/synapse/issues/4589)) - Fix 'no unique or exclusion constraint' error ([\#4591](https://github.com/matrix-org/synapse/issues/4591)) - Transfer Server ACLs on room upgrade. ([\#4608](https://github.com/matrix-org/synapse/issues/4608)) - Fix failure to start when not TLS certificate was given even if TLS was disabled. ([\#4618](https://github.com/matrix-org/synapse/issues/4618)) - Fix self-signed cert notice from generate-config. ([\#4625](https://github.com/matrix-org/synapse/issues/4625)) - Fix performance of `user_ips` table deduplication background update ([\#4626](https://github.com/matrix-org/synapse/issues/4626), [\#4627](https://github.com/matrix-org/synapse/issues/4627)) Internal Changes ---------------- - Change the user directory state query to use a filtered call to the db instead of a generic one. ([\#4462](https://github.com/matrix-org/synapse/issues/4462)) - Reject federation transactions if they include more than 50 PDUs or 100 EDUs. ([\#4513](https://github.com/matrix-org/synapse/issues/4513)) - Reduce duplication of ``synapse.app`` code. ([\#4567](https://github.com/matrix-org/synapse/issues/4567)) - Fix docker upload job to push -py2 images. ([\#4576](https://github.com/matrix-org/synapse/issues/4576)) - Add port configuration information to ACME instructions. ([\#4578](https://github.com/matrix-org/synapse/issues/4578)) - Update MSC1711 FAQ to calrify .well-known usage ([\#4584](https://github.com/matrix-org/synapse/issues/4584)) - Clean up default listener configuration ([\#4586](https://github.com/matrix-org/synapse/issues/4586)) - Clarifications for reverse proxy docs ([\#4607](https://github.com/matrix-org/synapse/issues/4607)) - Move ClientTLSOptionsFactory init out of `refresh_certificates` ([\#4611](https://github.com/matrix-org/synapse/issues/4611)) - Fail cleanly if listener config lacks a 'port' ([\#4616](https://github.com/matrix-org/synapse/issues/4616)) - Remove redundant entries from docker config ([\#4619](https://github.com/matrix-org/synapse/issues/4619)) - README updates ([\#4621](https://github.com/matrix-org/synapse/issues/4621))	2019-02-14 14:41:40 +00:00
Richard van der Hoff	c475275926	Clarifications for reverse proxy docs (#4607) ... Factor out the reverse proxy info to a separate file, add some more info on reverse-proxying the federation port.	2019-02-11 11:44:28 +00:00
Erik Johnston	4588b0d64a	Update MSC1711_certificates_FAQ.md ... Fix incorrect heading level	2019-02-08 09:37:16 +00:00
Erik Johnston	acb2ac5863	Update MSC1711 FAQ to be explicit about well-known (#4584) ... A surprising number of people are using the well-known method, and are simply copying the example configuration. This is problematic as the example includes an explicit port, which causes inbound federation requests to have the HTTP Host header include the port, upsetting some reverse proxies. Given that, we update the well-known example to be more explicit about the various ways you can set it up, and the consequence of using an explict port.	2019-02-07 19:30:32 +00:00
Richard van der Hoff	7cadc4c918	cleanups	2019-02-07 19:29:20 +00:00
Richard van der Hoff	188ad47e73	Merge branch 'master' into erikj/msc1711_faq	2019-02-07 19:27:42 +00:00
Erik Johnston	9285d5c2ce	Update MSC1711 FAQ to be explicit about well-known ... A surprising number of people are using the well-known method, and are simply copying the example configuration. This is problematic as the example includes an explicit port, which causes inbound federation requests to have the HTTP Host header include the port, upsetting some reverse proxies. Given that, we update the well-known example to be more explicit about the various ways you can set it up, and the consequence of using an explict port.	2019-02-07 19:24:11 +00:00
Richard van der Hoff	624b172e08	Merge remote-tracking branch 'origin/release-v0.99.0'	2019-02-07 19:18:26 +00:00
Andrew Morgan	c17b128b83	Update ACME docs to include port instructions (#4578)	2019-02-07 19:18:08 +00:00
Richard van der Hoff	9b7aa543d9	clarify option 1	2019-02-07 18:46:02 +00:00
Richard van der Hoff	b05dd4ac06	faq cleanups	2019-02-05 18:59:57 +00:00
Richard van der Hoff	39bf0ea2e8	Add notes on SRV and .well-known (#4573)	2019-02-05 18:11:26 +00:00
Andrew Morgan	4a7524ffd3	Merge pull request #4570 from matrix-org/anoa/self_signed_upgrade ... Add ACME docs and link to it from README and INSTALL	2019-02-05 17:34:43 +00:00
Neil Johnson	6585ef4799	Neilj/1711faq (#4572) ... MSC1711 certificates FAQ	2019-02-05 17:19:28 +00:00
Andrew Morgan	a6345009f9	Add TL;DR and final step details to ACME	2019-02-05 17:04:34 +00:00
Andrew Morgan	56cb34ba8b	Merge branch 'anoa/self_signed_upgrade' of github.com:matrix-org/synapse into anoa/self_signed_upgrade	2019-02-05 16:53:05 +00:00
Andrew Morgan	2ca63df83b	Update ACME	2019-02-05 16:50:00 +00:00
Richard van der Hoff	13828f7d58	Update docs/ACME.md ... Co-Authored-By: anoadragon453 <1342360+anoadragon453@users.noreply.github.com>	2019-02-05 16:46:28 +00:00
Andrew Morgan	ffcbd80982	Actually add ACME docs	2019-02-05 15:50:18 +00:00
Andrew Morgan	cbdc01cc3b	Convert ACME docs to md	2019-02-05 15:38:27 +00:00
Andrew Morgan	08b26afeee	Move ACME docs to docs/ACME.rst and link from UPGRADE.	2019-02-05 15:33:23 +00:00
Krithin Sitaram	98df67a8de	Remove mention of lt-cred-mech in the sample coturn config. (#4333) ... * Remove mention of lt-cred-mech in the sample coturn config. See https://github.com/coturn/coturn/pull/262 for more context. Also clean up some minor formatting issues while I'm here. * Add changelog. Signed-off-by: Krithin Sitaram <krithin@gmail.com>	2018-12-28 23:31:49 +00:00
Neil Johnson	d2f7c4e6b1	create support user (#4141) ... Allow for the creation of a support user. A support user can access the server, join rooms, interact with other users, but does not appear in the user directory nor does it contribute to monthly active user limits.	2018-12-14 18:20:59 +00:00
Richard van der Hoff	de8772a655	Do a GC after each test to fix logcontext leaks (#4227) ... * Some words about garbage collections and logcontexts * Do a GC after each test to fix logcontext leaks This feels like an awful hack, but... * changelog	2018-11-27 13:00:33 +11:00
Aaron Raimist	f6cbef6332	Add a note saying you need to manually reclaim disk space ... People keep asking why their database hasn't gotten smaller after using this API. Signed-off-by: Aaron Raimist <aaron@raim.ist>	2018-11-18 12:38:04 -06:00
Travis Ralston	0f5e51f726	Add config variables for enabling terms auth and the policy name (#4142) ... So people can still collect consent the old way if they want to.	2018-11-06 10:32:34 +00:00
Travis Ralston	22a2004428	Update documentation and templates for new consent	2018-10-12 17:53:14 -06:00
Travis Ralston	1ca2744621	Merge pull request #3734 from matrix-org/travis/worker-docs ... Reference that the federation_reader needs the HTTP replication port set	2018-08-23 07:51:46 -06:00
Erik Johnston	764030cf63	Merge pull request #3659 from matrix-org/erikj/split_profiles ... Allow profile updates to happen on workers	2018-08-22 11:35:55 +01:00
Travis Ralston	dd0ac1614c	Reference that the federation_reader needs the HTTP replication port set	2018-08-21 23:35:50 -06:00