1951 Commits

Author	SHA1	Message	Date
Patrick Cloke	e259d63f73	Stop shadow-banned users from sending invites. (#8095)	2020-08-20 15:07:42 -04:00
Patrick Cloke	f594e434c3	Switch the JSON byte producer from a pull to a push producer. (#8116)	2020-08-19 08:07:57 -04:00
Patrick Cloke	d294f0e7e1	Remove the unused inlineCallbacks code-paths in the caching code (#8119)	2020-08-19 07:09:07 -04:00
Erik Johnston	76d21d14a0	Separate get_current_token into two. (#8113) ... The function is used for two purposes: 1) for subscribers of streams to get a token they can use to get further updates with, and 2) for replication to track position of the writers of the stream. For streams with a single writer the two scenarios produce the same result, however the situation becomes complicated for streams with multiple writers. The current `MultiWriterIdGenerator` does not correctly handle the first case (which is not an issue as its only used for the `caches` stream which nothing subscribes to outside of replication).	2020-08-19 10:39:31 +01:00
Patrick Cloke	f40645e60b	Convert events worker database to async/await. (#8071)	2020-08-18 16:20:49 -04:00
Andrew Morgan	5cf7c12995	Remove : from allowed client_secret chars (#8101) ... Closes: https://github.com/matrix-org/synapse/issues/6766 Equivalent Sydent PR: https://github.com/matrix-org/sydent/pull/309 I believe it's now time to remove the extra allowed `:` from `client_secret` parameters.	2020-08-18 14:14:27 +01:00
Patrick Cloke	2f4d60a5ba	Iteratively encode JSON responses to avoid blocking the reactor. (#8013)	2020-08-18 08:49:59 -04:00
Patrick Cloke	050e20e7ca	Convert some of the general database methods to async (#8100)	2020-08-17 12:18:01 -04:00
Andrew Morgan	e04e465b4d	Use the default templates when a custom template file cannot be found (#8037) ... Fixes https://github.com/matrix-org/synapse/issues/6583	2020-08-17 17:05:00 +01:00
Patrick Cloke	ad6190c925	Convert stream database to async/await. (#8074)	2020-08-17 07:24:46 -04:00
Patrick Cloke	ac77cdb64e	Add a shadow-banned flag to users. (#8092)	2020-08-14 12:37:59 -04:00
Patrick Cloke	dd8f28bd3f	Fix unawaited coroutine error in tests. (#8072)	2020-08-13 07:11:39 -04:00
Patrick Cloke	fbe930dad2	Convert the roommember database to async/await. (#8070)	2020-08-12 12:14:34 -04:00
Patrick Cloke	5ecc8b5825	Convert devices database to async/await. (#8069)	2020-08-12 10:51:42 -04:00
Erik Johnston	5dd73d029e	Add type hints to handlers.message and events.builder (#8067)	2020-08-12 15:05:50 +01:00
Patrick Cloke	d68e10f308	Convert account data, device inbox, and censor events databases to async/await (#8063)	2020-08-12 09:29:06 -04:00
Patrick Cloke	a3a59bab7b	Convert appservice, group server, profile and more databases to async (#8066)	2020-08-12 09:28:48 -04:00
Patrick Cloke	04faa0bfa9	Convert tags and metrics databases to async/await (#8062)	2020-08-11 17:21:20 -04:00
Patrick Cloke	a0acdfa9e9	Converts event_federation and registration databases to async/await (#8061)	2020-08-11 17:21:13 -04:00
Richard van der Hoff	fcbab08cbd	Add an assertion on prev_events in create_new_client_event (#8041) ... I think this would have caught all the cases in https://github.com/matrix-org/synapse/issues/7642 - and I think a 500 makes more sense here than a 403	2020-08-10 12:29:47 +01:00
Patrick Cloke	7f837959ea	Convert directory, e2e_room_keys, end_to_end_keys, monthly_active_users database to async (#8042)	2020-08-07 13:36:29 -04:00
Patrick Cloke	f3fe6961b2	Convert additional database stores to async/await (#8045)	2020-08-07 12:17:17 -04:00
Erik Johnston	7620912d84	Add health check endpoint (#8048)	2020-08-07 14:21:24 +01:00
Brendan Abolivier	2ffd6783c7	Revert #7736 (#8039)	2020-08-06 17:15:35 +01:00
Patrick Cloke	fe6cfc80ec	Convert some util functions to async (#8035)	2020-08-06 08:39:35 -04:00
Patrick Cloke	d4a7829b12	Convert synapse.api to async/await (#8031)	2020-08-06 08:30:06 -04:00
Patrick Cloke	66f24449dd	Improve performance of the register endpoint (#8009)	2020-08-06 08:09:55 -04:00
Erik Johnston	a7bdf98d01	Rename database classes to make some sense (#8033)	2020-08-05 21:38:57 +01:00
Patrick Cloke	e19de43eb5	Convert streams to async. (#8014)	2020-08-04 07:21:47 -04:00
Andrew Morgan	5d92a1428c	Prevent join->join membership transitions changing member count (#7977) ... `StatsHandler` handles updates to the `current_state_delta_stream`, and updates room stats such as the amount of state events, joined users, etc. However, it counts every new join membership as a new user entering a room (and that user being in another room), whereas it's possible for a user's membership status to go from join -> join, for instance when they change their per-room profile information. This PR adds a check for join->join membership transitions, and bails out early, as none of the further checks are necessary at that point. Due to this bug, membership stats in many rooms have ended up being wildly larger than their true values. I am not sure if we also want to include a migration step which recalculates these statistics (possibly using the `_populate_stats_process_rooms` bg update). Bug introduced in the initial implementation https://github.com/matrix-org/synapse/pull/4338.	2020-08-03 21:54:24 +01:00
Patrick Cloke	6812509807	Implement handling of HTTP HEAD requests. (#7999)	2020-08-03 08:45:42 -04:00
Patrick Cloke	2a89ce8cd4	Convert the crypto module to async/await. (#8003)	2020-08-03 08:29:01 -04:00
Erik Johnston	faba873d4b	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/add_rate_limiting_to_joins	2020-07-31 15:07:01 +01:00
Erik Johnston	18de00adb4	Add ratelimiting on joins	2020-07-31 15:06:56 +01:00
Patrick Cloke	c978f6c451	Convert federation client to async/await. (#7975)	2020-07-30 08:01:33 -04:00
Patrick Cloke	4cce8ef74e	Convert appservice to async. (#7973)	2020-07-30 07:27:39 -04:00
Patrick Cloke	b3a97d6dac	Convert some of the data store to async. (#7976)	2020-07-30 07:20:41 -04:00
Brendan Abolivier	8dff4a1242	Re-implement unread counts (#7736)	2020-07-29 18:26:55 +01:00
Patrick Cloke	3345c166a4	Convert storage layer to async/await. (#7963)	2020-07-28 16:09:53 -04:00
Dirk Klimpel	e866e3b896	Add an option to disable purge in delete room admin API (#7964) ... Add option ```purge``` to ```POST /_synapse/admin/v1/rooms/<room_id>/delete``` Fixes: #3761 Signed-off-by: Dirk Klimpel dirk@klimpel.org	2020-07-28 20:08:23 +01:00
lugino-emeritus	3857de2194	Option to allow server admins to join complex rooms (#7902) ... Fixes #7901. Signed-off-by: Niklas Tittjung <nik_t.01@web.de>	2020-07-28 13:41:44 +01:00
Patrick Cloke	8553f46498	Convert a synapse.events to async/await. (#7949)	2020-07-27 13:40:22 -04:00
Patrick Cloke	5f65e62681	Convert groups and visibility code to async / await. (#7951)	2020-07-27 12:32:08 -04:00
Patrick Cloke	8144bc26a7	Convert push to async/await. (#7948)	2020-07-27 12:21:34 -04:00
Andrew Morgan	c4268e3da6	Convert tests/rest/admin/test_room.py to unix file endings (#7953) ... Converts tests/rest/admin/test_room.py to have unix file endings after they were accidentally changed in #7613. Keeping the same changelog as #7613 as it hasn't gone out in a release yet.	2020-07-27 13:22:52 +01:00
Patrick Cloke	3fc8fdd150	Support oEmbed for media previews. (#7920) ... Fixes previews of Twitter URLs by using their oEmbed endpoint to grab content.	2020-07-27 07:50:44 -04:00
Patrick Cloke	b975fa2e99	Convert state resolution to async/await (#7942)	2020-07-24 10:59:51 -04:00
Patrick Cloke	5ea29d7f85	Convert more of the media code to async/await (#7873)	2020-07-24 09:39:02 -04:00
Patrick Cloke	6a080ea184	Return an empty body for OPTIONS requests. (#7886)	2020-07-24 07:08:07 -04:00
Richard van der Hoff	1ec688bf21	Downgrade warning on client disconnect to INFO (#7928) ... Clients disconnecting before we finish processing the request happens from time to time. We don't need to yell about it	2020-07-24 09:55:47 +01:00
Patrick Cloke	68cd935826	Convert the federation agent and related code to async/await. (#7874)	2020-07-23 07:05:57 -04:00
Patrick Cloke	cc9bb3dc3f	Convert the message handler to async/await. (#7884)	2020-07-22 12:29:15 -04:00
Patrick Cloke	6b3ac3b8cd	Convert device handler to async/await (#7871)	2020-07-17 07:09:25 -04:00
Patrick Cloke	6fca1b3506	Convert _base, profile, and _receipts handlers to async/await (#7860)	2020-07-17 07:08:30 -04:00
Michael Albert	fff483ea96	Add admin endpoint to get members in a room. (#7842)	2020-07-16 16:43:23 -04:00
Erik Johnston	649a7ead5c	Add ability to run multiple pusher instances (#7855) ... This reuses the same scheme as federation sender sharding	2020-07-16 14:06:28 +01:00
Erik Johnston	97e1159ac1	Merge branch 'erikj/faster_typing' of github.com:matrix-org/synapse into develop	2020-07-15 16:54:30 +01:00
Patrick Cloke	8c7d0f163d	Allow accounts to be re-activated from the admin APIs. (#7847)	2020-07-15 11:00:21 -04:00
Erik Johnston	9006e125af	Fix tests	2020-07-15 15:47:27 +01:00
Erik Johnston	f13061d515	Fix client reader sharding tests (#7853) ... * Fix client reader sharding tests * Newsfile * Fix typing * Update changelog.d/7853.misc Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com> * Move mocking of http_client to tests Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>	2020-07-15 15:27:35 +01:00
Patrick Cloke	b11450dedc	Convert E2E key and room key handlers to async/await. (#7851)	2020-07-15 08:48:58 -04:00
Patrick Cloke	111e70d75c	Return the proper 403 Forbidden error during errors with JWT logins. (#7844)	2020-07-15 07:10:21 -04:00
Dirk Klimpel	491f0dab1b	Add delete room admin endpoint (#7613) ... The Delete Room admin API allows server admins to remove rooms from server and block these rooms. `DELETE /_synapse/admin/v1/rooms/<room_id>` It is a combination and improvement of "[Shutdown room](https://github.com/matrix-org/synapse/blob/develop/docs/admin_api/shutdown_room.md)" and "[Purge room](https://github.com/matrix-org/synapse/blob/develop/docs/admin_api/purge_room.md)" API. Fixes: #6425 It also fixes a bug in [synapse/storage/data_stores/main/room.py](synapse/storage/data_stores/main/room.py) in ` get_room_with_stats`. It should return `None` if the room is unknown. But it returns an `IndexError`. 901b1fa561/synapse/storage/data_stores/main/room.py (L99-L105) Related to: - #5575 - https://github.com/Awesome-Technologies/synapse-admin/issues/17 Signed-off-by: Dirk Klimpel dirk@klimpel.org	2020-07-14 12:36:23 +01:00
Patrick Cloke	77d2c05410	Add the option to validate the iss and aud claims for JWT logins. (#7827)	2020-07-14 07:16:43 -04:00
Patrick Cloke	457096e6df	Support handling registration requests across multiple client readers. (#7830)	2020-07-13 13:31:46 -04:00
Patrick Cloke	66a4af8d96	Do not use canonicaljson to magically handle decoding bytes from JSON. (#7802)	2020-07-10 14:30:08 -04:00
Erik Johnston	f299441cc6	Add ability to shard the federation sender (#7798)	2020-07-10 18:26:36 +01:00
Patrick Cloke	38e1fac886	Fix some spelling mistakes / typos. (#7811)	2020-07-09 09:52:58 -04:00
Richard van der Hoff	67593b1728	Add HomeServer.signing_key property (#7805) ... ... instead of duplicating `config.signing_key[0]` everywhere	2020-07-08 17:51:56 +01:00
Richard van der Hoff	76dbd7b8d6	Stop populating unused table local_invites. (#7793) ... This table is no longer used, so we may as well stop populating it. Removing it would prevent people rolling back to older releases of Synapse, so that can happen in a future release.	2020-07-07 14:20:40 +01:00
Patrick Cloke	6d687ebba1	Convert the appservice handler to async/await. (#7775)	2020-07-06 07:40:35 -04:00
reivilibre	57feeab364	Don't ignore set_tweak actions with no explicit value. (#7766) ... * Fix spec compliance; tweaks without values are valid (default to True, which is only concretely specified for `highlight`, but it seems only reasonable to generalise) * Changelog for 7766. * Add documentation to `tweaks_for_actions` May as well tidy up when I'm here. * Add a test for `tweaks_for_actions`	2020-07-06 11:43:41 +01:00
Will Hunt	62b1ce8539	isort 5 compatibility (#7786) ... The CI appears to use the latest version of isort, which is a problem when isort gets a major version bump. Rather than try to pin the version, I've done the necessary to make isort5 happy with synapse.	2020-07-05 16:32:02 +01:00
Erik Johnston	5cdca53aa0	Merge different Resource implementation classes (#7732)	2020-07-03 19:02:19 +01:00
Dirk Klimpel	21a212f8e5	Fix inconsistent handling of upper and lower cases of email addresses. (#7021) ... fixes #7016	2020-07-03 14:03:13 +01:00
reivilibre	e5808c4cfb	Hack to add push priority to push notifications (#7765) ... * Remove obsolete comment about ancient temporary code Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net> * Implement hack to set push priority based on whether the tweaks indicate the event might cause effects. * Changelog for 7765 Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net> * Antilint * Add tests for push priority Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net> * Update synapse/push/httppusher.py Co-authored-by: Brendan Abolivier <babolivier@matrix.org> * Antilint * Remove needless invites from tests. Co-authored-by: Brendan Abolivier <babolivier@matrix.org>	2020-07-01 17:02:31 +01:00
Brendan Abolivier	74d3e177f0	Back out MSC2625 implementation (#7761)	2020-07-01 11:08:25 +01:00
Patrick Cloke	71cccf1593	Additional configuration options for auto-join rooms (#7763)	2020-06-30 15:41:36 -04:00
Erik Johnston	0e0a2817a2	Yield during large v2 state res. (#7735) ... State res v2 across large data sets can be very CPU intensive, and if all the relevant events are in the cache the algorithm will run from start to finish within a single reactor tick. This can result in blocking the reactor tick for several seconds, which can have major repercussions on other requests. To fix this we simply add the occaisonal `sleep(0)` during iterations to yield execution until the next reactor tick. The aim is to only do this for large data sets so that we don't impact otherwise quick resolutions.=	2020-06-24 18:48:18 +01:00
Sorunome	6920e58136	add org.matrix.login.jwt so that m.login.jwt can be deprecated (#7675)	2020-06-24 10:23:55 +01:00
Patrick Cloke	95e41f368b	Allow local media to be marked as safe from being quarantined. (#7718)	2020-06-22 08:04:14 -04:00
Patrick Cloke	3630825612	Convert the typing handler to async/await. (#7679)	2020-06-17 10:37:59 -04:00
Patrick Cloke	434716e1d3	Fetch from the r0 media path instead of the unspecced v1. (#7714)	2020-06-17 08:36:46 -04:00
Brendan Abolivier	46613aaf79	Implement unread counter (MSC2625) (#7673) ... Implementation of https://github.com/matrix-org/matrix-doc/pull/2625	2020-06-17 10:58:32 +01:00
Patrick Cloke	ac51bd581a	Include a user agent in federation requests. (#7677)	2020-06-16 10:43:29 -04:00
Dagfinn Ilmari Mannsåker	a3f11567d9	Replace all remaining six usage with native Python 3 equivalents (#7704)	2020-06-16 08:51:47 -04:00
Richard van der Hoff	03619324fc	Create a ListenerConfig object (#7681) ... This ended up being a bit more invasive than I'd hoped for (not helped by generic_worker duplicating some of the code from homeserver), but hopefully it's an improvement. The idea is that, rather than storing unstructured `dict`s in the config for the listener configurations, we instead parse it into a structured `ListenerConfig` object.	2020-06-16 12:44:07 +01:00
Brendan Abolivier	789606577a	Merge branch 'master' into develop	2020-06-16 10:36:29 +01:00
Andrew Morgan	3e6b5bba71	Wrap register_device coroutine in an ensureDeferred (#7684) ... Fixes https://github.com/matrix-org/synapse/issues/7683 Broke in: #7649 We had a `yield` acting on a coroutine. To be fair this one is a bit difficult to notice as there's a function in the middle that just passes the coroutine along.	2020-06-16 10:13:59 +01:00
Patrick Cloke	cc32fa7358	Ensure the body is a string before comparing push rules. (#7701)	2020-06-15 16:20:34 -04:00
Hubert Chathi	2b2344652b	Ensure etag is a string for GET room_keys/version response (#7691)	2020-06-15 13:42:44 -04:00
Brendan Abolivier	6efb2b0ad4	Merge branch 'develop' into babolivier/mark_unread	2020-06-15 16:37:52 +01:00
Brendan Abolivier	c2b4621630	Merge branch 'develop' into babolivier/mark_unread	2020-06-15 16:37:08 +01:00
Patrick Cloke	7d2532be36	Discard RDATA from already seen positions. (#7648)	2020-06-15 08:44:54 -04:00
Brendan Abolivier	7e80c84902	Lint	2020-06-12 11:31:11 +01:00
Brendan Abolivier	6b1fa3293d	Test that a mark_unread action updates the right counter when using a slave store	2020-06-12 11:28:26 +01:00
Brendan Abolivier	63d9a00bf1	Remove debug logging	2020-06-12 11:13:30 +01:00
Brendan Abolivier	2a07c5ded6	Test that a mark_unread action updates the right counter	2020-06-12 11:08:05 +01:00
Brendan Abolivier	476a89707a	Fix tests	2020-06-10 17:55:03 +01:00
Andrew Morgan	fcd6961441	Add option to enable encryption by default for new rooms (#7639) ... Fixes https://github.com/matrix-org/synapse/issues/2431 Adds config option `encryption_enabled_by_default_for_room_type`, which determines whether encryption should be enabled with the default encryption algorithm in private or public rooms upon creation. Whether the room is private or public is decided based upon the room creation preset that is used. Part of this PR is also pulling out all of the individual instances of `m.megolm.v1.aes-sha2` into a constant variable to eliminate typos ala https://github.com/matrix-org/synapse/pull/7637 Based on #7637	2020-06-10 17:44:34 +01:00