forked-synapse

mirror of https://mau.dev/maunium/synapse.git synced 2024-10-01 01:36:05 -04:00

Author	SHA1	Message	Date
Matthew Hodgson	4a6d5de98c	Make /sync attempt to return device updates for both joined and invited users (#3484 )	2019-05-16 13:23:43 +01:00
David Baker	fafb936de5	Merge pull request #5187 from matrix-org/dbkr/only_check_threepid_not_in_use_if_actually_registering Only check 3pids not in use when registering	2019-05-16 10:58:09 +01:00
Erik Johnston	b5c62c6b26	Fix relations in worker mode	2019-05-16 10:38:13 +01:00
Erik Johnston	33453419b0	Add cache to relations	2019-05-16 10:02:14 +01:00
Erik Johnston	a0603523d2	Add aggregations API	2019-05-16 09:37:20 +01:00
Amber Brown	f1e5b41388	Make all the rate limiting options more consistent (#5181 )	2019-05-15 12:06:04 -05:00
Richard van der Hoff	5f027a315f	Drop support for v2_alpha API prefix (#5190 )	2019-05-15 17:37:46 +01:00
Erik Johnston	5be34fc3e3	Actually check for None rather falsey	2019-05-15 17:30:23 +01:00
Erik Johnston	e6459c26b4	Actually implement idempotency	2019-05-15 17:28:33 +01:00
Richard van der Hoff	1757e2d7c3	Merge branch 'master' into develop	2019-05-15 14:09:30 +01:00
Richard van der Hoff	4a926f528e	0.99.4	2019-05-15 13:58:45 +01:00
Erik Johnston	b50641e357	Add simple pagination API	2019-05-15 13:36:51 +01:00
Erik Johnston	efe3c7977a	Add simple send_relation API and track in DB	2019-05-15 13:36:51 +01:00
Erik Johnston	54d77107c1	Make generating SQL bounds for pagination generic This will allow us to reuse the same structure when we paginate e.g. relations	2019-05-15 11:30:05 +01:00
Erik Johnston	0aba6c8251	Merge pull request #5183 from matrix-org/erikj/async_serialize_event Allow client event serialization to be async	2019-05-15 10:36:30 +01:00
Erik Johnston	d94544051b	Merge pull request #5184 from matrix-org/erikj/expose_get_events_as_array Expose DataStore._get_events as get_events_as_list	2019-05-15 10:17:38 +01:00
Erik Johnston	8ed2f182f7	Update docstring with correct return type Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>	2019-05-15 09:52:52 +01:00
Erik Johnston	52ddc6c0ed	Update docstring with correct type Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>	2019-05-15 09:52:15 +01:00
David Baker	6ca88c4693	Only check 3pids not in use when registering We checked that 3pids were not already in use before we checked if we were going to return the account previously registered in the same UI auth session, in which case the 3pids will definitely be in use. https://github.com/vector-im/riot-web/issues/9586	2019-05-14 19:04:59 +01:00
Richard van der Hoff	daa2fb6317	comment about user_joined_room	2019-05-14 18:53:09 +01:00
Erik Johnston	4fb44fb5b9	Expose DataStore._get_events as get_events_as_list This is in preparation for reaction work which requires it.	2019-05-14 13:37:44 +01:00
Erik Johnston	a80e6b53f9	Newsfile	2019-05-14 13:12:23 +01:00
Erik Johnston	b54b03f9e1	Allow client event serialization to be async	2019-05-14 11:58:01 +01:00
Amber Brown	df2ebd75d3	Migrate all tests to use the dict-based config format instead of hanging items off HomeserverConfig (#5171 )	2019-05-13 15:01:14 -05:00
Andrew Morgan	5a4b328f52	Add ability to blacklist ip ranges for federation traffic (#5043 )	2019-05-13 19:05:06 +01:00
Brendan Abolivier	2e1129b5f7	0.99.4rc1	2019-05-13 16:11:21 +01:00
David Baker	516a5fb64b	Merge remote-tracking branch 'origin/develop' into dbkr/add_dummy_flow_to_recaptcha_only	2019-05-13 15:54:25 +01:00
David Baker	9e99143c47	Merge remote-tracking branch 'origin/develop' into dbkr/add_dummy_flow_to_recaptcha_only	2019-05-13 15:37:03 +01:00
Brendan Abolivier	bb93757b32	Fix CI after new release of isort	2019-05-13 15:19:44 +01:00
Andrew Morgan	2f48c4e1ae	URL preview blacklisting fixes (#5155 ) Prevents a SynapseError being raised inside of a IResolutionReceiver and instead opts to just return 0 results. This thus means that we have to lump a failed lookup and a blacklisted lookup together with the same error message, but the substitute should be generic enough to cover both cases.	2019-05-10 10:32:44 -07:00
David Baker	04299132af	Re-order flows so that email auth is done last It's more natural for the user if the bit that takes them away from the registration flow comes last. Adding the dummy stage allows us to do the stages in this order without the ambiguity.	2019-05-10 13:58:03 +01:00
David Baker	9c61dce3c8	Comment	2019-05-10 11:14:55 +01:00
David Baker	8714ff6d51	Add a DUMMY stage to captcha-only registration flow This allows the client to complete the email last which is more natual for the user. Without this stage, if the client would complete the recaptcha (and terms, if enabled) stages and then the registration request would complete because you've now completed a flow, even if you were intending to complete the flow that's the same except has email auth at the end. Adding a dummy auth stage to the recaptcha-only flow means it's always unambiguous which flow the client was trying to complete. Longer term we should think about changing the protocol so the client explicitly says which flow it's trying to complete. vector-im/riot-web#9586	2019-05-10 11:09:53 +01:00
David Baker	c2bb7476c9	Revert `085ae346ac` Accidentally went straight to develop	2019-05-10 11:08:01 +01:00
David Baker	085ae346ac	Add a DUMMY stage to captcha-only registration flow This allows the client to complete the email last which is more natual for the user. Without this stage, if the client would complete the recaptcha (and terms, if enabled) stages and then the registration request would complete because you've now completed a flow, even if you were intending to complete the flow that's the same except has email auth at the end. Adding a dummy auth stage to the recaptcha-only flow means it's always unambiguous which flow the client was trying to complete. Longer term we should think about changing the protocol so the client explicitly says which flow it's trying to complete. https://github.com/vector-im/riot-web/issues/9586	2019-05-10 10:52:24 +01:00
Richard van der Hoff	130f932cbc	Run `black` on per_destination_queue ... mostly to fix pep8 fails	2019-05-09 16:27:02 +01:00
Quentin Dufour	11ea16777f	Limit the number of EDUs in transactions to 100 as expected by receiver (#5138 ) Fixes #3951.	2019-05-09 11:01:41 +01:00
Matthew Hodgson	c0e0740bef	add options to require an access_token to GET /profile and /publicRooms on CS API (#5083 ) This commit adds two config options: * `restrict_public_rooms_to_local_users` Requires auth to fetch the public rooms directory through the CS API and disables fetching it through the federation API. * `require_auth_for_profile_requests` When set to `true`, requires that requests to `/profile` over the CS API are authenticated, and only returns the user's profile if the requester shares a room with the profile's owner, as per MSC1301. MSC1301 also specifies a behaviour for federation (only returning the profile if the server asking for it shares a room with the profile's owner), but that's currently really non-trivial to do in a not too expensive way. Next step is writing down a MSC that allows a HS to specify which user sent the profile query. In this implementation, Synapse won't send a profile query over federation if it doesn't believe it already shares a room with the profile's owner, though. Groups have been intentionally omitted from this commit.	2019-05-08 18:26:56 +01:00
Erik Johnston	c8c069db92	Merge pull request #5037 from matrix-org/erikj/limit_inflight_dns Limit in flight DNS requests	2019-05-08 17:11:03 +01:00
Brendan Abolivier	1473058b5e	Do checks on aliases for incoming m.room.aliases events (#5128 ) Follow-up to #5124 Also added a bunch of checks to make sure everything (both the stuff added on #5124 and this PR) works as intended.	2019-05-08 17:01:30 +01:00
Erik Johnston	de655e669a	Merge pull request #5104 from matrix-org/erikj/ratelimit_3pid_invite Ratelimit 3pid invites	2019-05-07 10:12:49 +01:00
Richard van der Hoff	59e2d2694d	Remove the requirement to authenticate for /admin/server_version. (#5122 ) This endpoint isn't much use for its intended purpose if you first need to get yourself an admin's auth token. I've restricted it to the `/_synapse/admin` path to make it a bit easier to lock down for those concerned about exposing this information. I don't imagine anyone is using it in anger currently.	2019-05-07 09:29:30 +01:00
Richard van der Hoff	836d3adcce	Merge branch 'master' into develop	2019-05-03 19:25:01 +01:00
Richard van der Hoff	fa21455e08	0.99.3.2	2019-05-03 18:56:24 +01:00
Richard van der Hoff	e3281d7d26	pin urllib3 to <1.25	2019-05-03 18:33:10 +01:00
Richard van der Hoff	863ec09622	0.99.3.1	2019-05-03 16:03:24 +01:00
Richard van der Hoff	a845abbf3a	Merge pull request #5134 from matrix-org/rav/url_preview_blacklist Blacklist 0.0.0.0 and :: by default for URL previews	2019-05-03 15:59:20 +01:00
Richard van der Hoff	1565ebec2c	more config comment updates	2019-05-03 15:50:59 +01:00
Richard van der Hoff	1acfb9e9f0	Merge pull request #5133 from matrix-org/rav/systemrandom Use SystemRandom for token generation.	2019-05-03 15:39:30 +01:00
Richard van der Hoff	1a7104fde3	Blacklist 0.0.0.0 and :: by default for URL previews	2019-05-03 15:35:49 +01:00
Richard van der Hoff	247dc1bd0b	Use SystemRandom for token generation	2019-05-03 13:02:55 +01:00
Erik Johnston	176f31c2e3	Rate limit early	2019-05-02 15:23:08 +01:00
Richard van der Hoff	12f9d51e82	Add admin api for sending server_notices (#5121 )	2019-05-02 11:59:16 +01:00
Brendan Abolivier	c193b39134	Merge pull request #5124 from matrix-org/babolivier/aliases Add some limitations to alias creation	2019-05-02 11:22:40 +01:00
Brendan Abolivier	84196cb231	Add some limitations to alias creation	2019-05-02 11:05:11 +01:00
Richard van der Hoff	0836cbb9f5	Factor out an "assert_requester_is_admin" function (#5120 ) Rather than copying-and-pasting the same four lines hundreds of times	2019-05-02 10:45:52 +01:00
Richard van der Hoff	f203c98794	fix examples	2019-05-01 17:49:56 +01:00
Richard van der Hoff	40e576e29c	Move admin api impl to its own package It doesn't really belong under rest/client/v1 any more.	2019-05-01 15:44:30 +01:00
Richard van der Hoff	8e9ca83537	Move admin API to a new prefix	2019-05-01 15:44:30 +01:00
Richard van der Hoff	579b637b6c	Move admin API away from ClientV1RestServlet	2019-05-01 15:16:04 +01:00
Travis Ralston	8c5b1e30d4	Add a default .m.rule.tombstone push rule (#4867 ) * Add a default .m.rule.tombstone push rule In support of MSC1930: https://github.com/matrix-org/matrix-doc/pull/1930 * changelog * Appease the changelog linter	2019-04-29 15:40:31 -06:00
Richard van der Hoff	b31cc1c613	Merge pull request #5100 from matrix-org/rav/verification_hackery Improve logging when event-signature checking fails	2019-04-29 13:19:32 +01:00
Erik Johnston	d6118c5be6	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/ratelimit_3pid_invite	2019-04-26 18:14:23 +01:00
Erik Johnston	28a81ed62f	Ratelimit 3pid invites We do ratelimit sending the 3PID invite events, but that happens after spamming the identity server.	2019-04-26 18:06:25 +01:00
Richard van der Hoff	bd0d45ca69	Fix infinite loop in presence handler Fixes #5102	2019-04-26 11:14:49 +01:00
Richard van der Hoff	837d7f85a9	more logging improvements	2019-04-25 22:17:59 +01:00
Richard van der Hoff	fd8fb32bdd	remove extraneous exception logging	2019-04-25 22:02:03 +01:00
Richard van der Hoff	7ca638c761	Clarify logging when PDU signature checking fails	2019-04-25 20:55:12 +01:00
Andrew Morgan	6824ddd93d	Config option for verifying federation certificates (MSC 1711) (#4967 )	2019-04-25 14:22:49 +01:00
Michael Kaye	788163e204	Remove log error for .well-known/matrix/client (#4972 )	2019-04-24 17:44:06 +01:00
Katie Wolfe	7e07dc429f	Lint I probably should've just run autopep8 in the first place... Signed-off-by: Katie Wolfe <katie@dnaf.moe>	2019-04-24 12:43:18 -04:00
Amber Brown	6b2b9a58c4	Prevent "producer not unregistered" message (#5009 )	2019-04-24 17:37:32 +01:00
Katie Wolfe	b3e5db402d	Clean up code Signed-off-by: Katie Wolfe <katie@dnaf.moe>	2019-04-24 12:26:55 -04:00
Katie Wolfe	5d3ed79944	Show heroes if room name or canonical alias are empty Fixes #4194 Signed-off-by: Katie Wolfe <katie@dnaf.moe>	2019-04-24 12:26:53 -04:00
Katie Wolfe	7f025eb425	Show heroes if room name or canonical alias are empty Fixes #4194 Signed-off-by: Katie Wolfe <katie@dnaf.moe>	2019-04-24 12:26:29 -04:00
Brendan Abolivier	f8826d31cd	Don't crash on lack of expiry templates	2019-04-18 14:50:05 +01:00
Erik Johnston	ca90336a69	Merge branch 'develop' of github.com:matrix-org/synapse into babolivier/account_expiration	2019-04-17 19:44:40 +01:00
Brendan Abolivier	eaf41a943b	Add management endpoints for account validity	2019-04-17 19:34:45 +01:00
Brendan Abolivier	91934025b9	Merge pull request #5047 from matrix-org/babolivier/account_expiration Send out emails with links to extend an account's validity period	2019-04-17 14:57:39 +01:00
Brendan Abolivier	20f0617e87	Send out emails with links to extend an account's validity period	2019-04-17 14:42:20 +01:00
Brendan Abolivier	49ff74da9b	Merge pull request #5071 from matrix-org/babolivier/3pid-check Make sure we're not registering the same 3pid twice	2019-04-17 14:37:42 +01:00
Brendan Abolivier	600ec04739	Make sure we're not registering the same 3pid twice	2019-04-17 14:23:01 +01:00
Erik Johnston	fd2fcb817c	Merge pull request #5070 from matrix-org/erikj/postpath Remove usage of request.postpath	2019-04-17 09:03:26 +01:00
Erik Johnston	a1eb4c6d2f	Merge pull request #5065 from matrix-org/erikj/fix_versions VersionRestServlet doesn't take a param	2019-04-16 17:52:36 +01:00
Erik Johnston	ad010f6306	Remove usage of request.postpath This is an undocumented variable in twisted, and relies on the servlet being mounted in the right way. This also breaks getting push rules on workers.	2019-04-16 17:42:50 +01:00
Silke Hofstra	a137f4eac0	Add systemd-python to optional dependencies (#4339 ) Using systemd-python allows for logging to the systemd journal, as is documented in: `synapse/contrib/systemd/log_config.yaml`. Signed-off-by: Silke Hofstra <silke@slxh.eu>	2019-04-16 20:41:17 +10:00
Erik Johnston	38642614cf	VersionRestServlet doesn't take a param	2019-04-15 19:39:47 +01:00
Erik Johnston	6e27a8620f	Merge pull request #5063 from matrix-org/erikj/move_endpoints Move some rest endpoints to client reader	2019-04-15 18:55:01 +01:00
Erik Johnston	ec638a1602	Only handle GET requests for /push_rules	2019-04-15 18:51:48 +01:00
Erik Johnston	d5adf297e6	Move some rest endpoints to client reader	2019-04-15 17:21:03 +01:00
Andrew Morgan	caa76e6021	Remove periods from copyright headers (#5046 )	2019-04-11 17:08:13 +01:00
Erik Johnston	c132c8e505	Handle the case of `get_missing_events` failing Currently if a call to `/get_missing_events` fails we log an exception and stop processing the top level event we received over federation. Instead let's try and handle it sensibly given it is a somewhat expected failure mode.	2019-04-10 10:39:54 +01:00
Erik Johnston	e446921def	Merge pull request #5033 from matrix-org/erikj/fix_schema_delta Fix schema upgrade when dropping tables	2019-04-10 10:22:35 +01:00
Richard van der Hoff	329688c161	Fix disappearing exceptions in manhole. (#5035 ) Avoid sending syntax errors from the manhole to sentry.	2019-04-10 07:23:48 +01:00
Erik Johnston	a0fc256d65	Limit in flight DNS requests This is to work around a bug in twisted where a large number of concurrent DNS requests cause it to tight loop forever. c.f. https://twistedmatrix.com/trac/ticket/9620#ticket	2019-04-09 17:23:42 +01:00
Brendan Abolivier	bfc8fdf1fc	Merge pull request #5027 from matrix-org/babolivier/account_expiration Add time-based account expiration	2019-04-09 17:02:41 +01:00
Brendan Abolivier	747aa9f8ca	Add account expiration feature	2019-04-09 16:46:04 +01:00
Erik Johnston	50d2a3059d	Fix schema upgrade when dropping tables We need to drop tables in the correct order due to foreign table constraints (on `application_services`), otherwise the DROP TABLE command will fail. Introduced in #4992.	2019-04-09 14:39:18 +01:00
Richard van der Hoff	644b86677f	Merge pull request #5030 from matrix-org/rav/rewrite_g_s_v_k Rewrite Datastore.get_server_verify_keys	2019-04-09 11:30:23 +01:00
Richard van der Hoff	4abf5aa81a	Bump psycopg requirement (#5032 )	2019-04-09 11:29:50 +01:00
Richard van der Hoff	18b69be00f	Rewrite Datastore.get_server_verify_keys Rewrite this so that it doesn't hammer the database.	2019-04-09 00:00:10 +01:00
Richard van der Hoff	f50efcb65d	Replace SlavedKeyStore with a shim since we're pulling everything out of KeyStore anyway, we may as well simplify it.	2019-04-08 23:59:07 +01:00
Richard van der Hoff	f88a9e6323	Remove redundant merged_keys dict There's no point in collecting a merged dict of keys: it is sufficient to consider just the new keys which have been fetched by the most recent key_fetch_fns.	2019-04-08 22:36:18 +01:00
Richard van der Hoff	3352baac4b	Remove unused server_tls_certificates functions (#5028 ) These have been unused since #4120, and with the demise of perspectives, it is unlikely that they will ever be used again.	2019-04-08 21:50:18 +01:00
Neil Johnson	b25e387c0d	add context to phonehome stats (#5020 ) add context to phonehome stats	2019-04-08 15:47:39 +01:00
Richard van der Hoff	67d7b44784	Merge pull request #5024 from matrix-org/rav/record_correct_server_in_serverkeys Fix from_server buglet in get_keys_from_perspectives	2019-04-08 15:40:37 +01:00
Neil Johnson	2d951686a7	drop tables listed in #1830 (#4992 ) Tables dropped: * application_services, * application_services_regex, * transaction_id_to_pdu, * stats_reporting * current_state_resets * event_content_hashes * event_destinations * event_edge_hashes * event_signatures * feedback * room_hosts * state_forward_extremities	2019-04-08 15:37:26 +01:00
Richard van der Hoff	7d2a0c848e	Fix from_server buglet in get_keys_from_perspectives make sure we store the name of the server the keys came from, rather than the origin server, after doing a fetch-from-perspectives.	2019-04-08 12:51:16 +01:00
Richard van der Hoff	7fc1e17f4c	Merge pull request #5001 from matrix-org/rav/keyring_cleanups Cleanups in the Keyring	2019-04-08 12:47:09 +01:00
Richard van der Hoff	6ae9361510	Hoist server_name check out of process_v2_response It's easier to check it in the caller than to complicate the interface with an extra param.	2019-04-04 19:12:54 +01:00
Richard van der Hoff	ef27d434d1	Clean up Keyring.process_v2_response Make this just return the key dict, rather than a single-entry dict mapping the server name to the key dict. It's easy for the caller to get the server name from from the response object anyway.	2019-04-04 19:12:54 +01:00
Richard van der Hoff	b43d9a920b	Fix docstring on get_server_keys_json	2019-04-04 18:54:03 +01:00
Brendan Abolivier	8e85493b0c	Add config option to block users from looking up 3PIDs (#5010 )	2019-04-04 17:25:47 +01:00
Amber Brown	a33a5abc4c	Clean up the database pagination code (#5007 ) * rewrite & simplify * changelog * cleanup potential sql injection	2019-04-05 00:21:16 +11:00
Erik Johnston	616e6a10bd	Merge pull request #5002 from matrix-org/erikj/delete_group Add delete group admin API	2019-04-04 14:15:41 +01:00
Andrew Morgan	db265f0642	Prevent kicking users who aren't in the room (#4999 ) Prevent kick events from succeeding if the user is not currently in the room.	2019-04-04 13:05:51 +01:00
Marcel Krüger	9f5d206c4a	Avoid redundant URL encoding (#4555 ) * Do not double encode fallback redirect URL Signed-off-by: Marcel Fabian Krüger <zauguin@gmail.com>	2019-04-04 12:05:56 +01:00
Erik Johnston	c192bf8970	Add admin API for group deletion	2019-04-03 16:29:52 +01:00
Erik Johnston	4a2e13631d	Add functions to delete a group	2019-04-03 16:29:52 +01:00
Andrew Morgan	4a4d5c4fd6	Fix grammar and document get_current_users_in_room (#4998 )	2019-04-03 14:32:20 +01:00
Neil Johnson	e8419554ff	Remove presence lists (#4989 ) Remove presence list support as per MSC 1819	2019-04-03 11:11:15 +01:00
Erik Johnston	8f549c1177	Merge pull request #4982 from matrix-org/erikj/msc1915 Implement MSC1915 - 3PID unbind APIs	2019-04-03 11:07:09 +01:00
Amber Brown	7efd1d87c2	Run black on the rest of the storage module (#4996 )	2019-04-03 10:07:29 +01:00
Erik Johnston	3039d61baf	Merge pull request #4991 from matrix-org/erikj/stagger_push_startup Make starting pushers faster during start up	2019-04-02 18:23:32 +01:00
Andrew Morgan	66e78700a2	Transfer related groups on room upgrade (#4990 ) Transfers the m.room.related_groups state event on room upgrade.	2019-04-02 17:15:24 +01:00
Richard van der Hoff	8530090b16	Add config.signing_key_path. (#4974 ) As requested by @andrewshadura	2019-04-02 16:59:27 +01:00
Erik Johnston	5bec8d660d	Make starting pushers faster during start up We start all pushers on start up and immediately start a background process to fetch push to send. This makes start up incredibly painful when dealing with many pushers. Instead, let's do a quick fast DB check to see if there may be push to send and only start the background processes for those pushers. We also stagger starting up and doing those checks so that we don't try and handle all pushers at once.	2019-04-02 16:59:13 +01:00
Richard van der Hoff	297bf2547e	Fix sync bug when accepting invites (#4956 ) Hopefully this time we really will fix #4422. We need to make sure that the cache on `get_rooms_for_user_with_stream_ordering` is invalidated before the SyncHandler is notified for the new events, and we can now do so reliably via the `events` stream.	2019-04-02 12:42:39 +01:00
Erik Johnston	4ef5d17b96	Correctly handle id_server param	2019-04-02 11:20:09 +01:00
Erik Johnston	24232514bf	Remove threepid binding if id server returns 400/404/501	2019-04-02 11:20:09 +01:00
Erik Johnston	c75e2017f1	Fixup docstrings	2019-04-02 11:20:06 +01:00
Erik Johnston	862d6e5ba5	Add unbind API to /r0 as it is now stabalised	2019-04-01 15:25:19 +01:00
Erik Johnston	3715c124b3	Grandfather in existing user threepids We assume, as we did before, that users bound their threepid to one of the trusted identity servers. So we simply fill the new table with all threepids in `user_threepids` joined with the trusted identity servers.	2019-04-01 15:25:19 +01:00
Erik Johnston	057715aaa2	Allowing specifying IS to use in unbind API. By default the homeserver will use the identity server used during the binding of the 3PID to unbind the 3PID. However, we need to allow clients to explicitly ask the homeserver to unbind via a particular identity server, for the case where the 3PID was bound out of band from the homeserver. Implements MSC915.	2019-04-01 15:25:18 +01:00
Erik Johnston	9fbbc3d9e5	For unbind poke IS used during binding of 3PID This changes the behaviour from using the server specified trusted identity server to using the IS that used during the binding of the 3PID, if known. This is the behaviour specified by MSC1915.	2019-04-01 15:23:30 +01:00
Erik Johnston	1666c0696a	Track IS used to bind 3PIDs This will then be used to know which IS to default to when unbinding the threepid.	2019-04-01 15:23:01 +01:00
Neil Johnson	62988f73fd	Merge branch 'master' into develop	2019-04-01 14:08:53 +01:00
Neil Johnson	35442efb75	0.99.3	2019-04-01 12:49:03 +00:00
Richard van der Hoff	54a87a7b08	Collect room-version variations into one place (#4969 ) Collect all the things that make room-versions different to one another into one place, so that it's easier to define new room versions.	2019-04-01 10:24:38 +01:00
Richard van der Hoff	902cdc63b6	Merge pull request #4955 from matrix-org/rav/merge_state_into_events Combine the CurrentStateDeltaStream into the EventStream	2019-03-28 18:32:13 +00:00
Richard van der Hoff	d688a51736	Merge pull request #4954 from matrix-org/rav/refactor_parse_row Refactors to replication stream row update/parsing	2019-03-28 18:31:17 +00:00
Neil Johnson	c7296bcb98	remove log line for password (#4965 ) Remove log line for password.	2019-03-28 17:38:01 +00:00
Andrew Morgan	7a91b9d81c	Allow password providers to bind emails (#4947 ) This PR allows password provider modules to bind email addresses when a user is registering and is motivated by matrix-org/matrix-synapse-ldap3#58	2019-03-28 15:48:07 +00:00
Erik Johnston	248014379e	Merge pull request #4942 from matrix-org/erikj/fix_presence Use event streams to calculate presence	2019-03-28 14:38:31 +00:00
Erik Johnston	4e5f0f7ca0	Use an assert	2019-03-28 14:05:05 +00:00
Erik Johnston	40e56997bc	Review comments	2019-03-28 13:48:41 +00:00
Richard van der Hoff	d035d62f6b	Merge remote-tracking branch 'origin/develop' into rav/refactor_parse_row	2019-03-28 13:45:14 +00:00
Richard van der Hoff	4eeb2c2f07	Merge pull request #4953 from matrix-org/rav/refactor_replication_streams Split up replication.tcp.streams into smaller files	2019-03-28 13:43:25 +00:00
Amber Brown	2e060774ad	Run `black` on some storage modules that the stats branch touches (#4959 )	2019-03-29 00:37:16 +11:00
Richard van der Hoff	4b91c313a9	Combine the CurrentStateDeltaStream into the EventStream	2019-03-27 22:07:05 +00:00
Richard van der Hoff	1f6d6f918a	Make EventStream rows have a type ... as a precursor to combining it with the CurrentStateDelta stream.	2019-03-27 22:07:05 +00:00
Richard van der Hoff	015b3622eb	Skip building a ROW_TYPE when building updates We're about to turn it straight into a JSON object anyway so building a ROW_TYPE is a bit pointless, and reduces flexibility in the update_function.	2019-03-27 21:58:03 +00:00
Richard van der Hoff	f570916a3e	Add parse_row method to replication stream class This will allow individual stream classes to override how a row is parsed.	2019-03-27 21:32:33 +00:00
Richard van der Hoff	71dcb275f1	move FederationStream out to its own file	2019-03-27 21:13:14 +00:00
Richard van der Hoff	aa1e017864	move EventsStream out to its own file	2019-03-27 21:13:14 +00:00
Richard van der Hoff	a5798de067	Move replication.tcp.streams into a package	2019-03-27 21:13:14 +00:00
Richard van der Hoff	acaa18f7dd	Fix/improve some docstrings in the replication code. (#4949 )	2019-03-27 21:12:36 +00:00
Erik Johnston	197fae1639	Use event streams to calculate presence Primarily this fixes a bug in the handling of remote users joining a room where the server sent out the presence for all local users in the room to all servers in the room. We also change to using the state delta stream, rather than the distributor, as it will make it easier to split processing out of the master process (as well as being more flexible). Finally, when sending presence states to newly joined servers we filter out old presence states to reduce the number sent. Initially we filter out states that are offline and have a last active more than a week ago, though this can be changed down the line. Fixes #3962	2019-03-27 13:41:36 +00:00
Neil Johnson	4aa914369b	bump version	2019-03-27 10:23:03 +00:00
Andrew Morgan	bbd244c7b2	Support 3PID login in password providers (#4931 ) Adds a new method, check_3pid_auth, which gives password providers the chance to allow authentication with third-party identifiers such as email or msisdn.	2019-03-26 17:48:30 +00:00
Amber Brown	903f04c21f	Use the state event amount for userdir import batching, not room count (#4944 )	2019-03-27 02:49:28 +11:00
Andrew Morgan	4a125be138	Make federation endpoints more tolerant of trailing slashes v2 (#4935 ) Redo of https://github.com/matrix-org/synapse/pull/4840	2019-03-26 11:35:29 +00:00
Richard van der Hoff	8cbbedaa2b	Fix ClientReplicationStreamProtocol.__str__ (#4929 ) `__str__` depended on `self.addr`, which was absent from ClientReplicationStreamProtocol, so attempting to call str on such an object would raise an exception. We can calculate the peer addr from the transport, so there is no need for addr anyway.	2019-03-25 16:41:51 +00:00
Richard van der Hoff	9bde730ef8	Fix bug where read-receipts lost their timestamps (#4927 ) Make sure that they are sent correctly over the replication stream. Fixes: #4898	2019-03-25 16:38:05 +00:00
Amber Brown	ac396a0d32	Refactor out state delta handling into its own class (#4917 )	2019-03-25 20:37:08 +11:00
Erik Johnston	5fee9d8067	Merge pull request #4869 from matrix-org/erikj/yaml_load Fix yaml warnings by using safe_load	2019-03-22 11:58:13 +00:00
Erik Johnston	3677548a82	Use yaml safe_load	2019-03-22 10:20:17 +00:00
Andrew Morgan	7bef97dfb7	Remove trailing slashes from outbound federation requests and retry on 400 (#4840 ) As per #3622, we remove trailing slashes from outbound federation requests. However, to ensure that we remain backwards compatible with previous versions of Synapse, if we receive a HTTP 400 with `M_UNRECOGNIZED`, then we are likely talking to an older version of Synapse in which case we retry with a trailing slash appended to the request path.	2019-03-21 15:07:28 +00:00
Andrew Morgan	b41c2eaadc	Clean up backoff_on_404 and metehod calls	2019-03-21 14:32:47 +00:00
Erik Johnston	01e6b405be	Merge pull request #4908 from matrix-org/erikj/block_peek_on_blocked_rooms Deny peeking into rooms that have been blocked	2019-03-21 14:07:17 +00:00
Erik Johnston	3959858eaa	Merge pull request #4904 from matrix-org/erikj/fix_shutdown Fixup shutdown room API	2019-03-21 11:24:42 +00:00
Erik Johnston	536a266520	Deny peeking into rooms that have been blocked	2019-03-21 11:20:13 +00:00
Erik Johnston	3ecec5ede2	Fix upsert	2019-03-21 10:21:15 +00:00
Erik Johnston	09f991a63d	Merge pull request #4896 from matrix-org/erikj/disable_room_directory Add option to disable search room lists	2019-03-21 10:16:54 +00:00
Erik Johnston	cd62981a6a	Revert spurious delete	2019-03-20 17:51:27 +00:00
Erik Johnston	8d8834d3e7	comment block_room	2019-03-20 17:49:56 +00:00
Erik Johnston	aa959a6c07	Use flags	2019-03-20 17:40:29 +00:00
Erik Johnston	7d47cc1305	Move requester check into assert_accepted_privacy_policy	2019-03-20 17:08:36 +00:00
Erik Johnston	72a14860ab	Gracefully handle failing to kick user	2019-03-20 16:54:21 +00:00
Erik Johnston	6b28890543	Log new room ID	2019-03-20 16:52:28 +00:00
Erik Johnston	74c46d81fa	Only require consent for events with an associated request There are a number of instances where a server or admin may puppet a user to join/leave rooms, which we don't want to fail if the user has not consented to the privacy policy. We fix this by adding a check to test if the requester has an associated access_token, which is used as a proxy to answer the question of whether the action is being done on behalf of a real request from the user.	2019-03-20 16:50:23 +00:00
Erik Johnston	67d618e111	Allow blocking a room multiple times	2019-03-20 16:50:05 +00:00
Erik Johnston	263f2c9ce1	Merge pull request #4895 from matrix-org/erikj/disable_user_search Add option to disable searching in the user dir	2019-03-20 16:47:15 +00:00
Amber Brown	4d53017432	Batching in the user directory import (#4900 )	2019-03-21 03:06:36 +11:00
Richard van der Hoff	cdb8036161	Add a config option for torture-testing worker replication. (#4902 ) Setting this to 50 or so makes a bunch of sytests fail in worker mode.	2019-03-20 16:04:35 +00:00
Richard van der Hoff	a902d13180	Batch up outgoing read-receipts to reduce federation traffic. (#4890 ) Rate-limit outgoing read-receipts as per #4730.	2019-03-20 16:02:25 +00:00
Erik Johnston	cd8c5b91ad	Fix up sample config	2019-03-20 14:35:41 +00:00
Richard van der Hoff	ab20f85c59	Update synapse/config/user_directory.py Co-Authored-By: erikjohnston <erikj@jki.re>	2019-03-20 14:33:11 +00:00
Erik Johnston	cc197a61a1	Disable publishing to room list when its disabled	2019-03-20 14:30:36 +00:00
Erik Johnston	2c90422146	Pull out config option	2019-03-20 14:25:58 +00:00
Erik Johnston	7529038e66	Return before we log	2019-03-20 14:25:28 +00:00
Erik Johnston	926f29ea6d	Fix up config comments	2019-03-20 14:24:53 +00:00
Andrew Morgan	2150151abe	kwargs doesn't like commas on calling funcs either. TIL	2019-03-20 14:13:32 +00:00
Andrew Morgan	bb52a2e653	lint	2019-03-20 14:08:57 +00:00
Andrew Morgan	cd36a1283b	New test, fix issues	2019-03-20 14:00:39 +00:00
Andrew Morgan	c69df5d5d3	Fix comments. v0.99.2 -> v0.99.3	2019-03-20 11:27:18 +00:00
Andrew Morgan	551ea11559	Just return if not doing any trailing slash shennanigans	2019-03-20 11:07:36 +00:00
Erik Johnston	213c98c00a	Add option to disable search room lists This disables both local and remote room list searching.	2019-03-19 17:10:52 +00:00
Erik Johnston	320667a479	Add option to disable searching in the user dir We still populate it, as it can still be accessed via the admin API.	2019-03-19 16:40:19 +00:00
Richard van der Hoff	13bc1e0746	Use a regular HomeServerConfig object for unit tests Rather than using a Mock for the homeserver config, use a genuine HomeServerConfig object. This makes for a more realistic test, and means that we don't have to keep remembering to add things to the mock config every time we add a new config setting.	2019-03-19 11:44:43 +00:00
Richard van der Hoff	45c4e19c74	Merge remote-tracking branch 'origin/develop' into HEAD	2019-03-19 11:43:46 +00:00
Neil Johnson	88f0675967	fix test_auto_create_auto_join_where_no_consent (#4886 )	2019-03-19 11:38:59 +00:00
Richard van der Hoff	0dbfae03f9	Enforce hs_disabled_message correctly Fixes a bug where hs_disabled_message was not enforced for 3pid-based requests if there was no server_notices_mxid configured.	2019-03-19 11:30:54 +00:00
Erik Johnston	e9eeca1314	Fix user directory background update (#4887 )	2019-03-19 22:13:53 +11:00
Richard van der Hoff	fd463b4f5d	Comment out most options in the generated config. (#4863 ) Make it so that most options in the config are optional, and commented out in the generated config. The reasons this is a good thing are as follows: * If we decide that we should change the default for an option, we can do so, and only those admins that have deliberately chosen to override that option will be stuck on the old setting. * It moves us towards a point where we can get rid of the super-surprising feature of synapse where the default settings for the config come from the generated yaml. * It makes setting up a test config for unit testing an order of magnitude easier (see forthcoming PR). * It makes the generated config more consistent, and hopefully easier for users to understand.	2019-03-19 10:06:40 +00:00
Amber Brown	282c97327f	Migrate the user directory initial population to a background task (#4864 )	2019-03-19 04:50:24 +11:00
Andrew Morgan	a8ad39eec7	lint	2019-03-18 17:47:39 +00:00
Andrew Morgan	621e7f37f1	Better exception handling	2019-03-18 17:45:54 +00:00
Brendan Abolivier	651ad8bc96	Add ratelimiting on failed login attempts (#4865 )	2019-03-18 12:57:20 +00:00
Brendan Abolivier	899e523d6d	Add ratelimiting on login (#4821 ) Add two ratelimiters on login (per-IP address and per-userID).	2019-03-15 17:46:16 +00:00
Richard van der Hoff	3b7ceb2c69	Merge pull request #4855 from matrix-org/rav/refactor_transaction_queue Split TransactionQueue up	2019-03-15 12:32:11 +00:00
Richard van der Hoff	2dee441bdb	Merge pull request #4852 from matrix-org/rav/move_rr_sending_to_worker Move client receipt processing to federation sender worker.	2019-03-15 12:30:30 +00:00
Erik Johnston	b0fa3f6ff3	Merge pull request #4853 from matrix-org/erikj/worker_docker_ci Allow passing --daemonize to workers	2019-03-15 10:35:38 +00:00
Erik Johnston	271cb1998b	Revert "Make federation endpoints more tolerant of trailing slashes for some endpoints (#4793 )" This reverts commit `290552fd83`.	2019-03-14 14:30:54 +00:00
Erik Johnston	9073cfc8bd	Merge pull request #4846 from matrix-org/hawkowl/userdir-search Improve searching in the userdir	2019-03-14 13:54:15 +00:00
Erik Johnston	9ad448c1e5	Correctly handle all command line options	2019-03-14 13:32:14 +00:00
Andrew Morgan	ecea5af491	Correct var name	2019-03-13 21:21:03 +00:00
Andrew Morgan	b2df0e8e2c	receiving a 400 caused an exception. handle it	2019-03-13 21:08:10 +00:00
Andrew Morgan	9a2e22fd41	is this what purgatory feels like	2019-03-13 20:29:38 +00:00
Andrew Morgan	45524f2f5e	i should have given up x2	2019-03-13 20:17:39 +00:00
Andrew Morgan	8d16ffaf7a	i should have given up	2019-03-13 20:03:10 +00:00
Richard van der Hoff	02e23b36bc	Rename and move the classes	2019-03-13 20:02:56 +00:00
Andrew Morgan	26f8e2d099	there comes a time when you should give up. but you dont	2019-03-13 19:49:08 +00:00
Andrew Morgan	5ca857ad84	as above	2019-03-13 19:35:23 +00:00
Andrew Morgan	7c0295f13c	no kwargs today	2019-03-13 19:27:10 +00:00
Andrew Morgan	66cdb840a6	Or perhaps I was the one who was drunk	2019-03-13 19:18:25 +00:00
Andrew Morgan	bec313818c	go home python, you're drunk	2019-03-13 19:10:56 +00:00
Andrew Morgan	c991e7aec7	Syntax checker is bork	2019-03-13 19:08:08 +00:00
Andrew Morgan	c2d848b80d	Destructure again	2019-03-13 19:04:43 +00:00
Andrew Morgan	ee8ba397e8	Are you happy now	2019-03-13 18:48:31 +00:00
Andrew Morgan	9dd0e34679	Syntax test	2019-03-13 18:45:17 +00:00
Andrew Morgan	220607a618	Remove testing code	2019-03-13 18:43:40 +00:00
Andrew Morgan	660b77f362	Add missing docstring detail	2019-03-13 18:38:16 +00:00
Andrew Morgan	5526b054aa	Fix syntax issues	2019-03-13 18:35:21 +00:00
Andrew Morgan	09626bfd39	Switch to wrapper function around _send_request	2019-03-13 18:26:06 +00:00
Richard van der Hoff	5d89a526f1	Factor per-destination stuff out of TransactionQueue This is easier than having to have a million fields keyed on destination.	2019-03-13 17:48:29 +00:00
Erik Johnston	72bfaf746d	Allow passing --daemonize to workers	2019-03-13 17:33:54 +00:00
Richard van der Hoff	fdcad8eabd	Move client receipt processing to federation sender worker. This is mostly a prerequisite for #4730, but also fits with the general theme of "move everything off the master that we possibly can".	2019-03-13 17:21:19 +00:00
Richard van der Hoff	eed7271b3b	declare a ReadReceipt class I'm going to use this in queues and things, so it'll be useful to give it more of a structure.	2019-03-13 17:20:55 +00:00
Andrew Morgan	7998ca3a66	Document using a certificate with a full chain (#4849 )	2019-03-13 15:26:29 +00:00
Andrew Morgan	7d053cfe10	Retry on 400:M_UNRECOGNIZED	2019-03-13 12:10:33 +00:00
Andrew Morgan	7e75d9644b	Fix paranthesis indent	2019-03-13 11:15:23 +00:00
Andrew Morgan	cf301e37d8	Add workaround note	2019-03-13 11:14:43 +00:00
Andrew Morgan	d42c81d724	Transfer local user's push rules on room upgrade (#4838 ) Transfer push rules (notifications) on room upgrade	2019-03-12 14:42:53 +00:00
Amber Brown	c0332d095f	fixup	2019-03-13 01:30:54 +11:00
Amber Brown	797b6a63fc	fixup	2019-03-13 01:17:51 +11:00
Andrew Morgan	0ea8582f8b	Cleaner way of implementing trailing slashes	2019-03-12 14:11:11 +00:00
Amber Brown	81d9d1bee6	fixup	2019-03-12 22:28:48 +11:00
Amber Brown	6f5890b2fa	fixup	2019-03-12 22:27:56 +11:00
Amber Brown	10480c4348	fixup	2019-03-12 21:47:14 +11:00
Amber Brown	8b618041ef	fixup	2019-03-12 18:06:28 +11:00
Amber Brown	c980c7e31f	use the old method	2019-03-12 17:51:14 +11:00
Aaron Raimist	8ea1b41a0e	Clarify what registration_shared_secret allows for (#2885 ) (#4844 ) * Clarify what registration_shared_secret allows for (#2885) Signed-off-by: Aaron Raimist <aaron@raim.ist> * Add changelog Signed-off-by: Aaron Raimist <aaron@raim.ist>	2019-03-11 18:21:52 +00:00
Andrew Morgan	f18dca26da	Merge branch 'develop' into anoa/trailing_slashes_client	2019-03-11 17:44:29 +00:00
Andrew Morgan	290552fd83	Make federation endpoints more tolerant of trailing slashes for some endpoints (#4793 ) Server side of a solution towards #3622.	2019-03-11 17:44:03 +00:00
Andrew Morgan	a8a028dfce	Merge branch 'develop' into anoa/trailing_slashes_client	2019-03-11 17:07:31 +00:00
Erik Johnston	78c563b77c	Correctly log expected errors when fetching server keys	2019-03-11 14:11:10 +00:00
Amber Brown	78a6b950b3	fix	2019-03-12 00:50:28 +11:00
Amber Brown	1b77bd69fb	pep8	2019-03-12 00:39:12 +11:00
Amber Brown	5ba8ceab4c	fixes	2019-03-12 00:35:31 +11:00
Amber Brown	8da22e2b53	master startup	2019-03-11 21:13:35 +11:00
Amber Brown	30a8deeb68	Merge remote-tracking branch 'origin/develop' into hawkowl/userdir-search	2019-03-11 21:12:30 +11:00
Amber Brown	4abc988c6a	initial	2019-03-11 21:11:36 +11:00
Matthew Hodgson	2326e00bc4	fix incorrect encoding of filenames with spaces in (#2090 ) fixes https://github.com/vector-im/riot-web/issues/3155	2019-03-11 09:53:45 +00:00
Andrew Morgan	a5dd335cd8	lint	2019-03-08 18:25:59 +00:00
Andrew Morgan	64ff11019e	Retry certain federation requests on 404	2019-03-08 18:22:47 +00:00
Andrew Morgan	525dd02bbe	Remove trailing slashes from outbound federation requests	2019-03-08 16:55:52 +00:00
Erik Johnston	fe6c12e6cd	Add comment to schema	2019-03-08 16:38:23 +00:00
Erik Johnston	5536ddba75	Make `prev_state` field optional The `prev_state` field on events is not specced and so synapse shouldn't explode if an event is missing the field. Fixes #4787	2019-03-08 15:05:32 +00:00
Erik Johnston	8c4896668f	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/soft_fail_impl	2019-03-08 11:44:20 +00:00
Erik Johnston	0ff8163eae	Factor out soft fail checks	2019-03-08 11:26:33 +00:00
Erik Johnston	469b66c102	Merge pull request #4829 from matrix-org/erikj/device_list_seen_updates When re-syncing device lists reset the state	2019-03-08 09:54:54 +00:00
Erik Johnston	d42b41544a	When re-syncing device lists reset the state We keep track of what stream IDs we've seen so that we know what updates we've handled or missed. If we re-sync we don't know if the updates we've seen are included in the re-sync (there may be a race), so we should reset the seen updates.	2019-03-07 16:04:24 +00:00
Erik Johnston	c633fc02d7	Add some debug logging for device list handling	2019-03-07 15:58:03 +00:00
Amber Brown	f6135d06cf	Rewrite userdir to be faster (#4537 )	2019-03-07 01:22:53 -08:00
Travis Ralston	06cbf79fe3	Merge pull request #4779 from matrix-org/travis/icons Use static locations for Riot icons	2019-03-06 15:11:41 -07:00
Erik Johnston	1f85c2c0ce	Merge pull request #4818 from matrix-org/erikj/prefill_client_ips Prefill client IPs cache on workers	2019-03-06 19:17:39 +00:00
Erik Johnston	0a6e716600	Merge pull request #4815 from matrix-org/erikj/docstrings Add docstrings from matrix-org-hotfixes	2019-03-06 18:59:28 +00:00
Erik Johnston	c665b637de	Merge pull request #4816 from matrix-org/erikj/4422_debug Port #4422 debug logging from hotfixes	2019-03-06 18:59:12 +00:00
Erik Johnston	face0c5b3c	Prefill client IPs cache on workers	2019-03-06 17:39:32 +00:00
Erik Johnston	b879870b2d	Send message after room has been shutdown Currently the explanation message is sent to the abuse room before any users are forced joined, which means it tends to get lost in the backlog of joins. So instead we send the message after we've forced joined everyone.	2019-03-06 17:35:11 +00:00
Erik Johnston	8b7790e68f	Port #4422 debug logging from hotfixes	2019-03-06 17:29:15 +00:00
Erik Johnston	6d13bdec91	Add docstrings from matrix-org-hotfixes	2019-03-06 17:24:26 +00:00
Erik Johnston	a9de04be72	Implement soft fail	2019-03-06 16:22:16 +00:00
Andrew Morgan	7b8a157b79	Merge pull request #4792 from matrix-org/anoa/replication_tokens Support batch updates in the worker sender	2019-03-06 15:48:29 +00:00
Brendan Abolivier	6f3cde8b25	Make registration ratelimiter separate from the main events one	2019-03-06 11:02:42 +00:00
Brendan Abolivier	f4195f4118	Revert "Split ratelimiters in two (one for events, one for registration)" This reverts commit `d7dbad3526`.	2019-03-06 10:55:22 +00:00
Brendan Abolivier	d7dbad3526	Split ratelimiters in two (one for events, one for registration)	2019-03-05 18:41:27 +00:00
Brendan Abolivier	067ce795c0	Move settings from registration to ratelimiting in config file	2019-03-05 18:03:14 +00:00
Erik Johnston	16c8b4ecbd	Merge pull request #4772 from jbweston/jbweston/server-version-api Add 'server_version' endpoint to admin API	2019-03-05 16:31:00 +00:00
Brendan Abolivier	a4c3a361b7	Add rate-limiting on registration (#4735 ) * Rate-limiting for registration * Add unit test for registration rate limiting * Add config parameters for rate limiting on auth endpoints * Doc * Fix doc of rate limiting function Co-Authored-By: babolivier <contact@brendanabolivier.com> * Incorporate review * Fix config parsing * Fix linting errors * Set default config for auth rate limiting * Fix tests * Add changelog * Advance reactor instead of mocked clock * Move parameters to registration specific config and give them more sensible default values * Remove unused config options * Don't mock the rate limiter un MAU tests * Rename _register_with_store into register_with_store * Make CI happy * Remove unused import * Update sample config * Fix ratelimiting test for py2 * Add non-guest test	2019-03-05 14:25:33 +00:00
Andrew Morgan	b9f6163092	Simplify token replication logic	2019-03-05 13:58:30 +00:00
Andrew Morgan	ae90531036	Merge pull request #4794 from matrix-org/anoa/erroneous_dollahs Remove unnecessary dollar signs	2019-03-05 09:46:59 +00:00
Erik Johnston	b050a10871	Merge pull request #4699 from matrix-org/erikj/stop_fed_not_in_room Stop backpaginating when events not visible	2019-03-05 09:32:33 +00:00
Erik Johnston	9e8bca5667	Merge pull request #4799 from matrix-org/rav/clean_up_replication_code Clean ups in replication notifier	2019-03-05 09:19:48 +00:00
Erik Johnston	aa06d26ae0	clarify comments	2019-03-05 09:16:35 +00:00
Erik Johnston	c3c542bb4a	Merge pull request #4796 from matrix-org/erikj/factor_out_e2e_keys Allow /keys/{changes,query} API to run on worker	2019-03-05 09:06:25 +00:00
Richard van der Hoff	c7325776a7	Remove redundant PreserveLoggingContext Both (!) things that register as replication listeners do the right thing wrt logcontexts, so this is redundant.	2019-03-04 18:31:18 +00:00
Erik Johnston	bfa7d46a10	Allow /keys/{changes,query} API to run on worker	2019-03-04 18:30:01 +00:00
Erik Johnston	157e5a8f27	Split DeviceHandler into master and worker	2019-03-04 18:29:26 +00:00
Richard van der Hoff	daa10e3e66	Remove unused `wait_for_replication` method I guess this was used once? It's not now, anyway.	2019-03-04 18:27:32 +00:00
Richard van der Hoff	2db49ea476	Add some debug about processing read receipts. I'm hoping to establish which rooms are having lots of RRs sent for them, and how old the events are when they are sent.	2019-03-04 18:19:40 +00:00
Richard van der Hoff	b29693a30b	Clean up read-receipt handling. Remove a call to run_as_background_process: there is no need to run this as a background process, because build_and_send_edu does not block. We may as well inline the whole of _push_remotes.	2019-03-04 18:16:43 +00:00
Erik Johnston	a84b8d56c2	Fixup slave stores	2019-03-04 18:04:57 +00:00
Richard van der Hoff	8e28bc5eee	Include a default configuration file in the 'docs' directory. (#4791 )	2019-03-04 17:14:58 +00:00
Erik Johnston	0d2d046709	Fix missing null guard	2019-03-04 16:04:04 +00:00
Andrew Morgan	336de1d45b	Remove unnecessary dollar signs A dollar sign is already appended to the end of each PATH, so there's no need to add one in the PATH declaration as well.	2019-03-04 15:25:12 +00:00
Andrew Morgan	fe7bd23a85	Clean up logic and add comments	2019-03-04 15:08:15 +00:00
Erik Johnston	d1523aed6b	Only check history visibility when filtering When filtering events to send to server we check more than just history visibility. However when deciding whether to backfill or not we only care about the history visibility.	2019-03-04 14:43:42 +00:00
Andrew Morgan	9f7cdf3da1	Clearer branching, fix missing list clear	2019-03-04 14:36:52 +00:00
Andrew Morgan	5f0c449dd5	Prevent replication wedging	2019-03-04 14:03:18 +00:00
Richard van der Hoff	856c83f5f8	Avoid rebuilding Edu objects in worker mode (#4770 ) In worker mode, on the federation sender, when we receive an edu for sending over the replication socket, it is parsed into an Edu object. There is no point extracting the contents of it so that we can then immediately build another Edu.	2019-03-04 12:57:44 +00:00
Erik Johnston	8b63fe4c26	s/get_forward_events/get_successor_events/	2019-03-04 11:56:03 +00:00
Erik Johnston	fbc047f2a5	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/stop_fed_not_in_room	2019-03-04 11:54:58 +00:00
Travis Ralston	4dfbae18fe	Use static locations for Riot icons See https://github.com/vector-im/riot-web/issues/9009	2019-03-01 15:06:21 -07:00
Richard van der Hoff	3064952939	Fix incorrect log about not persisting duplicate state event. (#4776 ) We were logging this when it was not true.	2019-03-01 16:47:12 +00:00
Richard van der Hoff	1beebe916f	Merge branch 'master' into develop	2019-03-01 10:58:39 +00:00
Richard van der Hoff	9ac72d9543	0.99.2	2019-03-01 10:55:44 +00:00
Joseph Weston	1e8388b311	Add 'server_version' endpoint to admin API This is required because the 'Server' HTTP header is not always passed through proxies.	2019-03-01 09:56:58 +01:00
Andrew Morgan	ac61b45a75	Minor docstring fixes for MatrixFederationAgent (#4765 )	2019-02-28 16:24:01 +00:00
Amber Brown	b131cc77df	Make 'event_id' a required parameter in federated state requests (#4741 ) * make 'event_id' a required parameter in federated state requests As per the spec: https://matrix.org/docs/spec/server_server/r0.1.1.html#id40 Signed-off-by: Joseph Weston <joseph@weston.cloud> * add changelog entry for bugfix Signed-off-by: Joseph Weston <joseph@weston.cloud> * Update server.py	2019-02-27 14:35:47 -08:00
Richard van der Hoff	68f47d6744	Fix parsing of Content-Disposition headers (#4763 ) * Fix parsing of Content-Disposition headers TIL: filenames in content-dispostion headers can contain semicolons, and aren't %-encoded. * fix python2 incompatibility * Fix docstrings	2019-02-27 14:29:10 -08:00
Amber Brown	f2a753ea38	Move from TravisCI to BuildKite (#4752 )	2019-02-27 13:03:14 -08:00
Erik Johnston	76550c58d2	Merge pull request #4759 from matrix-org/erikj/3pid_client_reader Move /account/3pid to client_reader	2019-02-27 16:11:21 +00:00
Erik Johnston	8267034a63	Merge pull request #4758 from matrix-org/erikj/use_presence_replication When presence is disabled don't send over replication	2019-02-27 15:46:26 +00:00
Erik Johnston	54f9ce11a7	Move /account/3pid to client_reader	2019-02-27 14:26:08 +00:00
Erik Johnston	1e315017d3	When presence is enabled don't send over replication	2019-02-27 13:53:46 +00:00
Erik Johnston	4cff9376f7	Move server key queries to federation reader	2019-02-27 13:43:53 +00:00
Erik Johnston	b183fef9ac	Update comments	2019-02-27 13:06:10 +00:00
Erik Johnston	7590e9fa28	Merge pull request #4749 from matrix-org/erikj/replication_connection_backoff Fix tightloop over connecting to replication server	2019-02-27 11:00:59 +00:00
Richard van der Hoff	44a4d65586	0.99.2rc1	2019-02-27 10:48:34 +00:00
Erik Johnston	6bb1c028f1	Limit cache invalidation replication line length (#4748 )	2019-02-27 10:28:37 +00:00
Erik Johnston	6870fc496f	Move connecting logic into ClientReplicationStreamProtocol	2019-02-27 10:23:51 +00:00
Richard van der Hoff	f191be822b	Add database version to phonehome stats. (#4753 )	2019-02-27 10:21:49 +00:00
Erik Johnston	25814921f1	Increase the max delay between retry attempts Otherwise if you have many workers they can easily take out master with their connection attempts	2019-02-26 15:12:33 +00:00
Erik Johnston	313987187e	Fix tightloop over connecting to replication server If the client failed to process incoming commands during the initial set up of the replication connection it would immediately disconnect and reconnect, resulting in a tightloop. This can happen, for example, when subscribing to a stream that has a row that is too long in the backlog. The fix here is to not consider the connection successfully set up until the client has succesfully subscribed and caught up with the streams. This ensures that the retry logic timers aren't reset until then, meaning that if an error does happen during start up the client will continue backing off before retrying again.	2019-02-26 15:05:41 +00:00
Andrew Morgan	802884d4ee	Merge branch 'develop' of github.com:matrix-org/synapse into anoa/public_rooms_federate_develop	2019-02-26 14:23:40 +00:00
Andrew Morgan	6fcb25202f	Put function def back to the way it was	2019-02-26 14:17:38 +00:00
Andrew Morgan	7a4632af9c	Prevent showing non-fed rooms in fed /publicRooms	2019-02-26 13:37:24 +00:00
Andrew Morgan	c74624a633	Revert "Prevent showing non-fed rooms in fed /publicRooms"	2019-02-26 13:20:38 +00:00
Andrew Morgan	a1a6473293	Merge pull request #4736 from matrix-org/anoa/public_rooms_federate Config option to prevent showing non-fed rooms in fed /publicRooms	2019-02-26 13:07:15 +00:00
Andrew Morgan	c4414768af	Cleaner chunk logic	2019-02-26 12:22:34 +00:00
Andrew Morgan	a712aa3a9c	Correct indent	2019-02-26 12:13:55 +00:00
Andrew Morgan	16565e67db	Correct docstring types and chunk logic	2019-02-26 12:12:48 +00:00
Andrew Morgan	40c2271680	Clean up room chunk logic	2019-02-26 12:04:34 +00:00
Andrew Morgan	6728bf3940	Make not showing non-federated rooms the default	2019-02-26 11:52:52 +00:00
Andrew Morgan	6946c20111	Result may be None	2019-02-26 11:27:19 +00:00
Andrew Morgan	71669a0fba	Address rich comments	2019-02-26 11:25:00 +00:00
Erik Johnston	899a119c2b	Don't log stack trace when client has gone away during media download (#4738 ) * Don't log stack trace when client has gone away during media download * Newsfile * Fixup newsfile	2019-02-25 11:17:22 -08:00
Richard van der Hoff	641c409e4e	Fix ACME config for python 2. (#4717 ) Fixes #4675.	2019-02-25 11:16:33 -08:00
Matthew Hodgson	70ea2f4e1d	switch from google.com to recaptcha.net for reCAPTCHA (#4731 ) * add trivial clarification about jemalloc * switch from google.com to recaptcha.net because https://developers.google.com/recaptcha/docs/faq#can-i-use-recaptcha-globally	2019-02-25 11:15:36 -08:00
Andrew Morgan	96c408273e	Fix group's call to generate_room_entry	2019-02-25 18:00:17 +00:00
Andrew Morgan	1330aa4a8f	elif not else if	2019-02-25 17:28:19 +00:00
Erik Johnston	65f3fbfbf7	Merge pull request #4737 from matrix-org/erikj/failure_log_tb Log tracebacks correctly	2019-02-25 17:26:30 +00:00
Erik Johnston	c7b333c545	Log tracebacks correctly	2019-02-25 16:56:41 +00:00
Erik Johnston	69efe6fb16	Merge pull request #4718 from matrix-org/erikj/fix_backfill_state_shred Fix backfill storing incorrect state for events	2019-02-25 16:48:52 +00:00
Erik Johnston	9c598dddcb	Fix typo	2019-02-25 16:32:02 +00:00
Erik Johnston	b1a90da82e	Merge pull request #4721 from matrix-org/erikj/msc_1866 MSC 1866 - Use M_UNSUPPORTED_ROOM_VERSION for invite API	2019-02-25 15:53:10 +00:00
Erik Johnston	16c7afa94c	Merge pull request #4722 from matrix-org/erikj/correctly_handle_keyring_exceptions Handle errors when fetching remote server keys	2019-02-25 15:53:02 +00:00
Andrew Morgan	8aaf7ffc44	syntax derp	2019-02-25 15:27:17 +00:00
Andrew Morgan	84c0a20dfe	Simplify call to generate_room_entry	2019-02-25 15:23:27 +00:00
Erik Johnston	4b9e5076c4	Merge branch 'develop' of github.com:matrix-org/synapse into anoa/public_rooms_federate	2019-02-25 15:08:18 +00:00
Andrew Morgan	07493607a8	Docs and arg name clarification	2019-02-25 15:04:46 +00:00
Andrew Morgan	bd398b874e	Don't restrict non-fed rooms over client APIs	2019-02-25 15:04:46 +00:00
Andrew Morgan	e4b078a600	Config option to prevent showing non-fed rooms in fed /publicRooms	2019-02-25 15:04:46 +00:00
Erik Johnston	d730c2c22b	More comments	2019-02-25 14:45:02 +00:00
Erik Johnston	890cb048fd	Assert rather than clobber the values	2019-02-25 14:42:11 +00:00
Erik Johnston	65d1003d01	raise_from already raises	2019-02-25 14:34:03 +00:00
Erik Johnston	9342cc6ab1	Add comments and paranoia	2019-02-25 10:02:12 +00:00
Erik Johnston	47a7e3928d	Correctly proxy exception in frontend_proxy worker	2019-02-23 15:17:57 +00:00
Erik Johnston	41285ffe5b	Handle errors when fetching remote server keys	2019-02-23 15:09:39 +00:00
Erik Johnston	59e0112209	MSC 1866 - Use M_UNSUPPORTED_ROOM_VERSION for invite API	2019-02-23 14:50:43 +00:00
Erik Johnston	d14e94bae4	Merge pull request #4716 from matrix-org/erikj/pusher_logging Fix up pusher logging a bit	2019-02-22 15:29:16 +00:00
Erik Johnston	b82c9cf462	Add missing return	2019-02-22 15:27:40 +00:00
Erik Johnston	f2891d2487	Correctly handle PusherConfigException	2019-02-22 15:18:19 +00:00
Erik Johnston	a164134a53	Drop logging level of creating a pusher	2019-02-22 14:48:06 +00:00
Erik Johnston	1d9df51ff1	Correctly handle null data in HttpPusher	2019-02-22 14:47:48 +00:00
Erik Johnston	80467bbac3	Fix state cache invalidation on workers	2019-02-22 14:38:14 +00:00
Erik Johnston	7b288826b7	Fix backfill storing incorrect state for events	2019-02-22 11:33:51 +00:00
Richard van der Hoff	e07384c4e1	Add prometheus metrics for number of badge update pushes. (#4709 ) We're counting the number of push notifications, but not the number of badges; I'd like to see if they are significant.	2019-02-22 10:57:15 +00:00
Richard van der Hoff	0abb094f1a	bail out early in on_new_receipts if no pushers (#4706 )	2019-02-21 17:51:21 +00:00
Richard van der Hoff	6d65659b62	Run push_receipts_to_remotes as background job (#4707 ) I suspect the CPU usage metrics for this are going to /dev/null at the moment.	2019-02-21 17:50:30 +00:00
Erik Johnston	b9d6756b14	Merge pull request #4263 from rkfg/develop Prevent crash on pagination.	2019-02-21 17:42:15 +00:00
Erik Johnston	71b625d808	Stop backpaginating when events not visible	2019-02-20 18:14:12 +00:00
Erik Johnston	82fca11fc1	Merge pull request #4694 from matrix-org/erikj/fix_sentry_config_format Fixup generated metrics config	2019-02-20 14:13:38 +00:00
Richard van der Hoff	82ca6d1f9f	Add metrics for number of outgoing EDUs, by type (#4695 )	2019-02-20 14:13:14 +00:00
Erik Johnston	3d672fec51	Fixup generated metrics config	2019-02-20 13:39:37 +00:00
Amber Brown	a06614bd2a	UPSERT many functionality (#4644 )	2019-02-20 23:03:30 +11:00
Richard van der Hoff	b2200a8690	Logging improvements for the pusher (#4691 )	2019-02-20 11:36:50 +00:00
Richard van der Hoff	c88bc53903	Fix TaskStopped exceptions when outbound requests time out (#4690 )	2019-02-20 11:35:52 +00:00
Richard van der Hoff	86920ac266	Merge pull request #4678 from matrix-org/rav/tls_install_instructions Attempt to clarify installation/config instructions	2019-02-20 11:10:33 +00:00
Erik Johnston	dbdc565dfd	Fix registration on workers (#4682 ) * Move RegistrationHandler init to HomeServer * Move post registration actions to RegistrationHandler * Add post regisration replication endpoint * Newsfile	2019-02-20 18:47:31 +11:00
Richard van der Hoff	5f9bdf90fe	Attempt to make default config more consistent The general idea here is that config examples should just have a hash and no extraneous whitespace, both to make it easier for people who don't understand yaml, and to make the examples stand out from the comments.	2019-02-19 13:54:29 +00:00
Erik Johnston	c003450057	Merge pull request #4671 from matrix-org/erikj/state_cache_invalidation Batch cache invalidation over replication	2019-02-19 13:14:30 +00:00
Erik Johnston	62175a20e5	Docs	2019-02-19 11:38:40 +00:00
Erik Johnston	1bb35e3a83	Use itertools	2019-02-19 11:34:40 +00:00
Erik Johnston	bc8fa1509d	Documentation	2019-02-19 11:24:59 +00:00
Brendan Abolivier	a288bdf0b1	Merge pull request #4652 from matrix-org/babolivier/acme-delegated Support .well-known delegation when issuing certificates through ACME	2019-02-19 11:15:38 +00:00
Brendan Abolivier	5a707a2f9a	Improve config documentation	2019-02-19 10:59:26 +00:00
Brendan Abolivier	a8626901cd	Fetch ACME domain into an instance member	2019-02-19 10:54:33 +00:00
Erik Johnston	32590b7139	Merge pull request #4670 from matrix-org/erikj/register_login_split Split /login into client_reader	2019-02-19 10:46:22 +00:00
Richard van der Hoff	7c70b8f8a6	Try and make TLS federation client code faster (#4674 ) * fix to use makeContext so that we don't need to rebuild the certificateoptions each time	2019-02-19 10:19:16 +00:00
Andrew Morgan	968a30a75c	Merge pull request #4642 from matrix-org/anoa/bans_room_upgrade Transfer bans on room upgrade	2019-02-19 10:16:42 +00:00
Amber Brown	561eebe170	fix to use makeContext so that we don't need to rebuild the certificateoptions each time	2019-02-19 16:18:05 +11:00
Andrew Morgan	34ac75ce2c	lint	2019-02-18 18:23:37 +00:00
Erik Johnston	a9b5ea6fc1	Batch cache invalidation over replication Currently whenever the current state changes in a room invalidate a lot of caches, which cause a lot of traffic over replication. Instead, lets batch up all those invalidations and send a single poke down the replication streams. Hopefully this will reduce load on the master process by substantially reducing traffic.	2019-02-18 17:53:31 +00:00
Andrew Morgan	f8b9ca53ce	Move member event processing and changelog fix	2019-02-18 17:28:52 +00:00
Erik Johnston	d154f5a055	Merge pull request #4632 from matrix-org/erikj/basic_sentry Add basic optional sentry.io integration	2019-02-18 17:22:45 +00:00
Erik Johnston	4cc4400b4d	Split /login into client_reader	2019-02-18 17:19:01 +00:00
Erik Johnston	fc2c245a1f	Merge pull request #4666 from matrix-org/erikj/register_login_split Split out registration to worker	2019-02-18 17:18:06 +00:00
Erik Johnston	459d3d5046	Merge pull request #4668 from matrix-org/erikj/catch_exceptions Correctly handle HttpResponseException when handling device updates	2019-02-18 16:55:43 +00:00
Erik Johnston	d328a93b51	Fixup error handling and message	2019-02-18 16:53:56 +00:00
Erik Johnston	af691e415c	Move register_device into handler	2019-02-18 16:49:38 +00:00
Erik Johnston	028267acd2	Merge pull request #4669 from matrix-org/erikj/log_exception Cleanup top level request exception logging	2019-02-18 16:41:41 +00:00
Erik Johnston	d08bac4136	Merge pull request #4651 from matrix-org/matthew/well-known-cors set CORS on .well-known URI to unbreak modular	2019-02-18 16:38:11 +00:00
Erik Johnston	c30f73c86a	Merge pull request #4667 from matrix-org/erikj/fix_revoke_guest_access_workers Fix kicking guest users in worker mode	2019-02-18 15:55:48 +00:00
Erik Johnston	092b541401	Fixup	2019-02-18 15:52:26 +00:00
Brendan Abolivier	45bb55c6de	Use a configuration parameter to give the domain to generate a certificate for	2019-02-18 15:46:23 +00:00
Erik Johnston	8b9ae6d3a6	Update docs	2019-02-18 15:26:13 +00:00
Erik Johnston	94960cef03	pep8	2019-02-18 15:24:13 +00:00
Erik Johnston	fe725f7e45	Cleanup top level request exception logging Firstly, we always logged that the request was being handled via `JsonResource._async_render`, so we change that to use the servlet name we add to the request. Secondly, we pass the exception information to the logger rather than formatting it manually. This makes it consistent with other exception logging, allwoing logging hooks and formatters to access the exception information.	2019-02-18 15:11:04 +00:00
Matthew Hodgson	04dad5ac16	fix missig import	2019-02-18 15:59:23 +01:00
Erik Johnston	e07cc31cb8	Correctly handle HttpResponseException	2019-02-18 14:55:09 +00:00
Brendan Abolivier	68a53f825f	Merge branch 'develop' into babolivier/acme-delegated	2019-02-18 14:52:23 +00:00
Erik Johnston	32e54b472a	Fix kicking guest users in worker mode When guest_access changes from allowed to forbidden all local guest users should be kicked from the room. This did not happen when revocation was received from federation on a worker. Presumably broken in #4141	2019-02-18 14:16:07 +00:00
Andrew Morgan	915421065b	Membership events are done later	2019-02-18 14:02:09 +00:00
Erik Johnston	d1b060b492	Merge pull request #4643 from matrix-org/erikj/catch_exceptions Correctly handle RequestSendFailed exceptions	2019-02-18 13:53:24 +00:00
Andrew Morgan	9caab0c364	Transfer bans on room upgrade	2019-02-18 13:52:51 +00:00
Erik Johnston	dc5efc92a8	Fixup	2019-02-18 13:52:49 +00:00
Erik Johnston	eb2b8523ae	Split out registration to worker This allows registration to be handled by a worker, though the actual write to the database still happens on master. Note: due to the in-memory session map all registration requests must be handled by the same worker.	2019-02-18 12:12:57 +00:00
Andrew Morgan	6d02a13d81	Typo in info log Co-Authored-By: babolivier <contact@brendanabolivier.com>	2019-02-18 11:36:34 +00:00
Juuso "Linda" Lapinlampi	68d2869c8d	config: Remove a repeated word from a logger warning The warning for missing macaroon_secret_key was "missing missing".	2019-02-15 22:24:53 -07:00
Erik Johnston	bd4505f765	Merge pull request #4647 from matrix-org/erikj/add_room_publishing_rules Add configurable room list publishing rules	2019-02-15 22:11:01 +00:00
Brendan Abolivier	f86b695cbd	Various cosmetics to make TravisCI happy	2019-02-15 12:29:34 +00:00
Brendan Abolivier	af8a2f679b	Remove unused import	2019-02-15 12:27:43 +00:00
Brendan Abolivier	1895d14e12	Support .well-known delegation when issuing certificates through ACME	2019-02-15 12:05:08 +00:00
Erik Johnston	b99c532c1c	Move defaults up into code	2019-02-15 10:53:39 +00:00
Erik Johnston	02c729d6b0	Hoist up checks to reduce overall work	2019-02-15 10:20:02 +00:00
Erik Johnston	02c46acc6a	Fixup comments	2019-02-15 10:17:13 +00:00
Matthew Hodgson	bfcefbb230	tabs	2019-02-14 19:53:12 +00:00
Matthew Hodgson	6f47bc3fb2	set CORS on .well-known URI to unbreak modular otherwise a riot/web running on foo.riot.im can't query the .well-known on foo.modular.im...	2019-02-14 19:51:41 +00:00
Erik Johnston	8e32f26cb8	Clarify comments	2019-02-14 18:21:24 +00:00
Erik Johnston	cb12a37708	Clarify and fix behaviour when there are multiple aliases	2019-02-14 18:16:32 +00:00
Erik Johnston	f61b2068e6	Only fetch aliases when publishing rooms	2019-02-14 18:08:09 +00:00
Erik Johnston	f666fe36d7	Fixup comments	2019-02-14 18:07:24 +00:00
Richard van der Hoff	bf4fd14806	Merge branch 'master' into develop	2019-02-14 17:34:09 +00:00
Richard van der Hoff	f595d6ac57	0.99.1.1	2019-02-14 17:20:02 +00:00
Richard van der Hoff	f311018823	Fix errors in acme provisioning (#4648 ) * Better logging for errors on startup * Fix "TypeError: '>' not supported" when starting without an existing certificate * Fix a bug where an existing certificate would be reprovisoned every day	2019-02-14 17:10:36 +00:00
Erik Johnston	eaf4d11af9	Add configurable room list publishing rules This allows specifying who and what is allowed to be published onto the public room list	2019-02-14 16:02:23 +00:00
Richard van der Hoff	b02465b9db	Merge branch 'master' into develop	2019-02-14 14:42:03 +00:00
Richard van der Hoff	06cd757ae7	0.99.1	2019-02-14 14:24:24 +00:00
Erik Johnston	7fc1196a36	Correctly handle RequestSendFailed exceptions This mainly reduces the number of exceptions we log.	2019-02-14 14:01:04 +00:00
Erik Johnston	6cb415b63f	Fixup comments and add warning	2019-02-13 16:15:11 +00:00
Richard van der Hoff	c6e75c9f2d	Merge pull request #4450 from 14mRh4X0r/fix-dependency-message Fix error message for optional dependencies	2019-02-13 16:12:49 +00:00
Richard van der Hoff	3bc238629e	0.99.1rc2	2019-02-13 14:46:18 +00:00
Richard van der Hoff	c1dfd6a18a	Merge remote-tracking branch 'origin/release-v0.99.1' into develop	2019-02-13 14:27:45 +00:00
Erik Johnston	309f3bb322	Update synapse/app/_base.py Co-Authored-By: richvdh <1389908+richvdh@users.noreply.github.com>	2019-02-13 13:24:27 +00:00
Amber Brown	bb4fd8f927	Run `black` on user directory code (#4635 )	2019-02-13 23:05:32 +11:00
Richard van der Hoff	767686af48	Use `listen_tcp` for the replication listener Fixes the "can't listen on 0.0.0.0" error. Also makes it more consistent with what we do elsewhere.	2019-02-13 11:59:04 +00:00
Richard van der Hoff	2a5a15aff8	Improve logging around listening services I wanted to bring listen_tcp into line with listen_ssl in terms of returning a list of ports, and wanted to check that was a safe thing to do - hence the logging in `refresh_certificate`. Also, pull the 'Synapse now listening' message up to homeserver.py, because it was being duplicated everywhere else.	2019-02-13 11:58:54 +00:00
Richard van der Hoff	e3a0300431	Special-case the default bind_addresses for metrics listener turns out it doesn't really support ipv6, so let's hack around that by only listening on ipv4 by default.	2019-02-13 11:48:56 +00:00
Erik Johnston	93f7d2df3e	Comments	2019-02-12 16:03:40 +00:00
Erik Johnston	6a8f902edb	Raise an appropriate error message if sentry_sdk missing	2019-02-12 16:01:41 +00:00
Erik Johnston	ef2228c890	Basic sentry integration	2019-02-12 13:55:58 +00:00
Erik Johnston	d2fa7b7e99	Update changelog and version	2019-02-12 13:22:25 +00:00
Erik Johnston	ba3f27b69a	Merge pull request #4608 from matrix-org/anoa/acls_room_upgrade Transfer Server ACLs on room upgrade	2019-02-12 13:20:06 +00:00
Erik Johnston	cf82338930	Merge pull request #4627 from matrix-org/erikj/user_ips_analyze Analyze user_ips before running deduplication	2019-02-12 13:05:09 +00:00
Erik Johnston	3df8fcca25	Merge pull request #4626 from matrix-org/erikj/fixup_user_ips_dedupe Reduce user_ips bloat during dedupe background update	2019-02-12 13:02:58 +00:00
Erik Johnston	495ea92350	Fix pep8	2019-02-12 12:40:42 +00:00
Erik Johnston	483ba85c7a	Analyze user_ips before running deduplication Due to the table locks taken out by the naive upsert, the table statistics may be out of date. During deduplication it is important that the correct index is used as otherwise a full table scan may be incorrectly used, which can end up thrashing the database badly.	2019-02-12 11:55:27 +00:00
Erik Johnston	362d80b770	Reduce user_ips bloat during dedupe background update The background update to remove duplicate rows naively deleted and reinserted the duplicates. For large tables with a large number of duplicates this causes a lot of bloat (with postgres), as the inserted rows are appended to the table, since deleted rows will not be overwritten until a VACUUM has happened. This should hopefully also help ensure that the query in the last batch uses the correct index, as inserting a large number of new rows without analyzing will upset the query planner.	2019-02-12 11:39:34 +00:00
Erik Johnston	3c03c37883	Merge pull request #4625 from matrix-org/rav/fix_generate_config_warnings fix self-signed cert notice from generate-config	2019-02-12 11:24:45 +00:00
Richard van der Hoff	a4ce91396b	Disable TLS by default (#4614 )	2019-02-12 10:52:08 +00:00
Richard van der Hoff	32b781bfe2	Fix error when loading cert if tls is disabled (#4618 ) If TLS is disabled, it should not be an error if no cert is given. Fixes #4554.	2019-02-12 10:51:31 +00:00
Richard van der Hoff	dfc846a316	fix self-signed cert notice from generate-config fixes #4620	2019-02-12 10:37:59 +00:00
Richard van der Hoff	0ca2908653	fix tests	2019-02-11 22:01:27 +00:00
Richard van der Hoff	4fddf8fc77	Infer no_tls from presence of TLS listeners Rather than have to specify `no_tls` explicitly, infer whether we need to load the TLS keys etc from whether we have any TLS-enabled listeners.	2019-02-11 21:39:14 +00:00
Richard van der Hoff	15272f837c	Merge branch 'rav/no_create_server_contexts_if_no_tls' into rav/tls_cert/work	2019-02-11 21:34:19 +00:00
Richard van der Hoff	9645728619	Don't create server contexts when TLS is disabled we aren't going to use them anyway.	2019-02-11 21:32:01 +00:00
Richard van der Hoff	be794c7cf7	Merge branch 'rav/tls_config_logging_fixes' into rav/tls_cert/work	2019-02-11 21:16:00 +00:00
Richard van der Hoff	2129dd1a02	Fail cleanly if listener config lacks a 'port' ... otherwise we would fail with a mysterious KeyError or something later.	2019-02-11 21:15:01 +00:00
Richard van der Hoff	086f6f27d4	Logging improvements around TLS certs Log which file we're reading keys and certs from, and refactor the code a bit in preparation for other work	2019-02-11 21:02:06 +00:00
Richard van der Hoff	5d27730a73	Move ClientTLSOptionsFactory init out of refresh_certificates (#4611 ) It's nothing to do with refreshing the certificates. No idea why it was here.	2019-02-11 18:03:30 +00:00
Erik Johnston	719e073f00	Merge pull request #4580 from matrix-org/uhoreg/e2e_backup_add_updating add updating of backup versions	2019-02-11 13:45:49 +00:00
Richard van der Hoff	24b7f3916d	Clean up default listener configuration (#4586 ) Rearrange the comments to try to clarify them, and expand on what some of it means. Use a sensible default 'bind_addresses' setting. For the insecure port, only bind to localhost, and enable x_forwarded, since apparently it's for use behind a load-balancer.	2019-02-11 12:50:30 +00:00
Andrew Morgan	a126f86eec	Transfer Server ACLs on room upgrade	2019-02-11 11:30:37 +00:00
Amber Brown	6e2a5aa050	ACME Reprovisioning (#4522 )	2019-02-11 10:36:26 +00:00
Amber Brown	4ffd10f46d	Be tolerant of blank TLS fingerprints config (#4589 )	2019-02-11 10:04:27 +00:00
Erik Johnston	b201149c7e	Merge pull request #4420 from matrix-org/jaywink/openid-listener New listener resource for the federation API "openid/userinfo" endpoint	2019-02-11 09:44:00 +00:00
Valentin Anger	2dc2b6e9f1	Allow "unavailable" presence status for /sync (#4592 ) * Allow "unavailable" presence status for /sync Closes #3772, closes #3779 Signed-off-by: Valentin Anger <valentin.an.1999@gmail.com> * Add changelog for PR 4592	2019-02-08 21:09:56 +00:00
Richard van der Hoff	56710c7df5	Fix 'no unique or exclusion constraint' error (#4591 ) Add more tables to the list of tables which need a background update to complete before we can upsert into them, which fixes a race against the background updates.	2019-02-08 18:30:46 +00:00
Amber Brown	9cd33d2f4b	Deduplicate some code in synapse.app (#4567 )	2019-02-08 17:25:57 +00:00
Hubert Chathi	afae8442b5	make sure version is in body and wrap in linearizer queue also add tests	2019-02-08 01:32:45 -05:00
Hubert Chathi	d9e424bf64	re-try to make isort happy	2019-02-06 22:18:41 -05:00
Hubert Chathi	9ff620a518	fix import to make isort happy	2019-02-06 21:32:52 -05:00
Hubert Chathi	8248637173	add new endpoint to update backup versions	2019-02-06 17:57:10 -05:00
Hubert Chathi	664c81e8b7	return proper error codes for some 404s	2019-02-06 17:47:22 -05:00
Richard van der Hoff	2475434080	Merge branch 'master' into develop	2019-02-05 18:44:49 +00:00
Richard van der Hoff	3bd9daf4b8	v0.99.0	2019-02-05 18:33:02 +00:00
Andrew Morgan	627ecd358e	Filter user directory state query to a subset of state events (#4462 ) * Filter user directory state query to a subset of state events * Add changelog	2019-02-05 12:16:28 +00:00
Richard van der Hoff	bf1e4d96ad	Fix default ACME config for py2 (#4564 ) Fixes #4559	2019-02-05 11:37:33 +00:00
Richard van der Hoff	d7e27a1f08	fix typo in config comments (#4557 )	2019-02-05 11:32:45 +00:00
Richard van der Hoff	30fd2f89db	0.99.0rc4	2019-02-01 15:52:28 +00:00
Richard van der Hoff	ef43a03fc5	Merge pull request #4546 from matrix-org/rav/silence_critical_error_from_federation Fix noisy "twisted.internet.task.TaskStopped" errors in logs	2019-02-01 14:37:22 +00:00
Richard van der Hoff	f8db967d5a	Merge remote-tracking branch 'origin/release-v0.99.0' into develop	2019-02-01 13:20:15 +00:00
Richard van der Hoff	fa794980ec	Merge pull request #4544 from matrix-org/rav/skip_invalid_well_known Treat an invalid .well-known the same as an absent one	2019-02-01 13:18:36 +00:00
Richard van der Hoff	e9779a6f8f	Fix b'ab' noise in logs	2019-02-01 12:34:31 +00:00
Richard van der Hoff	9763a73af0	Merge branch 'release-v0.99.0' into develop	2019-02-01 12:30:22 +00:00
Richard van der Hoff	f0ba34f581	Fix noisy "twisted.internet.task.TaskStopped" errors in logs Fixes #4003	2019-02-01 12:22:57 +00:00
Richard van der Hoff	8a21b03fba	Treat an invalid .well-known the same as an absent one ... basically, carry on and fall back to SRV etc.	2019-02-01 11:37:31 +00:00
Richard van der Hoff	3c8a41140e	Cache failures to parse .well-known Also add a Measure block around the .well-known fetch	2019-02-01 00:37:52 +00:00
Richard van der Hoff	24d59c7568	better logging for federation connections	2019-01-31 23:18:20 +00:00
Richard van der Hoff	d428b46346	Update federation routing logic to check .well-known before SRV	2019-01-31 23:14:18 +00:00
Andrew Morgan	c45fd0dda0	Merge pull request #4530 from matrix-org/anoa/room_upgrade_federatable Copy over non-federatable trait on room upgrade	2019-01-31 20:28:37 +00:00
Richard van der Hoff	625385d684	Merge branch 'release-v0.99.0' into develop	2019-01-31 18:43:20 +00:00
Richard van der Hoff	85129d7068	v0.99.0rc3	2019-01-31 18:35:38 +00:00
Andrew Morgan	d239f67c25	Raise an exception instead of returning None	2019-01-31 18:34:15 +00:00
Richard van der Hoff	07dfe148de	Add some debug for membership syncing issues (#4538 ) I can't figure out what's going on with #4422 and #4436; perhaps this will help.	2019-01-31 18:30:40 +00:00
Andrew Morgan	3ed3cb4339	New function for getting room's create event	2019-01-31 18:21:39 +00:00
Andrew Morgan	bbb97a35fd	Merge branch 'develop' of github.com:matrix-org/synapse into anoa/room_upgrade_federatable	2019-01-31 18:11:00 +00:00
Richard van der Hoff	e707e7b38d	Fix infinite loop when an event is redacted in a v3 room (#4535 )	2019-01-31 15:34:17 +00:00
Andrew Morgan	563f6a832b	Reject large transactions on federation (#4513 ) * Reject large transactions on federation * Add changelog * lint * Simplify large transaction handling	2019-01-31 11:44:04 +00:00
Andrew Morgan	fb50934b8f	lint	2019-01-31 11:34:45 +00:00
Richard van der Hoff	6fba9fd20c	Merge remote-tracking branch 'origin/release-v0.99.0' into develop	2019-01-30 17:02:32 +00:00
Andrew Morgan	d621c5562e	Copy over non-federatable trait on room upgrade	2019-01-30 16:33:51 +00:00
Richard van der Hoff	b8b898666e	v0.99.0rc2	2019-01-30 16:31:07 +00:00
Matthew Hodgson	ad7ac8853c	by default include m.room.encryption on invites (#3902 ) * by default include m.room.encryption on invites * fix constant * changelog	2019-01-30 16:26:13 +00:00
Richard van der Hoff	c74b96755c	Merge remote-tracking branch 'origin/develop' into release-v0.99.0	2019-01-30 16:23:28 +00:00
Neil Johnson	81b7e7eed3	Update constants.py remove trailing ,	2019-01-30 16:11:36 +00:00
Erik Johnston	ed8c5e4cda	Fix remote invite rejections not comming down sync This was broken in PR #4405, commit `886e5ac`, where we changed remote rejections to be outliers. The fix is to explicitly add the leave event in when we know its an out of band invite. We can't always add the event as if the server is/was in the room there might be more events to send down the sync than just the leave.	2019-01-30 15:46:27 +00:00
Richard van der Hoff	a5d0c771a3	0.99.0rc1	2019-01-30 15:11:18 +00:00
Erik Johnston	6587b0b89b	Merge pull request #4472 from matrix-org/neilj/room_capabilities Server capabilities support	2019-01-30 14:26:56 +00:00
Erik Johnston	a4f52a33fe	Fix replication for room v3 (#4523 ) * Fix replication for room v3 We were not correctly quoting the path fragments over http replication, which meant that it exploded when the event IDs had a slash in them * Newsfile	2019-01-30 14:19:52 +00:00
Richard van der Hoff	7615a8ced1	ACME config cleanups (#4525 ) * Handle listening for ACME requests on IPv6 addresses the weird url-but-not-actually-a-url-string doesn't handle IPv6 addresses without extra quoting. Building a string which you are about to parse again seems like a weird choice. Let's just use listenTCP, which is consistent with what we do elsewhere. * Clean up the default ACME config make it look a bit more consistent with everything else, and tweak the defaults to listen on port 80. * newsfile	2019-01-30 14:17:55 +00:00
Erik Johnston	e87d7a4b0f	Raise ConfigError instead	2019-01-30 12:48:09 +00:00
Erik Johnston	270f212a2a	_listener_http should return a list	2019-01-30 12:14:50 +00:00
Richard van der Hoff	a79034aedf	Merge pull request #4521 from matrix-org/rav/fed_routing/cleanups Tiny .well-known fixes	2019-01-30 11:47:24 +00:00
Richard van der Hoff	c7b24ac3d0	Follow redirects on .well-known (#4520 )	2019-01-30 11:43:33 +00:00
Amber Brown	f6813919e8	SIGHUP for TLS cert reloading (#4495 )	2019-01-30 11:00:02 +00:00
Richard van der Hoff	09a1a6b55e	fix exception text	2019-01-30 10:58:52 +00:00
Richard van der Hoff	928c50b59a	Also jitter the invalid cache period	2019-01-30 10:58:52 +00:00
Neil Johnson	b37e8c9572	Merge branch 'neilj/room_capabilities' of github.com:matrix-org/synapse into neilj/room_capabilities	2019-01-30 10:56:47 +00:00
Neil Johnson	f834d98402	isort	2019-01-30 10:55:42 +00:00
Richard van der Hoff	bc5f6e1797	Add a caching layer to .well-known responses (#4516 )	2019-01-30 10:55:25 +00:00
Amber Brown	3f189c902e	Fix flake8 (#4519 )	2019-01-30 10:53:17 +00:00
Neil Johnson	ee4df7fd7a	Merge branch 'develop' into neilj/room_capabilities	2019-01-30 10:28:08 +00:00
Neil Johnson	c5a0f82cca	define room dispositions for use in exposing room capabilities	2019-01-30 10:24:24 +00:00
Neil Johnson	9c850d9d5e	formatting and use constants where available	2019-01-30 10:23:26 +00:00
Neil Johnson	2f46804055	Populate default room version from Constants	2019-01-30 09:39:10 +00:00
Neil Johnson	c7837dce24	reflect that rooms v3 is a stable room version	2019-01-30 09:33:30 +00:00

... 9 10 11 12 13 ...

10408 Commits