2044 Commits

Author	SHA1	Message	Date
Richard van der Hoff	93003aa172	add some tests	2019-06-01 11:14:37 +01:00
Erik Johnston	58cce39f3a	Merge pull request #5276 from matrix-org/babolivier/account_validity_job_delta ... Allow configuring a range for the account validity startup job	2019-05-31 12:11:56 +01:00
Brendan Abolivier	4d794dae21	Move delta from +10% to -10%	2019-05-31 11:09:39 +01:00
Brendan Abolivier	847b9dcd1c	Make max_delta equal to period * 10%	2019-05-31 09:54:46 +01:00
Erik Johnston	e9e5d3392d	Merge pull request #5278 from matrix-org/erikj/cleanup_bad_extremities ... Add DB bg update to cleanup extremities.	2019-05-30 14:28:26 +01:00
Amber Brown	46c8f7a517	Implement the SHHS complexity API (#5216)	2019-05-30 01:47:16 +10:00
Erik Johnston	d7add713a8	Add test	2019-05-29 15:59:06 +01:00
Richard van der Hoff	540f40f0cd	Merge pull request #5251 from matrix-org/rav/server_keys/01-check_sig ... Ensure that server_keys fetched via a notary server are correctly signed.	2019-05-28 21:32:17 +01:00
Brendan Abolivier	7e1c7cc274	Typo	2019-05-28 17:13:26 +01:00
Brendan Abolivier	52839886d6	Allow configuring a range for the account validity startup job ... When enabling the account validity feature, Synapse will look at startup for registered account without an expiration date, and will set one equals to 'now + validity_period' for them. On large servers, it can mean that a large number of users will have the same expiration date, which means that they will all be sent a renewal email at the same time, which isn't ideal. In order to mitigate this, this PR allows server admins to define a 'max_delta' so that the expiration date is a random value in the [now + validity_period ; now + validity_period + max_delta] range. This allows renewal emails to be progressively sent over a configured period instead of being sent all in one big batch.	2019-05-28 16:52:45 +01:00
Richard van der Hoff	fa1b293da2	Simplification to Keyring.wait_for_previous_lookups. (#5250) ... The list of server names was redundant, since it was equivalent to the keys on the server_to_deferred map. This reduces the number of large lists being passed around, and has the benefit of deduplicating the entries in `wait_on`.	2019-05-24 22:17:18 +01:00
Erik Johnston	8c41c04ee4	Merge pull request #5244 from matrix-org/rav/server_keys/00-factor-out-fetchers ... Factor out KeyFetchers from KeyRing	2019-05-23 16:28:51 +01:00
Richard van der Hoff	753b1270da	Require sig from origin server on perspectives responses	2019-05-23 15:01:09 +01:00
Andrew Morgan	6368150a74	Add config option for setting homeserver's default room version (#5223) ... Replaces DEFAULT_ROOM_VERSION constant with a method that first checks the config, then returns a hardcoded value if the option is not present. That hardcoded value is now located in the server.py config file.	2019-05-23 15:00:20 +01:00
Richard van der Hoff	895b79ac2e	Factor out KeyFetchers from KeyRing ... Rather than have three methods which have to have the same interface, factor out a separate interface which is provided by three implementations. I find it easier to grok the code this way.	2019-05-23 13:46:47 +01:00
Richard van der Hoff	b75537beaf	Store key validity time in the storage layer ... This is a first step to checking that the key is valid at the required moment. The idea here is that, rather than passing VerifyKey objects in and out of the storage layer, we instead pass FetchKeyResult objects, which simply wrap the VerifyKey and add a valid_until_ts field.	2019-05-23 11:52:22 +01:00
Richard van der Hoff	2e052110ee	Rewrite store_server_verify_key to store several keys at once (#5234) ... Storing server keys hammered the database a bit. This replaces the implementation which stored a single key, with one which can do many updates at once.	2019-05-23 11:45:39 +01:00
Amber Brown	4a30e4acb4	Room Statistics (#4338)	2019-05-21 11:36:50 -05:00
Erik Johnston	9526aa96a6	Merge pull request #5212 from matrix-org/erikj/deny_multiple_reactions ... Block attempts to annotate the same event twice	2019-05-21 17:08:14 +01:00
Brendan Abolivier	6a5a70edf0	Merge pull request #5204 from matrix-org/babolivier/account_validity_expiration_date ... Add startup background job for account validity	2019-05-21 14:55:15 +01:00
Erik Johnston	c7ec06e8a6	Block attempts to annotate the same event twice	2019-05-20 17:39:05 +01:00
Erik Johnston	1dff859d6a	Rename relation types to match MSC	2019-05-20 14:31:19 +01:00
Erik Johnston	57ba3451b6	Merge pull request #5209 from matrix-org/erikj/reactions_base ... Land basic reaction and edit support.	2019-05-20 14:06:40 +01:00
Erik Johnston	2ac9c965dd	Fixup comments	2019-05-20 12:32:26 +01:00
Erik Johnston	935af0da38	Correctly update aggregation counts after redaction	2019-05-20 12:09:27 +01:00
ReidAnderson	3787133c9e	Limit UserIds to a length that fits in a state key (#5198)	2019-05-20 11:20:08 +01:00
Brendan Abolivier	ad5b4074e1	Add startup background job for account validity ... If account validity is enabled in the server's configuration, this job will run at startup as a background job and will stick an expiration date to any registered account missing one.	2019-05-17 19:37:31 +01:00
Erik Johnston	d4ca533d70	Make tests use different user for each reaction it sends ... As users aren't allowed to react with the same emoji more than once.	2019-05-17 18:03:05 +01:00
Brendan Abolivier	fdeac1e984	Merge pull request #5196 from matrix-org/babolivier/per_room_profiles ... Add an option to disable per-room profiles	2019-05-17 12:10:49 +01:00
David Baker	07cff7b121	Merge pull request #5174 from matrix-org/dbkr/add_dummy_flow_to_recaptcha_only ... Re-order registration stages to do msisdn & email auth last	2019-05-16 17:27:39 +01:00
Erik Johnston	d46aab3fa8	Add basic editing support	2019-05-16 16:54:45 +01:00
Brendan Abolivier	8f9ce1a8a2	Lint	2019-05-16 15:25:54 +01:00
Brendan Abolivier	cc8c139a39	Lint	2019-05-16 15:20:59 +01:00
Brendan Abolivier	efdc55db75	Forgot copyright	2019-05-16 15:10:24 +01:00
Brendan Abolivier	54a582ed44	Add test case	2019-05-16 15:09:16 +01:00
Erik Johnston	95f3fcda3c	Check that event is visible in new APIs	2019-05-16 14:19:06 +01:00
Erik Johnston	a0603523d2	Add aggregations API	2019-05-16 09:37:20 +01:00
Amber Brown	f1e5b41388	Make all the rate limiting options more consistent (#5181)	2019-05-15 12:06:04 -05:00
Erik Johnston	b50641e357	Add simple pagination API	2019-05-15 13:36:51 +01:00
Erik Johnston	efe3c7977a	Add simple send_relation API and track in DB	2019-05-15 13:36:51 +01:00
Erik Johnston	d94544051b	Merge pull request #5184 from matrix-org/erikj/expose_get_events_as_array ... Expose DataStore._get_events as get_events_as_list	2019-05-15 10:17:38 +01:00
Erik Johnston	495e859e58	Merge branch 'erikj/fix_config_ratelimiting' into erikj/test	2019-05-14 14:42:47 +01:00
Erik Johnston	dc4f6d1b01	Use correct config option for ratelimiting in tests	2019-05-14 14:37:40 +01:00
Erik Johnston	4fb44fb5b9	Expose DataStore._get_events as get_events_as_list ... This is in preparation for reaction work which requires it.	2019-05-14 13:37:44 +01:00
Amber Brown	df2ebd75d3	Migrate all tests to use the dict-based config format instead of hanging items off HomeserverConfig (#5171)	2019-05-13 15:01:14 -05:00
Andrew Morgan	5a4b328f52	Add ability to blacklist ip ranges for federation traffic (#5043)	2019-05-13 19:05:06 +01:00
David Baker	822072b1bb	Terms might not be the last stage	2019-05-13 16:10:26 +01:00
David Baker	9e99143c47	Merge remote-tracking branch 'origin/develop' into dbkr/add_dummy_flow_to_recaptcha_only	2019-05-13 15:37:03 +01:00
David Baker	8782bfb783	And now I realise why the test is failing...	2019-05-13 15:34:11 +01:00
Andrew Morgan	2f48c4e1ae	URL preview blacklisting fixes (#5155) ... Prevents a SynapseError being raised inside of a IResolutionReceiver and instead opts to just return 0 results. This thus means that we have to lump a failed lookup and a blacklisted lookup together with the same error message, but the substitute should be generic enough to cover both cases.	2019-05-10 10:32:44 -07:00
Amber Brown	b36c82576e	Run Black on the tests again (#5170)	2019-05-10 00:12:11 -05:00
Brendan Abolivier	d216a36b37	Fix bogus imports in tests (#5154)	2019-05-08 21:57:03 +01:00
Matthew Hodgson	c0e0740bef	add options to require an access_token to GET /profile and /publicRooms on CS API (#5083) ... This commit adds two config options: * `restrict_public_rooms_to_local_users` Requires auth to fetch the public rooms directory through the CS API and disables fetching it through the federation API. * `require_auth_for_profile_requests` When set to `true`, requires that requests to `/profile` over the CS API are authenticated, and only returns the user's profile if the requester shares a room with the profile's owner, as per MSC1301. MSC1301 also specifies a behaviour for federation (only returning the profile if the server asking for it shares a room with the profile's owner), but that's currently really non-trivial to do in a not too expensive way. Next step is writing down a MSC that allows a HS to specify which user sent the profile query. In this implementation, Synapse won't send a profile query over federation if it doesn't believe it already shares a room with the profile's owner, though. Groups have been intentionally omitted from this commit.	2019-05-08 18:26:56 +01:00
Brendan Abolivier	1473058b5e	Do checks on aliases for incoming m.room.aliases events (#5128) ... Follow-up to #5124 Also added a bunch of checks to make sure everything (both the stuff added on #5124 and this PR) works as intended.	2019-05-08 17:01:30 +01:00
Richard van der Hoff	59e2d2694d	Remove the requirement to authenticate for /admin/server_version. (#5122) ... This endpoint isn't much use for its intended purpose if you first need to get yourself an admin's auth token. I've restricted it to the `/_synapse/admin` path to make it a bit easier to lock down for those concerned about exposing this information. I don't imagine anyone is using it in anger currently.	2019-05-07 09:29:30 +01:00
Richard van der Hoff	12f9d51e82	Add admin api for sending server_notices (#5121)	2019-05-02 11:59:16 +01:00
Richard van der Hoff	40e576e29c	Move admin api impl to its own package ... It doesn't really belong under rest/client/v1 any more.	2019-05-01 15:44:30 +01:00
Andrew Morgan	6824ddd93d	Config option for verifying federation certificates (MSC 1711) (#4967)	2019-04-25 14:22:49 +01:00
Erik Johnston	ca90336a69	Merge branch 'develop' of github.com:matrix-org/synapse into babolivier/account_expiration	2019-04-17 19:44:40 +01:00
Brendan Abolivier	eaf41a943b	Add management endpoints for account validity	2019-04-17 19:34:45 +01:00
Brendan Abolivier	91934025b9	Merge pull request #5047 from matrix-org/babolivier/account_expiration ... Send out emails with links to extend an account's validity period	2019-04-17 14:57:39 +01:00
Brendan Abolivier	20f0617e87	Send out emails with links to extend an account's validity period	2019-04-17 14:42:20 +01:00
Andrew Morgan	caa76e6021	Remove periods from copyright headers (#5046)	2019-04-11 17:08:13 +01:00
Brendan Abolivier	bfc8fdf1fc	Merge pull request #5027 from matrix-org/babolivier/account_expiration ... Add time-based account expiration	2019-04-09 17:02:41 +01:00
Brendan Abolivier	747aa9f8ca	Add account expiration feature	2019-04-09 16:46:04 +01:00
Richard van der Hoff	18b69be00f	Rewrite Datastore.get_server_verify_keys ... Rewrite this so that it doesn't hammer the database.	2019-04-09 00:00:10 +01:00
Richard van der Hoff	0084309cd2	Rewrite test_keys as a HomeserverTestCase	2019-04-08 23:59:57 +01:00
Richard van der Hoff	7d2a0c848e	Fix from_server buglet in get_keys_from_perspectives ... make sure we store the name of the server the keys came from, rather than the origin server, after doing a fetch-from-perspectives.	2019-04-08 12:51:16 +01:00
Richard van der Hoff	7fc1e17f4c	Merge pull request #5001 from matrix-org/rav/keyring_cleanups ... Cleanups in the Keyring	2019-04-08 12:47:09 +01:00
Richard van der Hoff	b2d574f126	test for get_keys_from_perspectives	2019-04-04 19:12:54 +01:00
Richard van der Hoff	30805237fa	add a test for get_keys_from_server	2019-04-04 19:12:54 +01:00
Brendan Abolivier	8e85493b0c	Add config option to block users from looking up 3PIDs (#5010)	2019-04-04 17:25:47 +01:00
Erik Johnston	c6a233a936	Add unit test for deleting groups	2019-04-03 16:29:52 +01:00
Andrew Morgan	4a4d5c4fd6	Fix grammar and document get_current_users_in_room (#4998)	2019-04-03 14:32:20 +01:00
Richard van der Hoff	e4d473d855	Rewrite KeyringTestCase as a HomeServerTestCase (#4986) ... This is a bit fiddly due to the keyring doing weird things with logcontexts.	2019-04-03 14:11:27 +01:00
Neil Johnson	e8419554ff	Remove presence lists (#4989) ... Remove presence list support as per MSC 1819	2019-04-03 11:11:15 +01:00
Richard van der Hoff	297bf2547e	Fix sync bug when accepting invites (#4956) ... Hopefully this time we really will fix #4422. We need to make sure that the cache on `get_rooms_for_user_with_stream_ordering` is invalidated *before* the SyncHandler is notified for the new events, and we can now do so reliably via the `events` stream.	2019-04-02 12:42:39 +01:00
Neil Johnson	4c552ed78a	Neilj/fix threepid auth check (with tests) (#4474) ... test threepid checking	2019-04-01 17:42:18 +01:00
Richard van der Hoff	54a87a7b08	Collect room-version variations into one place (#4969) ... Collect all the things that make room-versions different to one another into one place, so that it's easier to define new room versions.	2019-04-01 10:24:38 +01:00
Erik Johnston	248014379e	Merge pull request #4942 from matrix-org/erikj/fix_presence ... Use event streams to calculate presence	2019-03-28 14:38:31 +00:00
Erik Johnston	40e56997bc	Review comments	2019-03-28 13:48:41 +00:00
Richard van der Hoff	a5798de067	Move replication.tcp.streams into a package	2019-03-27 21:13:14 +00:00
Erik Johnston	b7fa834c40	Add unit tests	2019-03-27 13:41:36 +00:00
Andrew Morgan	4a125be138	Make federation endpoints more tolerant of trailing slashes v2 (#4935) ... Redo of https://github.com/matrix-org/synapse/pull/4840	2019-03-26 11:35:29 +00:00
Richard van der Hoff	9bde730ef8	Fix bug where read-receipts lost their timestamps (#4927) ... Make sure that they are sent correctly over the replication stream. Fixes: #4898	2019-03-25 16:38:05 +00:00
Richard van der Hoff	a54a44734f	Use an explicit dbname for postgres connections in the tests. (#4928) ... I don't have a database with the same name as my user, so leaving the database name unset fails. While we're at it, clear out some unused stuff in the test setup.	2019-03-25 16:36:56 +00:00
Erik Johnston	5fee9d8067	Merge pull request #4869 from matrix-org/erikj/yaml_load ... Fix yaml warnings by using safe_load	2019-03-22 11:58:13 +00:00
Erik Johnston	3677548a82	Use yaml safe_load	2019-03-22 10:20:17 +00:00
Amber Brown	a68e00fca8	Some more porting to HomeserverTestCase and remove old RESTHelper (#4913)	2019-03-22 02:10:21 +11:00
Andrew Morgan	7bef97dfb7	Remove trailing slashes from outbound federation requests and retry on 400 (#4840) ... As per #3622, we remove trailing slashes from outbound federation requests. However, to ensure that we remain backwards compatible with previous versions of Synapse, if we receive a HTTP 400 with `M_UNRECOGNIZED`, then we are likely talking to an older version of Synapse in which case we retry with a trailing slash appended to the request path.	2019-03-21 15:07:28 +00:00
Erik Johnston	01e6b405be	Merge pull request #4908 from matrix-org/erikj/block_peek_on_blocked_rooms ... Deny peeking into rooms that have been blocked	2019-03-21 14:07:17 +00:00
Erik Johnston	d3f640f0ac	isort	2019-03-21 11:29:48 +00:00
Erik Johnston	3959858eaa	Merge pull request #4904 from matrix-org/erikj/fix_shutdown ... Fixup shutdown room API	2019-03-21 11:24:42 +00:00
Erik Johnston	cd80cbffea	Fix typo and add description	2019-03-21 11:24:04 +00:00
Erik Johnston	536a266520	Deny peeking into rooms that have been blocked	2019-03-21 11:20:13 +00:00
Erik Johnston	4a8a1ac962	Rejig testcase to make it more extensible	2019-03-21 11:02:11 +00:00
Erik Johnston	9c9e618b93	Remove debug	2019-03-21 10:58:56 +00:00
Erik Johnston	5c6f61f81c	Add tests	2019-03-21 10:51:21 +00:00
Erik Johnston	09f991a63d	Merge pull request #4896 from matrix-org/erikj/disable_room_directory ... Add option to disable search room lists	2019-03-21 10:16:54 +00:00
Richard van der Hoff	a6f2d3053d	Log requests which are simulated by the unit tests. (#4905) ... Rather than stubbing out the access_log, make it actually log the requests, which makes it a lot more obvious what is going on during tests.	2019-03-20 18:00:02 +00:00
Erik Johnston	263f2c9ce1	Merge pull request #4895 from matrix-org/erikj/disable_user_search ... Add option to disable searching in the user dir	2019-03-20 16:47:15 +00:00
Richard van der Hoff	a902d13180	Batch up outgoing read-receipts to reduce federation traffic. (#4890) ... Rate-limit outgoing read-receipts as per #4730.	2019-03-20 16:02:25 +00:00
Erik Johnston	3660d24ebe	Add test	2019-03-20 15:16:36 +00:00
Erik Johnston	cc09685830	Add test	2019-03-20 14:53:44 +00:00
Andrew Morgan	bb52a2e653	lint	2019-03-20 14:08:57 +00:00
Andrew Morgan	cd36a1283b	New test, fix issues	2019-03-20 14:00:39 +00:00
Andrew Morgan	c69df5d5d3	Fix comments. v0.99.2 -> v0.99.3	2019-03-20 11:27:18 +00:00
Andrew Morgan	94cb7939e4	Federation test fixed!	2019-03-20 10:50:44 +00:00
Richard van der Hoff	b5d48560c7	Fix RegistrationTestCase ... turns out this relies on there being a `user_consent_version` set.	2019-03-19 12:05:05 +00:00
Richard van der Hoff	13bc1e0746	Use a regular HomeServerConfig object for unit tests ... Rather than using a Mock for the homeserver config, use a genuine HomeServerConfig object. This makes for a more realistic test, and means that we don't have to keep remembering to add things to the mock config every time we add a new config setting.	2019-03-19 11:44:43 +00:00
Richard van der Hoff	053c50bcb3	Fix resource limits tests ... Make sure that we have a `server_notices_mxid` set, given that we are relying on it.	2019-03-19 11:44:43 +00:00
Richard van der Hoff	45bb54a6c6	Fix registration test ... * Set allow_guest_access = True, since we rely on it * config doesn't have a `hostname` attribute; it is `server_name`	2019-03-19 11:44:43 +00:00
Richard van der Hoff	8c1774e821	Fix email test ... The Mailer expects the config object to have `email_smtp_pass` and `email_riot_base_url` attributes (and it won't by default, because the default config impl doesn't set any of the attributes unless email_enable_notifs is set).	2019-03-19 11:44:43 +00:00
Richard van der Hoff	45c4e19c74	Merge remote-tracking branch 'origin/develop' into HEAD	2019-03-19 11:43:46 +00:00
Neil Johnson	88f0675967	fix test_auto_create_auto_join_where_no_consent (#4886)	2019-03-19 11:38:59 +00:00
Richard van der Hoff	0dbfae03f9	Enforce hs_disabled_message correctly ... Fixes a bug where hs_disabled_message was not enforced for 3pid-based requests if there was no server_notices_mxid configured.	2019-03-19 11:30:54 +00:00
Amber Brown	282c97327f	Migrate the user directory initial population to a background task (#4864)	2019-03-19 04:50:24 +11:00
Andrew Morgan	621e7f37f1	Better exception handling	2019-03-18 17:45:54 +00:00
Brendan Abolivier	651ad8bc96	Add ratelimiting on failed login attempts (#4865)	2019-03-18 12:57:20 +00:00
Brendan Abolivier	899e523d6d	Add ratelimiting on login (#4821) ... Add two ratelimiters on login (per-IP address and per-userID).	2019-03-15 17:46:16 +00:00
Erik Johnston	271cb1998b	Revert "Make federation endpoints more tolerant of trailing slashes for some endpoints (#4793)" ... This reverts commit 290552fd83.	2019-03-14 14:30:54 +00:00
Erik Johnston	9073cfc8bd	Merge pull request #4846 from matrix-org/hawkowl/userdir-search ... Improve searching in the userdir	2019-03-14 13:54:15 +00:00
Andrew Morgan	ecea5af491	Correct var name	2019-03-13 21:21:03 +00:00
Andrew Morgan	86c60bda15	i should have given up x3	2019-03-13 20:19:07 +00:00
Andrew Morgan	45524f2f5e	i should have given up x2	2019-03-13 20:17:39 +00:00
Andrew Morgan	09626bfd39	Switch to wrapper function around _send_request	2019-03-13 18:26:06 +00:00
Andrew Morgan	97653ef1f4	Correct argument name	2019-03-12 14:30:26 +00:00
Amber Brown	797b6a63fc	fixup	2019-03-13 01:17:51 +11:00
Andrew Morgan	0ea8582f8b	Cleaner way of implementing trailing slashes	2019-03-12 14:11:11 +00:00
Amber Brown	d306bd1b26	fixup	2019-03-12 22:38:01 +11:00
Amber Brown	10480c4348	fixup	2019-03-12 21:47:14 +11:00
Andrew Morgan	4868b12029	and again	2019-03-11 18:22:26 +00:00
Andrew Morgan	802cb5dcf0	Fix syntax error	2019-03-11 18:08:28 +00:00
Andrew Morgan	66f205e93d	We're calling different functions now	2019-03-11 18:01:58 +00:00
Andrew Morgan	290552fd83	Make federation endpoints more tolerant of trailing slashes for some endpoints (#4793) ... Server side of a solution towards #3622.	2019-03-11 17:44:03 +00:00
Amber Brown	5ba8ceab4c	fixes	2019-03-12 00:35:31 +11:00
Amber Brown	26eefca3b7	setup master	2019-03-11 21:16:10 +11:00
Amber Brown	30a8deeb68	Merge remote-tracking branch 'origin/develop' into hawkowl/userdir-search	2019-03-11 21:12:30 +11:00
Amber Brown	4abc988c6a	initial	2019-03-11 21:11:36 +11:00
Andrew Morgan	d6e0be92fe	Disable captcha registration by default in tests	2019-03-08 15:49:38 +00:00
Amber Brown	f6135d06cf	Rewrite userdir to be faster (#4537)	2019-03-07 01:22:53 -08:00
Erik Johnston	16c8b4ecbd	Merge pull request #4772 from jbweston/jbweston/server-version-api ... Add 'server_version' endpoint to admin API	2019-03-05 16:31:00 +00:00
Brendan Abolivier	a4c3a361b7	Add rate-limiting on registration (#4735) ... * Rate-limiting for registration * Add unit test for registration rate limiting * Add config parameters for rate limiting on auth endpoints * Doc * Fix doc of rate limiting function Co-Authored-By: babolivier <contact@brendanabolivier.com> * Incorporate review * Fix config parsing * Fix linting errors * Set default config for auth rate limiting * Fix tests * Add changelog * Advance reactor instead of mocked clock * Move parameters to registration specific config and give them more sensible default values * Remove unused config options * Don't mock the rate limiter un MAU tests * Rename _register_with_store into register_with_store * Make CI happy * Remove unused import * Update sample config * Fix ratelimiting test for py2 * Add non-guest test	2019-03-05 14:25:33 +00:00
Richard van der Hoff	2c3548d9d8	Update test_typing to use HomeserverTestCase. (#4771)	2019-03-04 10:05:39 +00:00
Joseph Weston	1e8388b311	Add 'server_version' endpoint to admin API ... This is required because the 'Server' HTTP header is not always passed through proxies.	2019-03-01 09:56:58 +01:00
Joseph Weston	b136ee10df	Import 'admin' module rather than 'register_servlets' directly ... We will later need also to import 'register_servlets' from the 'login' module, so we un-pollute the namespace now to keep the logical changes separate.	2019-03-01 09:46:25 +01:00
Richard van der Hoff	68f47d6744	Fix parsing of Content-Disposition headers (#4763) ... * Fix parsing of Content-Disposition headers TIL: filenames in content-dispostion headers can contain semicolons, and aren't %-encoded. * fix python2 incompatibility * Fix docstrings	2019-02-27 14:29:10 -08:00
Amber Brown	f2a753ea38	Move from TravisCI to BuildKite (#4752)	2019-02-27 13:03:14 -08:00
Erik Johnston	d4dc527a1a	Fix unit tests	2019-02-27 14:24:45 +00:00
Amber Brown	a06614bd2a	UPSERT many functionality (#4644)	2019-02-20 23:03:30 +11:00
Erik Johnston	dbdc565dfd	Fix registration on workers (#4682) ... * Move RegistrationHandler init to HomeServer * Move post registration actions to RegistrationHandler * Add post regisration replication endpoint * Newsfile	2019-02-20 18:47:31 +11:00
Erik Johnston	af691e415c	Move register_device into handler	2019-02-18 16:49:38 +00:00
Erik Johnston	41c3f21c3b	Fix unit tests	2019-02-18 13:43:16 +00:00
Erik Johnston	cb12a37708	Clarify and fix behaviour when there are multiple aliases	2019-02-14 18:16:32 +00:00
Erik Johnston	eaf4d11af9	Add configurable room list publishing rules ... This allows specifying who and what is allowed to be published onto the public room list	2019-02-14 16:02:23 +00:00
Richard van der Hoff	32b781bfe2	Fix error when loading cert if tls is disabled (#4618) ... If TLS is disabled, it should not be an error if no cert is given. Fixes #4554.	2019-02-12 10:51:31 +00:00
Richard van der Hoff	0ca2908653	fix tests	2019-02-11 22:01:27 +00:00
Richard van der Hoff	5d27730a73	Move ClientTLSOptionsFactory init out of refresh_certificates (#4611) ... It's nothing to do with refreshing the certificates. No idea why it was here.	2019-02-11 18:03:30 +00:00
Erik Johnston	719e073f00	Merge pull request #4580 from matrix-org/uhoreg/e2e_backup_add_updating ... add updating of backup versions	2019-02-11 13:45:49 +00:00
Erik Johnston	b201149c7e	Merge pull request #4420 from matrix-org/jaywink/openid-listener ... New listener resource for the federation API "openid/userinfo" endpoint	2019-02-11 09:44:00 +00:00
Hubert Chathi	afae8442b5	make sure version is in body and wrap in linearizer queue ... also add tests	2019-02-08 01:32:45 -05:00
Richard van der Hoff	8a21b03fba	Treat an invalid .well-known the same as an absent one ... ... basically, carry on and fall back to SRV etc.	2019-02-01 11:37:31 +00:00
Richard van der Hoff	d428b46346	Update federation routing logic to check .well-known before SRV	2019-01-31 23:14:18 +00:00
Erik Johnston	6587b0b89b	Merge pull request #4472 from matrix-org/neilj/room_capabilities ... Server capabilities support	2019-01-30 14:26:56 +00:00
Richard van der Hoff	c7b24ac3d0	Follow redirects on .well-known (#4520)	2019-01-30 11:43:33 +00:00
Richard van der Hoff	bc5f6e1797	Add a caching layer to .well-known responses (#4516)	2019-01-30 10:55:25 +00:00
Amber Brown	3f189c902e	Fix flake8 (#4519)	2019-01-30 10:53:17 +00:00
Neil Johnson	ee4df7fd7a	Merge branch 'develop' into neilj/room_capabilities	2019-01-30 10:28:08 +00:00
Neil Johnson	e4bef9d470	rework format of change password capability	2019-01-29 18:04:56 +00:00
Richard van der Hoff	cc2d650ef7	Relax requirement for a content-type on .well-known (#4511)	2019-01-29 16:49:17 +00:00
Neil Johnson	f03b3a7a3a	support change_password in capabilities end-point	2019-01-29 15:58:37 +00:00
Amber Brown	6bd4374636	Do not generate self-signed TLS certificates by default. (#4509)	2019-01-29 14:09:10 +00:00
Erik Johnston	b8d75ef53e	Merge pull request #4481 from matrix-org/erikj/event_builder ... Refactor event building into EventBuilder	2019-01-29 14:07:23 +00:00
Richard van der Hoff	99e36d5e24	Implement MSC1708 (.well-known lookups for server routing) (#4489)	2019-01-29 13:53:02 +00:00
Neil Johnson	4eeb2fb215	isort	2019-01-29 12:44:10 +00:00
Richard van der Hoff	5488cadaae	Enable configuring test log level via env var (#4506) ... I got fed up with always adding '@unittest.DEBUG' every time I needed to debug a test.	2019-01-29 12:07:00 +00:00
Neil Johnson	a124025dab	enforce auth for capabilities endpoint	2019-01-29 11:37:56 +00:00
Erik Johnston	aee39f7de8	Fix test to use valid event format	2019-01-29 11:19:50 +00:00
Erik Johnston	554ca58ea1	Make add_hashes_and_signatures operate on dicts	2019-01-29 11:12:38 +00:00
Richard van der Hoff	f2b553d656	Use SimpleResolverComplexifier in tests (#4497) ... two reasons for this. One, it saves a bunch of boilerplate. Two, it squashes unicode to IDNA-in-a-`str` (even on python 3) in a way that it turns out we rely on to give consistent behaviour between python 2 and 3.	2019-01-29 09:38:29 +00:00
Erik Johnston	f1a04462eb	Merge pull request #4482 from matrix-org/erikj/event_auth_room_version ... Pass through room version to event auth	2019-01-28 20:09:38 +00:00
Amber Brown	7072fe3084	Fix UPSERTs on SQLite 3.24+ (#4477)	2019-01-28 15:43:32 +00:00
Richard van der Hoff	0fd5b3b53e	Handle IP literals explicitly ... We don't want to be doing .well-known lookups on these guys.	2019-01-28 10:34:30 +00:00
Richard van der Hoff	d840019192	Fix idna and ipv6 literal handling in MatrixFederationAgent (#4487) ... Turns out that the library does a better job of parsing URIs than our reinvented wheel. Who knew. There are two things going on here. The first is that, unlike parse_server_name, URI.fromBytes will strip off square brackets from IPv6 literals, which means that it is valid input to ClientTLSOptionsFactory and HostnameEndpoint. The second is that we stay in `bytes` throughout (except for the argument to ClientTLSOptionsFactory), which avoids the weirdness of (sometimes) ending up with idna-encoded values being held in `unicode` variables. TBH it probably would have been ok but it made the tests fragile.	2019-01-28 09:56:59 +00:00
Erik Johnston	ae2a957dba	Pass through room version to event auth	2019-01-25 18:31:41 +00:00
Erik Johnston	b6dce9b9fd	Merge pull request #4470 from matrix-org/erikj/require_format_version ... Require event format version to parse or create events	2019-01-25 15:59:36 +00:00
Richard van der Hoff	8520bc3109	Fix Host header sent by MatrixFederationAgent (#4468) ... Move the Host header logic down here so that (a) it is used if we reuse the agent elsewhere, and (b) we can mess about with it with .well-known.	2019-01-25 12:38:16 +00:00
Neil Johnson	95f871fc0d	Support room version capabilities in CS API (MSC1804)	2019-01-25 11:16:29 +00:00
Erik Johnston	9770ed91c2	Fix tests	2019-01-25 10:32:26 +00:00
Erik Johnston	be6a7e47fa	Revert "Require event format version to parse or create events"	2019-01-25 10:23:51 +00:00
Erik Johnston	efb8ed1d45	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/require_format_version	2019-01-24 18:52:34 +00:00
Richard van der Hoff	afd69a0920	Look up the right SRV record	2019-01-24 13:31:43 +00:00
Richard van der Hoff	e1c8440e0c	lots more tests for MatrixFederationAgent	2019-01-24 13:28:07 +00:00
Neil Johnson	10b89d5c2e	Merge pull request #4435 from matrix-org/neilj/fix_threepid_auth_check ... Neilj/fix threepid auth check	2019-01-24 13:02:50 +00:00
Amber Brown	58f6c48183	Use native UPSERTs where possible (#4306)	2019-01-24 21:31:54 +11:00
Richard van der Hoff	97fd29c019	Don't send IP addresses as SNI (#4452) ... The problem here is that we have cut-and-pasted an impl from Twisted, and then failed to maintain it. It was fixed in Twisted in https://github.com/twisted/twisted/pull/1047/files; let's do the same here.	2019-01-24 09:34:44 +00:00
Erik Johnston	e79ba9eb34	Fix tests	2019-01-24 09:28:16 +00:00
Richard van der Hoff	6b574f3df7	fix python2 test failure	2019-01-23 11:25:36 +00:00
Jason Robinson	6f680241bd	Fix flake8 issues ... Signed-off-by: Jason Robinson <jasonr@matrix.org>	2019-01-23 10:53:48 +02:00
Jason Robinson	1838ef1ac3	Fix openid tests after rebase ... Signed-off-by: Jason Robinson <jasonr@matrix.org>	2019-01-23 10:38:13 +02:00
Jason Robinson	a47fac9af6	Fix sorting of imports in tests. Remove an unnecessary mock ... Signed-off-by: Jason Robinson <jasonr@matrix.org>	2019-01-23 10:32:41 +02:00
Jason Robinson	5336e49b39	Add tests for the openid lister for SynapseHomeServer ... Check all possible variants of openid and federation listener on/off possibilities. Signed-off-by: Jason Robinson <jasonr@matrix.org>	2019-01-23 10:32:41 +02:00
Jason Robinson	6d25599098	Add tests for the openid lister for FederationReaderServer ... Check all possible variants of openid and federation listener on/off possibilities. Signed-off-by: Jason Robinson <jasonr@matrix.org>	2019-01-23 10:32:41 +02:00
Jason Robinson	ab97b6e33c	Fix a test docstring in frontend proxy tests ... Signed-off-by: Jason Robinson <jasonr@matrix.org>	2019-01-23 10:32:41 +02:00
Richard van der Hoff	d02c4532c0	Add a test for MatrixFederationAgent	2019-01-22 20:35:12 +00:00
Richard van der Hoff	7021784d46	put resolve_service in an object ... this makes it easier to stub things out for tests.	2019-01-22 20:35:12 +00:00
Richard van der Hoff	53a327b4d5	Require that service_name be a byte string ... it is only ever a bytes now, so let's enforce that.	2019-01-22 20:35:12 +00:00
Richard van der Hoff	7871146667	Make MatrixFederationClient use MatrixFederationAgent ... ... instead of the matrix_federation_endpoint	2019-01-22 20:34:35 +00:00
Neil Johnson	d619b113ed	Fix None guard in config.server.is_threepid_reserved	2019-01-22 16:52:29 +00:00
Richard van der Hoff	33a55289cb	Refactor and bugfix for resove_service (#4427)	2019-01-22 10:59:27 +00:00
Amber Brown	23b0813599	Require ECDH key exchange & remove dh_params (#4429) ... * remove dh_params and set better cipher string	2019-01-22 21:58:50 +11:00
Richard van der Hoff	de6888e7ce	Remove redundant WrappedConnection (#4409) ... * Remove redundant WrappedConnection The matrix federation client uses an HTTP connection pool, which times out its idle HTTP connections, so there is no need for any of this business.	2019-01-18 23:07:38 +11:00
Richard van der Hoff	676cf2ee26	Fix incorrect logcontexts after a Deferred was cancelled (#4407)	2019-01-17 14:00:23 +00:00
Amber Brown	522dada206	Merge remote-tracking branch 'origin/master' into develop	2019-01-11 02:22:48 +11:00
Richard van der Hoff	e0910d0145	Merge branch rav/macaroon_key_fix_0.34 into rav/macaroon_key_fix_0.34.1 ... Fixes #4371	2019-01-10 14:12:50 +00:00
Richard van der Hoff	aa70d24125	Merge branch 'rav/macaroon_key_fix' into rav/macaroon_key_fix_0.34	2019-01-10 12:58:33 +00:00
Richard van der Hoff	566947ff34	Skip macaroon check for access tokens in the db	2019-01-10 12:57:21 +00:00
Amber Brown	7960c26fda	Fix adding new rows instead of updating them if one of the key values is a NULL in upserts. (#4369)	2019-01-09 22:26:25 +11:00
Erik Johnston	b970cb0e96	Refactor request sending to have better excpetions (#4358) ... * Correctly retry and back off if we get a HTTPerror response * Refactor request sending to have better excpetions MatrixFederationHttpClient blindly reraised exceptions to the caller without differentiating "expected" failures (e.g. connection timeouts etc) versus more severe problems (e.g. programming errors). This commit adds a RequestSendFailed exception that is raised when "expected" failures happen, allowing the TransactionQueue to log them as warnings while allowing us to log other exceptions as actual exceptions.	2019-01-08 11:04:28 +00:00
Amber Brown	d7843f47b6	Remove v1 only REST APIs now we don't ship matrix console (#4334)	2018-12-29 23:12:30 +11:00
Richard van der Hoff	f85676cc93	Return well_known in /login response (#4319) ... ... as per MSC1730.	2018-12-24 10:44:33 +01:00
Richard van der Hoff	3355c1a9ec	Merge pull request #4317 from de-vri-es/test-metric-prometheus-0.5 ... Fix test_metrics.py compatibility prometheus_client 0.5	2018-12-23 00:14:47 +01:00
Amber Brown	ea6abf6724	Fix IP URL previews on Python 3 (#4215)	2018-12-22 01:56:13 +11:00
Maarten de Vries	48b7ff7a35	Fix test_metrics.py compatibility prometheus_client 0.5 ... prometheus_client 0.5 has a named-tuple Sample type with more member than the old plain tuple had. This commit makes sure the unit test detects this and changes the way it reads the sample. Signed-off-by: Maarten de Vries <maarten@de-vri.es>	2018-12-21 01:53:57 +01:00
Neil Johnson	d2f7c4e6b1	create support user (#4141) ... Allow for the creation of a support user. A support user can access the server, join rooms, interact with other users, but does not appear in the user directory nor does it contribute to monthly active user limits.	2018-12-14 18:20:59 +00:00
Richard van der Hoff	30da50a5b8	Initialise user displayname from SAML2 data (#4272) ... When we register a new user from SAML2 data, initialise their displayname correctly.	2018-12-07 14:44:46 +01:00
Richard van der Hoff	c7401a697f	Implement SAML2 authentication (#4267) ... This implements both a SAML2 metadata endpoint (at `/_matrix/saml2/metadata.xml`), and a SAML2 response receiver (at `/_matrix/saml2/authn_response`). If the SAML2 response matches what's been configured, we complete the SSO login flow by redirecting to the client url (aka `RelayState` in SAML2 jargon) with a login token. What we don't yet have is anything to build a SAML2 request and redirect the user to the identity provider. That is left as an exercise for the reader.	2018-12-07 13:11:11 +01:00
Richard van der Hoff	c588b9b9e4	Factor SSO success handling out of CAS login (#4264) ... This is mostly factoring out the post-CAS-login code to somewhere we can reuse it for other SSO flows, but it also fixes the userid mapping while we're at it.	2018-12-07 13:10:07 +01:00
Richard van der Hoff	b0c24a66ec	Rip out half-implemented m.login.saml2 support (#4265) ... * Rip out half-implemented m.login.saml2 support This was implemented in an odd way that left most of the work to the client, in a way that I really didn't understand. It's going to be a pain to maintain, so let's start by ripping it out. * drop undocumented dependency on dateutil It turns out we were relying on dateutil being pulled in transitively by pysaml2. There's no need for that bloat.	2018-12-06 19:44:38 +11:00
Richard van der Hoff	e8d98466b0	Implement .well-known handling (#4262) ... Sometimes it's useful for synapse to generate its own .well-known file.	2018-12-05 14:38:58 +01:00
Richard van der Hoff	48972ce9d1	Patch defer.inlineCallbacks to check logcontexts in tests (#4205)	2018-12-04 11:30:32 +01:00
Neil Johnson	7039ece8fb	Neilj/fix autojoin (#4223) ... * Fix auto join failures for servers that require user consent * Fix auto join failures for servers that require user consent	2018-11-28 22:24:57 +11:00
Richard van der Hoff	a44c0a096f	Check logcontexts before and after each test (#4190) ... * Add better diagnostics to flakey keyring test * fix interpolation fail * Check logcontexts before and after each test * update changelog * update changelog	2018-11-27 13:47:18 +11:00
Richard van der Hoff	80527b568d	Fix more logcontext leaks in tests (#4209)	2018-11-27 13:01:04 +11:00
Richard van der Hoff	de8772a655	Do a GC after each test to fix logcontext leaks (#4227) ... * Some words about garbage collections and logcontexts * Do a GC after each test to fix logcontext leaks This feels like an awful hack, but... * changelog	2018-11-27 13:00:33 +11:00
Richard van der Hoff	416c671474	Merge pull request #4204 from matrix-org/rav/logcontext_leak_fixes ... Fix some logcontext leaks	2018-11-20 12:19:19 +01:00
Amber Brown	80cac86b2c	Fix fallback auth on Python 3 (#4197)	2018-11-19 12:27:33 -06:00
Richard van der Hoff	828f18bd8b	Fix logcontext leak in test_url_preview	2018-11-19 17:07:01 +00:00
Richard van der Hoff	a267c2e3ed	Fix logcontext leak in http pusher test	2018-11-19 17:07:01 +00:00
Richard van der Hoff	884a561447	Fix some tests which leaked logcontexts	2018-11-19 17:07:01 +00:00
Travis Ralston	3da9781c98	Fix the terms UI auth tests ... By setting the config value directly, we skip the block that adds the slash automatically for us.	2018-11-15 23:00:28 -07:00
Amber Brown	8b1affe7d5	Fix Content-Disposition in media repository (#4176)	2018-11-15 15:55:58 -06:00
Travis Ralston	835779f7fb	Add option to track MAU stats (but not limit people) (#3830)	2018-11-15 18:08:27 +00:00
Amber Brown	df758e155d	Use <meta> tags to discover the per-page encoding of html previews (#4183)	2018-11-15 11:05:08 -06:00
Richard van der Hoff	c1efcd7c6a	Add a test for the public T&Cs form	2018-11-14 10:46:27 +00:00
Erik Johnston	abaa93c158	Add test to assert set_e2e_device_keys correctly returns False on no-op	2018-11-08 14:06:44 +00:00
Amber Brown	b3708830b8	Fix URL preview bugs (type error when loading cache from db, content-type including quotes) (#4157)	2018-11-08 01:37:43 +11:00
Amber Brown	e62f7f17b3	Remove some boilerplate in tests (#4156)	2018-11-07 03:00:00 +11:00
Travis Ralston	0f5e51f726	Add config variables for enabling terms auth and the policy name (#4142) ... So people can still collect consent the old way if they want to.	2018-11-06 10:32:34 +00:00
Hubert Chathi	f1087106cf	handle empty backups according to latest spec proposal (#4123) ... fixes #4056	2018-11-05 17:59:29 -05:00
Amber Brown	efdcbbe46b	Tests for user consent resource (#4140)	2018-11-06 05:53:44 +11:00
Amber Brown	5a63589e80	Add some tests for the HTTP pusher (#4149)	2018-11-06 05:53:24 +11:00
Erik Johnston	bc80b3f454	Add helpers for getting prev and auth events (#4139) ... * Add helpers for getting prev and auth events This is in preparation for allowing the event format to change between room versions.	2018-11-06 00:35:15 +11:00
Erik Johnston	90d713b8c6	Merge pull request #4137 from matrix-org/erikj/clean_up_events ... Clean up event accesses and tests	2018-11-02 14:12:49 +00:00
Erik Johnston	b86d05a279	Clean up event accesses and tests ... This is in preparation to refactor FrozenEvent to support different event formats for different room versions	2018-11-02 13:44:14 +00:00
Amber Brown	cb7a6b2379	Fix typing being reset causing infinite syncs (#4127)	2018-11-03 00:19:23 +11:00
Erik Johnston	b199534518	Merge pull request #4135 from matrix-org/erikj/fix_state_res_none ... Fix None exception in state res v2	2018-11-02 10:45:57 +00:00
Erik Johnston	54aec35867	Fix None exception in state res v2	2018-11-02 10:29:19 +00:00
Travis Ralston	a8c9faa9a2	The tests also need a version parameter	2018-10-31 13:28:08 -06:00
Travis Ralston	d1e7b9c44c	Merge branch 'develop' into travis/login-terms	2018-10-31 13:15:14 -06:00
Amber Brown	3bade14ec0	Fix search 500ing (#4122)	2018-10-31 04:33:41 +11:00
Amber Brown	0dce9e1379	Write some tests for the email pusher (#4095)	2018-10-30 23:55:43 +11:00
Richard van der Hoff	bf33eed609	Merge pull request #4091 from matrix-org/rav/room_version_upgrades ... Room version upgrade support	2018-10-29 12:47:20 +00:00
Amber Brown	77d70a7646	Port register_new_matrix_user to Python 3 and add tests (#4085)	2018-10-26 22:05:22 +11:00
Richard van der Hoff	474810d9d5	fix broken test ... This test stubbed out some stuff in a very weird way. I have no idea why. It broke.	2018-10-25 23:15:03 +01:00
Erik Johnston	cb53ce9d64	Refactor state group lookup to reduce DB hits (#4011) ... Currently when fetching state groups from the data store we make two hits two the database: once for members and once for non-members (unless request is filtered to one or the other). This adds needless load to the datbase, so this PR refactors the lookup to make only a single database hit.	2018-10-25 17:49:55 +01:00
Erik Johnston	c85e063302	Merge pull request #4051 from matrix-org/erikj/alias_disallow_list ... Add config option to control alias creation	2018-10-25 17:04:59 +01:00
Neil Johnson	95ad128851	Merge pull request #4081 from matrix-org/neilj/fix_mau_init ... fix race condiftion in calling initialise_reserved_users	2018-10-25 16:33:40 +01:00
Erik Johnston	b94a43d5b5	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/alias_disallow_list	2018-10-25 15:25:31 +01:00
Erik Johnston	e5481b22aa	Use allow/deny	2018-10-25 15:25:21 +01:00
Neil Johnson	f7f487e14c	Merge branch 'develop' of github.com:matrix-org/synapse into matthew/autocreate_autojoin	2018-10-25 14:40:06 +01:00
Travis Ralston	a5468eaadf	pep8	2018-10-24 13:54:38 -06:00
Travis Ralston	81880beff4	It helps to import things	2018-10-24 13:32:13 -06:00
Travis Ralston	4acb6fe8a3	Move test to where the other integration tests are	2018-10-24 13:24:24 -06:00
Travis Ralston	9283987f7e	Fix test ... Debug tests Try printing the channel fix Import and use six Remove debugging Disable captcha Add some mocks Define the URL Fix the clock? Less rendering? use the other render Complete the dummy auth stage Fix last stage of the test Remove mocks we don't need	2018-10-24 13:23:08 -06:00
Travis Ralston	54def42c19	Merge branch 'develop' into travis/login-terms	2018-10-24 13:22:59 -06:00
Neil Johnson	ea69a84bbb	fix style inconsistencies	2018-10-24 17:18:08 +01:00
Neil Johnson	07126e43a4	Merge branch 'develop' of github.com:matrix-org/synapse into neilj/fix_mau_init	2018-10-24 16:25:39 +01:00
Richard van der Hoff	3ad359e5be	Merge remote-tracking branch 'origin/develop' into rav/fix_event_filter_validation	2018-10-24 11:23:49 +01:00
Richard van der Hoff	7328039117	Merge pull request #4082 from matrix-org/rav/fix_pep8 ... Fix a number of flake8 errors	2018-10-24 11:23:35 +01:00
Erik Johnston	3904cbf307	Merge pull request #4040 from matrix-org/erikj/states_res_v2_rebase ... Add v2 state resolution algorithm	2018-10-24 11:12:12 +01:00
Richard van der Hoff	7e07d25ed6	Allow backslashes in event field filters ... Fixes a bug introduced in https://github.com/matrix-org/synapse/pull/1783 which meant that single backslashes were not allowed in event field filters. The intention here is to allow single-backslashes, but disallow double-backslashes.	2018-10-24 11:11:24 +01:00
Richard van der Hoff	ef771cc4c2	Fix a number of flake8 errors ... Broadly three things here: * disable W504 which seems a bit whacko * remove a bunch of `as e` expressions from exception handlers that don't use them * use `r""` for strings which include backslashes Also, we don't use pep8 any more, so we can get rid of the duplicate config there.	2018-10-24 10:39:03 +01:00
Erik Johnston	b313b9b009	isort	2018-10-24 10:02:41 +01:00
Erik Johnston	810715f79a	Rename resolve_events_with_factory	2018-10-24 09:44:22 +01:00
Neil Johnson	6105c6101f	fix race condiftion in calling initialise_reserved_users	2018-10-23 15:24:58 +01:00
Erik Johnston	3c580c2b47	Add tests for alias creation rules	2018-10-19 10:22:45 +01:00
Erik Johnston	1b4bf232b9	Add tests for config generation	2018-10-19 10:22:45 +01:00
Travis Ralston	dba84fa69c	Fix terms UI auth test	2018-10-18 12:45:21 -06:00
Travis Ralston	88c5ffec33	Test for terms UI auth	2018-10-18 12:35:30 -06:00
Will Hunt	d6a7797dd1	Fix roomlist since tokens on Python 3 (#4046) ... Thanks @Half-Shot !!!	2018-10-17 23:04:55 +11:00
Erik Johnston	947c7443eb	Add some state res v2 tests	2018-10-16 16:28:39 +01:00
Neil Johnson	a2bfb778c8	improve auto room join logic, comments and tests	2018-10-12 18:17:36 +01:00
David Baker	dc045ef202	Merge remote-tracking branch 'origin/develop' into dbkr/e2e_backups	2018-10-09 10:05:02 +01:00
Neil Johnson	ed82043efb	Merge branch 'develop' into matthew/autocreate_autojoin	2018-10-04 17:26:59 +01:00
Neil Johnson	2dadc092b8	move logic into register, fix room alias localpart bug, tests	2018-10-04 17:00:27 +01:00
Erik Johnston	9c834b8ee9	Add tests	2018-10-02 16:22:39 +01:00
Amber Brown	7232917f12	Disable frozen dicts by default (#3987)	2018-10-02 22:53:47 +10:00
Richard van der Hoff	b5b93f45d5	Merge pull request #3968 from matrix-org/rav/fix_federation_errors ... Fix exceptions when handling incoming transactions	2018-10-01 15:54:24 +01:00
Amber Brown	6e05fd032c	Fix userconsent on Python 3 (#3938)	2018-10-02 00:11:58 +10:00
Richard van der Hoff	f094f715cf	Merge remote-tracking branch 'origin/develop' into rav/fix_federation_errors	2018-09-27 15:18:21 +01:00
Amber Brown	a512e637ac	Merge pull request #3970 from schnuffle/develop-py3 ... Replaced all occurences of e.message with str(e)	2018-09-27 23:39:45 +10:00
Richard van der Hoff	484a9b8c81	Remove redundant, failing, test ... This test didn't do what it claimed to do, and what it claimed to do was the same as test_cant_hide_direct_ancestors anyway. This stuff is tested by sytest anyway.	2018-09-27 13:11:23 +01:00
Schnuffle	dc5db01ff2	Replaced all occurences of e.message with str(e) ... Signed-off-by: Schnuffle <schnuffle@github.com>	2018-09-27 13:38:50 +02:00
Richard van der Hoff	4a15a3e4d5	Include eventid in log lines when processing incoming federation transactions (#3959) ... when processing incoming transactions, it can be hard to see what's going on, because we process a bunch of stuff in parallel, and because we may end up recursively working our way through a chain of three or four events. This commit creates a way to use logcontexts to add the relevant event ids to the log lines.	2018-09-27 11:25:34 +01:00
Richard van der Hoff	ae6ad4cf41	docstrings and unittests for storage.state (#3958) ... I spent ages trying to figure out how I was going mad...	2018-09-27 11:22:25 +01:00
Richard van der Hoff	4c3e7eeec5	Merge pull request #3932 from matrix-org/erikj/auto_start_expiring_caches ... Fix some instances of ExpiringCache not expiring cache items	2018-09-25 12:02:57 +01:00
Amber Brown	fbe5ba25f6	Merge branch 'master' into develop	2018-09-25 03:10:01 +10:00
Oleg Girko	7d3f639844	Fix compatibility issue with older Twisted in tests. ... Older Twisted (18.4.0) returns TimeoutError instead of ConnectingCancelledError when connection times out. This change allows tests to be compatible with this behaviour. Signed-off-by: Oleg Girko <ol@infoserver.lv>	2018-09-25 02:51:18 +10:00
Erik Johnston	8601c24287	Fix some instances of ExpiringCache not expiring cache items ... ExpiringCache required that `start()` be called before it would actually start expiring entries. A number of places didn't do that. This PR removes `start` from ExpiringCache, and automatically starts backround reaping process on creation instead.	2018-09-21 14:19:46 +01:00
Amber Brown	1f3f5fcf52	Fix client IPs being broken on Python 3 (#3908)	2018-09-20 20:14:34 +10:00
Amber Brown	aeca5a5ed5	Add a regression test for logging on failed connections (#3912)	2018-09-20 16:28:18 +10:00
Richard van der Hoff	31c15dcb80	Refactor matrixfederationclient to fix logging (#3906) ... We want to wait until we have read the response body before we log the request as complete, otherwise a confusing thing happens where the request appears to have completed, but we later fail it. To do this, we factor the salient details of a request out to a separate object, which can then keep track of the txn_id, so that it can be logged.	2018-09-18 18:17:15 +01:00
Erik Johnston	f6e82dcddb	Tests	2018-09-14 15:08:37 +01:00
Amber Brown	bfa0b759e0	Attempt to figure out what's going on with timeouts (#3857)	2018-09-14 00:15:51 +10:00
Neil Johnson	8decd6233d	improve naming	2018-09-12 16:22:15 +01:00
Neil Johnson	0ddf486724	expose number of real reserved users	2018-09-12 11:58:52 +01:00
Amber Brown	52ec6e9dfa	Port tests/ to Python 3 (#3808)	2018-09-07 02:58:18 +10:00
Neil Johnson	c5440b2ca0	Merge pull request #3800 from matrix-org/neilj/remove-guests-from-mau-count ... guest users should not be part of mau total	2018-09-06 17:45:55 +01:00
Neil Johnson	84a750e0c3	ensure guests never enter mau list	2018-09-06 17:22:53 +01:00
Hubert Chathi	3801b8aa03	try to make flake8 and isort happy	2018-09-06 11:35:19 -04:00
Richard van der Hoff	4f8baab0c4	Merge branch 'master' into develop	2018-09-06 13:05:22 +01:00
Neil Johnson	61b05727fa	guest users should not be part of mau total	2018-09-05 22:30:36 +01:00
Richard van der Hoff	c127c8d042	Fix origin handling for pushed transactions ... Use the actual origin for push transactions, rather than whatever the remote server claimed.	2018-09-05 13:08:07 +01:00
Amber Brown	77055dba92	Fix tests on postgresql (#3740)	2018-09-04 02:21:48 +10:00
Neil Johnson	0b01281e77	move threepid checker to config, add missing yields	2018-08-31 17:11:11 +01:00
Neil Johnson	ea068d6f3c	fix bug where preserved threepid user comes to sign up and server is mau blocked	2018-08-31 10:49:14 +01:00
Amber Brown	14e4d4f4bf	Port storage/ to Python 3 (#3725)	2018-08-31 00:19:58 +10:00
Richard van der Hoff	414fa36f3e	Fix up tests	2018-08-28 17:21:05 +01:00
Hubert Chathi	16a31c6fce	update to newer Synapse APIs	2018-08-24 22:51:25 -04:00
Erik Johnston	9db2476991	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/admin_contact	2018-08-24 17:00:37 +01:00
Erik Johnston	05077e06fa	Change admin_uri to admin_contact in config and errors	2018-08-24 16:51:27 +01:00
Hubert Chathi	83caead95a	Merge branch 'develop' into e2e_backups	2018-08-24 11:44:26 -04:00
Erik Johnston	01a5a8b9e3	Fix checking if service notice room is already tagged ... This manifested in synapse repeatedly setting the tag for the room	2018-08-24 16:22:37 +01:00
Erik Johnston	f2cbbda956	Unit tests	2018-08-23 19:17:19 +01:00
Erik Johnston	cd77270a66	Implement trail users	2018-08-23 19:17:19 +01:00
Erik Johnston	7e6e588e60	Fix bug where we resent "limit exceeded" server notices ... This was due to a bug where we mutated a cached event's contents	2018-08-23 16:21:20 +01:00
Erik Johnston	c5842dff1a	Actually run the tests	2018-08-23 10:35:54 +01:00
Erik Johnston	fd2dbf1836	Merge branch 'develop' of github.com:matrix-org/synapse into neilj/server_notices_on_blocking	2018-08-22 17:06:10 +01:00
Erik Johnston	9643a6f7f2	Update notice format	2018-08-22 17:00:29 +01:00
Erik Johnston	764030cf63	Merge pull request #3659 from matrix-org/erikj/split_profiles ... Allow profile updates to happen on workers	2018-08-22 11:35:55 +01:00
Erik Johnston	3bf8bab8f9	Merge pull request #3673 from matrix-org/erikj/refactor_state_handler ... Refactor state module to support multiple room versions	2018-08-22 10:04:55 +01:00
Matthew Hodgson	bb81e78ec6	Split the state_group_cache in two (#3726) ... Splits the state_group_cache in two. One half contains normal state events; the other contains member events. The idea is that the lazyloading common case of: "I want a subset of member events plus all of the other state" can be accomplished efficiently by splitting the cache into two, and asking for "all events" from the non-members cache, and "just these keys" from the members cache. This means we can avoid having to make DictionaryCache aware of these sort of complicated queries, whilst letting LL requests benefit from the caching. Previously we were unable to sensibly use the caching and had to pull all state from the DB irrespective of the filtering, which made things slow. Hopefully fixes https://github.com/matrix-org/synapse/issues/3720.	2018-08-22 00:56:37 +02:00
Erik Johnston	4d664278af	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/refactor_state_handler	2018-08-20 14:49:43 +01:00
Neil Johnson	e07970165f	rename error code	2018-08-18 14:39:45 +01:00
Neil Johnson	ba1fbf7d5b	special case server_notices_mxid	2018-08-18 12:31:08 +01:00
Amber Brown	c334ca67bb	Integrate presence from hotfixes (#3694)	2018-08-18 01:08:45 +10:00
Neil Johnson	d49b77404b	clean up, no functional changes	2018-08-17 15:21:34 +01:00
Richard van der Hoff	63260397c6	Merge pull request #3701 from matrix-org/rav/use_producer_for_responses ... Use a producer to stream back responses	2018-08-17 14:58:45 +01:00
Neil Johnson	3ee57bdcbb	Merge branch 'develop' of github.com:matrix-org/synapse into neilj/server_notices_on_blocking	2018-08-17 14:34:10 +01:00
Erik Johnston	782689bd40	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/split_profiles	2018-08-17 14:15:48 +01:00
Erik Johnston	ca87ad1def	Split ProfileHandler into master and worker	2018-08-17 14:15:14 +01:00
Neil Johnson	b5f638f1f4	Merge branch 'develop' of github.com:matrix-org/synapse into neilj/server_notices_on_blocking	2018-08-17 14:04:15 +01:00
Neil Johnson	69c49d3fa3	Merge branch 'develop' into neilj/limit_exceeded_error	2018-08-17 12:44:26 +00:00
Neil Johnson	bcfeb44afe	call reap on start up and fix under reaping bug	2018-08-16 22:55:32 +01:00
Neil Johnson	13ad9930c8	add new error type ResourceLimit	2018-08-16 18:02:02 +01:00
Neil Johnson	3c1080b6e4	refactor for readability, and reuse caching for setting tags	2018-08-16 17:02:04 +01:00
Neil Johnson	a675f9c556	check for room state before deciding on action	2018-08-16 14:53:35 +01:00
Neil Johnson	8cfad2e686	Merge branch 'develop' of github.com:matrix-org/synapse into neilj/server_notices_on_blocking	2018-08-15 17:20:38 +01:00
Neil Johnson	81d727efa9	Merge pull request #3689 from matrix-org/neilj/fix_off_by_1+maus ... Fix Mau off by one errors	2018-08-15 16:19:41 +00:00
Matthew Hodgson	2f78f432c4	speed up /members and add at= and membership params (#3568)	2018-08-15 16:35:22 +01:00
Neil Johnson	fc5d937550	Merge branch 'develop' of github.com:matrix-org/synapse into neilj/server_notices_on_blocking	2018-08-15 16:31:40 +01:00
Neil Johnson	86a00e05e1	Merge branch 'develop' of github.com:matrix-org/synapse into neilj/fix_off_by_1+maus	2018-08-15 16:27:08 +01:00
Amber Brown	a87af25fbb	Fix the tests	2018-08-15 15:12:23 +01:00
Neil Johnson	75c663c7b9	update error codes	2018-08-15 11:27:48 +01:00
Neil Johnson	b8429c7c81	update error codes for resource limiting	2018-08-15 10:19:48 +01:00
Neil Johnson	ab035bdeac	replace admin_email with admin_uri for greater flexibility	2018-08-15 10:16:41 +01:00
Neil Johnson	19b433e3f4	Merge branch 'develop' of github.com:matrix-org/synapse into neilj/admin_email	2018-08-14 17:44:46 +01:00
Neil Johnson	b586b8b986	Merge branch 'develop' of github.com:matrix-org/synapse into neilj/fix_off_by_1+maus	2018-08-14 17:43:22 +01:00
Neil Johnson	7277216d01	fix setup_test_homeserver to be postgres compatible	2018-08-14 17:14:39 +01:00
Neil Johnson	1522ed9c07	in case max_mau is less than I think	2018-08-14 16:52:30 +01:00
Neil Johnson	414d54b61a	Merge pull request #3670 from matrix-org/neilj/mau_sync_block ... Block ability to read via sync if mau limit exceeded	2018-08-14 15:21:31 +00:00
Neil Johnson	06b331ff40	fix off by 1 errors	2018-08-14 15:28:15 +01:00
Neil Johnson	8f9a7eb58d	support admin_email config and pass through into blocking errors, return AuthError in all cases	2018-08-14 15:11:54 +01:00
Neil Johnson	ed4bc3d2fc	fix off by 1s on mau	2018-08-14 15:04:48 +01:00
Neil Johnson	9b5bf3d858	Merge branch 'neilj/admin_email' of github.com:matrix-org/synapse into neilj/fix_off_by_1+maus	2018-08-14 14:51:38 +01:00
Neil Johnson	e25d87d97b	Merge branch 'neilj/mau_sync_block' of github.com:matrix-org/synapse into neilj/fix_off_by_1+maus	2018-08-14 14:32:18 +01:00
Amber Brown	bdfbd934d6	Implement a new test baseclass to cut down on boilerplate (#3684)	2018-08-14 20:53:43 +10:00
Neil Johnson	f4b49152e2	support admin_email config and pass through into blocking errors, return AuthError in all cases	2018-08-13 21:09:47 +01:00
Neil Johnson	ce7de9ae6b	Revert "support admin_email config and pass through into blocking errors, return AuthError in all cases" ... This reverts commit 0d43f991a1.	2018-08-13 18:06:18 +01:00
Neil Johnson	0d43f991a1	support admin_email config and pass through into blocking errors, return AuthError in all cases	2018-08-13 18:00:23 +01:00
Amber Brown	99dd975dae	Run tests under PostgreSQL (#3423)	2018-08-13 16:47:46 +10:00
Matthew Hodgson	66a4ca1d28	404 nicely if you try to interact with a missing current version	2018-08-12 19:14:31 -04:00
Matthew Hodgson	edc427a351	flake8	2018-08-12 19:14:31 -04:00
Matthew Hodgson	fe87890b18	implement remaining tests and make them work	2018-08-12 19:14:31 -04:00
Matthew Hodgson	f6a3067868	linting	2018-08-12 19:14:31 -04:00
Matthew Hodgson	15d513f16f	fix idiocies and so make tests pass	2018-08-12 19:14:31 -04:00
Matthew Hodgson	174be586e5	first cut at a UT	2018-08-12 19:14:31 -04:00
Neil Johnson	6c6aba76e1	implementation of server notices to alert on hitting resource limits	2018-08-10 15:12:59 +01:00
Amber Brown	807449d8f2	fix up a forced long line	2018-08-11 00:01:43 +10:00
black	8b3d9b6b19	Run black.	2018-08-10 23:54:09 +10:00
Amber Brown	b37c472419	Rename async to async_helpers because async is a keyword on Python 3.7 (#3678)	2018-08-10 23:50:21 +10:00
Richard van der Hoff	3c0213a217	Merge pull request #3439 from vojeroen/send_sni_for_federation_requests ... send SNI for federation requests	2018-08-10 12:23:54 +01:00
Neil Johnson	885ea9c602	rename _user_last_seen_monthly_active	2018-08-09 18:02:12 +01:00
Neil Johnson	04df714259	fix imports	2018-08-09 17:41:52 +01:00
Neil Johnson	09cf130898	only block on sync where user is not part of the mau cohort	2018-08-09 17:39:12 +01:00
Erik Johnston	3e19beb941	Fix tests	2018-08-09 14:58:49 +01:00
Neil Johnson	69ce057ea6	block sync if auth checks fail	2018-08-09 12:26:27 +01:00
Neil Johnson	3dce9050cf	Merge branch 'develop' of github.com:matrix-org/synapse into neilj/mau_sync_block	2018-08-09 11:42:02 +01:00
Neil Johnson	0ad98e38d0	Merge pull request #3655 from matrix-org/neilj/disable_hs ... Flag to disable HS without disabling federation	2018-08-09 10:41:43 +00:00
Neil Johnson	a5ef110749	Merge branch 'develop' of github.com:matrix-org/synapse into neilj/mau_sync_block	2018-08-09 11:40:37 +01:00
Amber Brown	2511f3f8a0	Test fixes for Python 3 (#3647)	2018-08-09 12:22:01 +10:00
Neil Johnson	e92fb00f32	sync auth blocking	2018-08-08 17:54:49 +01:00
Neil Johnson	5298d79fb5	Merge branch 'develop' into neilj/disable_hs	2018-08-08 16:13:03 +00:00
Richard van der Hoff	8521ae13e3	Merge pull request #3654 from matrix-org/rav/room_versions ... Support for room versioning	2018-08-08 17:10:53 +01:00
Neil Johnson	d2f3ef98ac	Merge branch 'develop' into neilj/disable_hs	2018-08-08 15:55:47 +00:00
Neil Johnson	7f3d897e7a	mock config.max_mau_value	2018-08-08 11:46:23 +01:00
Neil Johnson	e8eba2b4e3	implement reserved users for mau limits	2018-08-07 17:49:43 +01:00
Neil Johnson	fbe255f9a4	add default mau_limits_reserved_threepids	2018-08-06 23:24:54 +01:00
Neil Johnson	42c6823827	disable HS from config	2018-08-04 22:07:04 +01:00
Neil Johnson	d08296f9f2	remove unused import	2018-08-03 23:00:16 +01:00
Neil Johnson	886be75ad1	bug fixes	2018-08-03 22:29:03 +01:00
Neil Johnson	e10830e976	wip commit - tests failing	2018-08-03 17:55:50 +01:00
Richard van der Hoff	0ca459ea33	Basic support for room versioning ... This is the first tranche of support for room versioning. It includes: * setting the default room version in the config file * new room_version param on the createRoom API * storing the version of newly-created rooms in the m.room.create event * fishing the version of existing rooms out of the m.room.create event	2018-08-03 16:08:32 +01:00
Neil Johnson	5593ff6773	fix (lots of) py3 test failures	2018-08-03 14:59:17 +01:00
Neil Johnson	b2aab04d2c	fix py3 test failure	2018-08-03 14:12:56 +01:00
Neil Johnson	950807d93a	fix caching and tests	2018-08-03 13:49:53 +01:00
Neil Johnson	74b1d46ad9	do mau checks based on monthly_active_users table	2018-08-02 16:57:35 +01:00
Neil Johnson	9180061b49	remove unused count_monthly_users	2018-08-02 15:55:29 +01:00
Neil Johnson	c4ffbecb68	fix test, update constructor call	2018-08-02 13:51:05 +01:00
Neil Johnson	00f99f74b1	insertion into monthly_active_users	2018-08-02 13:47:19 +01:00
Neil Johnson	165e067033	Revert "change monthly_active_users table to be a single column" ... This reverts commit ec716a35b2.	2018-08-02 10:59:58 +01:00
Neil Johnson	ec716a35b2	change monthly_active_users table to be a single column	2018-08-01 17:54:37 +01:00
Neil Johnson	d766f26de9	Merge branch 'develop' of github.com:matrix-org/synapse into neilj/mau_tracker	2018-08-01 17:49:41 +01:00
Neil Johnson	085435e13a	Merge pull request #3630 from matrix-org/neilj/mau_sign_in_log_in_limits ... Initial impl of capping MAU	2018-08-01 15:58:45 +00:00
Amber Brown	da7785147d	Python 3: Convert some unicode/bytes uses (#3569)	2018-08-02 00:54:06 +10:00
Neil Johnson	c480c4c962	fix isort	2018-08-01 14:25:58 +01:00
Neil Johnson	6eed16d8a2	fix test for py3	2018-08-01 14:02:10 +01:00
Neil Johnson	303f1c851f	Merge branch 'develop' of github.com:matrix-org/synapse into neilj/mau_sign_in_log_in_limits	2018-08-01 13:42:50 +01:00
Neil Johnson	4e6e00152c	fix known broken test	2018-08-01 11:48:37 +01:00
Richard van der Hoff	cab782c17e	Merge pull request #3384 from matrix-org/rav/rewrite_cachedlist_decorator ... Rewrite cache list decorator	2018-08-01 10:28:56 +01:00
Neil Johnson	7931393495	make count_monthly_users async synapse/handlers/auth.py	2018-08-01 10:21:56 +01:00
Neil Johnson	6ef983ce5c	api into monthly_active_users table	2018-07-31 16:36:24 +01:00
Neil Johnson	df2235e7fa	coding style	2018-07-31 13:16:20 +01:00
Neil Johnson	7d05406a07	fix user_ips counting	2018-07-31 12:03:23 +01:00
Travis Ralston	e908b86832	Remove pdu_failures from transactions ... The field is never read from, and all the opportunities given to populate it are not utilized. It should be very safe to remove this.	2018-07-30 16:28:47 -06:00
Neil Johnson	cefac79c10	monthly_active_tests	2018-07-30 22:08:09 +01:00
Neil Johnson	251e6c1210	limit register and sign in on number of monthly users	2018-07-30 15:55:57 +01:00
Richard van der Hoff	a8cbce0ced	fix invalidation	2018-07-27 16:17:17 +01:00
Richard van der Hoff	f102c05856	Rewrite cache list decorator ... Because it was complicated and annoyed me. I suspect this will be more efficient too.	2018-07-27 13:47:04 +01:00
Richard van der Hoff	7041cd872b	Merge branch 'develop' into send_sni_for_federation_requests	2018-07-27 09:17:11 +01:00
Matthew Hodgson	2565804030	Merge branch 'develop' into matthew/filter_members	2018-07-25 17:27:49 +01:00
Matthew Hodgson	0620d27f4d	flake8	2018-07-25 17:21:17 +01:00
Matthew Hodgson	0a7ee0ab8b	add tests for _get_some_state_from_cache	2018-07-25 16:33:52 +01:00
Richard van der Hoff	f59be4eb0e	Fix unit tests ... on_notifier_poke no longer runs synchonously, so we have to do a different hack to make sure that the replication data has been sent. Let's actually listen for its arrival.	2018-07-25 10:30:36 +01:00
Matthew Hodgson	cd241d6bda	incorporate more review	2018-07-24 12:39:40 +01:00
Matthew Hodgson	adfe29ec0b	Merge branch 'develop' into matthew/filter_members	2018-07-23 19:21:37 +01:00
Erik Johnston	8fbe418777	Fix unit tests	2018-07-23 13:33:49 +01:00
Amber Brown	3132b89f12	Make the rest of the .iterwhatever go away (#3562)	2018-07-21 15:47:18 +10:00
Richard van der Hoff	3d6df84658	Test and fix support for cancellation in Linearizer	2018-07-20 13:59:55 +01:00
Richard van der Hoff	4f67623674	Merge pull request #3571 from matrix-org/rav/limiter_fixes ... A set of improvements to the Limiter	2018-07-20 13:53:03 +01:00
Amber Brown	e1a237eaab	Admin API for creating new users (#3415)	2018-07-20 22:41:13 +10:00
Richard van der Hoff	7c712f95bb	Combine Limiter and Linearizer ... Linearizer was effectively a Limiter with max_count=1, so rather than maintaining two sets of code, let's combine them.	2018-07-20 13:11:43 +01:00
Richard van der Hoff	d7275eecf3	Add a sleep to the Limiter to fix stack overflows. ... Fixes #3570	2018-07-20 12:37:12 +01:00
Matthew Hodgson	650daf5628	make test work	2018-07-19 20:49:44 +01:00
Matthew Hodgson	1fa4f7e03e	first cut of a UT for testing state store (untested)	2018-07-19 20:19:32 +01:00
Richard van der Hoff	c754e006f4	Merge pull request #3556 from matrix-org/rav/background_processes ... Run things as background processes	2018-07-19 11:04:18 +01:00
Amber Brown	a97c845271	Move v1-only APIs into their own module & isolate deprecated ones (#3460)	2018-07-19 20:03:33 +10:00
Richard van der Hoff	8c69b735e3	Make Distributor run its processes as a background process ... This is more involved than it might otherwise be, because the current implementation just drops its logcontexts and runs everything in the sentinel context. It turns out that we aren't actually using a bunch of the functionality here (notably suppress_failures and the fact that Distributor.fire returns a deferred), so the easiest way to fix this is actually by simplifying a bunch of code.	2018-07-18 20:55:05 +01:00
Richard van der Hoff	d897be6a98	Fix visibility of events from erased users over federation	2018-07-17 14:02:07 +01:00
Richard van der Hoff	9c04b4abf9	Merge pull request #3541 from matrix-org/rav/optimize_filter_events_for_server ... Refactor and optimze filter_events_for_server	2018-07-17 14:01:39 +01:00
Richard van der Hoff	94440ae994	fix imports	2018-07-17 11:51:26 +01:00
Amber Brown	bc006b3c9d	Refactor REST API tests to use explicit reactors (#3351)	2018-07-17 20:43:18 +10:00
Erik Johnston	850238b4ef	Add unit test	2018-07-17 10:59:02 +01:00
Amber Brown	3fe0938b76	Merge pull request #3530 from matrix-org/erikj/stream_cache ... Don't return unknown entities in get_entities_changed	2018-07-17 13:44:46 +10:00
Richard van der Hoff	15b13b537f	Add a test which profiles filter_events_for_server in a large room	2018-07-16 14:06:09 +01:00
Richard van der Hoff	ea69d35651	Move filter_events_for_server out of FederationHandler ... for easier unit testing.	2018-07-16 13:06:24 +01:00
Jeroen	505530f36a	Merge remote-tracking branch 'upstream/develop' into send_sni_for_federation_requests ... # Conflicts: # synapse/crypto/context_factory.py	2018-07-14 20:24:46 +02:00
Amber Brown	33b60c01b5	Make auth & transactions more testable (#3499)	2018-07-14 07:34:49 +10:00
Erik Johnston	bc832f822f	Fixup unit test	2018-07-13 17:03:04 +01:00
Richard van der Hoff	3b391d9c45	Fix unit tests	2018-07-13 16:28:04 +01:00
Oleg Girko	6c1ec5a1bd	Use more portable syntax using attrs package. ... Newer syntax attr.ib(factory=dict) is just a syntactic sugar for attr.ib(default=attr.Factory(dict)) It was introduced in newest version of attrs package (18.1.0) and doesn't work with older versions. We should either require minimum version of attrs to be 18.1.0, or use older (slightly more verbose) syntax. Requiring newest version is not a good solution because Linux distributions may have older version of attrs (17.4.0 in Fedora 28), and requiring to build (and package) newer version just to use newer syntactic sugar in only one test is just too much. It's much better to fix that test to use older syntax. Signed-off-by: Oleg Girko <ol@infoserver.lv>	2018-07-10 00:38:49 +01:00
Jeroen	b5e157d895	Merge branch 'develop' into send_sni_for_federation_requests ... # Conflicts: # synapse/http/endpoint.py	2018-07-09 08:51:11 +02:00
Amber Brown	49af402019	run isort	2018-07-09 16:09:20 +10:00
Neil Johnson	feef8461d1	Merge remote-tracking branch 'hera/rav/server_acls' into develop	2018-07-05 11:04:01 +01:00
Erik Johnston	1a88640677	Merge pull request #3483 from matrix-org/rav/more_server_name_validation ... More server_name validation	2018-07-05 10:04:20 +01:00
Richard van der Hoff	3cf3e08a97	Implementation of server_acls ... ... as described at https://docs.google.com/document/d/1EttUVzjc2DWe2ciw4XPtNpUpIl9lWXGEsy2ewDS7rtw.	2018-07-04 19:06:20 +01:00
Richard van der Hoff	546bc9e28b	More server_name validation ... We need to do a bit more validation when we get a server name, but don't want to be re-doing it all over the shop, so factor out a separate parse_and_validate_server_name, and do the extra validation. Also, use it to verify the server name in the config file.	2018-07-04 18:59:51 +01:00
Richard van der Hoff	ea555d5633	Reinstate lost run_on_reactor in unit test ... a61738b removed a call to run_on_reactor from a unit test, but that call was doing something useful, in making the function in question asynchronous. Reinstate the call and add a check that we are testing what we wanted to be testing.	2018-07-04 09:40:01 +01:00
Richard van der Hoff	508196e08a	Reject invalid server names (#3480) ... Make sure that server_names used in auth headers are sane, and reject them with a sensible error code, before they disappear off into the depths of the system.	2018-07-03 14:36:14 +01:00
Erik Johnston	e3b4043800	Merge pull request #3456 from matrix-org/hawkowl/federation-prevevent-checking ... Check the state of prev_events a bit more thoroughly when coming over federation	2018-06-29 13:55:02 +01:00
Matthew Hodgson	f82cf3c7df	add test	2018-06-28 21:14:16 +01:00
Matthew Hodgson	e72234f6bd	fix tests	2018-06-28 20:56:07 +01:00
Amber Brown	f03a5d1a17	pep8	2018-06-27 11:38:14 +01:00
Amber Brown	94f09618e5	cleanups	2018-06-27 11:38:03 +01:00
Amber Brown	35cc3e8b14	stylistic cleanup	2018-06-27 11:32:09 +01:00
Amber Brown	8d62baa48c	cleanups	2018-06-27 11:31:48 +01:00
Amber Brown	77078d6c8e	handle federation not telling us about prev_events	2018-06-27 11:27:32 +01:00
Amber Brown	cd6bcdaf87	Better testing framework for homeserver-using things (#3446)	2018-06-27 10:37:24 +01:00
Jeroen	3d605853c8	send SNI for federation requests	2018-06-24 22:38:43 +02:00
Erik Johnston	f79abda87f	Merge pull request #3427 from matrix-org/erikj/remove_filters ... remove dead filter_events_for_clients	2018-06-22 14:45:42 +01:00
Richard van der Hoff	43e02c409d	Disable partial state group caching for wildcard lookups ... When _get_state_for_groups is given a wildcard filter, just do a complete lookup. Hopefully this will give us the best of both worlds by not filling up the ram if we only need one or two keys, but also making the cache still work for the federation reader usecase.	2018-06-22 11:52:07 +01:00
Amber Brown	77ac14b960	Pass around the reactor explicitly (#3385)	2018-06-22 09:37:10 +01:00
Richard van der Hoff	9a793f861c	Merge branch 'master' into develop	2018-06-14 16:36:01 +01:00
Richard van der Hoff	1e77ac66e3	Fix broken unit test ... We need power levels for this test to do what it is supposed to do.	2018-06-14 14:21:29 +01:00
Richard van der Hoff	a502cfec00	remove spurious debug	2018-06-14 14:20:53 +01:00
Richard van der Hoff	5c9afd6f80	Make default state_default 50 ... Make it so that, before there is a power-levels event in the room, you need a power level of at least 50 to send state. Partially addresses https://github.com/matrix-org/matrix-doc/issues/1192	2018-06-14 12:38:09 +01:00
Amber Brown	a61738b316	Remove run_on_reactor (#3395)	2018-06-14 18:27:37 +10:00
Richard van der Hoff	bd348f0af6	remove dead filter_events_for_clients ... This is only used by filter_events_for_client, so we can simplify the whole thing by just doing one user at a time, and removing a dead storage function to boot.	2018-06-12 09:51:31 +01:00
Amber Brown	f7869f8f8b	Port to sortedcontainers (with tests!) (#3332)	2018-06-06 00:13:57 +10:00
Amber Brown	5dbf305444	Put python's logs into Trial when running unit tests (#3319)	2018-06-04 16:06:06 +10:00
Richard van der Hoff	a995fdae39	fix tests	2018-05-29 20:19:29 +01:00
Matthew Hodgson	adb6bac4d5	fix another dumb typo	2018-05-29 02:29:22 +01:00
Matthew Hodgson	284e36ad04	fix dumb typo	2018-05-29 02:23:11 +01:00
Amber Brown	a8990fa2ec	Merge remote-tracking branch 'origin/develop' into 3218-official-prom	2018-05-22 10:50:26 -05:00
Richard van der Hoff	a5e2941aad	Reject attempts to send event before privacy consent is given ... Returns an M_CONSENT_NOT_GIVEN error (cf https://github.com/matrix-org/matrix-doc/issues/1252) if consent is not yet given.	2018-05-22 12:00:47 +01:00
Richard van der Hoff	d5dca9a04f	Move consent config parsing into ConsentConfig ... turns out we need to reuse this, so it's better in the config class.	2018-05-22 11:54:51 +01:00
Richard van der Hoff	9ea219c514	Send users a server notice about consent ... When a user first syncs, we will send them a server notice asking them to consent to the privacy policy if they have not already done so.	2018-05-22 11:54:51 +01:00
Amber Brown	fcc525b0b7	rest of the changes	2018-05-21 19:48:57 -05:00
Amber Brown	df9f72d9e5	replacing portions	2018-05-21 19:47:37 -05:00
Richard van der Hoff	6e1cb54a05	Fix logcontext leak in HttpTransactionCache ... ONE DAY I WILL PURGE THE WORLD OF THIS EVIL	2018-05-21 16:58:20 +01:00
Richard van der Hoff	6d6e7288fe	Stop the transaction cache caching failures ... The transaction cache has some code which tries to stop it caching failures, but if the callback function failed straight away, then things would happen backwards and we'd end up with the failure stuck in the cache.	2018-05-21 16:49:59 +01:00
Erik Johnston	6406b70aeb	Use stream rather depth ordering for push actions ... This simplifies things as it is, but will also allow us to change the way we traverse topologically without having to update the way push actions work.	2018-05-11 15:30:11 +01:00
Richard van der Hoff	902673e356	Merge pull request #3161 from NotAFile/remove-v1auth ... Make Client-Server API return 403 for invalid token	2018-05-03 10:10:57 +01:00
Erik Johnston	53a5fdf312	Merge pull request #3175 from matrix-org/erikj/escape_metric_values ... Escape label values in prometheus metrics	2018-05-03 10:01:04 +01:00
Erik Johnston	32015e1109	Escape label values in prometheus metrics	2018-05-02 16:52:42 +01:00
Richard van der Hoff	415c6b672e	Merge branch 'develop' into rav/more_logcontext_leaks	2018-05-02 16:16:01 +01:00
Richard van der Hoff	11607006d9	Remove spurious unittest.DEBUG	2018-05-02 15:48:47 +01:00
Richard van der Hoff	46beeb9a30	Fix a couple of logcontext leaks in unit tests ... ... which were making other, innocent, tests, fail. Plus remove a spurious unittest.DEBUG which was making the output noisy.	2018-05-02 15:46:22 +01:00
Richard van der Hoff	f22e7cda2c	Fix a class of logcontext leaks ... So, it turns out that if you have a first `Deferred` `D1`, you can add a callback which returns another `Deferred` `D2`, and `D2` must then complete before any further callbacks on `D1` will execute (and later callbacks on `D1` get the *result* of `D2` rather than `D2` itself). So, `D1` might have `called=True` (as in, it has started running its callbacks), but any new callbacks added to `D1` won't get run until `D2` completes - so if you `yield D1` in an `inlineCallbacks` function, your `yield` will 'block'. In conclusion: some of our assumptions in `logcontext` were invalid. We need to make sure that we don't optimise out the logcontext juggling when this situation happens. Fortunately, it is easy to detect by checking `D1.paused`.	2018-05-02 11:58:00 +01:00
Richard van der Hoff	e482f8cd85	Fix incorrect reference to StringIO ... This was introduced in 4f2f5171	2018-05-02 09:12:26 +01:00
Adrian Tschira	6495dbb326	Burminate v1auth ... This closes #2602 v1auth was created to account for the differences in status code between the v1 and v2_alpha revisions of the protocol (401 vs 403 for invalid tokens). However since those protocols were merged, this makes the r0 version/endpoint internally inconsistent, and violates the specification for the r0 endpoint. This might break clients that rely on this inconsistency with the specification. This is said to affect the legacy angular reference client. However, I feel that restoring parity with the spec is more important. Either way, it is critical to inform developers about this change, in case they rely on the illegal behaviour. Signed-off-by: Adrian Tschira <nota@notafile.com>	2018-04-30 22:20:43 +02:00
Richard van der Hoff	9e2601f830	Merge pull request #3108 from NotAFile/py3-six-urlparse ... Use six.moves.urlparse	2018-04-30 00:33:05 +01:00
Richard van der Hoff	3d1ae61399	Merge branch 'develop' into rav/deferred_timeout	2018-04-27 12:54:43 +01:00
Richard van der Hoff	286e20f2bc	Merge pull request #3109 from NotAFile/py3-tests-fix ... Make tests py3 compatible	2018-04-23 11:59:03 +01:00
Richard van der Hoff	1ea904b9f0	Use deferred.addTimeout instead of time_bound_deferred ... This doesn't feel like a wheel we need to reinvent.	2018-04-23 00:53:18 +01:00
Richard van der Hoff	d06a9ea5f7	Merge pull request #3104 from NotAFile/py3-unittest-config ... Add some more variables to the unittest config	2018-04-20 15:35:58 +01:00
Richard van der Hoff	692a3cc806	Merge pull request #3103 from NotAFile/py3-baseexcepton-message ... Use str(e) instead of e.message	2018-04-20 15:34:49 +01:00
Richard van der Hoff	639480e14a	Avoid creating events with huge numbers of prev_events ... In most cases, we limit the number of prev_events for a given event to 10 events. This fixes a particular code path which created events with huge numbers of prev_events.	2018-04-16 18:41:37 +01:00
Adrian Tschira	a1a3c9660f	Make tests py3 compatible ... This is a mixed commit that fixes various small issues * print parentheses * 01 is invalid syntax (it was octal in py2) * [x for i in 1, 2] is invalid syntax * six moves Signed-off-by: Adrian Tschira <nota@notafile.com>	2018-04-16 00:39:32 +02:00
Adrian Tschira	2a3c33ff03	Use six.moves.urlparse ... The imports were shuffled around a bunch in py3 Signed-off-by: Adrian Tschira <nota@notafile.com>	2018-04-15 21:22:43 +02:00
Adrian Tschira	cb9cdfecd0	Add some more variables to the unittest config ... These worked accidentally before (python2 doesn't complain if you compare incompatible types) but under py3 this blows up spectacularly Signed-off-by: Adrian Tschira <nota@notafile.com>	2018-04-15 20:36:39 +02:00
Adrian Tschira	1515560f5c	Use str(e) instead of e.message ... Doing this I learned e.message was pretty shortlived, added in 2.6, they realized it was a bad idea and deprecated it in 2.7 Signed-off-by: Adrian Tschira <nota@notafile.com>	2018-04-15 20:32:42 +02:00
Erik Johnston	e939f3bca6	Fix tests	2018-04-11 14:37:11 +01:00
Richard van der Hoff	a3599dda97	Merge pull request #2996 from krombel/allow_auto_join_rooms ... move handling of auto_join_rooms to RegisterHandler	2018-04-10 01:11:00 +01:00
Richard van der Hoff	87478c5a60	Merge pull request #3061 from NotAFile/add-some-byte-strings ... Add b prefixes to some strings that are bytes in py3	2018-04-09 23:54:05 +01:00
Richard van der Hoff	664adb4236	Merge pull request #3016 from silkeh/improve-service-lookups ... Improve handling of SRV records for federation connections	2018-04-09 23:40:06 +01:00
Richard van der Hoff	01afc563c3	Fix overzealous cache invalidation ... Fixes an issue where a cache invalidation would invalidate *all* pending entries, rather than just the entry that we intended to invalidate.	2018-04-05 16:24:04 +01:00
Adrian Tschira	6168351877	Add b prefixes to some strings that are bytes in py3 ... This has no effect on python2 Signed-off-by: Adrian Tschira <nota@notafile.com>	2018-04-04 13:48:51 +02:00
Silke	72251d1b97	Remove address resolution of hosts in SRV records ... Signed-off-by: Silke Hofstra <silke@slxh.eu>	2018-04-04 12:26:50 +02:00
Krombel	91ea0202e6	move handling of auto_join_rooms to RegisterHandler ... Currently the handling of auto_join_rooms only works when a user registers itself via public register api. Registrations via registration_shared_secret and ModuleApi do not work This auto_joins the users in the registration handler which enables the auto join feature for all 3 registration paths. This is related to issue #2725 Signed-Off-by: Matthias Kesler <krombel@krombel.de>	2018-03-14 16:45:37 +01:00
Erik Johnston	cb9f8e527c	s/replication_client/federation_client/	2018-03-13 13:26:52 +00:00
Erik Johnston	cea462e285	s/replication_server/federation_server	2018-03-13 13:22:21 +00:00
Erik Johnston	6ea27fafad	Fix tests	2018-03-13 10:55:47 +00:00
Erik Johnston	631a73f7ef	Fix tests	2018-03-13 10:39:19 +00:00
Richard van der Hoff	2c911d75e8	Fix comment typo	2018-03-05 12:24:49 +00:00
Richard van der Hoff	c818fcab11	Test and fix find_first_stream_ordering_after_ts ... It seemed to suffer from a bunch of off-by-one errors.	2018-03-05 12:04:02 +00:00
Erik Johnston	784f036306	Move RoomMemberHandler out of Handlers	2018-03-01 14:36:50 +00:00
Erik Johnston	e5b4a208ce	Merge pull request #2892 from matrix-org/erikj/batch_inserts_push_actions ... Batch inserts into event_push_actions_staging	2018-02-26 14:45:40 +00:00
Erik Johnston	25634ed152	Fix test	2018-02-20 12:40:44 +00:00
Erik Johnston	e440e28456	Fix unit tests	2018-02-20 11:41:40 +00:00
Erik Johnston	3a061cae26	Fix unit test	2018-02-15 16:31:59 +00:00
Erik Johnston	4810f7effd	Remove context.push_actions	2018-02-15 15:47:06 +00:00
Erik Johnston	617199d73d	Merge pull request #2847 from matrix-org/erikj/separate_event_creation ... Split event creation into a separate handler	2018-02-06 17:01:17 +00:00
Erik Johnston	770b2252ca	s/_create_new_client_event/create_new_client_event/	2018-02-06 16:40:30 +00:00
Erik Johnston	3d33eef6fc	Store state groups separately from events (#2784) ... * Split state group persist into seperate storage func * Add per database engine code for state group id gen * Move store_state_group to StateReadStore This allows other workers to use it, and so resolve state. * Hook up store_state_group * Fix tests * Rename _store_mult_state_groups_txn * Rename StateGroupReadStore * Remove redundant _have_persisted_state_group_txn * Update comments * Comment compute_event_context * Set start val for state_group_id_seq ... otherwise we try to recreate old state groups * Update comments * Don't store state for outliers * Update comment * Update docstring as state groups are ints	2018-02-06 14:31:24 +00:00
Erik Johnston	25c0a020f4	Updates tests	2018-02-05 16:01:48 +00:00
Richard van der Hoff	db91e72ade	Merge pull request #2844 from matrix-org/rav/evicted_metrics ... montoring metrics for number of cache evictions	2018-02-05 16:34:36 +01:00
Richard van der Hoff	bc496df192	report metrics on number of cache evictions	2018-02-05 15:34:01 +00:00
Erik Johnston	a1beca0e25	Fix broken unit test for media storage	2018-02-05 12:44:03 +00:00
Erik Johnston	1f881e0746	Merge pull request #2791 from matrix-org/erikj/media_storage_refactor ... Ensure media is in local cache before thumbnailing	2018-02-05 11:28:52 +00:00
Richard van der Hoff	6da4c4d3bd	Factor out resolve_state_groups to a separate handler ... We extract the storage-independent bits of the state group resolution out to a separate functiom, and stick it in a new handler, in preparation for its use from the storage layer.	2018-02-01 12:40:04 +00:00
Richard van der Hoff	d1f3490e75	Add tests for user directory search	2018-01-27 17:21:57 +00:00
Richard van der Hoff	88b9c5cbf0	Make it possible to run tests against postgres	2018-01-27 17:15:24 +00:00
Richard van der Hoff	d7eacc4f87	Create dbpool as normal in tests ... ... instead of creating our own special SQLiteMemoryDbPool, whose purpose was a bit of a mystery. For some reason this makes one of the tests run slightly slower, so bump the sleep(). Sorry.	2018-01-27 17:15:15 +00:00
Richard van der Hoff	b178eca261	Run on_new_connection for unit tests ... Configure the connectionpool used for unit tests to run the `on_new_connection` function.	2018-01-27 17:06:04 +00:00
Richard van der Hoff	6e9bf67f18	Remove unused/bitrotted MemoryDataStore ... This isn't used, and looks thoroughly bitrotted.	2018-01-26 00:35:15 +00:00
Richard van der Hoff	2b91846497	Remove spurious unittest.DEBUG	2018-01-26 00:34:27 +00:00
Richard van der Hoff	b2da272b77	Merge pull request #2821 from matrix-org/rav/matthew_test_fixes ... Matthew's fixes to the unit tests	2018-01-22 20:43:56 +00:00
Matthew Hodgson	ab9f844aaf	Add federation_domain_whitelist option (#2820) ... Add federation_domain_whitelist gives a way to restrict which domains your HS is allowed to federate with. useful mainly for gracefully preventing a private but internet-connected HS from trying to federate to the wider public Matrix network	2018-01-22 19:11:18 +01:00
Richard van der Hoff	5c431f421c	Matthew's fixes to the unit tests ... Extracted from https://github.com/matrix-org/synapse/pull/2820	2018-01-22 16:46:16 +00:00
Matthew Hodgson	d84f65255e	Merge pull request #2813 from matrix-org/matthew/registrations_require_3pid ... add registrations_require_3pid and allow_local_3pids	2018-01-22 13:57:22 +00:00
Matthew Hodgson	81d037dbd8	mock registrations_require_3pid	2018-01-19 00:28:08 +00:00
Erik Johnston	ce4f66133e	Add unit tests	2018-01-18 16:32:11 +00:00
Erik Johnston	b6dc7044a9	Merge pull request #2804 from matrix-org/erikj/file_consumer ... Add decent impl of a FileConsumer	2018-01-18 16:31:33 +00:00
Erik Johnston	1432f7ccd5	Move test stuff to tests	2018-01-18 11:57:57 +00:00
Erik Johnston	bc67e7d260	Add decent impl of a FileConsumer ... Twisted core doesn't have a general purpose one, so we need to write one ourselves. Features: - All writing happens in background thread - Supports both push and pull producers - Push producers get paused if the consumer falls behind	2018-01-17 16:43:03 +00:00
Richard van der Hoff	44a498418c	Optimise LoggingContext creation and copying ... It turns out that the only thing we use the __dict__ of LoggingContext for is `request`, and given we create lots of LoggingContexts and then copy them every time we do a db transaction or log line, using the __dict__ seems a bit redundant. Let's try to optimise things by making the request attribute explicit.	2018-01-16 15:49:42 +00:00
Richard van der Hoff	a767f06e3f	Merge pull request #2765 from matrix-org/rav/fix_room_uts ... Fix flaky test_rooms UTs	2018-01-09 14:21:03 +00:00
Richard van der Hoff	f8fa5ae4af	enable twisted delayedcall debugging in UTs	2018-01-09 12:06:45 +00:00
Richard van der Hoff	374c4d4ced	Remove dead code ... pointless function is pointless	2018-01-09 12:06:45 +00:00
Richard van der Hoff	142fb0a7d4	Disable user_directory updates for UTs ... Fix flakiness in the UTs caused by the user_directory being updated in the background	2018-01-09 12:06:45 +00:00
Richard van der Hoff	0211464ba2	Fix broken config UTs ... https://github.com/matrix-org/synapse/pull/2755 broke log-config generation, which in turn broke the unit tests.	2018-01-09 11:28:33 +00:00
Matthew Hodgson	33cb7ef0b7	Merge pull request #2723 from matrix-org/matthew/search-all-local-users ... Add all local users to the user_directory and optionally search them	2017-12-05 11:09:47 +00:00
Richard van der Hoff	d5f9fb06b0	Refactor UI auth implementation ... Instead of returning False when auth is incomplete, throw an exception which can be caught with a wrapper.	2017-12-05 09:40:05 +00:00
Matthew Hodgson	7b86c1fdcd	try make tests work a bit more...	2017-12-04 17:10:03 +00:00
Matthew Hodgson	95f8a713dc	erik told me to	2017-12-04 16:56:25 +00:00
Luke Barnard	ab1dc84779	Add extra space before inline comment	2017-11-16 18:22:40 +00:00
Luke Barnard	e836bdf734	Fix tests	2017-11-16 18:14:39 +00:00
Richard van der Hoff	63ef607f1f	Fix tests for Store.__init__ update ... Fix the test to pass the right number of args to the Store constructors	2017-11-13 10:46:08 +00:00
Ilya Zhuravlev	8a4a0ddea6	Fix appservice tests to account for new behavior of notify_interested_services	2017-11-02 23:19:57 +03:00
Richard van der Hoff	02237ce725	Fix tests for refresh_token removal	2017-11-01 10:19:42 +00:00
Richard van der Hoff	9ded00f221	fix tests	2017-10-31 14:21:13 +00:00
Richard van der Hoff	eaaabc6c4f	replace 'except:' with 'except Exception:' ... what could possibly go wrong	2017-10-23 15:52:32 +01:00
Richard van der Hoff	1135193dfd	Validate group ids when parsing ... May as well do it whenever we parse a Group ID. We check the sigil and basic structure here so it makes sense to check the grammar in the same place.	2017-10-21 00:30:39 +01:00
Richard van der Hoff	dbdfd8967d	Merge pull request #2546 from matrix-org/rav/remove_dead_event_injector ... Remove dead class	2017-10-17 13:21:22 +01:00
Richard van der Hoff	b8e40d146f	Merge pull request #2547 from matrix-org/rav/test_make_deferred_yieldable ... Add some tests for make_deferred_yieldable	2017-10-17 13:21:09 +01:00
Richard van der Hoff	a6ad8148b9	Fix name of test_logcontext ... The file under test is logcontext.py, not log_context.py	2017-10-17 10:53:34 +01:00
Richard van der Hoff	5b5f35ccc0	Add some tests for make_deferred_yieldable	2017-10-17 10:52:31 +01:00
Richard van der Hoff	9b714abf35	Remove dead class ... This isn't used anywhere.	2017-10-17 10:43:36 +01:00
David Baker	33122c5a1b	Fix test	2017-10-17 10:39:50 +01:00
Richard van der Hoff	cc794d60e7	Merge pull request #2532 from matrix-org/rav/fix_linearizer ... Fix stackoverflow and logcontexts from linearizer	2017-10-11 17:29:32 +01:00
Richard van der Hoff	4fad8efbfb	Fix stackoverflow and logcontexts from linearizer ... 1. make it not blow out the stack when there are more than 50 things waiting for a lock. Fixes https://github.com/matrix-org/synapse/issues/2505. 2. Make it not mess up the log contexts.	2017-10-11 15:05:05 +01:00
David Baker	27955056e0	Merge branch 'develop' into erikj/groups_merged	2017-10-02 16:20:41 +01:00
Richard van der Hoff	f65e31d22f	Do an AAAA lookup on SRV record targets (#2462) ... Support SRV records which point at AAAA records, as well as A records. Fixes https://github.com/matrix-org/synapse/issues/2405	2017-09-22 20:26:47 +01:00
Richard van der Hoff	72472456d8	Add some more tests for Keyring	2017-09-20 01:32:42 +01:00
Richard van der Hoff	9864efa532	Fix concurrent server_key requests (#2458) ... Fix a bug where we could end up firing off multiple requests for server_keys for the same server at the same time.	2017-09-19 23:25:44 +01:00
Richard van der Hoff	aa620d09a0	Add a config option to block all room invites (#2457) ... - allows sysadmins the ability to lock down their servers so that people can't send their users room invites.	2017-09-19 16:08:14 +01:00
Erik Johnston	b5feaa5a49	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/groups_merged	2017-09-19 11:07:45 +01:00
Richard van der Hoff	290777b3d9	Clean up and document handling of logcontexts in Keyring (#2452) ... I'm still unclear on what the intended behaviour for `verify_json_objects_for_server` is, but at least I now understand the behaviour of most of the things it calls...	2017-09-18 18:31:01 +01:00
Erik Johnston	bf81f3cf2c	Split out profile handler to fix tests	2017-08-25 14:34:56 +01:00
Erik Johnston	c544188ee3	Add groups to sync stream	2017-07-20 16:36:42 +01:00
Erik Johnston	ed9a7f5436	Merge pull request #2309 from matrix-org/erikj/user_ip_repl ... Fix up user_ip replication commands	2017-07-06 14:33:14 +01:00
Erik Johnston	29a4066a4d	Update test	2017-07-04 10:21:25 +01:00
Erik Johnston	8c23221666	Fix up	2017-06-27 15:53:45 +01:00
Erik Johnston	a0a561ae85	Fix up client ips to read from pending data	2017-06-27 14:46:12 +01:00
Erik Johnston	052c5d19d5	Merge pull request #2281 from matrix-org/erikj/phone_home_stats ... Fix phone home stats	2017-06-15 12:46:23 +01:00
Erik Johnston	93e7a38370	Remove unhelpful test	2017-06-15 09:30:54 +01:00
Erik Johnston	23da638360	Fix typing tests	2017-05-26 10:02:04 +01:00
Erik Johnston	11c2a3655f	Only load jinja2 templates once ... Instead of every time a new email pusher is created, as loading jinja2 templates is slow.	2017-05-22 17:48:58 +01:00
Erik Johnston	bbfe4e996c	Make get_state_groups_from_groups faster. ... Most of the time was spent copying a dict to filter out sentinel values that indicated that keys did not exist in the dict. The sentinel values were added to ensure that we cached the non-existence of keys. By updating DictionaryCache to keep track of which keys were known to not exist itself we can remove a dictionary copy.	2017-05-17 15:12:15 +01:00
Richard van der Hoff	de042b3b88	Do some logging when one-time-keys get claimed ... might help us figure out if https://github.com/vector-im/riot-web/issues/3868 has happened.	2017-05-09 19:04:56 +01:00
Richard van der Hoff	a7e9d8762d	Allow clients to upload one-time-keys with new sigs ... When a client retries a key upload, don't give an error if the signature has changed (but the key is the same). Fixes https://github.com/vector-im/riot-android/issues/1208, hopefully.	2017-05-09 19:04:56 +01:00
Erik Johnston	3e5a62ecd8	Add more granular event send metrics	2017-05-02 14:23:26 +01:00
Erik Johnston	85a0d6c7ab	Remove test of replication resource	2017-04-11 10:59:27 +01:00
Erik Johnston	a5c401bd12	Merge pull request #2097 from matrix-org/erikj/repl_tcp_client ... Move to using TCP replication	2017-04-05 09:36:21 +01:00
Erik Johnston	3a1f3f8388	Change slave storage to use new replication interface ... As the TCP replication uses a slightly different API and streams than the HTTP replication. This breaks HTTP replication.	2017-04-03 15:34:19 +01:00
Erik Johnston	9cee0ce7db	Merge pull request #2075 from matrix-org/erikj/cache_speed ... Speed up cached function access	2017-03-31 10:10:56 +01:00
Erik Johnston	350333a09a	Merge pull request #2076 from matrix-org/erikj/as_perf ... Make AS's faster	2017-03-31 09:43:10 +01:00
Erik Johnston	eefd9fee81	Fix up tests	2017-03-30 14:14:46 +01:00
Richard van der Hoff	f9b4bb05e0	Fix the logcontext handling in the cache wrappers (#2077) ... The cache wrappers had a habit of leaking the logcontext into the reactor while the lookup function was running, and then not restoring it correctly when the lookup function had completed. It's all the fault of `preserve_context_over_{fn,deferred}` which are basically a bit broken.	2017-03-30 13:22:24 +01:00
Erik Johnston	650f0e69f2	Compile the regex's used in ASes	2017-03-28 13:27:21 +01:00
Erik Johnston	57cfa513f5	Merge pull request #2054 from matrix-org/erikj/user_iter_cursor ... Reduce some CPU work on DB threads	2017-03-24 11:40:08 +00:00
Richard van der Hoff	9397edb28b	Merge pull request #2050 from matrix-org/rav/federation_backoff ... push federation retry limiter down to matrixfederationclient	2017-03-23 22:27:01 +00:00
Richard van der Hoff	06ce7335e9	Merge pull request #2052 from matrix-org/rav/time_bound_deferred ... Fix time_bound_deferred to throw the right exception	2017-03-23 22:22:54 +00:00
Erik Johnston	00957d1aa4	User Cursor.__iter__ instead of fetchall ... This prevents unnecessary construction of lists	2017-03-23 17:53:49 +00:00
pik	566641a0b5	use jsonschema.FormatChecker for RoomID and UserID strings ... * use a valid filter in rest/client/v2_alpha test Signed-off-by: pik <alexander.maznev@gmail.com>	2017-03-23 11:42:41 -03:00
pik	acafcf1c5b	Add valid filter tests, flake8, fix typo ... Signed-off-by: pik <alexander.maznev@gmail.com>	2017-03-23 11:42:10 -03:00
pik	e56c79c114	check_valid_filter using JSONSchema ... * add invalid filter tests Signed-off-by: pik <alexander.maznev@gmail.com>	2017-03-23 11:42:07 -03:00
Richard van der Hoff	0bfea9a2be	fix tests	2017-03-23 13:20:08 +00:00
Richard van der Hoff	b88a323ffb	Fix time_bound_deferred to throw the right exception ... Due to a failure to instantiate DeferredTimedOutError, time_bound_deferred would throw a CancelledError when the deferred timed out, which was rather confusing.	2017-03-23 12:07:11 +00:00
Richard van der Hoff	4bd597d9fc	push federation retry limiter down to matrixfederationclient ... rather than having to instrument everywhere we make a federation call, make the MatrixFederationHttpClient manage the retry limiter.	2017-03-23 09:28:46 +00:00
Richard van der Hoff	95f21c7a66	Fix caching of remote servers' signature keys ... The `@cached` decorator on `KeyStore._get_server_verify_key` was missing its `num_args` parameter, which meant that it was returning the wrong key for any server which had more than one recorded key. By way of a fix, change the default for `num_args` to be *all* arguments. To implement that, factor out a common base class for `CacheDescriptor` and `CacheListDescriptor`.	2017-03-22 15:11:30 +00:00
Erik Johnston	37a187bfab	Merge pull request #2033 from matrix-org/erikj/repl_speed ... Don't send the full event json over replication	2017-03-21 13:11:15 +00:00
Erik Johnston	737f283a07	Fix unit test	2017-03-20 14:03:58 +00:00
Richard van der Hoff	f40c2db05a	Stop preserve_fn leaking context into the reactor ... Fix a bug in ``logcontext.preserve_fn`` which made it leak context into the reactor, and add a test for it. Also, get rid of ``logcontext.reset_context_after_deferred``, which tried to do the same thing but had its own, different, set of bugs.	2017-03-18 00:07:43 +00:00
Will Hunt	c144292373	Modify test_user_sync so it doesn't look at last_active_ts over last_user_sync_ts	2017-03-15 15:38:57 +00:00
Erik Johnston	095b45c165	Aggregate event push actions	2017-02-14 13:39:41 +00:00
Erik Johnston	51adaac953	Fix email push in pusher worker ... This was broken when device list updates were implemented, as Mailer could no longer instantiate an AuthHandler due to a dependency on federation sending.	2017-02-02 10:53:36 +00:00
Erik Johnston	692daf6f54	Remote membership tests for replication ... This is because it now relies of the caches stream, which only works on postgres. We are trying to test with sqlite.	2017-01-31 16:10:16 +00:00
Erik Johnston	c7a26b7c32	Fix unit tests	2017-01-30 17:11:24 +00:00
Erik Johnston	b3e1f2aa7a	Fix unit tests	2017-01-26 17:16:24 +00:00
Erik Johnston	c974116f19	Implement device key caching over federation	2017-01-26 16:07:24 +00:00
Erik Johnston	2367c5568c	Add basic implementation of local device list changes	2017-01-25 14:27:27 +00:00
Erik Johnston	a55fa2047f	Insert delta of current_state_events to be more efficient	2017-01-20 17:10:18 +00:00
Erik Johnston	09eb08f910	Derive current_state_events from state groups	2017-01-20 11:52:51 +00:00
Erik Johnston	15f012032c	Merge pull request #1818 from matrix-org/erikj/state_auth_splitout_split ... Optimise state resolution	2017-01-18 10:53:00 +00:00
Erik Johnston	5d6bad1b3c	Optimise state resolution	2017-01-17 13:22:19 +00:00
Erik Johnston	9e8e236d98	Tidy up test	2017-01-17 11:50:18 +00:00
Erik Johnston	f85b6ca494	Speed up cache size calculation ... Instead of calculating the size of the cache repeatedly, which can take a long time now that it can use a callback, instead cache the size and update that on insertion and deletion. This requires changing the cache descriptors to have two caches, one for pending deferreds and the other for the actual values. There's no reason to evict from the pending deferreds as they won't take up any more memory.	2017-01-17 11:18:13 +00:00
Erik Johnston	f2f179dce2	Add ExpiringCache tests	2017-01-16 15:33:34 +00:00
Erik Johnston	2fae34bd2c	Optionally measure size of cache by sum of length of values	2017-01-13 17:46:17 +00:00
Erik Johnston	6823fe5241	Linearize updates to membership via PUT /state/	2017-01-09 18:25:13 +00:00
Marcin Bachry	24c16fc349	Fix crash in url preview when html tag has no text ... Signed-off-by: Marcin Bachry <hegel666@gmail.com>	2016-12-14 22:38:18 +01:00
Erik Johnston	d53a80af25	Merge pull request #1620 from matrix-org/erikj/concurrent_room_access ... Limit the number of events that can be created on a given room concurrently	2016-12-12 10:30:23 +00:00
Erik Johnston	8b34f71bea	Fix unit tests	2016-12-09 16:48:48 +00:00
Richard van der Hoff	1529c19675	Prevent user tokens being used as guest tokens (#1675) ... Make sure that a user cannot pretend to be a guest by adding 'guest = True' caveats.	2016-12-06 15:31:37 +00:00
Johannes Löthberg	6c9a0ba415	test_preview: Fix incorrect wrapping ... The old test expected an incorrect wrapping due to the preview function not using unicode properly, so it got the wrong length. Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>	2016-12-05 16:33:57 +01:00
Johannes Löthberg	0697bb2247	Make test_preview use unicode strings ... Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>	2016-12-05 16:33:57 +01:00
Richard van der Hoff	12f3b9000c	fix imports	2016-11-30 17:45:49 +00:00
Richard van der Hoff	aa09d6b8f0	Rip out more refresh_token code ... We might as well treat all refresh_tokens as invalid. Just return a 403 from /tokenrefresh, so that we don't have a load of dead, untestable code hanging around. Still TODO: removing the table from the schema.	2016-11-30 17:40:18 +00:00
Richard van der Hoff	dc4b23e1a1	Merge branch 'develop' into rav/no_more_refresh_tokens	2016-11-30 17:10:04 +00:00
Richard van der Hoff	1c4f05db41	Stop putting a time caveat on access tokens ... The 'time' caveat on the access tokens was something of a lie, since we weren't enforcing it; more pertinently its presence stops us ever adding useful time caveats. Let's move in the right direction by not lying in our caveats.	2016-11-29 16:49:41 +00:00
Richard van der Hoff	5c4edc83b5	Stop generating refresh tokens ... Since we're not doing refresh tokens any more, we should start killing off the dead code paths. /tokenrefresh itself is a bit of a thornier subject, since there might be apps out there using it, but we can at least not generate refresh tokens on new logins.	2016-11-28 10:13:01 +00:00
Erik Johnston	feec718265	Shuffle receipt handler around so that worker apps don't need to load it	2016-11-23 15:14:24 +00:00
Erik Johnston	ee5e8d71ac	Fix tests	2016-11-23 14:57:07 +00:00
Erik Johnston	b69f76c106	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/split_out_fed_txn	2016-11-23 11:31:53 +00:00
Erik Johnston	54fed21c04	Fix tests and flake8	2016-11-22 18:18:31 +00:00
Kegan Dougal	c3d963ac24	Review comments	2016-11-22 13:42:11 +00:00
Kegan Dougal	0a8b0eeca1	More tests	2016-11-22 09:59:27 +00:00
Kegan Dougal	53b27bbf06	Add remaining tests	2016-11-21 17:58:22 +00:00
Kegan Dougal	70a2157b64	Start adding some tests	2016-11-21 17:52:45 +00:00
Kegan Dougal	f97511a1f3	Move event_fields filtering to serialize_event ... Also make it an inclusive not exclusive filter, as the spec demands.	2016-11-21 17:42:16 +00:00
Erik Johnston	524d61bf7e	Fix tests	2016-11-21 11:53:02 +00:00
Kegan Dougal	3991b4cbdb	Clean transactions based on time. Add HttpTransactionCache tests.	2016-11-14 11:19:24 +00:00
Erik Johnston	d073cb7ead	Add Limiter: limit concurrent access to resource	2016-11-10 16:29:51 +00:00
Mark Haines	177f104432	Merge pull request #1098 from matrix-org/markjh/bearer_token ... Allow clients to supply access_tokens as headers	2016-10-25 17:33:15 +01:00
Erik Johnston	78c083f159	Merge pull request #1164 from pik/error-codes ... Clarify Error codes for GET /filter/	2016-10-19 14:26:17 +01:00
pik	e8b1d2a452	Refactor test_filter to use real DataStore ... * add tests for filter api errors	2016-10-18 12:17:38 -05:00
pik	d43b63818c	Fix MockHttpRequest always returning M_UNKNOWN errcode in testing	2016-10-14 15:46:54 -05:00
Alexander Maznev	d9350b0db8	Error codes for filters ... * add tests Signed-off-by: Alexander Maznev <alexander.maznev@gmail.com>	2016-10-14 10:18:28 -05:00
Erik Johnston	35e2cc8b52	Merge pull request #1155 from matrix-org/erikj/pluggable_pwd_auth ... Implement pluggable password auth	2016-10-12 11:41:20 +01:00
Erik Johnston	3061dac53e	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/replication_noop	2016-10-11 14:08:29 +01:00
Patrik Oldsberg	7b5546d077	rest/client/v1/register: use the correct requester in createUser ... Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>	2016-10-06 22:12:32 +02:00
Patrik Oldsberg	9bfc617791	storage/appservice: make appservice methods only relying on the cache synchronous	2016-10-06 15:24:59 +02:00
Erik Johnston	850b103b36	Implement pluggable password auth ... Allows delegating the password auth to an external module. This also moves the LDAP auth to using this system, allowing it to be removed from the synapse tree entirely in the future.	2016-10-03 10:36:40 +01:00
Erik Johnston	748d8fdc7b	Reduce DB hits for replication ... Some streams will occaisonally advance their positions without actually having any new rows to send over federation. Currently this means that the token will not advance on the workers, leading to them repeatedly sending a slightly out of date token. This in turns requires the master to hit the DB to check if there are any new rows, rather than hitting the no op logic where we check if the given token matches the current token. This commit changes the API to always return an entry if the position for a stream has changed, allowing workers to advance their tokens correctly.	2016-09-23 16:49:21 +01:00
Erik Johnston	22578545a0	Time out typing over federation	2016-09-23 14:00:52 +01:00
Mark Haines	ec609f8094	Fix unit tests	2016-09-12 10:46:02 +01:00
Mark Haines	74cbfdc7de	Fix unit tests	2016-09-06 18:30:03 +01:00
Erik Johnston	c10cb581c6	Correctly handle the difference between prev and current state	2016-08-31 14:26:22 +01:00
Erik Johnston	bc1a8b1f7a	Don't notify for online -> online transitions. ... Specifically, if currently_active remains true then we should not notify if only the last active time changes.	2016-08-30 15:05:32 +01:00
Erik Johnston	55fc17cf4b	Merge pull request #1049 from matrix-org/erikj/presence_users_in_room ... Use state handler instead of get_users_in_room/get_joined_hosts	2016-08-30 10:50:37 +01:00
Erik Johnston	93b32d4515	Fix unit tests	2016-08-26 15:40:27 +01:00
Mark Haines	4bbef62124	Merge remote-tracking branch 'origin/develop' into markjh/direct_to_device	2016-08-26 14:35:31 +01:00
Erik Johnston	3f11953fcb	Fix tests	2016-08-26 10:15:52 +01:00
Mark Haines	3b8d0ceb22	More 0_0 in tests	2016-08-25 18:42:46 +01:00
Erik Johnston	a3dc1e9cbe	Replace context.current_state with context.current_state_ids	2016-08-25 17:32:22 +01:00
Erik Johnston	45fd2c8942	Ensure invalidation list does not grow unboundedly	2016-08-19 16:09:16 +01:00
Erik Johnston	c0d7d9d642	Rename to on_invalidate	2016-08-19 15:13:58 +01:00
Erik Johnston	dc76a3e909	Make cache_context an explicit option	2016-08-19 15:02:38 +01:00
Erik Johnston	ba214a5e32	Remove lru option	2016-08-19 14:17:11 +01:00
Erik Johnston	4161ff2fc4	Add concept of cache contexts	2016-08-19 14:17:07 +01:00
Erik Johnston	9da84a9a1e	Make AppserviceHandler stream events from database ... This is for two reasons: 1. Suppresses duplicates correctly, as the notifier doesn't do any duplicate suppression. 2. Makes it easier to connect the AppserviceHandler to the replication stream.	2016-08-18 11:54:41 +01:00
Erik Johnston	320dfe523c	Make notify_interested_services faster	2016-08-17 17:20:50 +01:00
Erik Johnston	7321f45457	Clean up _ServiceQueuer	2016-08-17 12:03:04 +01:00
Erik Johnston	b835ebcc79	Update unit tests	2016-08-17 11:22:11 +01:00
Erik Johnston	48b5829aea	Fix up preview URL API. Add tests. ... This includes: - Splitting out methods of a class into stand alone functions, to make them easier to test. - Adding unit tests to split out functions, testing HTML -> preview. - Handle the fact that elements in lxml may have tail text.	2016-08-16 14:53:24 +01:00
Richard van der Hoff	a8bcc7274d	PEP8	2016-08-08 17:20:38 +01:00
Richard van der Hoff	6fe6a6f029	Fix login with m.login.token ... login with token (as used by CAS auth) was broken by 067596d, such that it always returned a 401.	2016-08-08 16:40:39 +01:00
Erik Johnston	e97648c4e2	Test summarization	2016-08-04 16:09:09 +01:00
Richard van der Hoff	98385888b8	PEP8	2016-08-03 15:42:08 +01:00
Richard van der Hoff	68264d7404	Include device name in /keys/query response ... Add an 'unsigned' section which includes the device display name.	2016-08-03 15:42:08 +01:00
Richard van der Hoff	91fa69e029	keys/query: return all users which were asked for ... In the situation where all of a user's devices get deleted, we want to indicate this to a client, so we want to return an empty dictionary, rather than nothing at all.	2016-08-03 15:41:44 +01:00
Mark Haines	0a7d3cd00f	Create separate methods for getting messages to push ... for the email and http pushers rather than trying to make a single method that will work with their conflicting requirements. The http pusher needs to get the messages in ascending stream order, and doesn't want to miss a message. The email pusher needs to get the messages in descending timestamp order, and doesn't mind if it misses messages.	2016-07-28 20:24:24 +01:00
Richard van der Hoff	eb359eced4	Add create_requester function ... Wrap the `Requester` constructor with a function which provides sensible defaults, and use it throughout	2016-07-26 16:46:53 +01:00
Richard van der Hoff	8e02494166	Delete refresh tokens when deleting devices	2016-07-26 11:10:37 +01:00
Richard van der Hoff	012b4c1913	Implement updating devices ... You can update the displayname of devices now.	2016-07-26 07:35:48 +01:00
Richard van der Hoff	436bffd15f	Implement deleting devices	2016-07-26 07:35:48 +01:00
Richard van der Hoff	bf3de7b90b	Merge pull request #945 from matrix-org/rav/background_reindex ... Create index on user_ips in the background	2016-07-25 14:04:05 +01:00
Richard van der Hoff	42f4feb2b7	PEP8	2016-07-25 12:25:06 +01:00
Richard van der Hoff	f16f0e169d	Slightly saner logging for unittests ... 1. Give the handler used for logging in unit tests a formatter, so that the output is slightly more meaningful 2. Log some synapse.storage stuff, because it's useful.	2016-07-25 12:12:47 +01:00
Richard van der Hoff	465117d7ca	Fix background_update tests ... A bit of a cleanup for background_updates, and make sure that the real background updates have run before we start the unit tests, so that they don't interfere with the tests.	2016-07-25 12:10:42 +01:00
Richard van der Hoff	55abbe1850	make /devices return a list ... Turns out I specced this to return a list of devices rather than a dict of them	2016-07-21 15:57:28 +01:00
Richard van der Hoff	1c3c202b96	Fix PEP8 errors	2016-07-21 13:15:15 +01:00
Richard van der Hoff	406f7aa0f6	Implement GET /device/{deviceId}	2016-07-21 12:00:29 +01:00
Richard van der Hoff	40a1c96617	Fix PEP8 errors	2016-07-20 18:06:28 +01:00
Richard van der Hoff	7314bf4682	Merge branch 'develop' into rav/get_devices_api ... (pick up PR #938 in the hope of fixing the UTs)	2016-07-20 17:40:00 +01:00
Richard van der Hoff	e9e3eaa67d	Merge pull request #938 from matrix-org/rav/add_device_id_to_client_ips ... Record device_id in client_ips	2016-07-20 17:38:45 +01:00
Richard van der Hoff	bc8f265f0a	GET /devices endpoint ... implement a GET /devices endpoint which lists all of the user's devices. It also returns the last IP where we saw that device, so there is some dancing to fish that out of the user_ips table.	2016-07-20 16:42:32 +01:00
Richard van der Hoff	ec041b335e	Record device_id in client_ips ... Record the device_id when we add a client ip; it's somewhat redundant as we could get it via the access_token, but it will make querying rather easier.	2016-07-20 16:41:03 +01:00
Richard van der Hoff	b97a1356b1	Register a device_id in the /v2/register flow. ... This doesn't cover *all* of the registration flows, but it does cover the most common ones: in particular: shared_secret registration, appservice registration, and normal user/pass registration. Pull device_id from the registration parameters. Register the device in the devices table. Associate the device with the returned access and refresh tokens. Profit.	2016-07-20 16:38:27 +01:00
David Baker	e967bc86e7	Merge pull request #932 from matrix-org/rav/register_refactor ... Further registration refactoring	2016-07-20 11:03:33 +01:00
Erik Johnston	1e2a7f18a1	Merge pull request #922 from matrix-org/erikj/file_api2 ... Feature: Add filter to /messages. Add 'contains_url' to filter.	2016-07-20 10:40:48 +01:00
Richard van der Hoff	40cbffb2d2	Further registration refactoring ... * `RegistrationHandler.appservice_register` no longer issues an access token: instead it is left for the caller to do it. (There are two of these, one in `synapse/rest/client/v1/register.py`, which now simply calls `AuthHandler.issue_access_token`, and the other in `synapse/rest/client/v2_alpha/register.py`, which is covered below). * In `synapse/rest/client/v2_alpha/register.py`, move the generation of access_tokens into `_create_registration_details`. This means that the normal flow no longer needs to call `AuthHandler.issue_access_token`; the shared-secret flow can tell `RegistrationHandler.register` not to generate a token; and the appservice flow continues to work despite the above change.	2016-07-19 18:46:19 +01:00
David Baker	b9e997f561	Merge pull request #931 from matrix-org/rav/refactor_register ... rest/client/v2_alpha/register.py: Refactor flow somewhat.	2016-07-19 16:13:45 +01:00
Richard van der Hoff	0da0d0a29d	rest/client/v2_alpha/register.py: Refactor flow somewhat. ... This is meant to be an *almost* non-functional change, with the exception that it fixes what looks a lot like a bug in that it only calls `auth_handler.add_threepid` and `add_pusher` once instead of three times. The idea is to move the generation of the `access_token` out of `registration_handler.register`, because `access_token`s now require a device_id, and we only want to generate a device_id once registration has been successful.	2016-07-19 13:12:22 +01:00
Richard van der Hoff	f863a52cea	Add device_id support to /login ... Add a 'devices' table to the storage, as well as a 'device_id' column to refresh_tokens. Allow the client to pass a device_id, and initial_device_display_name, to /login. If login is successful, then register the device in the devices table if it wasn't known already. If no device_id was supplied, make one up. Associate the device_id with the access token and refresh token, so that we can get at it again later. Ensure that the device_id is copied from the refresh token to the access_token when the token is refreshed.	2016-07-18 16:39:44 +01:00
Erik Johnston	a98d215204	Add filter param to /messages API	2016-07-14 16:30:56 +01:00
Negar Fazeli	0136a522b1	Bug fix: expire invalid access tokens	2016-07-13 15:00:37 +02:00