synapse-product

mirror of https://git.anonymousland.org/anonymousland/synapse-product.git synced 2024-12-25 16:39:29 -05:00

Author	SHA1	Message	Date
Travis Ralston	dba84fa69c	Fix terms UI auth test	2018-10-18 12:45:21 -06:00
Travis Ralston	88c5ffec33	Test for terms UI auth	2018-10-18 12:35:30 -06:00
Will Hunt	d6a7797dd1	Fix roomlist since tokens on Python 3 (#4046 ) Thanks @Half-Shot !!!	2018-10-17 23:04:55 +11:00
Erik Johnston	947c7443eb	Add some state res v2 tests	2018-10-16 16:28:39 +01:00
Neil Johnson	a2bfb778c8	improve auto room join logic, comments and tests	2018-10-12 18:17:36 +01:00
David Baker	dc045ef202	Merge remote-tracking branch 'origin/develop' into dbkr/e2e_backups	2018-10-09 10:05:02 +01:00
Neil Johnson	ed82043efb	Merge branch 'develop' into matthew/autocreate_autojoin	2018-10-04 17:26:59 +01:00
Neil Johnson	2dadc092b8	move logic into register, fix room alias localpart bug, tests	2018-10-04 17:00:27 +01:00
Erik Johnston	9c834b8ee9	Add tests	2018-10-02 16:22:39 +01:00
Amber Brown	7232917f12	Disable frozen dicts by default (#3987 )	2018-10-02 22:53:47 +10:00
Richard van der Hoff	b5b93f45d5	Merge pull request #3968 from matrix-org/rav/fix_federation_errors Fix exceptions when handling incoming transactions	2018-10-01 15:54:24 +01:00
Amber Brown	6e05fd032c	Fix userconsent on Python 3 (#3938 )	2018-10-02 00:11:58 +10:00
Richard van der Hoff	f094f715cf	Merge remote-tracking branch 'origin/develop' into rav/fix_federation_errors	2018-09-27 15:18:21 +01:00
Amber Brown	a512e637ac	Merge pull request #3970 from schnuffle/develop-py3 Replaced all occurences of e.message with str(e)	2018-09-27 23:39:45 +10:00
Richard van der Hoff	484a9b8c81	Remove redundant, failing, test This test didn't do what it claimed to do, and what it claimed to do was the same as test_cant_hide_direct_ancestors anyway. This stuff is tested by sytest anyway.	2018-09-27 13:11:23 +01:00
Schnuffle	dc5db01ff2	Replaced all occurences of e.message with str(e) Signed-off-by: Schnuffle <schnuffle@github.com>	2018-09-27 13:38:50 +02:00
Richard van der Hoff	4a15a3e4d5	Include eventid in log lines when processing incoming federation transactions (#3959 ) when processing incoming transactions, it can be hard to see what's going on, because we process a bunch of stuff in parallel, and because we may end up recursively working our way through a chain of three or four events. This commit creates a way to use logcontexts to add the relevant event ids to the log lines.	2018-09-27 11:25:34 +01:00
Richard van der Hoff	ae6ad4cf41	docstrings and unittests for storage.state (#3958 ) I spent ages trying to figure out how I was going mad...	2018-09-27 11:22:25 +01:00
Richard van der Hoff	4c3e7eeec5	Merge pull request #3932 from matrix-org/erikj/auto_start_expiring_caches Fix some instances of ExpiringCache not expiring cache items	2018-09-25 12:02:57 +01:00
Amber Brown	fbe5ba25f6	Merge branch 'master' into develop	2018-09-25 03:10:01 +10:00
Oleg Girko	7d3f639844	Fix compatibility issue with older Twisted in tests. Older Twisted (18.4.0) returns TimeoutError instead of ConnectingCancelledError when connection times out. This change allows tests to be compatible with this behaviour. Signed-off-by: Oleg Girko <ol@infoserver.lv>	2018-09-25 02:51:18 +10:00
Erik Johnston	8601c24287	Fix some instances of ExpiringCache not expiring cache items ExpiringCache required that `start()` be called before it would actually start expiring entries. A number of places didn't do that. This PR removes `start` from ExpiringCache, and automatically starts backround reaping process on creation instead.	2018-09-21 14:19:46 +01:00
Amber Brown	1f3f5fcf52	Fix client IPs being broken on Python 3 (#3908 )	2018-09-20 20:14:34 +10:00
Amber Brown	aeca5a5ed5	Add a regression test for logging on failed connections (#3912 )	2018-09-20 16:28:18 +10:00
Richard van der Hoff	31c15dcb80	Refactor matrixfederationclient to fix logging (#3906 ) We want to wait until we have read the response body before we log the request as complete, otherwise a confusing thing happens where the request appears to have completed, but we later fail it. To do this, we factor the salient details of a request out to a separate object, which can then keep track of the txn_id, so that it can be logged.	2018-09-18 18:17:15 +01:00
Erik Johnston	f6e82dcddb	Tests	2018-09-14 15:08:37 +01:00
Amber Brown	bfa0b759e0	Attempt to figure out what's going on with timeouts (#3857 )	2018-09-14 00:15:51 +10:00
Neil Johnson	8decd6233d	improve naming	2018-09-12 16:22:15 +01:00
Neil Johnson	0ddf486724	expose number of real reserved users	2018-09-12 11:58:52 +01:00
Amber Brown	52ec6e9dfa	Port tests/ to Python 3 (#3808 )	2018-09-07 02:58:18 +10:00
Neil Johnson	c5440b2ca0	Merge pull request #3800 from matrix-org/neilj/remove-guests-from-mau-count guest users should not be part of mau total	2018-09-06 17:45:55 +01:00
Neil Johnson	84a750e0c3	ensure guests never enter mau list	2018-09-06 17:22:53 +01:00
Hubert Chathi	3801b8aa03	try to make flake8 and isort happy	2018-09-06 11:35:19 -04:00
Richard van der Hoff	4f8baab0c4	Merge branch 'master' into develop	2018-09-06 13:05:22 +01:00
Neil Johnson	61b05727fa	guest users should not be part of mau total	2018-09-05 22:30:36 +01:00
Richard van der Hoff	c127c8d042	Fix origin handling for pushed transactions Use the actual origin for push transactions, rather than whatever the remote server claimed.	2018-09-05 13:08:07 +01:00
Amber Brown	77055dba92	Fix tests on postgresql (#3740 )	2018-09-04 02:21:48 +10:00
Neil Johnson	0b01281e77	move threepid checker to config, add missing yields	2018-08-31 17:11:11 +01:00
Neil Johnson	ea068d6f3c	fix bug where preserved threepid user comes to sign up and server is mau blocked	2018-08-31 10:49:14 +01:00
Amber Brown	14e4d4f4bf	Port storage/ to Python 3 (#3725 )	2018-08-31 00:19:58 +10:00
Richard van der Hoff	414fa36f3e	Fix up tests	2018-08-28 17:21:05 +01:00
Hubert Chathi	16a31c6fce	update to newer Synapse APIs	2018-08-24 22:51:25 -04:00
Erik Johnston	9db2476991	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/admin_contact	2018-08-24 17:00:37 +01:00
Erik Johnston	05077e06fa	Change admin_uri to admin_contact in config and errors	2018-08-24 16:51:27 +01:00
Hubert Chathi	83caead95a	Merge branch 'develop' into e2e_backups	2018-08-24 11:44:26 -04:00
Erik Johnston	01a5a8b9e3	Fix checking if service notice room is already tagged This manifested in synapse repeatedly setting the tag for the room	2018-08-24 16:22:37 +01:00
Erik Johnston	f2cbbda956	Unit tests	2018-08-23 19:17:19 +01:00
Erik Johnston	cd77270a66	Implement trail users	2018-08-23 19:17:19 +01:00
Erik Johnston	7e6e588e60	Fix bug where we resent "limit exceeded" server notices This was due to a bug where we mutated a cached event's contents	2018-08-23 16:21:20 +01:00
Erik Johnston	c5842dff1a	Actually run the tests	2018-08-23 10:35:54 +01:00
Erik Johnston	fd2dbf1836	Merge branch 'develop' of github.com:matrix-org/synapse into neilj/server_notices_on_blocking	2018-08-22 17:06:10 +01:00
Erik Johnston	9643a6f7f2	Update notice format	2018-08-22 17:00:29 +01:00
Erik Johnston	764030cf63	Merge pull request #3659 from matrix-org/erikj/split_profiles Allow profile updates to happen on workers	2018-08-22 11:35:55 +01:00
Erik Johnston	3bf8bab8f9	Merge pull request #3673 from matrix-org/erikj/refactor_state_handler Refactor state module to support multiple room versions	2018-08-22 10:04:55 +01:00
Matthew Hodgson	bb81e78ec6	Split the state_group_cache in two (#3726 ) Splits the state_group_cache in two. One half contains normal state events; the other contains member events. The idea is that the lazyloading common case of: "I want a subset of member events plus all of the other state" can be accomplished efficiently by splitting the cache into two, and asking for "all events" from the non-members cache, and "just these keys" from the members cache. This means we can avoid having to make DictionaryCache aware of these sort of complicated queries, whilst letting LL requests benefit from the caching. Previously we were unable to sensibly use the caching and had to pull all state from the DB irrespective of the filtering, which made things slow. Hopefully fixes https://github.com/matrix-org/synapse/issues/3720.	2018-08-22 00:56:37 +02:00
Erik Johnston	4d664278af	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/refactor_state_handler	2018-08-20 14:49:43 +01:00
Neil Johnson	e07970165f	rename error code	2018-08-18 14:39:45 +01:00
Neil Johnson	ba1fbf7d5b	special case server_notices_mxid	2018-08-18 12:31:08 +01:00
Amber Brown	c334ca67bb	Integrate presence from hotfixes (#3694 )	2018-08-18 01:08:45 +10:00
Neil Johnson	d49b77404b	clean up, no functional changes	2018-08-17 15:21:34 +01:00
Richard van der Hoff	63260397c6	Merge pull request #3701 from matrix-org/rav/use_producer_for_responses Use a producer to stream back responses	2018-08-17 14:58:45 +01:00
Neil Johnson	3ee57bdcbb	Merge branch 'develop' of github.com:matrix-org/synapse into neilj/server_notices_on_blocking	2018-08-17 14:34:10 +01:00
Erik Johnston	782689bd40	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/split_profiles	2018-08-17 14:15:48 +01:00
Erik Johnston	ca87ad1def	Split ProfileHandler into master and worker	2018-08-17 14:15:14 +01:00
Neil Johnson	b5f638f1f4	Merge branch 'develop' of github.com:matrix-org/synapse into neilj/server_notices_on_blocking	2018-08-17 14:04:15 +01:00
Neil Johnson	69c49d3fa3	Merge branch 'develop' into neilj/limit_exceeded_error	2018-08-17 12:44:26 +00:00
Neil Johnson	bcfeb44afe	call reap on start up and fix under reaping bug	2018-08-16 22:55:32 +01:00
Neil Johnson	13ad9930c8	add new error type ResourceLimit	2018-08-16 18:02:02 +01:00
Neil Johnson	3c1080b6e4	refactor for readability, and reuse caching for setting tags	2018-08-16 17:02:04 +01:00
Neil Johnson	a675f9c556	check for room state before deciding on action	2018-08-16 14:53:35 +01:00
Neil Johnson	8cfad2e686	Merge branch 'develop' of github.com:matrix-org/synapse into neilj/server_notices_on_blocking	2018-08-15 17:20:38 +01:00
Neil Johnson	81d727efa9	Merge pull request #3689 from matrix-org/neilj/fix_off_by_1+maus Fix Mau off by one errors	2018-08-15 16:19:41 +00:00
Matthew Hodgson	2f78f432c4	speed up /members and add at= and membership params (#3568 )	2018-08-15 16:35:22 +01:00
Neil Johnson	fc5d937550	Merge branch 'develop' of github.com:matrix-org/synapse into neilj/server_notices_on_blocking	2018-08-15 16:31:40 +01:00
Neil Johnson	86a00e05e1	Merge branch 'develop' of github.com:matrix-org/synapse into neilj/fix_off_by_1+maus	2018-08-15 16:27:08 +01:00
Amber Brown	a87af25fbb	Fix the tests	2018-08-15 15:12:23 +01:00
Neil Johnson	75c663c7b9	update error codes	2018-08-15 11:27:48 +01:00
Neil Johnson	b8429c7c81	update error codes for resource limiting	2018-08-15 10:19:48 +01:00
Neil Johnson	ab035bdeac	replace admin_email with admin_uri for greater flexibility	2018-08-15 10:16:41 +01:00
Neil Johnson	19b433e3f4	Merge branch 'develop' of github.com:matrix-org/synapse into neilj/admin_email	2018-08-14 17:44:46 +01:00
Neil Johnson	b586b8b986	Merge branch 'develop' of github.com:matrix-org/synapse into neilj/fix_off_by_1+maus	2018-08-14 17:43:22 +01:00
Neil Johnson	7277216d01	fix setup_test_homeserver to be postgres compatible	2018-08-14 17:14:39 +01:00
Neil Johnson	1522ed9c07	in case max_mau is less than I think	2018-08-14 16:52:30 +01:00
Neil Johnson	414d54b61a	Merge pull request #3670 from matrix-org/neilj/mau_sync_block Block ability to read via sync if mau limit exceeded	2018-08-14 15:21:31 +00:00
Neil Johnson	06b331ff40	fix off by 1 errors	2018-08-14 15:28:15 +01:00
Neil Johnson	8f9a7eb58d	support admin_email config and pass through into blocking errors, return AuthError in all cases	2018-08-14 15:11:54 +01:00
Neil Johnson	ed4bc3d2fc	fix off by 1s on mau	2018-08-14 15:04:48 +01:00
Neil Johnson	9b5bf3d858	Merge branch 'neilj/admin_email' of github.com:matrix-org/synapse into neilj/fix_off_by_1+maus	2018-08-14 14:51:38 +01:00
Neil Johnson	e25d87d97b	Merge branch 'neilj/mau_sync_block' of github.com:matrix-org/synapse into neilj/fix_off_by_1+maus	2018-08-14 14:32:18 +01:00
Amber Brown	bdfbd934d6	Implement a new test baseclass to cut down on boilerplate (#3684 )	2018-08-14 20:53:43 +10:00
Neil Johnson	f4b49152e2	support admin_email config and pass through into blocking errors, return AuthError in all cases	2018-08-13 21:09:47 +01:00
Neil Johnson	ce7de9ae6b	Revert "support admin_email config and pass through into blocking errors, return AuthError in all cases" This reverts commit `0d43f991a1`.	2018-08-13 18:06:18 +01:00
Neil Johnson	0d43f991a1	support admin_email config and pass through into blocking errors, return AuthError in all cases	2018-08-13 18:00:23 +01:00
Amber Brown	99dd975dae	Run tests under PostgreSQL (#3423 )	2018-08-13 16:47:46 +10:00
Matthew Hodgson	66a4ca1d28	404 nicely if you try to interact with a missing current version	2018-08-12 19:14:31 -04:00
Matthew Hodgson	edc427a351	flake8	2018-08-12 19:14:31 -04:00
Matthew Hodgson	fe87890b18	implement remaining tests and make them work	2018-08-12 19:14:31 -04:00
Matthew Hodgson	f6a3067868	linting	2018-08-12 19:14:31 -04:00
Matthew Hodgson	15d513f16f	fix idiocies and so make tests pass	2018-08-12 19:14:31 -04:00
Matthew Hodgson	174be586e5	first cut at a UT	2018-08-12 19:14:31 -04:00
Neil Johnson	6c6aba76e1	implementation of server notices to alert on hitting resource limits	2018-08-10 15:12:59 +01:00
Amber Brown	807449d8f2	fix up a forced long line	2018-08-11 00:01:43 +10:00
black	8b3d9b6b19	Run black.	2018-08-10 23:54:09 +10:00
Amber Brown	b37c472419	Rename async to async_helpers because `async` is a keyword on Python 3.7 (#3678 )	2018-08-10 23:50:21 +10:00
Richard van der Hoff	3c0213a217	Merge pull request #3439 from vojeroen/send_sni_for_federation_requests send SNI for federation requests	2018-08-10 12:23:54 +01:00
Neil Johnson	885ea9c602	rename _user_last_seen_monthly_active	2018-08-09 18:02:12 +01:00
Neil Johnson	04df714259	fix imports	2018-08-09 17:41:52 +01:00
Neil Johnson	09cf130898	only block on sync where user is not part of the mau cohort	2018-08-09 17:39:12 +01:00
Erik Johnston	3e19beb941	Fix tests	2018-08-09 14:58:49 +01:00
Neil Johnson	69ce057ea6	block sync if auth checks fail	2018-08-09 12:26:27 +01:00
Neil Johnson	3dce9050cf	Merge branch 'develop' of github.com:matrix-org/synapse into neilj/mau_sync_block	2018-08-09 11:42:02 +01:00
Neil Johnson	0ad98e38d0	Merge pull request #3655 from matrix-org/neilj/disable_hs Flag to disable HS without disabling federation	2018-08-09 10:41:43 +00:00
Neil Johnson	a5ef110749	Merge branch 'develop' of github.com:matrix-org/synapse into neilj/mau_sync_block	2018-08-09 11:40:37 +01:00
Amber Brown	2511f3f8a0	Test fixes for Python 3 (#3647 )	2018-08-09 12:22:01 +10:00
Neil Johnson	e92fb00f32	sync auth blocking	2018-08-08 17:54:49 +01:00
Neil Johnson	5298d79fb5	Merge branch 'develop' into neilj/disable_hs	2018-08-08 16:13:03 +00:00
Richard van der Hoff	8521ae13e3	Merge pull request #3654 from matrix-org/rav/room_versions Support for room versioning	2018-08-08 17:10:53 +01:00
Neil Johnson	d2f3ef98ac	Merge branch 'develop' into neilj/disable_hs	2018-08-08 15:55:47 +00:00
Neil Johnson	7f3d897e7a	mock config.max_mau_value	2018-08-08 11:46:23 +01:00
Neil Johnson	e8eba2b4e3	implement reserved users for mau limits	2018-08-07 17:49:43 +01:00
Neil Johnson	fbe255f9a4	add default mau_limits_reserved_threepids	2018-08-06 23:24:54 +01:00
Neil Johnson	42c6823827	disable HS from config	2018-08-04 22:07:04 +01:00
Neil Johnson	d08296f9f2	remove unused import	2018-08-03 23:00:16 +01:00
Neil Johnson	886be75ad1	bug fixes	2018-08-03 22:29:03 +01:00
Neil Johnson	e10830e976	wip commit - tests failing	2018-08-03 17:55:50 +01:00
Richard van der Hoff	0ca459ea33	Basic support for room versioning This is the first tranche of support for room versioning. It includes: * setting the default room version in the config file * new room_version param on the createRoom API * storing the version of newly-created rooms in the m.room.create event * fishing the version of existing rooms out of the m.room.create event	2018-08-03 16:08:32 +01:00
Neil Johnson	5593ff6773	fix (lots of) py3 test failures	2018-08-03 14:59:17 +01:00
Neil Johnson	b2aab04d2c	fix py3 test failure	2018-08-03 14:12:56 +01:00
Neil Johnson	950807d93a	fix caching and tests	2018-08-03 13:49:53 +01:00
Neil Johnson	74b1d46ad9	do mau checks based on monthly_active_users table	2018-08-02 16:57:35 +01:00
Neil Johnson	9180061b49	remove unused count_monthly_users	2018-08-02 15:55:29 +01:00
Neil Johnson	c4ffbecb68	fix test, update constructor call	2018-08-02 13:51:05 +01:00
Neil Johnson	00f99f74b1	insertion into monthly_active_users	2018-08-02 13:47:19 +01:00
Neil Johnson	165e067033	Revert "change monthly_active_users table to be a single column" This reverts commit `ec716a35b2`.	2018-08-02 10:59:58 +01:00
Neil Johnson	ec716a35b2	change monthly_active_users table to be a single column	2018-08-01 17:54:37 +01:00
Neil Johnson	d766f26de9	Merge branch 'develop' of github.com:matrix-org/synapse into neilj/mau_tracker	2018-08-01 17:49:41 +01:00
Neil Johnson	085435e13a	Merge pull request #3630 from matrix-org/neilj/mau_sign_in_log_in_limits Initial impl of capping MAU	2018-08-01 15:58:45 +00:00
Amber Brown	da7785147d	Python 3: Convert some unicode/bytes uses (#3569 )	2018-08-02 00:54:06 +10:00
Neil Johnson	c480c4c962	fix isort	2018-08-01 14:25:58 +01:00
Neil Johnson	6eed16d8a2	fix test for py3	2018-08-01 14:02:10 +01:00
Neil Johnson	303f1c851f	Merge branch 'develop' of github.com:matrix-org/synapse into neilj/mau_sign_in_log_in_limits	2018-08-01 13:42:50 +01:00
Neil Johnson	4e6e00152c	fix known broken test	2018-08-01 11:48:37 +01:00
Richard van der Hoff	cab782c17e	Merge pull request #3384 from matrix-org/rav/rewrite_cachedlist_decorator Rewrite cache list decorator	2018-08-01 10:28:56 +01:00
Neil Johnson	7931393495	make count_monthly_users async synapse/handlers/auth.py	2018-08-01 10:21:56 +01:00
Neil Johnson	6ef983ce5c	api into monthly_active_users table	2018-07-31 16:36:24 +01:00
Neil Johnson	df2235e7fa	coding style	2018-07-31 13:16:20 +01:00
Neil Johnson	7d05406a07	fix user_ips counting	2018-07-31 12:03:23 +01:00
Travis Ralston	e908b86832	Remove pdu_failures from transactions The field is never read from, and all the opportunities given to populate it are not utilized. It should be very safe to remove this.	2018-07-30 16:28:47 -06:00
Neil Johnson	cefac79c10	monthly_active_tests	2018-07-30 22:08:09 +01:00
Neil Johnson	251e6c1210	limit register and sign in on number of monthly users	2018-07-30 15:55:57 +01:00
Richard van der Hoff	a8cbce0ced	fix invalidation	2018-07-27 16:17:17 +01:00
Richard van der Hoff	f102c05856	Rewrite cache list decorator Because it was complicated and annoyed me. I suspect this will be more efficient too.	2018-07-27 13:47:04 +01:00
Richard van der Hoff	7041cd872b	Merge branch 'develop' into send_sni_for_federation_requests	2018-07-27 09:17:11 +01:00
Matthew Hodgson	2565804030	Merge branch 'develop' into matthew/filter_members	2018-07-25 17:27:49 +01:00
Matthew Hodgson	0620d27f4d	flake8	2018-07-25 17:21:17 +01:00
Matthew Hodgson	0a7ee0ab8b	add tests for _get_some_state_from_cache	2018-07-25 16:33:52 +01:00
Richard van der Hoff	f59be4eb0e	Fix unit tests on_notifier_poke no longer runs synchonously, so we have to do a different hack to make sure that the replication data has been sent. Let's actually listen for its arrival.	2018-07-25 10:30:36 +01:00
Matthew Hodgson	cd241d6bda	incorporate more review	2018-07-24 12:39:40 +01:00
Matthew Hodgson	adfe29ec0b	Merge branch 'develop' into matthew/filter_members	2018-07-23 19:21:37 +01:00
Erik Johnston	8fbe418777	Fix unit tests	2018-07-23 13:33:49 +01:00
Amber Brown	3132b89f12	Make the rest of the .iterwhatever go away (#3562 )	2018-07-21 15:47:18 +10:00
Richard van der Hoff	3d6df84658	Test and fix support for cancellation in Linearizer	2018-07-20 13:59:55 +01:00
Richard van der Hoff	4f67623674	Merge pull request #3571 from matrix-org/rav/limiter_fixes A set of improvements to the Limiter	2018-07-20 13:53:03 +01:00
Amber Brown	e1a237eaab	Admin API for creating new users (#3415 )	2018-07-20 22:41:13 +10:00
Richard van der Hoff	7c712f95bb	Combine Limiter and Linearizer Linearizer was effectively a Limiter with max_count=1, so rather than maintaining two sets of code, let's combine them.	2018-07-20 13:11:43 +01:00
Richard van der Hoff	d7275eecf3	Add a sleep to the Limiter to fix stack overflows. Fixes #3570	2018-07-20 12:37:12 +01:00
Matthew Hodgson	650daf5628	make test work	2018-07-19 20:49:44 +01:00
Matthew Hodgson	1fa4f7e03e	first cut of a UT for testing state store (untested)	2018-07-19 20:19:32 +01:00
Richard van der Hoff	c754e006f4	Merge pull request #3556 from matrix-org/rav/background_processes Run things as background processes	2018-07-19 11:04:18 +01:00
Amber Brown	a97c845271	Move v1-only APIs into their own module & isolate deprecated ones (#3460 )	2018-07-19 20:03:33 +10:00
Richard van der Hoff	8c69b735e3	Make Distributor run its processes as a background process This is more involved than it might otherwise be, because the current implementation just drops its logcontexts and runs everything in the sentinel context. It turns out that we aren't actually using a bunch of the functionality here (notably suppress_failures and the fact that Distributor.fire returns a deferred), so the easiest way to fix this is actually by simplifying a bunch of code.	2018-07-18 20:55:05 +01:00
Richard van der Hoff	d897be6a98	Fix visibility of events from erased users over federation	2018-07-17 14:02:07 +01:00
Richard van der Hoff	9c04b4abf9	Merge pull request #3541 from matrix-org/rav/optimize_filter_events_for_server Refactor and optimze filter_events_for_server	2018-07-17 14:01:39 +01:00
Richard van der Hoff	94440ae994	fix imports	2018-07-17 11:51:26 +01:00
Amber Brown	bc006b3c9d	Refactor REST API tests to use explicit reactors (#3351 )	2018-07-17 20:43:18 +10:00
Erik Johnston	850238b4ef	Add unit test	2018-07-17 10:59:02 +01:00
Amber Brown	3fe0938b76	Merge pull request #3530 from matrix-org/erikj/stream_cache Don't return unknown entities in get_entities_changed	2018-07-17 13:44:46 +10:00
Richard van der Hoff	15b13b537f	Add a test which profiles filter_events_for_server in a large room	2018-07-16 14:06:09 +01:00
Richard van der Hoff	ea69d35651	Move filter_events_for_server out of FederationHandler for easier unit testing.	2018-07-16 13:06:24 +01:00
Jeroen	505530f36a	Merge remote-tracking branch 'upstream/develop' into send_sni_for_federation_requests # Conflicts: # synapse/crypto/context_factory.py	2018-07-14 20:24:46 +02:00
Amber Brown	33b60c01b5	Make auth & transactions more testable (#3499 )	2018-07-14 07:34:49 +10:00
Erik Johnston	bc832f822f	Fixup unit test	2018-07-13 17:03:04 +01:00
Richard van der Hoff	3b391d9c45	Fix unit tests	2018-07-13 16:28:04 +01:00
Oleg Girko	6c1ec5a1bd	Use more portable syntax using attrs package. Newer syntax attr.ib(factory=dict) is just a syntactic sugar for attr.ib(default=attr.Factory(dict)) It was introduced in newest version of attrs package (18.1.0) and doesn't work with older versions. We should either require minimum version of attrs to be 18.1.0, or use older (slightly more verbose) syntax. Requiring newest version is not a good solution because Linux distributions may have older version of attrs (17.4.0 in Fedora 28), and requiring to build (and package) newer version just to use newer syntactic sugar in only one test is just too much. It's much better to fix that test to use older syntax. Signed-off-by: Oleg Girko <ol@infoserver.lv>	2018-07-10 00:38:49 +01:00
Jeroen	b5e157d895	Merge branch 'develop' into send_sni_for_federation_requests # Conflicts: # synapse/http/endpoint.py	2018-07-09 08:51:11 +02:00
Amber Brown	49af402019	run isort	2018-07-09 16:09:20 +10:00
Neil Johnson	feef8461d1	Merge remote-tracking branch 'hera/rav/server_acls' into develop	2018-07-05 11:04:01 +01:00
Erik Johnston	1a88640677	Merge pull request #3483 from matrix-org/rav/more_server_name_validation More server_name validation	2018-07-05 10:04:20 +01:00
Richard van der Hoff	3cf3e08a97	Implementation of server_acls ... as described at https://docs.google.com/document/d/1EttUVzjc2DWe2ciw4XPtNpUpIl9lWXGEsy2ewDS7rtw.	2018-07-04 19:06:20 +01:00
Richard van der Hoff	546bc9e28b	More server_name validation We need to do a bit more validation when we get a server name, but don't want to be re-doing it all over the shop, so factor out a separate parse_and_validate_server_name, and do the extra validation. Also, use it to verify the server name in the config file.	2018-07-04 18:59:51 +01:00
Richard van der Hoff	ea555d5633	Reinstate lost run_on_reactor in unit test `a61738b` removed a call to run_on_reactor from a unit test, but that call was doing something useful, in making the function in question asynchronous. Reinstate the call and add a check that we are testing what we wanted to be testing.	2018-07-04 09:40:01 +01:00
Richard van der Hoff	508196e08a	Reject invalid server names (#3480 ) Make sure that server_names used in auth headers are sane, and reject them with a sensible error code, before they disappear off into the depths of the system.	2018-07-03 14:36:14 +01:00
Erik Johnston	e3b4043800	Merge pull request #3456 from matrix-org/hawkowl/federation-prevevent-checking Check the state of prev_events a bit more thoroughly when coming over federation	2018-06-29 13:55:02 +01:00
Matthew Hodgson	f82cf3c7df	add test	2018-06-28 21:14:16 +01:00
Matthew Hodgson	e72234f6bd	fix tests	2018-06-28 20:56:07 +01:00
Amber Brown	f03a5d1a17	pep8	2018-06-27 11:38:14 +01:00
Amber Brown	94f09618e5	cleanups	2018-06-27 11:38:03 +01:00
Amber Brown	35cc3e8b14	stylistic cleanup	2018-06-27 11:32:09 +01:00
Amber Brown	8d62baa48c	cleanups	2018-06-27 11:31:48 +01:00
Amber Brown	77078d6c8e	handle federation not telling us about prev_events	2018-06-27 11:27:32 +01:00
Amber Brown	cd6bcdaf87	Better testing framework for homeserver-using things (#3446 )	2018-06-27 10:37:24 +01:00
Jeroen	3d605853c8	send SNI for federation requests	2018-06-24 22:38:43 +02:00
Erik Johnston	f79abda87f	Merge pull request #3427 from matrix-org/erikj/remove_filters remove dead filter_events_for_clients	2018-06-22 14:45:42 +01:00
Richard van der Hoff	43e02c409d	Disable partial state group caching for wildcard lookups When _get_state_for_groups is given a wildcard filter, just do a complete lookup. Hopefully this will give us the best of both worlds by not filling up the ram if we only need one or two keys, but also making the cache still work for the federation reader usecase.	2018-06-22 11:52:07 +01:00
Amber Brown	77ac14b960	Pass around the reactor explicitly (#3385 )	2018-06-22 09:37:10 +01:00
Richard van der Hoff	9a793f861c	Merge branch 'master' into develop	2018-06-14 16:36:01 +01:00
Richard van der Hoff	1e77ac66e3	Fix broken unit test We need power levels for this test to do what it is supposed to do.	2018-06-14 14:21:29 +01:00
Richard van der Hoff	a502cfec00	remove spurious debug	2018-06-14 14:20:53 +01:00
Richard van der Hoff	5c9afd6f80	Make default state_default 50 Make it so that, before there is a power-levels event in the room, you need a power level of at least 50 to send state. Partially addresses https://github.com/matrix-org/matrix-doc/issues/1192	2018-06-14 12:38:09 +01:00
Amber Brown	a61738b316	Remove run_on_reactor (#3395 )	2018-06-14 18:27:37 +10:00
Richard van der Hoff	bd348f0af6	remove dead filter_events_for_clients This is only used by filter_events_for_client, so we can simplify the whole thing by just doing one user at a time, and removing a dead storage function to boot.	2018-06-12 09:51:31 +01:00
Amber Brown	f7869f8f8b	Port to sortedcontainers (with tests!) (#3332 )	2018-06-06 00:13:57 +10:00
Amber Brown	5dbf305444	Put python's logs into Trial when running unit tests (#3319 )	2018-06-04 16:06:06 +10:00
Richard van der Hoff	a995fdae39	fix tests	2018-05-29 20:19:29 +01:00
Matthew Hodgson	adb6bac4d5	fix another dumb typo	2018-05-29 02:29:22 +01:00
Matthew Hodgson	284e36ad04	fix dumb typo	2018-05-29 02:23:11 +01:00
Amber Brown	a8990fa2ec	Merge remote-tracking branch 'origin/develop' into 3218-official-prom	2018-05-22 10:50:26 -05:00
Richard van der Hoff	a5e2941aad	Reject attempts to send event before privacy consent is given Returns an M_CONSENT_NOT_GIVEN error (cf https://github.com/matrix-org/matrix-doc/issues/1252) if consent is not yet given.	2018-05-22 12:00:47 +01:00
Richard van der Hoff	d5dca9a04f	Move consent config parsing into ConsentConfig turns out we need to reuse this, so it's better in the config class.	2018-05-22 11:54:51 +01:00
Richard van der Hoff	9ea219c514	Send users a server notice about consent When a user first syncs, we will send them a server notice asking them to consent to the privacy policy if they have not already done so.	2018-05-22 11:54:51 +01:00
Amber Brown	fcc525b0b7	rest of the changes	2018-05-21 19:48:57 -05:00
Amber Brown	df9f72d9e5	replacing portions	2018-05-21 19:47:37 -05:00
Richard van der Hoff	6e1cb54a05	Fix logcontext leak in HttpTransactionCache ONE DAY I WILL PURGE THE WORLD OF THIS EVIL	2018-05-21 16:58:20 +01:00
Richard van der Hoff	6d6e7288fe	Stop the transaction cache caching failures The transaction cache has some code which tries to stop it caching failures, but if the callback function failed straight away, then things would happen backwards and we'd end up with the failure stuck in the cache.	2018-05-21 16:49:59 +01:00
Erik Johnston	6406b70aeb	Use stream rather depth ordering for push actions This simplifies things as it is, but will also allow us to change the way we traverse topologically without having to update the way push actions work.	2018-05-11 15:30:11 +01:00
Richard van der Hoff	902673e356	Merge pull request #3161 from NotAFile/remove-v1auth Make Client-Server API return 403 for invalid token	2018-05-03 10:10:57 +01:00
Erik Johnston	53a5fdf312	Merge pull request #3175 from matrix-org/erikj/escape_metric_values Escape label values in prometheus metrics	2018-05-03 10:01:04 +01:00
Erik Johnston	32015e1109	Escape label values in prometheus metrics	2018-05-02 16:52:42 +01:00
Richard van der Hoff	415c6b672e	Merge branch 'develop' into rav/more_logcontext_leaks	2018-05-02 16:16:01 +01:00
Richard van der Hoff	11607006d9	Remove spurious unittest.DEBUG	2018-05-02 15:48:47 +01:00
Richard van der Hoff	46beeb9a30	Fix a couple of logcontext leaks in unit tests ... which were making other, innocent, tests, fail. Plus remove a spurious unittest.DEBUG which was making the output noisy.	2018-05-02 15:46:22 +01:00
Richard van der Hoff	f22e7cda2c	Fix a class of logcontext leaks So, it turns out that if you have a first `Deferred` `D1`, you can add a callback which returns another `Deferred` `D2`, and `D2` must then complete before any further callbacks on `D1` will execute (and later callbacks on `D1` get the result of `D2` rather than `D2` itself). So, `D1` might have `called=True` (as in, it has started running its callbacks), but any new callbacks added to `D1` won't get run until `D2` completes - so if you `yield D1` in an `inlineCallbacks` function, your `yield` will 'block'. In conclusion: some of our assumptions in `logcontext` were invalid. We need to make sure that we don't optimise out the logcontext juggling when this situation happens. Fortunately, it is easy to detect by checking `D1.paused`.	2018-05-02 11:58:00 +01:00
Richard van der Hoff	e482f8cd85	Fix incorrect reference to StringIO This was introduced in `4f2f5171`	2018-05-02 09:12:26 +01:00
Adrian Tschira	6495dbb326	Burminate v1auth This closes #2602 v1auth was created to account for the differences in status code between the v1 and v2_alpha revisions of the protocol (401 vs 403 for invalid tokens). However since those protocols were merged, this makes the r0 version/endpoint internally inconsistent, and violates the specification for the r0 endpoint. This might break clients that rely on this inconsistency with the specification. This is said to affect the legacy angular reference client. However, I feel that restoring parity with the spec is more important. Either way, it is critical to inform developers about this change, in case they rely on the illegal behaviour. Signed-off-by: Adrian Tschira <nota@notafile.com>	2018-04-30 22:20:43 +02:00
Richard van der Hoff	9e2601f830	Merge pull request #3108 from NotAFile/py3-six-urlparse Use six.moves.urlparse	2018-04-30 00:33:05 +01:00
Richard van der Hoff	3d1ae61399	Merge branch 'develop' into rav/deferred_timeout	2018-04-27 12:54:43 +01:00
Richard van der Hoff	286e20f2bc	Merge pull request #3109 from NotAFile/py3-tests-fix Make tests py3 compatible	2018-04-23 11:59:03 +01:00
Richard van der Hoff	1ea904b9f0	Use deferred.addTimeout instead of time_bound_deferred This doesn't feel like a wheel we need to reinvent.	2018-04-23 00:53:18 +01:00
Richard van der Hoff	d06a9ea5f7	Merge pull request #3104 from NotAFile/py3-unittest-config Add some more variables to the unittest config	2018-04-20 15:35:58 +01:00
Richard van der Hoff	692a3cc806	Merge pull request #3103 from NotAFile/py3-baseexcepton-message Use str(e) instead of e.message	2018-04-20 15:34:49 +01:00
Richard van der Hoff	639480e14a	Avoid creating events with huge numbers of prev_events In most cases, we limit the number of prev_events for a given event to 10 events. This fixes a particular code path which created events with huge numbers of prev_events.	2018-04-16 18:41:37 +01:00
Adrian Tschira	a1a3c9660f	Make tests py3 compatible This is a mixed commit that fixes various small issues * print parentheses * 01 is invalid syntax (it was octal in py2) * [x for i in 1, 2] is invalid syntax * six moves Signed-off-by: Adrian Tschira <nota@notafile.com>	2018-04-16 00:39:32 +02:00
Adrian Tschira	2a3c33ff03	Use six.moves.urlparse The imports were shuffled around a bunch in py3 Signed-off-by: Adrian Tschira <nota@notafile.com>	2018-04-15 21:22:43 +02:00
Adrian Tschira	cb9cdfecd0	Add some more variables to the unittest config These worked accidentally before (python2 doesn't complain if you compare incompatible types) but under py3 this blows up spectacularly Signed-off-by: Adrian Tschira <nota@notafile.com>	2018-04-15 20:36:39 +02:00
Adrian Tschira	1515560f5c	Use str(e) instead of e.message Doing this I learned e.message was pretty shortlived, added in 2.6, they realized it was a bad idea and deprecated it in 2.7 Signed-off-by: Adrian Tschira <nota@notafile.com>	2018-04-15 20:32:42 +02:00
Erik Johnston	e939f3bca6	Fix tests	2018-04-11 14:37:11 +01:00
Richard van der Hoff	a3599dda97	Merge pull request #2996 from krombel/allow_auto_join_rooms move handling of auto_join_rooms to RegisterHandler	2018-04-10 01:11:00 +01:00
Richard van der Hoff	87478c5a60	Merge pull request #3061 from NotAFile/add-some-byte-strings Add b prefixes to some strings that are bytes in py3	2018-04-09 23:54:05 +01:00
Richard van der Hoff	664adb4236	Merge pull request #3016 from silkeh/improve-service-lookups Improve handling of SRV records for federation connections	2018-04-09 23:40:06 +01:00
Richard van der Hoff	01afc563c3	Fix overzealous cache invalidation Fixes an issue where a cache invalidation would invalidate all pending entries, rather than just the entry that we intended to invalidate.	2018-04-05 16:24:04 +01:00
Adrian Tschira	6168351877	Add b prefixes to some strings that are bytes in py3 This has no effect on python2 Signed-off-by: Adrian Tschira <nota@notafile.com>	2018-04-04 13:48:51 +02:00
Silke	72251d1b97	Remove address resolution of hosts in SRV records Signed-off-by: Silke Hofstra <silke@slxh.eu>	2018-04-04 12:26:50 +02:00
Krombel	91ea0202e6	move handling of auto_join_rooms to RegisterHandler Currently the handling of auto_join_rooms only works when a user registers itself via public register api. Registrations via registration_shared_secret and ModuleApi do not work This auto_joins the users in the registration handler which enables the auto join feature for all 3 registration paths. This is related to issue #2725 Signed-Off-by: Matthias Kesler <krombel@krombel.de>	2018-03-14 16:45:37 +01:00
Erik Johnston	cb9f8e527c	s/replication_client/federation_client/	2018-03-13 13:26:52 +00:00
Erik Johnston	cea462e285	s/replication_server/federation_server	2018-03-13 13:22:21 +00:00
Erik Johnston	6ea27fafad	Fix tests	2018-03-13 10:55:47 +00:00
Erik Johnston	631a73f7ef	Fix tests	2018-03-13 10:39:19 +00:00
Richard van der Hoff	2c911d75e8	Fix comment typo	2018-03-05 12:24:49 +00:00
Richard van der Hoff	c818fcab11	Test and fix find_first_stream_ordering_after_ts It seemed to suffer from a bunch of off-by-one errors.	2018-03-05 12:04:02 +00:00
Erik Johnston	784f036306	Move RoomMemberHandler out of Handlers	2018-03-01 14:36:50 +00:00
Erik Johnston	e5b4a208ce	Merge pull request #2892 from matrix-org/erikj/batch_inserts_push_actions Batch inserts into event_push_actions_staging	2018-02-26 14:45:40 +00:00
Erik Johnston	25634ed152	Fix test	2018-02-20 12:40:44 +00:00
Erik Johnston	e440e28456	Fix unit tests	2018-02-20 11:41:40 +00:00
Erik Johnston	3a061cae26	Fix unit test	2018-02-15 16:31:59 +00:00
Erik Johnston	4810f7effd	Remove context.push_actions	2018-02-15 15:47:06 +00:00
Erik Johnston	617199d73d	Merge pull request #2847 from matrix-org/erikj/separate_event_creation Split event creation into a separate handler	2018-02-06 17:01:17 +00:00
Erik Johnston	770b2252ca	s/_create_new_client_event/create_new_client_event/	2018-02-06 16:40:30 +00:00
Erik Johnston	3d33eef6fc	Store state groups separately from events (#2784 ) * Split state group persist into seperate storage func * Add per database engine code for state group id gen * Move store_state_group to StateReadStore This allows other workers to use it, and so resolve state. * Hook up store_state_group * Fix tests * Rename _store_mult_state_groups_txn * Rename StateGroupReadStore * Remove redundant _have_persisted_state_group_txn * Update comments * Comment compute_event_context * Set start val for state_group_id_seq ... otherwise we try to recreate old state groups * Update comments * Don't store state for outliers * Update comment * Update docstring as state groups are ints	2018-02-06 14:31:24 +00:00
Erik Johnston	25c0a020f4	Updates tests	2018-02-05 16:01:48 +00:00
Richard van der Hoff	db91e72ade	Merge pull request #2844 from matrix-org/rav/evicted_metrics montoring metrics for number of cache evictions	2018-02-05 16:34:36 +01:00
Richard van der Hoff	bc496df192	report metrics on number of cache evictions	2018-02-05 15:34:01 +00:00
Erik Johnston	a1beca0e25	Fix broken unit test for media storage	2018-02-05 12:44:03 +00:00
Erik Johnston	1f881e0746	Merge pull request #2791 from matrix-org/erikj/media_storage_refactor Ensure media is in local cache before thumbnailing	2018-02-05 11:28:52 +00:00
Richard van der Hoff	6da4c4d3bd	Factor out resolve_state_groups to a separate handler We extract the storage-independent bits of the state group resolution out to a separate functiom, and stick it in a new handler, in preparation for its use from the storage layer.	2018-02-01 12:40:04 +00:00
Richard van der Hoff	d1f3490e75	Add tests for user directory search	2018-01-27 17:21:57 +00:00
Richard van der Hoff	88b9c5cbf0	Make it possible to run tests against postgres	2018-01-27 17:15:24 +00:00
Richard van der Hoff	d7eacc4f87	Create dbpool as normal in tests ... instead of creating our own special SQLiteMemoryDbPool, whose purpose was a bit of a mystery. For some reason this makes one of the tests run slightly slower, so bump the sleep(). Sorry.	2018-01-27 17:15:15 +00:00
Richard van der Hoff	b178eca261	Run on_new_connection for unit tests Configure the connectionpool used for unit tests to run the `on_new_connection` function.	2018-01-27 17:06:04 +00:00
Richard van der Hoff	6e9bf67f18	Remove unused/bitrotted MemoryDataStore This isn't used, and looks thoroughly bitrotted.	2018-01-26 00:35:15 +00:00
Richard van der Hoff	2b91846497	Remove spurious unittest.DEBUG	2018-01-26 00:34:27 +00:00
Richard van der Hoff	b2da272b77	Merge pull request #2821 from matrix-org/rav/matthew_test_fixes Matthew's fixes to the unit tests	2018-01-22 20:43:56 +00:00
Matthew Hodgson	ab9f844aaf	Add federation_domain_whitelist option (#2820 ) Add federation_domain_whitelist gives a way to restrict which domains your HS is allowed to federate with. useful mainly for gracefully preventing a private but internet-connected HS from trying to federate to the wider public Matrix network	2018-01-22 19:11:18 +01:00
Richard van der Hoff	5c431f421c	Matthew's fixes to the unit tests Extracted from https://github.com/matrix-org/synapse/pull/2820	2018-01-22 16:46:16 +00:00
Matthew Hodgson	d84f65255e	Merge pull request #2813 from matrix-org/matthew/registrations_require_3pid add registrations_require_3pid and allow_local_3pids	2018-01-22 13:57:22 +00:00
Matthew Hodgson	81d037dbd8	mock registrations_require_3pid	2018-01-19 00:28:08 +00:00
Erik Johnston	ce4f66133e	Add unit tests	2018-01-18 16:32:11 +00:00
Erik Johnston	b6dc7044a9	Merge pull request #2804 from matrix-org/erikj/file_consumer Add decent impl of a FileConsumer	2018-01-18 16:31:33 +00:00
Erik Johnston	1432f7ccd5	Move test stuff to tests	2018-01-18 11:57:57 +00:00
Erik Johnston	bc67e7d260	Add decent impl of a FileConsumer Twisted core doesn't have a general purpose one, so we need to write one ourselves. Features: - All writing happens in background thread - Supports both push and pull producers - Push producers get paused if the consumer falls behind	2018-01-17 16:43:03 +00:00
Richard van der Hoff	44a498418c	Optimise LoggingContext creation and copying It turns out that the only thing we use the __dict__ of LoggingContext for is `request`, and given we create lots of LoggingContexts and then copy them every time we do a db transaction or log line, using the __dict__ seems a bit redundant. Let's try to optimise things by making the request attribute explicit.	2018-01-16 15:49:42 +00:00
Richard van der Hoff	a767f06e3f	Merge pull request #2765 from matrix-org/rav/fix_room_uts Fix flaky test_rooms UTs	2018-01-09 14:21:03 +00:00
Richard van der Hoff	f8fa5ae4af	enable twisted delayedcall debugging in UTs	2018-01-09 12:06:45 +00:00
Richard van der Hoff	374c4d4ced	Remove dead code pointless function is pointless	2018-01-09 12:06:45 +00:00
Richard van der Hoff	142fb0a7d4	Disable user_directory updates for UTs Fix flakiness in the UTs caused by the user_directory being updated in the background	2018-01-09 12:06:45 +00:00
Richard van der Hoff	0211464ba2	Fix broken config UTs https://github.com/matrix-org/synapse/pull/2755 broke log-config generation, which in turn broke the unit tests.	2018-01-09 11:28:33 +00:00
Matthew Hodgson	33cb7ef0b7	Merge pull request #2723 from matrix-org/matthew/search-all-local-users Add all local users to the user_directory and optionally search them	2017-12-05 11:09:47 +00:00
Richard van der Hoff	d5f9fb06b0	Refactor UI auth implementation Instead of returning False when auth is incomplete, throw an exception which can be caught with a wrapper.	2017-12-05 09:40:05 +00:00
Matthew Hodgson	7b86c1fdcd	try make tests work a bit more...	2017-12-04 17:10:03 +00:00
Matthew Hodgson	95f8a713dc	erik told me to	2017-12-04 16:56:25 +00:00
Luke Barnard	ab1dc84779	Add extra space before inline comment	2017-11-16 18:22:40 +00:00
Luke Barnard	e836bdf734	Fix tests	2017-11-16 18:14:39 +00:00
Richard van der Hoff	63ef607f1f	Fix tests for Store.__init__ update Fix the test to pass the right number of args to the Store constructors	2017-11-13 10:46:08 +00:00
Ilya Zhuravlev	8a4a0ddea6	Fix appservice tests to account for new behavior of notify_interested_services	2017-11-02 23:19:57 +03:00
Richard van der Hoff	02237ce725	Fix tests for refresh_token removal	2017-11-01 10:19:42 +00:00
Richard van der Hoff	9ded00f221	fix tests	2017-10-31 14:21:13 +00:00
Richard van der Hoff	eaaabc6c4f	replace 'except:' with 'except Exception:' what could possibly go wrong	2017-10-23 15:52:32 +01:00
Richard van der Hoff	1135193dfd	Validate group ids when parsing May as well do it whenever we parse a Group ID. We check the sigil and basic structure here so it makes sense to check the grammar in the same place.	2017-10-21 00:30:39 +01:00
Richard van der Hoff	dbdfd8967d	Merge pull request #2546 from matrix-org/rav/remove_dead_event_injector Remove dead class	2017-10-17 13:21:22 +01:00
Richard van der Hoff	b8e40d146f	Merge pull request #2547 from matrix-org/rav/test_make_deferred_yieldable Add some tests for make_deferred_yieldable	2017-10-17 13:21:09 +01:00
Richard van der Hoff	a6ad8148b9	Fix name of test_logcontext The file under test is logcontext.py, not log_context.py	2017-10-17 10:53:34 +01:00
Richard van der Hoff	5b5f35ccc0	Add some tests for make_deferred_yieldable	2017-10-17 10:52:31 +01:00
Richard van der Hoff	9b714abf35	Remove dead class This isn't used anywhere.	2017-10-17 10:43:36 +01:00
David Baker	33122c5a1b	Fix test	2017-10-17 10:39:50 +01:00
Richard van der Hoff	cc794d60e7	Merge pull request #2532 from matrix-org/rav/fix_linearizer Fix stackoverflow and logcontexts from linearizer	2017-10-11 17:29:32 +01:00
Richard van der Hoff	4fad8efbfb	Fix stackoverflow and logcontexts from linearizer 1. make it not blow out the stack when there are more than 50 things waiting for a lock. Fixes https://github.com/matrix-org/synapse/issues/2505. 2. Make it not mess up the log contexts.	2017-10-11 15:05:05 +01:00
David Baker	27955056e0	Merge branch 'develop' into erikj/groups_merged	2017-10-02 16:20:41 +01:00
Richard van der Hoff	f65e31d22f	Do an AAAA lookup on SRV record targets (#2462 ) Support SRV records which point at AAAA records, as well as A records. Fixes https://github.com/matrix-org/synapse/issues/2405	2017-09-22 20:26:47 +01:00
Richard van der Hoff	72472456d8	Add some more tests for Keyring	2017-09-20 01:32:42 +01:00
Richard van der Hoff	9864efa532	Fix concurrent server_key requests (#2458 ) Fix a bug where we could end up firing off multiple requests for server_keys for the same server at the same time.	2017-09-19 23:25:44 +01:00
Richard van der Hoff	aa620d09a0	Add a config option to block all room invites (#2457 ) - allows sysadmins the ability to lock down their servers so that people can't send their users room invites.	2017-09-19 16:08:14 +01:00
Erik Johnston	b5feaa5a49	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/groups_merged	2017-09-19 11:07:45 +01:00
Richard van der Hoff	290777b3d9	Clean up and document handling of logcontexts in Keyring (#2452 ) I'm still unclear on what the intended behaviour for `verify_json_objects_for_server` is, but at least I now understand the behaviour of most of the things it calls...	2017-09-18 18:31:01 +01:00
Erik Johnston	bf81f3cf2c	Split out profile handler to fix tests	2017-08-25 14:34:56 +01:00
Erik Johnston	c544188ee3	Add groups to sync stream	2017-07-20 16:36:42 +01:00
Erik Johnston	ed9a7f5436	Merge pull request #2309 from matrix-org/erikj/user_ip_repl Fix up user_ip replication commands	2017-07-06 14:33:14 +01:00
Erik Johnston	29a4066a4d	Update test	2017-07-04 10:21:25 +01:00
Erik Johnston	8c23221666	Fix up	2017-06-27 15:53:45 +01:00
Erik Johnston	a0a561ae85	Fix up client ips to read from pending data	2017-06-27 14:46:12 +01:00
Erik Johnston	052c5d19d5	Merge pull request #2281 from matrix-org/erikj/phone_home_stats Fix phone home stats	2017-06-15 12:46:23 +01:00
Erik Johnston	93e7a38370	Remove unhelpful test	2017-06-15 09:30:54 +01:00
Erik Johnston	23da638360	Fix typing tests	2017-05-26 10:02:04 +01:00
Erik Johnston	11c2a3655f	Only load jinja2 templates once Instead of every time a new email pusher is created, as loading jinja2 templates is slow.	2017-05-22 17:48:58 +01:00
Erik Johnston	bbfe4e996c	Make get_state_groups_from_groups faster. Most of the time was spent copying a dict to filter out sentinel values that indicated that keys did not exist in the dict. The sentinel values were added to ensure that we cached the non-existence of keys. By updating DictionaryCache to keep track of which keys were known to not exist itself we can remove a dictionary copy.	2017-05-17 15:12:15 +01:00
Richard van der Hoff	de042b3b88	Do some logging when one-time-keys get claimed might help us figure out if https://github.com/vector-im/riot-web/issues/3868 has happened.	2017-05-09 19:04:56 +01:00
Richard van der Hoff	a7e9d8762d	Allow clients to upload one-time-keys with new sigs When a client retries a key upload, don't give an error if the signature has changed (but the key is the same). Fixes https://github.com/vector-im/riot-android/issues/1208, hopefully.	2017-05-09 19:04:56 +01:00
Erik Johnston	3e5a62ecd8	Add more granular event send metrics	2017-05-02 14:23:26 +01:00
Erik Johnston	85a0d6c7ab	Remove test of replication resource	2017-04-11 10:59:27 +01:00
Erik Johnston	a5c401bd12	Merge pull request #2097 from matrix-org/erikj/repl_tcp_client Move to using TCP replication	2017-04-05 09:36:21 +01:00
Erik Johnston	3a1f3f8388	Change slave storage to use new replication interface As the TCP replication uses a slightly different API and streams than the HTTP replication. This breaks HTTP replication.	2017-04-03 15:34:19 +01:00
Erik Johnston	9cee0ce7db	Merge pull request #2075 from matrix-org/erikj/cache_speed Speed up cached function access	2017-03-31 10:10:56 +01:00
Erik Johnston	350333a09a	Merge pull request #2076 from matrix-org/erikj/as_perf Make AS's faster	2017-03-31 09:43:10 +01:00
Erik Johnston	eefd9fee81	Fix up tests	2017-03-30 14:14:46 +01:00
Richard van der Hoff	f9b4bb05e0	Fix the logcontext handling in the cache wrappers (#2077 ) The cache wrappers had a habit of leaking the logcontext into the reactor while the lookup function was running, and then not restoring it correctly when the lookup function had completed. It's all the fault of `preserve_context_over_{fn,deferred}` which are basically a bit broken.	2017-03-30 13:22:24 +01:00
Erik Johnston	650f0e69f2	Compile the regex's used in ASes	2017-03-28 13:27:21 +01:00
Erik Johnston	57cfa513f5	Merge pull request #2054 from matrix-org/erikj/user_iter_cursor Reduce some CPU work on DB threads	2017-03-24 11:40:08 +00:00
Richard van der Hoff	9397edb28b	Merge pull request #2050 from matrix-org/rav/federation_backoff push federation retry limiter down to matrixfederationclient	2017-03-23 22:27:01 +00:00
Richard van der Hoff	06ce7335e9	Merge pull request #2052 from matrix-org/rav/time_bound_deferred Fix time_bound_deferred to throw the right exception	2017-03-23 22:22:54 +00:00
Erik Johnston	00957d1aa4	User Cursor.__iter__ instead of fetchall This prevents unnecessary construction of lists	2017-03-23 17:53:49 +00:00
pik	566641a0b5	use jsonschema.FormatChecker for RoomID and UserID strings * use a valid filter in rest/client/v2_alpha test Signed-off-by: pik <alexander.maznev@gmail.com>	2017-03-23 11:42:41 -03:00
pik	acafcf1c5b	Add valid filter tests, flake8, fix typo Signed-off-by: pik <alexander.maznev@gmail.com>	2017-03-23 11:42:10 -03:00
pik	e56c79c114	check_valid_filter using JSONSchema * add invalid filter tests Signed-off-by: pik <alexander.maznev@gmail.com>	2017-03-23 11:42:07 -03:00
Richard van der Hoff	0bfea9a2be	fix tests	2017-03-23 13:20:08 +00:00
Richard van der Hoff	b88a323ffb	Fix time_bound_deferred to throw the right exception Due to a failure to instantiate DeferredTimedOutError, time_bound_deferred would throw a CancelledError when the deferred timed out, which was rather confusing.	2017-03-23 12:07:11 +00:00
Richard van der Hoff	4bd597d9fc	push federation retry limiter down to matrixfederationclient rather than having to instrument everywhere we make a federation call, make the MatrixFederationHttpClient manage the retry limiter.	2017-03-23 09:28:46 +00:00
Richard van der Hoff	95f21c7a66	Fix caching of remote servers' signature keys The `@cached` decorator on `KeyStore._get_server_verify_key` was missing its `num_args` parameter, which meant that it was returning the wrong key for any server which had more than one recorded key. By way of a fix, change the default for `num_args` to be all arguments. To implement that, factor out a common base class for `CacheDescriptor` and `CacheListDescriptor`.	2017-03-22 15:11:30 +00:00
Erik Johnston	37a187bfab	Merge pull request #2033 from matrix-org/erikj/repl_speed Don't send the full event json over replication	2017-03-21 13:11:15 +00:00
Erik Johnston	737f283a07	Fix unit test	2017-03-20 14:03:58 +00:00
Richard van der Hoff	f40c2db05a	Stop preserve_fn leaking context into the reactor Fix a bug in ``logcontext.preserve_fn`` which made it leak context into the reactor, and add a test for it. Also, get rid of ``logcontext.reset_context_after_deferred``, which tried to do the same thing but had its own, different, set of bugs.	2017-03-18 00:07:43 +00:00
Will Hunt	c144292373	Modify test_user_sync so it doesn't look at last_active_ts over last_user_sync_ts	2017-03-15 15:38:57 +00:00
Erik Johnston	095b45c165	Aggregate event push actions	2017-02-14 13:39:41 +00:00
Erik Johnston	51adaac953	Fix email push in pusher worker This was broken when device list updates were implemented, as Mailer could no longer instantiate an AuthHandler due to a dependency on federation sending.	2017-02-02 10:53:36 +00:00
Erik Johnston	692daf6f54	Remote membership tests for replication This is because it now relies of the caches stream, which only works on postgres. We are trying to test with sqlite.	2017-01-31 16:10:16 +00:00
Erik Johnston	c7a26b7c32	Fix unit tests	2017-01-30 17:11:24 +00:00
Erik Johnston	b3e1f2aa7a	Fix unit tests	2017-01-26 17:16:24 +00:00
Erik Johnston	c974116f19	Implement device key caching over federation	2017-01-26 16:07:24 +00:00
Erik Johnston	2367c5568c	Add basic implementation of local device list changes	2017-01-25 14:27:27 +00:00
Erik Johnston	a55fa2047f	Insert delta of current_state_events to be more efficient	2017-01-20 17:10:18 +00:00
Erik Johnston	09eb08f910	Derive current_state_events from state groups	2017-01-20 11:52:51 +00:00
Erik Johnston	15f012032c	Merge pull request #1818 from matrix-org/erikj/state_auth_splitout_split Optimise state resolution	2017-01-18 10:53:00 +00:00
Erik Johnston	5d6bad1b3c	Optimise state resolution	2017-01-17 13:22:19 +00:00
Erik Johnston	9e8e236d98	Tidy up test	2017-01-17 11:50:18 +00:00
Erik Johnston	f85b6ca494	Speed up cache size calculation Instead of calculating the size of the cache repeatedly, which can take a long time now that it can use a callback, instead cache the size and update that on insertion and deletion. This requires changing the cache descriptors to have two caches, one for pending deferreds and the other for the actual values. There's no reason to evict from the pending deferreds as they won't take up any more memory.	2017-01-17 11:18:13 +00:00
Erik Johnston	f2f179dce2	Add ExpiringCache tests	2017-01-16 15:33:34 +00:00
Erik Johnston	2fae34bd2c	Optionally measure size of cache by sum of length of values	2017-01-13 17:46:17 +00:00
Erik Johnston	6823fe5241	Linearize updates to membership via PUT /state/	2017-01-09 18:25:13 +00:00
Marcin Bachry	24c16fc349	Fix crash in url preview when html tag has no text Signed-off-by: Marcin Bachry <hegel666@gmail.com>	2016-12-14 22:38:18 +01:00
Erik Johnston	d53a80af25	Merge pull request #1620 from matrix-org/erikj/concurrent_room_access Limit the number of events that can be created on a given room concurrently	2016-12-12 10:30:23 +00:00
Erik Johnston	8b34f71bea	Fix unit tests	2016-12-09 16:48:48 +00:00
Richard van der Hoff	1529c19675	Prevent user tokens being used as guest tokens (#1675 ) Make sure that a user cannot pretend to be a guest by adding 'guest = True' caveats.	2016-12-06 15:31:37 +00:00
Johannes Löthberg	6c9a0ba415	test_preview: Fix incorrect wrapping The old test expected an incorrect wrapping due to the preview function not using unicode properly, so it got the wrong length. Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>	2016-12-05 16:33:57 +01:00
Johannes Löthberg	0697bb2247	Make test_preview use unicode strings Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>	2016-12-05 16:33:57 +01:00
Richard van der Hoff	12f3b9000c	fix imports	2016-11-30 17:45:49 +00:00
Richard van der Hoff	aa09d6b8f0	Rip out more refresh_token code We might as well treat all refresh_tokens as invalid. Just return a 403 from /tokenrefresh, so that we don't have a load of dead, untestable code hanging around. Still TODO: removing the table from the schema.	2016-11-30 17:40:18 +00:00
Richard van der Hoff	dc4b23e1a1	Merge branch 'develop' into rav/no_more_refresh_tokens	2016-11-30 17:10:04 +00:00
Richard van der Hoff	1c4f05db41	Stop putting a time caveat on access tokens The 'time' caveat on the access tokens was something of a lie, since we weren't enforcing it; more pertinently its presence stops us ever adding useful time caveats. Let's move in the right direction by not lying in our caveats.	2016-11-29 16:49:41 +00:00
Richard van der Hoff	5c4edc83b5	Stop generating refresh tokens Since we're not doing refresh tokens any more, we should start killing off the dead code paths. /tokenrefresh itself is a bit of a thornier subject, since there might be apps out there using it, but we can at least not generate refresh tokens on new logins.	2016-11-28 10:13:01 +00:00
Erik Johnston	feec718265	Shuffle receipt handler around so that worker apps don't need to load it	2016-11-23 15:14:24 +00:00
Erik Johnston	ee5e8d71ac	Fix tests	2016-11-23 14:57:07 +00:00
Erik Johnston	b69f76c106	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/split_out_fed_txn	2016-11-23 11:31:53 +00:00
Erik Johnston	54fed21c04	Fix tests and flake8	2016-11-22 18:18:31 +00:00
Kegan Dougal	c3d963ac24	Review comments	2016-11-22 13:42:11 +00:00
Kegan Dougal	0a8b0eeca1	More tests	2016-11-22 09:59:27 +00:00
Kegan Dougal	53b27bbf06	Add remaining tests	2016-11-21 17:58:22 +00:00
Kegan Dougal	70a2157b64	Start adding some tests	2016-11-21 17:52:45 +00:00
Kegan Dougal	f97511a1f3	Move event_fields filtering to serialize_event Also make it an inclusive not exclusive filter, as the spec demands.	2016-11-21 17:42:16 +00:00
Erik Johnston	524d61bf7e	Fix tests	2016-11-21 11:53:02 +00:00
Kegan Dougal	3991b4cbdb	Clean transactions based on time. Add HttpTransactionCache tests.	2016-11-14 11:19:24 +00:00
Erik Johnston	d073cb7ead	Add Limiter: limit concurrent access to resource	2016-11-10 16:29:51 +00:00
Mark Haines	177f104432	Merge pull request #1098 from matrix-org/markjh/bearer_token Allow clients to supply access_tokens as headers	2016-10-25 17:33:15 +01:00
Erik Johnston	78c083f159	Merge pull request #1164 from pik/error-codes Clarify Error codes for GET /filter/	2016-10-19 14:26:17 +01:00
pik	e8b1d2a452	Refactor test_filter to use real DataStore * add tests for filter api errors	2016-10-18 12:17:38 -05:00
pik	d43b63818c	Fix MockHttpRequest always returning M_UNKNOWN errcode in testing	2016-10-14 15:46:54 -05:00
Alexander Maznev	d9350b0db8	Error codes for filters * add tests Signed-off-by: Alexander Maznev <alexander.maznev@gmail.com>	2016-10-14 10:18:28 -05:00
Erik Johnston	35e2cc8b52	Merge pull request #1155 from matrix-org/erikj/pluggable_pwd_auth Implement pluggable password auth	2016-10-12 11:41:20 +01:00
Erik Johnston	3061dac53e	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/replication_noop	2016-10-11 14:08:29 +01:00
Patrik Oldsberg	7b5546d077	rest/client/v1/register: use the correct requester in createUser Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>	2016-10-06 22:12:32 +02:00
Patrik Oldsberg	9bfc617791	storage/appservice: make appservice methods only relying on the cache synchronous	2016-10-06 15:24:59 +02:00
Erik Johnston	850b103b36	Implement pluggable password auth Allows delegating the password auth to an external module. This also moves the LDAP auth to using this system, allowing it to be removed from the synapse tree entirely in the future.	2016-10-03 10:36:40 +01:00
Erik Johnston	748d8fdc7b	Reduce DB hits for replication Some streams will occaisonally advance their positions without actually having any new rows to send over federation. Currently this means that the token will not advance on the workers, leading to them repeatedly sending a slightly out of date token. This in turns requires the master to hit the DB to check if there are any new rows, rather than hitting the no op logic where we check if the given token matches the current token. This commit changes the API to always return an entry if the position for a stream has changed, allowing workers to advance their tokens correctly.	2016-09-23 16:49:21 +01:00
Erik Johnston	22578545a0	Time out typing over federation	2016-09-23 14:00:52 +01:00
Mark Haines	ec609f8094	Fix unit tests	2016-09-12 10:46:02 +01:00
Mark Haines	74cbfdc7de	Fix unit tests	2016-09-06 18:30:03 +01:00
Erik Johnston	c10cb581c6	Correctly handle the difference between prev and current state	2016-08-31 14:26:22 +01:00
Erik Johnston	bc1a8b1f7a	Don't notify for online -> online transitions. Specifically, if currently_active remains true then we should not notify if only the last active time changes.	2016-08-30 15:05:32 +01:00
Erik Johnston	55fc17cf4b	Merge pull request #1049 from matrix-org/erikj/presence_users_in_room Use state handler instead of get_users_in_room/get_joined_hosts	2016-08-30 10:50:37 +01:00
Erik Johnston	93b32d4515	Fix unit tests	2016-08-26 15:40:27 +01:00
Mark Haines	4bbef62124	Merge remote-tracking branch 'origin/develop' into markjh/direct_to_device	2016-08-26 14:35:31 +01:00
Erik Johnston	3f11953fcb	Fix tests	2016-08-26 10:15:52 +01:00
Mark Haines	3b8d0ceb22	More 0_0 in tests	2016-08-25 18:42:46 +01:00
Erik Johnston	a3dc1e9cbe	Replace context.current_state with context.current_state_ids	2016-08-25 17:32:22 +01:00
Erik Johnston	45fd2c8942	Ensure invalidation list does not grow unboundedly	2016-08-19 16:09:16 +01:00
Erik Johnston	c0d7d9d642	Rename to on_invalidate	2016-08-19 15:13:58 +01:00
Erik Johnston	dc76a3e909	Make cache_context an explicit option	2016-08-19 15:02:38 +01:00
Erik Johnston	ba214a5e32	Remove lru option	2016-08-19 14:17:11 +01:00
Erik Johnston	4161ff2fc4	Add concept of cache contexts	2016-08-19 14:17:07 +01:00
Erik Johnston	9da84a9a1e	Make AppserviceHandler stream events from database This is for two reasons: 1. Suppresses duplicates correctly, as the notifier doesn't do any duplicate suppression. 2. Makes it easier to connect the AppserviceHandler to the replication stream.	2016-08-18 11:54:41 +01:00
Erik Johnston	320dfe523c	Make notify_interested_services faster	2016-08-17 17:20:50 +01:00
Erik Johnston	7321f45457	Clean up _ServiceQueuer	2016-08-17 12:03:04 +01:00
Erik Johnston	b835ebcc79	Update unit tests	2016-08-17 11:22:11 +01:00
Erik Johnston	48b5829aea	Fix up preview URL API. Add tests. This includes: - Splitting out methods of a class into stand alone functions, to make them easier to test. - Adding unit tests to split out functions, testing HTML -> preview. - Handle the fact that elements in lxml may have tail text.	2016-08-16 14:53:24 +01:00
Richard van der Hoff	a8bcc7274d	PEP8	2016-08-08 17:20:38 +01:00
Richard van der Hoff	6fe6a6f029	Fix login with m.login.token login with token (as used by CAS auth) was broken by `067596d`, such that it always returned a 401.	2016-08-08 16:40:39 +01:00
Erik Johnston	e97648c4e2	Test summarization	2016-08-04 16:09:09 +01:00
Richard van der Hoff	98385888b8	PEP8	2016-08-03 15:42:08 +01:00
Richard van der Hoff	68264d7404	Include device name in /keys/query response Add an 'unsigned' section which includes the device display name.	2016-08-03 15:42:08 +01:00
Richard van der Hoff	91fa69e029	keys/query: return all users which were asked for In the situation where all of a user's devices get deleted, we want to indicate this to a client, so we want to return an empty dictionary, rather than nothing at all.	2016-08-03 15:41:44 +01:00
Mark Haines	0a7d3cd00f	Create separate methods for getting messages to push for the email and http pushers rather than trying to make a single method that will work with their conflicting requirements. The http pusher needs to get the messages in ascending stream order, and doesn't want to miss a message. The email pusher needs to get the messages in descending timestamp order, and doesn't mind if it misses messages.	2016-07-28 20:24:24 +01:00
Richard van der Hoff	eb359eced4	Add `create_requester` function Wrap the `Requester` constructor with a function which provides sensible defaults, and use it throughout	2016-07-26 16:46:53 +01:00
Richard van der Hoff	8e02494166	Delete refresh tokens when deleting devices	2016-07-26 11:10:37 +01:00
Richard van der Hoff	012b4c1913	Implement updating devices You can update the displayname of devices now.	2016-07-26 07:35:48 +01:00
Richard van der Hoff	436bffd15f	Implement deleting devices	2016-07-26 07:35:48 +01:00
Richard van der Hoff	bf3de7b90b	Merge pull request #945 from matrix-org/rav/background_reindex Create index on user_ips in the background	2016-07-25 14:04:05 +01:00
Richard van der Hoff	42f4feb2b7	PEP8	2016-07-25 12:25:06 +01:00
Richard van der Hoff	f16f0e169d	Slightly saner logging for unittests 1. Give the handler used for logging in unit tests a formatter, so that the output is slightly more meaningful 2. Log some synapse.storage stuff, because it's useful.	2016-07-25 12:12:47 +01:00
Richard van der Hoff	465117d7ca	Fix background_update tests A bit of a cleanup for background_updates, and make sure that the real background updates have run before we start the unit tests, so that they don't interfere with the tests.	2016-07-25 12:10:42 +01:00
Richard van der Hoff	55abbe1850	make /devices return a list Turns out I specced this to return a list of devices rather than a dict of them	2016-07-21 15:57:28 +01:00
Richard van der Hoff	1c3c202b96	Fix PEP8 errors	2016-07-21 13:15:15 +01:00
Richard van der Hoff	406f7aa0f6	Implement GET /device/{deviceId}	2016-07-21 12:00:29 +01:00
Richard van der Hoff	40a1c96617	Fix PEP8 errors	2016-07-20 18:06:28 +01:00
Richard van der Hoff	7314bf4682	Merge branch 'develop' into rav/get_devices_api (pick up PR #938 in the hope of fixing the UTs)	2016-07-20 17:40:00 +01:00
Richard van der Hoff	e9e3eaa67d	Merge pull request #938 from matrix-org/rav/add_device_id_to_client_ips Record device_id in client_ips	2016-07-20 17:38:45 +01:00
Richard van der Hoff	bc8f265f0a	GET /devices endpoint implement a GET /devices endpoint which lists all of the user's devices. It also returns the last IP where we saw that device, so there is some dancing to fish that out of the user_ips table.	2016-07-20 16:42:32 +01:00
Richard van der Hoff	ec041b335e	Record device_id in client_ips Record the device_id when we add a client ip; it's somewhat redundant as we could get it via the access_token, but it will make querying rather easier.	2016-07-20 16:41:03 +01:00
Richard van der Hoff	b97a1356b1	Register a device_id in the /v2/register flow. This doesn't cover all of the registration flows, but it does cover the most common ones: in particular: shared_secret registration, appservice registration, and normal user/pass registration. Pull device_id from the registration parameters. Register the device in the devices table. Associate the device with the returned access and refresh tokens. Profit.	2016-07-20 16:38:27 +01:00
David Baker	e967bc86e7	Merge pull request #932 from matrix-org/rav/register_refactor Further registration refactoring	2016-07-20 11:03:33 +01:00
Erik Johnston	1e2a7f18a1	Merge pull request #922 from matrix-org/erikj/file_api2 Feature: Add filter to /messages. Add 'contains_url' to filter.	2016-07-20 10:40:48 +01:00
Richard van der Hoff	40cbffb2d2	Further registration refactoring * `RegistrationHandler.appservice_register` no longer issues an access token: instead it is left for the caller to do it. (There are two of these, one in `synapse/rest/client/v1/register.py`, which now simply calls `AuthHandler.issue_access_token`, and the other in `synapse/rest/client/v2_alpha/register.py`, which is covered below). * In `synapse/rest/client/v2_alpha/register.py`, move the generation of access_tokens into `_create_registration_details`. This means that the normal flow no longer needs to call `AuthHandler.issue_access_token`; the shared-secret flow can tell `RegistrationHandler.register` not to generate a token; and the appservice flow continues to work despite the above change.	2016-07-19 18:46:19 +01:00
David Baker	b9e997f561	Merge pull request #931 from matrix-org/rav/refactor_register rest/client/v2_alpha/register.py: Refactor flow somewhat.	2016-07-19 16:13:45 +01:00
Richard van der Hoff	0da0d0a29d	rest/client/v2_alpha/register.py: Refactor flow somewhat. This is meant to be an almost non-functional change, with the exception that it fixes what looks a lot like a bug in that it only calls `auth_handler.add_threepid` and `add_pusher` once instead of three times. The idea is to move the generation of the `access_token` out of `registration_handler.register`, because `access_token`s now require a device_id, and we only want to generate a device_id once registration has been successful.	2016-07-19 13:12:22 +01:00
Richard van der Hoff	f863a52cea	Add device_id support to /login Add a 'devices' table to the storage, as well as a 'device_id' column to refresh_tokens. Allow the client to pass a device_id, and initial_device_display_name, to /login. If login is successful, then register the device in the devices table if it wasn't known already. If no device_id was supplied, make one up. Associate the device_id with the access token and refresh token, so that we can get at it again later. Ensure that the device_id is copied from the refresh token to the access_token when the token is refreshed.	2016-07-18 16:39:44 +01:00
Erik Johnston	a98d215204	Add filter param to /messages API	2016-07-14 16:30:56 +01:00
Negar Fazeli	0136a522b1	Bug fix: expire invalid access tokens	2016-07-13 15:00:37 +02:00
Erik Johnston	7335f0adda	Add ReadWriteLock	2016-07-05 15:23:17 +01:00
David Baker	2455ad8468	Remove room name & alias test as get_room_name_and_alias is now gone	2016-06-24 13:34:20 +01:00
Martin Weinelt	0a32208e5d	Rework ldap integration with ldap3 Use the pure-python ldap3 library, which eliminates the need for a system dependency. Offer both a `search` and `simple_bind` mode, for more sophisticated ldap scenarios. - `search` tries to find a matching DN within the `user_base` while employing the `user_filter`, then tries the bind when a single matching DN was found. - `simple_bind` tries the bind against a specific DN by combining the localpart and `user_base` Offer support for STARTTLS on a plain connection. The configuration was changed to reflect these new possibilities. Signed-off-by: Martin Weinelt <hexa@darmstadt.ccc.de>	2016-06-22 17:51:59 +02:00
Erik Johnston	0113ad36ee	Enable use_frozen_events in tests	2016-06-17 15:13:13 +01:00
Mark Haines	7dbb473339	Add function to load config without generating it Renames ``load_config`` to ``load_or_generate_config`` Adds a method called ``load_config`` that just loads the config. The main synapse.app.homeserver will continue to use ``load_or_generate_config`` to retain backwards compat. However new worker processes can use ``load_config`` to load the config avoiding some of the cruft needed to generate the config. As the new ``load_config`` method is expected to be used by new configs it removes support for the legacy commandline overrides that ``load_or_generate_config`` supports	2016-06-09 18:50:38 +01:00
Negar Fazeli	6e7dc7c7dd	Fix a bug caused by a change in auth_handler function Fix the relevant unit test cases	2016-06-08 23:22:39 +02:00
Erik Johnston	84379062f9	Fix AS retries, but with correct ordering	2016-06-07 10:24:50 +01:00
Erik Johnston	310197bab5	Fix AS retries	2016-06-07 09:34:50 +01:00
Erik Johnston	05e01f21d7	Remove event fetching from DB threads	2016-06-03 17:22:13 +01:00
Erik Johnston	73c7112433	Change CacheMetrics to be quicker We change it so that each cache has an individual CacheMetric, instead of having one global CacheMetric. This means that when a cache tries to increment a counter it does not need to go through so many indirections.	2016-06-03 11:26:52 +01:00
Mark Haines	56d15a0530	Store the typing users as user_id strings. (#819 ) Rather than storing them as UserID objects.	2016-06-02 16:28:54 +01:00
Mark Haines	70599ce925	Allow external processes to mark a user as syncing. (#812 ) * Add infrastructure to the presence handler to track sync requests in external processes * Expire stale entries for dead external processes * Add an http endpoint for making users as syncing Add some docstrings and comments. * Fixes	2016-06-02 15:20:15 +01:00
David Baker	4a10510cd5	Split out the auth handler	2016-06-02 13:31:45 +01:00
David Baker	195254cae8	Inject fake room list handler in tests Otherwise it tries to start the remote public room list updating looping call which breaks.	2016-06-01 11:14:16 +01:00
Mark Haines	c626fc576a	Move the AS handler out of the Handlers object. Access it directly from the homeserver itself. It already wasn't inheriting from BaseHandler storing it on the Handlers object was already somewhat dubious.	2016-05-31 13:53:48 +01:00
Negi Fazeli	6fe04ffef2	Fix set profile error with Requester. Replace flush_user with delete access token due to function removal Add a new test case for if the user is already registered	2016-05-23 19:50:28 +02:00
Mark Haines	0cb441fedd	Move typing handler out of the Handlers object	2016-05-17 15:58:46 +01:00
Mark Haines	425e6b4983	Merge branch 'develop' into markjh/member_cleanup	2016-05-17 10:13:16 +01:00
Mark Haines	816df9f267	get_room_members is unused now	2016-05-16 19:51:43 +01:00
Mark Haines	821306120a	Replaces calls to fetch_room_distributions_into with get_joined_hosts_for_room	2016-05-16 19:48:07 +01:00
Mark Haines	3b86ecfa79	Move the presence handler out of the Handlers object	2016-05-16 18:56:37 +01:00
Mark Haines	21018c2c13	Merge pull request #783 from matrix-org/markjh/slave_account_data Add a slaved datastore for account data	2016-05-13 15:56:04 +01:00
Mark Haines	3abab26458	Add a slaved datastore for account data	2016-05-13 15:34:06 +01:00
Negi Fazeli	40aa6e8349	Create user with expiry - Add unittests for client, api and handler Signed-off-by: Negar Fazeli <negar.fazeli@ericsson.com>	2016-05-13 15:34:15 +02:00
Mark Haines	c0d8e0eb63	Replicate push actions	2016-04-21 15:25:58 +01:00
Mark Haines	f52dd35ac3	Merge pull request #738 from matrix-org/markjh/slaved_receipts Add a slaved receipts store	2016-04-19 17:31:59 +01:00
Mark Haines	5bbd424ee0	Add a slaved receipts store	2016-04-19 17:14:08 +01:00
Mark Haines	e99365f601	Replicate get_invited_rooms_for_user	2016-04-19 15:22:14 +01:00
Erik Johnston	eb8619e256	Create log context in Measure if one doesn't exist	2016-04-18 16:08:32 +01:00
Erik Johnston	79fc4ff6f9	Merge pull request #677 from matrix-org/erikj/dns_cache Read from DNS cache if within TTL	2016-04-08 14:09:56 +01:00
Mark Haines	7e2f971c08	Remove some unused functions (#711 ) * Remove some unused functions * get_room_events_stream is only used in tests * is_exclusive_room might actually be something we want	2016-04-08 14:01:56 +01:00
Mark Haines	ceb599e789	Add tests for redactions	2016-04-07 16:52:07 +01:00
Mark Haines	8c82b06904	Merge pull request #704 from matrix-org/markh/slaveIII Add tests for get_latest_event_ids_in_room and get_current_state	2016-04-07 16:49:34 +01:00
Mark Haines	57fa1801c3	Add sensible __eq__ operators inside the tests. Rather than adding them globally. This limits the changes to only affect the tests.	2016-04-07 16:41:37 +01:00
Erik Johnston	af03ecf352	Deduplicate joins	2016-04-07 14:19:02 +01:00
Mark Haines	60ec9793fb	Add tests for get_latest_event_ids_in_room and get_current_state	2016-04-07 13:17:56 +01:00
Erik Johnston	a28d066732	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/dns_cache	2016-04-07 11:11:17 +01:00
Mark Haines	6bfec56796	Test that room membership is replicated	2016-04-06 16:20:13 +01:00
Mark Haines	e815763b7f	Merge pull request #697 from matrix-org/markjh/slaveI Add a slaved events store class	2016-04-06 16:19:25 +01:00
Mark Haines	75fb9ac1be	Add a slaved events store class Add a test to check that get_room_names_and_aliases does the same thing on both the master and on the slave data store.	2016-04-06 14:18:35 +01:00
Erik Johnston	8aab9d87fa	Don't require config to create database	2016-04-06 14:15:45 +01:00
Erik Johnston	11860637e1	Tests	2016-04-06 10:12:30 +01:00
Erik Johnston	c906f30661	Do checks for memberships before creating events	2016-04-01 16:17:32 +01:00
Erik Johnston	f699b8f997	Read from DNS cache if within TTL	2016-03-31 10:04:28 +01:00
Mark Haines	31a9eceda5	Add a replication stream for state groups	2016-03-30 16:01:58 +01:00
Mark Haines	673c96ce97	Remove dead code left over from presence changes	2016-03-17 11:03:47 +00:00
David Baker	ff7d3dc3a0	Fix tests	2016-03-16 14:25:14 +00:00
Mark Haines	b6e8420aee	Add replication stream for pushers	2016-03-15 17:33:10 +00:00
Mark Haines	e9c1cabac2	Use parse_json_object_from_request to parse JSON out of request bodies	2016-03-11 16:41:03 +00:00
blide	1be438f2a6	Flake8 fix	2016-03-10 12:13:35 +03:00
blide	40160e24ab	Register endpoint returns refresh_token Guest registration still doesn't return refresh_token	2016-03-10 10:29:19 +03:00
David Baker	316c00936f	Fix tests	2016-03-07 17:32:36 +00:00
Mark Haines	3110c37d02	Fix unit tests	2016-03-04 14:48:17 +00:00
Mark Haines	ec7460b4f2	Merge branch 'develop' into markjh/pushrule_stream	2016-03-04 14:44:34 +00:00
Patrik Oldsberg	bb0e82fff1	tests/utils: added room_invite_state_types to test config Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>	2016-03-04 10:43:17 +01:00
Daniel Wagner-Hall	b4022cc487	Pass whole requester to ratelimiting This will enable more detailed decisions	2016-03-03 16:43:42 +00:00
Daniel Wagner-Hall	95481e7ba7	Merge pull request #571 from matrix-org/daniel/asids Mark AS users with their AS's ID	2016-03-03 13:56:28 +00:00
Mark Haines	2223204eba	Hook push rules up to the replication API	2016-03-02 17:26:20 +00:00
Erik Johnston	2a78dac60d	Merge pull request #612 from matrix-org/erikj/cache_size Add environment variable SYNAPSE_CACHE_FACTOR, default it to 0.1	2016-03-02 14:36:10 +00:00
Mark Haines	60a0f81c7a	Add a /replication API for extracting the updates that happened on synapse This is necessary for replicating the data in synapse to be visible to a separate service because presence and typing notifications aren't stored in a database so won't be visible to another process. This API can be used to either get the raw data by requesting the tables themselves or to just receive notifications for updates by following the streams meta-stream. Returns updates for each table requested a JSON array of arrays with a row for each row in the table. Each table is prefixed by a header row with the: name of the table, current stream_id position for the table, number of rows, number of columns and the names of the columns. This is followed by the rows that have been added to the server since the requester last asked. The API has a timeout and is hooked up to the notifier so that a slave can long poll for updates.	2016-03-01 14:49:41 +00:00
Erik Johnston	ce2cdced61	Move cache size fiddling to descriptors only. Fix tests	2016-03-01 13:21:46 +00:00
Mark Haines	5e2890bd49	Check that the disable_registration config key is handled correctly	2016-02-22 16:01:29 +00:00
Mark Haines	7641a90c34	Add a test for TreeCache.__contains__	2016-02-22 15:22:38 +00:00
Mark Haines	5c79ef9396	Test Filter.filter_rooms Also check that the __repr__ method for FilterCollection does something sensible.	2016-02-19 17:55:28 +00:00
Mark Haines	700487a7c7	Fix flake8 warnings for tests	2016-02-19 15:34:38 +00:00
Erik Johnston	929cb0ed7d	Don't set currently_active for remote presence	2016-02-19 10:58:27 +00:00
Erik Johnston	b4796a62ee	Add unit test	2016-02-18 11:52:33 +00:00
Erik Johnston	9da9826b85	Remove old tests.	2016-02-18 10:46:16 +00:00
Erik Johnston	e5999bfb1a	Initial cut	2016-02-17 15:40:50 +00:00
Erik Johnston	9e7900da1e	Add wheeltimer impl	2016-02-17 14:29:28 +00:00
Daniel Wagner-Hall	763360594d	Mark AS users with their AS's ID	2016-02-11 17:26:42 +00:00
Erik Johnston	e557dc80b8	Merge pull request #566 from matrix-org/erikj/logcontext Don't bother copying records on parent context	2016-02-10 11:41:45 +00:00
Erik Johnston	78a5482267	Typo	2016-02-09 16:23:11 +00:00
Erik Johnston	e664e9737c	Fix test	2016-02-09 14:57:43 +00:00
Erik Johnston	13ba8d878c	Fix test	2016-02-09 14:55:21 +00:00
Daniel Wagner-Hall	6a9f1209df	Error if macaroon key is missing from config Currently we store all access tokens in the DB, and fall back to that check if we can't validate the macaroon, so our fallback works here, but for guests, their macaroons don't get persisted, so we don't get to find them in the database. Each restart, we generate a new ephemeral key, so guests lose access after each server restart. I tried to fix up the config stuff to be less insane, but gave up, so instead I bolt on yet another piece of custom one-off insanity. Also, add some basic tests for config generation and loading.	2016-02-05 01:58:23 +00:00
Daniel Wagner-Hall	66bb255fcd	Merge pull request #556 from matrix-org/daniel/config Rename config field to reflect yaml name	2016-02-03 14:55:54 +00:00
Daniel Wagner-Hall	5054806ec1	Rename config field to reflect yaml name	2016-02-03 14:42:01 +00:00
Erik Johnston	f8aae79a72	Simplify get_rooms	2016-02-03 13:24:35 +00:00
Mark Haines	f2d5ff5bf2	Fix the mock homserver used in the tests	2016-01-29 14:53:14 +00:00
Erik Johnston	b18114e19e	Merge pull request #536 from matrix-org/erikj/sync Make /sync "better".	2016-01-29 13:04:51 +00:00
Erik Johnston	02a9c3be6c	Merge pull request #538 from matrix-org/erikj/fix_lru_cache Fix LruCache. Make TreeCache track its own size.	2016-01-29 11:53:55 +00:00
Erik Johnston	4fce59f274	Add tests	2016-01-29 11:33:11 +00:00
Erik Johnston	35981c8b71	Fix test	2016-01-28 17:20:05 +00:00
Erik Johnston	167d1df699	Merge pull request #534 from matrix-org/erikj/setup Add a Homeserver.setup method	2016-01-28 14:45:36 +00:00
Erik Johnston	4e7948b47a	Allow paginating backwards from stream token	2016-01-28 11:52:34 +00:00
Erik Johnston	61eaa6ec64	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/sync	2016-01-28 10:50:57 +00:00
Richard van der Hoff	e26390ca46	Merge pull request #535 from matrix-org/rav/paginate_from_stream_token Make it possible to paginate forwards from stream tokens	2016-01-28 09:49:46 +00:00
Richard van der Hoff	5cba88ea7c	Make it possible to paginate forwards from stream tokens In order that we can fill the gap after a /sync, make it possible to paginate forwards from a stream token.	2016-01-27 17:42:45 +00:00
Erik Johnston	0487c9441f	Fix tests	2016-01-27 17:33:13 +00:00
Erik Johnston	9959d9ece8	Remove redundated BaseHomeServer	2016-01-26 13:52:29 +00:00
Erik Johnston	8c6012a4af	Fix tests	2016-01-25 13:12:35 +00:00
Erik Johnston	88baa3865e	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/sync	2016-01-22 15:57:12 +00:00
David Baker	d552861346	Revert all the bits changing keys of eeverything that used LRUCaches to tuples	2016-01-22 12:18:14 +00:00
David Baker	10f76dc5da	Make LRU cache not default to treecache & add options to use it	2016-01-22 12:10:33 +00:00
David Baker	31a051b677	Test treecache directly	2016-01-22 11:22:00 +00:00
Erik Johnston	8f9c74e9f1	Fix tests	2016-01-22 10:48:27 +00:00
David Baker	4efcaa43c8	Add tests for treecache directly and test del_multi at the LruCache level too.	2016-01-22 10:37:37 +00:00
David Baker	f1f8122120	Change LRUCache to be tree-based so we can delete subtrees.	2016-01-21 19:16:25 +00:00
Erik Johnston	191070123d	Cache dns lookups, and use the cache if we fail to lookup servers later	2016-01-20 11:34:09 +00:00
Erik Johnston	2c176e02ae	Make unit tests work	2016-01-18 14:48:50 +00:00
Daniel Wagner-Hall	ac5a4477ad	Require unbanning before other membership changes	2016-01-15 16:27:26 +00:00
Daniel Wagner-Hall	2680043bc6	Require ID and as_token be unique for ASs Defaults ID to as_token if not specified. This will change when IDs are fully supported.	2016-01-14 14:34:01 +00:00
Mark Haines	c0a279e808	Delete the table objects from TransactionStore	2016-01-13 11:15:20 +00:00
Daniel Wagner-Hall	42aa1f3f33	Merge pull request #478 from matrix-org/daniel/userobject Introduce a User object I'm sick of passing around more and more things as tuple items around the whole world, and needing to edit every call site every time there is more information about a user. So pass them around together as an object. This object has incredibly poorly named fields because we have a convention that `user` indicates a UserID object, and `user_id` indicates a string. I tried to clean up the whole repo to fix this, but gave up. So instead, I introduce a second convention. A user_object is a User, and a user_id_object is a UserId. I may have cried a little bit.	2016-01-11 17:50:22 +00:00
Daniel Wagner-Hall	2110e35fd6	Introduce a Requester object This tracks data about the entity which made the request. This is instead of passing around a tuple, which requires call-site modifications every time a new piece of optional context is passed around. I tried to introduce a User object. I gave up.	2016-01-11 17:48:45 +00:00
David Baker	c232780081	Merge pull request #456 from matrix-org/store_event_actions Send unread notification counts	2016-01-08 14:47:15 +00:00
Matthew Hodgson	6c28ac260c	copyrights	2016-01-07 04:26:29 +00:00
David Baker	0e48f7f245	fix tests	2016-01-06 16:46:41 +00:00
David Baker	eb03625626	Merge remote-tracking branch 'origin/develop' into store_event_actions	2016-01-05 18:39:50 +00:00
Daniel Wagner-Hall	cfd07aafff	Allow guests to upgrade their accounts	2016-01-05 18:01:18 +00:00
David Baker	92a1e74b20	fix tests	2016-01-04 14:17:35 +00:00
David Baker	928c575c6f	Merge remote-tracking branch 'origin/develop' into store_event_actions	2016-01-04 13:39:51 +00:00
Mark Haines	7fa71e3267	Add a unit test for the snapshot cache	2015-12-23 11:48:03 +00:00
David Baker	d79e90f078	Add mocks to make tests work again	2015-12-22 17:56:56 +00:00
Mark Haines	d547afeae0	Merge remote-tracking branch 'origin/master' into develop	2015-12-07 13:13:43 +00:00
Oleg Girko	4013216fcc	Fix mock import in tests. For some reason, one test imports Mock class from mock.mock rather than from mock. This change fixes this error. Signed-off-by: Oleg Girko <ol@infoserver.lv>	2015-12-06 20:50:11 +00:00
Daniel Wagner-Hall	14d7acfad4	Host /unstable and /r0 versions of r0 APIs	2015-12-01 17:34:32 +00:00
Mark Haines	7b593af7e1	rename the method in the tests as well	2015-12-01 16:06:17 +00:00
Erik Johnston	bd3de8f39a	Update tests	2015-11-17 18:38:48 +00:00
Daniel Wagner-Hall	468a2ed4ec	Return non-room events from guest /events calls	2015-11-12 16:45:28 +00:00
Richard van der Hoff	78f6010207	Fix an issue with ignoring power_level changes on divergent graphs Changes to m.room.power_levels events are supposed to be handled at a high priority; however a typo meant that the relevant bit of code was never executed, so they were handled just like any other state change - which meant that a bad person could cause room state changes by forking the graph from a point in history when they were allowed to do so.	2015-11-12 15:24:59 +00:00
Erik Johnston	e21cef9bb5	Merge pull request #359 from matrix-org/markjh/incremental_indexing Incremental background updates for db indexes	2015-11-11 17:19:51 +00:00
Daniel Wagner-Hall	cf437900e0	Return world_readable and guest_can_join in /publicRooms	2015-11-10 17:10:27 +00:00
Mark Haines	36c58b18a3	Test for background updates	2015-11-10 15:51:40 +00:00
Richard van der Hoff	9107ed23b7	Add a couple of unit tests for room/<x>/messages ... merely because I was trying to figure out how it worked, and couldn't.	2015-11-09 16:16:43 +00:00
Daniel Wagner-Hall	ca2f90742d	Open up /events to anonymous users for room events only Squash-merge of PR #345 from daniel/anonymousevents	2015-11-05 14:32:26 +00:00
Daniel Wagner-Hall	4e62ffdb21	Merge branch 'develop' of github.com:matrix-org/synapse into develop	2015-11-04 17:31:01 +00:00
Daniel Wagner-Hall	f522f50a08	Allow guests to register and call /events?room_id= This follows the same flows-based flow as regular registration, but as the only implemented flow has no requirements, it auto-succeeds. In the future, other flows (e.g. captcha) may be required, so clients should treat this like the regular registration flow choices.	2015-11-04 17:29:07 +00:00
Mark Haines	33b3e04049	Merge branch 'develop' into daniel/removesomelies Conflicts: synapse/notifier.py	2015-11-04 16:01:00 +00:00
Mark Haines	c452dabc3d	Remove the LockManager class because it wasn't being used	2015-11-04 14:08:15 +00:00
Daniel Wagner-Hall	771ca56c88	Remove more unused parameters	2015-11-02 15:31:57 +00:00
Mark Haines	0e36756383	Merge branch 'develop' into markjh/room_tags	2015-11-02 10:57:00 +00:00
Mark Haines	fb46937413	Support clients supplying older tokens, fix short poll test	2015-10-30 16:38:35 +00:00
Richard van der Hoff	f69a5c9134	Fix a 500 error resulting from empty room_ids POST /_matrix/client/api/v1/rooms//send/a.b.c gave a 500 error, because we assumed that rooms always had at least one character.	2015-10-26 18:44:03 +00:00
Erik Johnston	b0ac0a9438	Merge pull request #319 from matrix-org/erikj/filter_refactor Refactor api.filtering to have a Filter API	2015-10-22 13:17:10 +01:00
Mark Haines	87da71bace	Merge pull request #314 from matrix-org/paul/event-redaction Add some unit tests of prune_events()	2015-10-22 11:07:20 +01:00
Erik Johnston	45cd2b0233	Refactor api.filtering to have a Filter API	2015-10-20 15:33:25 +01:00
Daniel Wagner-Hall	f510586372	Merge branch 'develop' of github.com:matrix-org/synapse into develop	2015-10-20 12:00:22 +01:00
Daniel Wagner-Hall	137fafce4e	Allow rejecting invites This is done by using the same /leave flow as you would use if you had already accepted the invite and wanted to leave.	2015-10-20 11:58:58 +01:00
Paul "LeoNerd" Evans	9ed784098a	Invoke EventBuilder directly instead of going via the EventBuilderFactory	2015-10-19 17:42:34 +01:00
Paul "LeoNerd" Evans	531e3aa75e	Capture __init__.py	2015-10-19 17:37:35 +01:00
Paul "LeoNerd" Evans	a8795c9644	Use assertIn() instead of assertTrue on the 'in' operator	2015-10-19 15:24:49 +01:00
Paul "LeoNerd" Evans	07b58a431f	Another signing test vector using an 'm.room.message' with content, so that the implementation will have to redact it	2015-10-19 15:00:52 +01:00
Paul "LeoNerd" Evans	0aab34004b	Initial minimial hack at a test of event hashing and signing	2015-10-19 14:40:15 +01:00
Paul "LeoNerd" Evans	aff4d850bd	Add some unit tests of prune_events()	2015-10-16 19:56:46 +01:00
Mark Haines	8897781558	update filtering tests	2015-10-13 14:13:51 +01:00
Erik Johnston	17c80c8a3d	rename schema_prepare to prepare_database	2015-10-13 13:56:22 +01:00
Erik Johnston	ec398af41c	Expose error more nicely	2015-10-13 11:43:43 +01:00
Erik Johnston	d5e081c7ae	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/unfederatable	2015-10-02 10:33:49 +01:00
Mark Haines	1ee3d26432	synapse/storage/_base.py:_simple_selectupdate_one was unused	2015-09-23 10:30:03 +01:00
Mark Haines	bb4dddd6c4	Move NullSource out of synapse and into tests since it is only used by the tests	2015-09-22 18:33:34 +01:00
Daniel Wagner-Hall	7213588083	Implement configurable stats reporting SYN-287 This requires that HS owners either opt in or out of stats reporting. When --generate-config is passed, --report-stats must be specified If an already-generated config is used, and doesn't have the report_stats key, it is requested to be set.	2015-09-22 12:57:40 +01:00
Mark Haines	ee2d722f0f	Merge pull request #276 from matrix-org/markjh/history_for_rooms_that_have_been_left SPEC-216: Allow users to view the history of rooms that they have left.	2015-09-21 14:38:13 +01:00
Daniel Wagner-Hall	728d07c8c1	Merge pull request #256 from matrix-org/auth Attempt to validate macaroons	2015-09-14 18:09:33 +01:00
Mark Haines	e2054ce21a	Allow users to GET individual state events for rooms that they have left	2015-09-10 15:06:47 +01:00
Mark Haines	bc8b25eb56	Allow users that have left the room to view the member list from the point they left	2015-09-09 15:42:16 +01:00
Erik Johnston	187320b019	Merge branch 'erikj/check_room_exists' into erikj/unfederatable	2015-09-01 15:58:10 +01:00
Erik Johnston	b345853918	Check against sender rather than event_id	2015-09-01 15:57:35 +01:00
Erik Johnston	00149c063b	Fix tests	2015-09-01 15:42:03 +01:00
Daniel Wagner-Hall	e255c2c32f	s/user_id/user/g for consistency	2015-09-01 12:41:16 +01:00
Daniel Wagner-Hall	3063383547	Swap out bcrypt for md5 in tests This reduces our ~8 second sequential test time down to ~7 seconds	2015-08-26 15:59:32 +01:00
Daniel Wagner-Hall	81450fded8	Turn TODO into thing which actually will fail	2015-08-26 13:56:01 +01:00
Daniel Wagner-Hall	6a4b650d8a	Attempt to validate macaroons A couple of weird caveats: * If we can't validate your macaroon, we fall back to checking that your access token is in the DB, and ignoring the failure * Even if we can validate your macaroon, we still have to hit the DB to get the access token ID, which we pretend is a device ID all over the codebase. This mostly adds the interesting code, and points out the two pieces we need to delete (and necessary conditions) in order to fix the above caveats.	2015-08-26 13:22:23 +01:00
Daniel Wagner-Hall	a9d8bd95e7	Stop looking up "admin", which we never read	2015-08-25 16:29:39 +01:00
Daniel Wagner-Hall	a0b181bd17	Remove completely unused concepts from codebase Removes device_id and ClientInfo device_id is never actually written, and the matrix.org DB has no non-null entries for it. Right now, it's just cluttering up code. This doesn't remove the columns from the database, because that's fiddly.	2015-08-25 16:23:06 +01:00
Daniel Wagner-Hall	e8cf77fa49	Merge branch 'develop' into refresh Conflicts: synapse/rest/client/v1/login.py	2015-08-20 16:25:40 +01:00
Daniel Wagner-Hall	cecbd636e9	/tokenrefresh POST endpoint This allows refresh tokens to be exchanged for (access_token, refresh_token). It also starts issuing them on login, though no clients currently interpret them.	2015-08-20 16:21:35 +01:00
Daniel Wagner-Hall	13a6517d89	s/by_token/by_access_token/g We're about to have two kinds of token, access and refresh	2015-08-20 16:01:29 +01:00
Daniel Wagner-Hall	617501dd2a	Move token generation to auth handler I prefer the auth handler to worry about all auth, and register to call into it as needed, than to smatter auth logic between the two.	2015-08-20 11:35:56 +01:00
Mark Haines	a0b8e5f2fe	Merge pull request #211 from matrix-org/email_in_use Changes for unique emails	2015-08-20 10:04:04 +01:00
Daniel Wagner-Hall	70e265e695	Re-add whitespace around caveat operators	2015-08-19 14:30:31 +01:00
Daniel Wagner-Hall	ce832c38d4	Remove padding space around caveat operators	2015-08-18 17:39:26 +01:00
Daniel Wagner-Hall	42e858daeb	Fix units in test I made the non-test seconds instead of ms, but not the test	2015-08-18 17:38:37 +01:00
Daniel Wagner-Hall	3e6fdfda00	Fix some formatting to use tuples	2015-08-18 15:18:50 +01:00
Daniel Wagner-Hall	1469141023	Merge branch 'develop' into auth	2015-08-18 14:43:44 +01:00
Daniel Wagner-Hall	2d3462714e	Issue macaroons as opaque auth tokens This just replaces random bytes with macaroons. The macaroons are not inspected by the client or server. In particular, they claim to have an expiry time, but nothing verifies that they have not expired. Follow-up commits will actually enforce the expiration, and allow for token refresh. See https://bit.ly/matrix-auth for more information	2015-08-18 14:22:02 +01:00
Erik Johnston	d7451e0f22	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/dictionary_cache	2015-08-12 10:30:30 +01:00
Erik Johnston	2df8dd9b37	Move all the caches into their own package, synapse.util.caches	2015-08-11 18:00:59 +01:00
Daniel Wagner-Hall	4ff0228c25	Remove call to recently removed function in mock	2015-08-11 16:56:30 +01:00
Daniel Wagner-Hall	dcd5983fe4	Remove call to recently removed function in mock	2015-08-11 16:54:06 +01:00
Erik Johnston	20addfa358	Change Cache to not use *args in its interface	2015-08-07 18:32:47 +01:00
Erik Johnston	b8e386db59	Change Cache to not use *args in its interface	2015-08-07 11:52:21 +01:00
Erik Johnston	f596ff402e	Merge branch 'erikj/cache_deferreds' into erikj/dictionary_cache	2015-08-06 13:37:56 +01:00
Erik Johnston	2efb93af52	Merge branch 'erikj/cached_keyword_args' into erikj/cache_deferreds	2015-08-06 13:35:28 +01:00
Erik Johnston	7eea3e356f	Make @cached cache deferreds rather than the deferreds' values	2015-08-06 13:33:34 +01:00
Erik Johnston	07507643cb	Use dictionary cache to do group -> state fetching	2015-08-05 15:11:42 +01:00
Erik Johnston	c67ba143fa	Move DictionaryCache	2015-08-04 15:58:28 +01:00
Erik Johnston	e7768e77f5	Add basic dictionary cache	2015-08-04 15:56:56 +01:00
David Baker	07ad03d5df	Fix tests	2015-08-04 15:18:40 +01:00
Kegan Dougal	11b0a34074	Use the same reg paths as register v1 for ASes. Namely this means using registration_handler.appservice_register.	2015-07-29 10:00:54 +01:00
Kegan Dougal	a4d62ba36a	Fix v2_alpha registration. Add unit tests. V2 Registration forced everyone (including ASes) to create a password for a user, when ASes should be able to omit passwords. Also unbreak AS registration in general which checked too early if the given username was claimed by an AS; it was checked before knowing if the AS was the one doing the registration! Add unit tests for AS reg, user reg and disabled_registration flag.	2015-07-28 17:34:12 +01:00
Erik Johnston	f0dd6d4cbd	Fix test.	2015-07-07 16:18:36 +01:00
Erik Johnston	bd1236c0ee	Consolidate duplicate code in notifier	2015-07-02 11:46:05 +01:00
Erik Johnston	ddf7979531	Add receipts_key to StreamToken	2015-07-02 11:45:44 +01:00
Kegan Dougal	d88e20cdb9	Fix bug where synapse was sending AS user queries incorrectly. Bug introduced in `92b20713d7` which reversed the comparison when checking if a user existed in the users table. Added UTs to prevent this happening again.	2015-06-17 17:26:03 +01:00
Erik Johnston	261ccd7f5f	Fix tests	2015-06-12 17:17:29 +01:00
Erik Johnston	3bbd0d0e09	Merge pull request #180 from matrix-org/erikj/prev_state_context Don't needlessly compute prev_state	2015-06-03 17:20:56 +01:00
Erik Johnston	9dda396baa	Merge pull request #179 from matrix-org/erikj/state_group_outliers Don't compute EventContext for outliers.	2015-06-03 17:20:40 +01:00
Erik Johnston	55bf90b9e4	Don't needlessly compute prev_state	2015-06-03 16:44:24 +01:00
Erik Johnston	1c3d844e73	Don't needlessly compute context	2015-06-03 16:41:51 +01:00
Erik Johnston	d8866d7277	Caches should be bound to instances. Before, caches were global and so different instances of the stores would share caches. This caused problems in the unit tests.	2015-06-03 14:45:17 +01:00
Mark Haines	17167898c8	Fix the presence tests	2015-05-22 16:22:54 +01:00
Mark Haines	49a2c10279	Merge pull request #157 from matrix-org/markjh/presence_performance Improve presence performance in loadtest	2015-05-22 16:04:40 +01:00
Mark Haines	f43544eecc	Make the appservice use 'users_in_room' rather than get_room_members since it is cached	2015-05-22 11:01:28 +01:00
Mark Haines	88f1ea36ce	Oops, get_rooms_for_user returns a namedtuple, not a room_id	2015-05-21 15:23:58 +01:00
Erik Johnston	a551c5dad7	Merge pull request #155 from matrix-org/erikj/perf Bulk and batch retrieval of events.	2015-05-21 14:54:40 +01:00
Mark Haines	8eca5bd50a	Fix the presence tests	2015-05-20 13:22:18 +01:00
Mark Haines	0b0033c40b	Merge branch 'develop' into notifier_performance	2015-05-18 13:50:01 +01:00
Erik Johnston	131bdf9bb1	Merge branch 'erikj/events_move' of github.com:matrix-org/synapse into erikj/perf	2015-05-18 10:23:37 +01:00
Erik Johnston	a2c4f3f150	Fix daedlock	2015-05-15 10:54:04 +01:00
Mark Haines	c5d1b4986b	Remove unused arguments and doc PresenceHandler.push_update_to_clients	2015-05-14 14:59:31 +01:00
Mark Haines	f1b83d88a3	Discard unused NotifierUserStreams	2015-05-13 16:54:02 +01:00
Mark Haines	63878c0379	Don't bother checking for updates if the stream token hasn't advanced for a user	2015-05-13 13:42:21 +01:00
Erik Johnston	43c2e8deae	Add support for using executemany	2015-05-05 15:13:25 +01:00
Erik Johnston	327ca883ec	Merge branch 'develop' of github.com:matrix-org/synapse into postgres	2015-04-28 13:39:42 +01:00
Erik Johnston	6f8e2d517e	Merge branch 'develop' of github.com:matrix-org/synapse into postgres	2015-04-27 14:41:40 +01:00
David Baker	6532b6e607	Merge branch 'develop' into csauth Conflicts: synapse/http/server.py	2015-04-24 09:37:54 +01:00
Paul "LeoNerd" Evans	e1e5e53127	Remove users from the remote_offline_serials list (and clean up empty elements) when they go online again	2015-04-23 19:01:37 +01:00
Paul "LeoNerd" Evans	8a785c3006	Store a list of the presence serial number at which remote users went offline, so that when we delete them from the cachemap, we can still synthesize OFFLINE events for them (SYN-261)	2015-04-23 18:40:19 +01:00
David Baker	cb03fafdf1	Merge branch 'develop' into csauth	2015-04-17 13:51:10 +01:00
Erik Johnston	6ba2e3df4e	Merge branch 'develop' of github.com:matrix-org/synapse into postgres	2015-04-17 11:22:31 +01:00
Paul "LeoNerd" Evans	04c7f3576e	Various minor fixes to unit-test structure around typing notifications	2015-04-15 23:32:11 +01:00
Paul "LeoNerd" Evans	0268d40281	Have TypingNotificationEventSource.get_new_events_for_user() return a deferred, for consistency and extensibility	2015-04-15 23:09:35 +01:00
Erik Johnston	22d7a59306	Fix tests after commit 9a0579	2015-04-08 16:57:14 +01:00
Erik Johnston	4fe95094d1	Merge branch 'develop' of github.com:matrix-org/synapse into mysql	2015-04-07 18:05:39 +01:00
Erik Johnston	9236136f3a	Make work in both Maria and SQLite. Fix tests	2015-04-01 14:12:33 +01:00
Kegan Dougal	9fa8bda099	Merge branch 'develop' into application-services-registration-script	2015-04-01 10:19:17 +01:00
Kegan Dougal	09cbff174a	Fix thinko whereby events for the AS specifically were not passed on. This was caused by not explicitly checking the service.sender field. This has now been fixed and a regression test has been added.	2015-03-31 16:44:45 +01:00
Kegan Dougal	cf1fa59f4b	Use a sender localpart instead of a user ID. Form the user ID at runtime instead, This gives less room for error in AS config files since they cannot specify the domain of another HS.	2015-03-31 13:48:03 +01:00
Kegan Dougal	c217504949	Edit SQL schema to use string IDs not ints. Use token as ID. Update tests.	2015-03-31 12:07:56 +01:00
Kegan Dougal	b59aa74556	Fix tests and missing returns on deferreds.	2015-03-31 11:35:45 +01:00
Kegan Dougal	d33ae65efc	Remove more reg/unreg methods. Read config not database for cache.	2015-03-31 11:00:00 +01:00
Kegan Dougal	4edcbcee3b	Merge branch 'develop' into application-services-txn-reliability Conflicts: synapse/storage/__init__.py	2015-03-26 10:07:59 +00:00
Paul "LeoNerd" Evans	9ba6487b3f	Allow a choice of LRU behaviour for Cache() by using LruCache() or OrderedDict()	2015-03-25 19:05:34 +00:00
Paul "LeoNerd" Evans	7ab9f91a60	Unit-test that Cache() key eviction is ordered	2015-03-25 18:50:43 +00:00
David Baker	ce2766d19c	Fix tests	2015-03-24 18:56:51 +00:00
David Baker	438a21c87b	Don't test exact equality of the list: as long as it has the fields we expect, that's just fine. I added the user_id (as in database pkey) and it broke: no point testing what that comes out as: it's determined by the db.	2015-03-24 18:21:54 +00:00
Paul "LeoNerd" Evans	0f86312c4c	Pull out the cache logic from the @cached wrapper into its own class we can reuse	2015-03-20 18:25:42 +00:00
Erik Johnston	7e282a53a5	Tidy up _simple_... methods	2015-03-20 15:05:10 +00:00
Kegan Dougal	d04fa1f712	Implement ServiceQueuer with tests.	2015-03-16 14:03:16 +00:00
Kegan Dougal	6279285b2a	Replace EventGrouper for ServiceQueuer to move to push-based txns. Fix tests and add stub tests for ServiceQueuer.	2015-03-16 13:15:40 +00:00
Kegan Dougal	835e01fc70	Minor PR comment tweaks.	2015-03-16 10:16:59 +00:00
Kegan Dougal	f9232c7917	Merge branch 'develop' into application-services-txn-reliability Conflicts: synapse/storage/appservice.py	2015-03-16 10:09:15 +00:00
Paul "LeoNerd" Evans	c1cdd7954d	Add an .inc_by() method to CounterMetric; implement DistributionMetric a neater way	2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans	f1fbe3e09f	Rename TimerMetric to DistributionMetric; as it could count more than just time	2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans	cbc0406be8	Export CacheMetric as hits+total, rather than hits+misses, as it's easier to derive hit ratio from that	2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans	0e847540c3	Prometheus needs "escaped" label values	2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans	b3a0179d64	Bugfix to rendering output of vectored TimerMetrics	2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans	f9478e475b	Rename Metrics' "keys" to "labels"	2015-03-12 16:24:51 +00:00
Paul "LeoNerd" Evans	72625f2f4d	Initial hack at a TimerMetric; for storing counts + duration accumulators	2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans	23ab0c68c2	Implement vector CallbackMetrics	2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans	8664599af7	Rename CacheCounterMetric to just CacheMetric; add a CallbackMetric component to give the size of the cache	2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans	d8caa5454d	Initial attempt at a scalar callback-based metric to give instantaneous snapshot gauges	2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans	ce8b5769f7	Create the concept of a cachecounter metric; generating two counters specific to caches	2015-03-12 16:24:50 +00:00
Paul "LeoNerd" Evans	e7420a3bef	Initial tiny attempt at (vectorable) counter metrics	2015-03-12 16:24:50 +00:00
Kegan Dougal	b98cd03193	Use event IDs instead of dumping event content in the txns table.	2015-03-09 17:25:20 +00:00
Kegan Dougal	21fd84dcb8	Use seconds; start gluing in the AS scheduler into the AS handler.	2015-03-09 17:01:19 +00:00
Erik Johnston	6d74e46621	Fix tests	2015-03-09 17:01:11 +00:00
Kegan Dougal	0a60bbf4fa	Finish appservice txn storage impl and tests.	2015-03-09 15:53:03 +00:00
Kegan Dougal	1ead1caa18	Implement create_appservice_txn with tests.	2015-03-09 13:54:20 +00:00
Kegan Dougal	1c2dcf762a	Partially implement txn store methods with tests.	2015-03-09 13:10:31 +00:00
Kegan Dougal	10766f1e93	Update UTs	2015-03-06 16:17:01 +00:00
Kegan Dougal	2602ddc379	Apply clarity and docstrings	2015-03-06 16:16:14 +00:00
Kegan Dougal	0354659f9d	Finish synapse.appservice.scheduler implementation. With tests to assert behaviour. Not hooked up yet. Stub datastore methods not implemented yet.	2015-03-06 16:09:05 +00:00
Kegan Dougal	f260cb72cd	Flesh out more stub functions.	2015-03-06 15:12:24 +00:00
Kegan Dougal	0fbfe1b08a	Add more tests; fix bugs.	2015-03-06 14:36:52 +00:00
Kegan Dougal	192e228a98	Start adding some tests	2015-03-06 11:50:27 +00:00
Kegsay	8ad024ea80	Merge pull request #93 from matrix-org/application-services-exclusive Application services exclusive flag support	2015-03-02 14:56:32 +00:00
Erik Johnston	0a036944bd	Merge branch 'develop' of github.com:matrix-org/synapse into batched_get_pdu	2015-03-02 13:53:30 +00:00
Kegan Dougal	de190e49d5	Add more unit tests for exclusive namespaces.	2015-02-27 11:51:06 +00:00
Kegan Dougal	127efeeb68	Update unit tests to use new format.	2015-02-27 11:10:48 +00:00
Kegan Dougal	0ebd632d39	Fix unit tests	2015-02-27 09:46:38 +00:00
Paul "LeoNerd" Evans	f53fcbce97	Use cache.pop() instead of a separate membership test + del []	2015-02-23 18:30:45 +00:00
Paul "LeoNerd" Evans	bd84755e64	Merge remote-tracking branch 'origin/develop' into performance-cache-improvements	2015-02-23 17:16:03 +00:00
Paul "LeoNerd" Evans	e76d485e29	Allow @cached-wrapped functions to have a prefill method for setting entries	2015-02-23 15:41:54 +00:00
Erik Johnston	852816befe	Fix presence tests	2015-02-23 15:14:09 +00:00
Paul "LeoNerd" Evans	ebc3db295b	Take named arguments to @cached() decorator, add a 'max_entries' limit	2015-02-19 18:36:02 +00:00
Erik Johnston	5f4c28d313	Update tests	2015-02-19 14:34:32 +00:00
Erik Johnston	5358966a87	Use git aware version string in User-Agent and Server headers	2015-02-18 16:52:04 +00:00
Mark Haines	0d872f5aa6	Merge pull request #50 from matrix-org/application-services Application Services	2015-02-13 15:06:14 +00:00
Erik Johnston	021d93db11	Merge pull request #62 from matrix-org/state-chache State chache	2015-02-11 17:10:28 +00:00
Kegan Dougal	cb43fbeeb4	Fix tests which broke when event caching was introduced.	2015-02-11 16:46:01 +00:00
Kegan Dougal	f2fdcb7c4b	Merge branch 'develop' into application-services	2015-02-11 16:43:26 +00:00
Mark Haines	f5a70e0d2e	Add a cache for get_event	2015-02-11 15:01:15 +00:00
Mark Haines	d8324d5a2b	Add a lru cache class	2015-02-11 14:52:23 +00:00
Mark Haines	896253e085	Factor out some of the common homeserver setup code into a setup_test_homeserver function in utils.	2015-02-11 11:37:30 +00:00
Kegan Dougal	c7783d6fee	Notify ASes for events sent by other users in a room which an AS user is a part of.	2015-02-11 10:36:08 +00:00

... 13 14 15 16 17 ...

1756 Commits