7692 Commits

Author	SHA1	Message	Date
Erik Johnston	cc7a294e2e	Fix current_state_events table to not lie ... If we try and persist two state events that have the same ancestor we calculate the wrong current state when persisting those events.	2017-03-14 10:57:43 +00:00
David Baker	7b6ed9871e	Use extend instead of +=	2017-03-14 10:49:55 +00:00
David Baker	d79a687d85	Oops, remove print	2017-03-14 10:40:20 +00:00
Luke Barnard	f29d85d9e4	Merge pull request #1993 from matrix-org/luke/delete-devices ... Implement delete_devices API	2017-03-14 10:02:56 +00:00
Luke Barnard	bbeeb97f75	Implement _simple_delete_many_txn, use it to delete devices ... (But this doesn't implement the same for deleting access tokens or e2e keys. Also respond to code review.	2017-03-13 17:53:23 +00:00
David Baker	0a9945220e	Fix registration for broken clients ... Only offer msisdn flows if the x_show_msisdn option is given.	2017-03-13 17:29:38 +00:00
David Baker	73a5f06652	Support registration / login with phone number ... Changes from https://github.com/matrix-org/synapse/pull/1971	2017-03-13 17:27:51 +00:00
Luke Barnard	c077c3277b	Flake	2017-03-13 16:45:38 +00:00
Richard van der Hoff	31f3ca1b2b	Merge pull request #1990 from matrix-org/rav/log_config_comments ... Add helpful texts to logger config options	2017-03-13 16:42:12 +00:00
Luke Barnard	c81f33f73d	Implement delete_devices API ... This implements the proposal here https://docs.google.com/document/d/1C-25Gqz3TXy2jIAoeOKxpNtmme0jI4g3yFGqv5GlAAk for deleting multiple devices at once in a single request.	2017-03-13 16:33:51 +00:00
Richard van der Hoff	170ccc9de5	Fix routing loop when fetching remote media ... When we proxy a media request to a remote server, add a query-param, which will tell the remote server to 404 if it doesn't recognise the server_name. This should fix a routing loop where the server keeps forwarding back to itself. Also improves the error handling on remote media fetches, so that we don't always return a rather obscure 502.	2017-03-13 16:30:36 +00:00
Erik Johnston	45c7f12d2a	Add new storage function to slave store	2017-03-13 16:26:44 +00:00
Richard van der Hoff	6037a9804c	Add helpful texts to logger config options	2017-03-13 12:33:35 +00:00
Richard van der Hoff	3c69f32402	Merge remote-tracking branch 'origin/develop' into rav/refactor_received_pdu	2017-03-13 12:20:47 +00:00
Richard van der Hoff	6bfe8e32b5	Merge pull request #1983 from matrix-org/rav/no_redirect_stdio ... Add an option to disable stdio redirect	2017-03-13 12:20:07 +00:00
Richard van der Hoff	5fc9261929	Merge pull request #1982 from matrix-org/rav/sighup_for_logconfig ... Reread log config on SIGHUP	2017-03-13 12:19:54 +00:00
Erik Johnston	0162994983	Comments	2017-03-13 11:53:26 +00:00
Erik Johnston	254b7c5b15	Bump changelog and versions	2017-03-13 10:02:58 +00:00
Erik Johnston	672dcf59d3	Merge branch 'develop' of github.com:matrix-org/synapse into release-v0.19.3	2017-03-13 09:59:54 +00:00
Erik Johnston	7eae6eaa2f	Revert "Support registration & login with phone number"	2017-03-13 09:59:33 +00:00
Erik Johnston	79926e016e	Assume rooms likely haven't changed	2017-03-13 09:50:10 +00:00
Matthew Hodgson	a61dd408ed	enable guest access for the 3pl/3pid APIs	2017-03-12 19:30:45 +00:00
Erik Johnston	8ffbe43ba1	Get current state by using current_state_events table	2017-03-10 17:39:35 +00:00
Richard van der Hoff	bcfa5cd00c	Add an option to disable stdio redirect ... This makes it tractable to run synapse under pdb.	2017-03-10 15:38:29 +00:00
Richard van der Hoff	d84bd51e95	Refactor logger config for workers ... - to make it easier to add more config options.	2017-03-10 15:34:01 +00:00
Richard van der Hoff	9072a8c627	Reread log config on SIGHUP ... When we are using a log_config file, reread it on SIGHUP.	2017-03-10 15:29:55 +00:00
Erik Johnston	8f267fa8a8	Fix it for the workers	2017-03-10 11:22:25 +00:00
Erik Johnston	64d62e41b8	Noop repated delete device inbox calls from /sync	2017-03-10 10:36:43 +00:00
Erik Johnston	3545e17f43	Add setdefault key to ExpiringCache	2017-03-10 10:30:49 +00:00
Richard van der Hoff	29235901b8	Move FederationServer._handle_new_pdu to FederationHandler ... Unfortunately this significantly increases the size of the already-rather-big FederationHandler, but the code fits more naturally here, and it paves the way for the tighter integration that I need between handling incoming PDUs and doing the join dance. Other than renaming the existing `FederationHandler.on_receive_pdu` to `_process_received_pdu` to make way for it, this just consists of the move, and replacing `self.handler` with `self` and `self` with `self.replication_layer`.	2017-03-09 16:20:13 +00:00
Richard van der Hoff	e8b1721290	Move sig check out of _handle_new_pdu ... When we receive PDUs via `get_missing_events`, we have already checked their sigs, so there is no need to do it again.	2017-03-09 15:50:44 +00:00
Richard van der Hoff	3406333a58	Factor _get_missing_events_for_pdu out of _handle_new_pdu ... This should be functionally identical: it just seeks to improve readability by reducing indentation.	2017-03-09 15:50:44 +00:00
Richard van der Hoff	45d173a59a	Fix docstring	2017-03-09 15:50:29 +00:00
David Baker	663396e45d	Merge pull request #1971 from matrix-org/dbkr/msisdn_signin ... Support registration & login with phone number	2017-03-09 10:18:51 +00:00
David Baker	ece7e00048	Comment when our 3pids would be incomplete	2017-03-08 19:07:18 +00:00
David Baker	9d0d40fc15	Docs	2017-03-08 19:05:29 +00:00
David Baker	3edc57296d	Incorrectly copied copyright ... This file post-dates OM	2017-03-08 19:00:51 +00:00
David Baker	727124a762	Not any more, it doesn't	2017-03-08 19:00:23 +00:00
Richard van der Hoff	6ad71cc29d	Remove spurious SQL logging (#1972) ... looks like the upsert function was accidentally sending sql logging to the general logger. We already log the sql in `txn.execute`.	2017-03-08 18:00:44 +00:00
David Baker	d4d3629aaf	Better error message	2017-03-08 17:01:26 +00:00
Erik Johnston	3170c56e07	Bump changelog and version	2017-03-08 13:00:47 +00:00
David Baker	1c99934b28	pep8	2017-03-08 11:58:20 +00:00
David Baker	a9e2b9ec16	Add msisdn util file	2017-03-08 11:53:36 +00:00
David Baker	85bb322333	Pull out datastore in initialiser	2017-03-08 11:51:25 +00:00
David Baker	65d43f3ca5	Minor fixes from PR feedback	2017-03-08 11:48:43 +00:00
David Baker	0e0aee25c4	Fix log line	2017-03-08 11:46:22 +00:00
David Baker	82c5e7de25	Typos	2017-03-08 11:42:44 +00:00
David Baker	2e27339add	Refector out assert_params_in_request ... and replace requestEmailToken where we meant requestMsisdnToken	2017-03-08 11:37:34 +00:00
David Baker	88df6c0c9a	Factor out msisdn canonicalisation ... Plus a couple of other minor fixes	2017-03-08 11:03:39 +00:00
David Baker	402a7bf63d	Fix pep8	2017-03-08 09:33:40 +00:00
David Baker	00466e2feb	Support new login format ... https://docs.google.com/document/d/1-6ZSSW5YvCGhVFDyD2QExAUAdpCWjccvJT5xiyTTG2Y/edit#	2017-03-07 16:37:23 +00:00
Erik Johnston	ac5491f563	Select distinct devices from DB ... Otherwise we might pull out tonnes of duplicate user_ids and this can make synapse sad.	2017-03-06 11:10:14 +00:00
David Baker	b0effa2160	Add msisdns as 3pids during registration ... and support binding them with the bind_msisdn param	2017-03-03 18:34:39 +00:00
Erik Johnston	82f7f1543b	Merge pull request #1964 from matrix-org/erikj/device_list_update_fix ... Fix device list update to not constantly resync	2017-03-03 16:18:16 +00:00
Erik Johnston	f2581ee8b8	Don't keep around old stream IDs forever	2017-03-03 16:02:53 +00:00
Erik Johnston	9834367eea	Spelling	2017-03-03 15:31:57 +00:00
Erik Johnston	da52d3af31	Fix up	2017-03-03 15:29:13 +00:00
David Baker	ad882cd54d	Just return the deferred straight off ... defer.returnValue doth not maketh a generator: it would need a yield to be a generator, and this doesn't need a yield.	2017-03-01 18:08:51 +00:00
David Baker	3557cf34dc	Merge remote-tracking branch 'origin/develop' into dbkr/msisdn_signin	2017-03-01 17:20:37 +00:00
Erik Johnston	d766343668	Add index to device_lists_stream	2017-03-01 15:56:30 +00:00
Erik Johnston	36be39b8b3	Fix device list update to not constantly resync	2017-03-01 14:12:11 +00:00
Erik Johnston	3365117151	Clobber old device list stream entries	2017-03-01 10:21:30 +00:00
Erik Johnston	e933a2712d	Don't log unknown cache warnings in workers	2017-02-28 16:22:41 +00:00
Erik Johnston	d638a7484b	Merge pull request #1959 from matrix-org/erikj/intern_once ... Intern table column names once	2017-02-28 15:15:16 +00:00
Erik Johnston	b84907bdbb	Intern table column names once	2017-02-28 14:38:16 +00:00
Erik Johnston	e4919b9329	Add stream_id index to device_lists_outbound_pokes ... As this is used for replication streaming	2017-02-28 11:19:06 +00:00
Erik Johnston	8a12b6f1eb	Fix up txn name	2017-02-28 10:15:50 +00:00
Erik Johnston	848cf95ea0	Pop with default value to stop throwing	2017-02-28 10:02:54 +00:00
Erik Johnston	eda96586ca	Merge pull request #1955 from matrix-org/erikj/current_state_query_bypass ... Don't fetch current state in common case	2017-02-27 19:15:59 +00:00
Erik Johnston	64a2cef9bb	Pop rather than del from dict	2017-02-27 19:15:36 +00:00
Erik Johnston	a41dce8f8a	Remove needless check	2017-02-27 18:54:43 +00:00
Erik Johnston	c0d6045776	It should be all	2017-02-27 18:45:24 +00:00
Erik Johnston	49f4bc4709	Don't fetch current state in common case ... Currently we fetch the list of current state events whenever we send something in a room. This is overkill for the common case of persisting a simple chain of non-state events, so lets handle that case specially.	2017-02-27 18:33:41 +00:00
Erik Johnston	fc5d876dba	Merge pull request #1954 from matrix-org/erikj/cache_device2 ... Cache get_user_devices_from_cache	2017-02-27 16:36:40 +00:00
Erik Johnston	f58dbb02a6	Cache get_user_devices_from_cache	2017-02-27 16:22:12 +00:00
Erik Johnston	acf6d4d2e3	Merge pull request #1945 from jkolo/fix_dynamic_thumbnails_aspect ... Fix #1677 (dynamic thumbnails aspect)	2017-02-27 09:51:52 +00:00
Jurek	aea5461488	Fix dynamic thumbnails aspect	2017-02-24 22:43:27 +01:00
Erik Johnston	1a4f8022e6	Strip newlines from SQL queries	2017-02-23 11:15:31 +00:00
Erik Johnston	b2d20e94fa	Remove lock from rotate notifs	2017-02-22 14:24:02 +00:00
Erik Johnston	7455ba436a	Ensure we pass positive ints to delay function	2017-02-22 12:08:14 +00:00
Erik Johnston	b7442c3e2b	Store looping call	2017-02-21 13:59:25 +00:00
Erik Johnston	a3708a1885	Merge branch 'master' of github.com:matrix-org/synapse into develop	2017-02-21 13:46:27 +00:00
Erik Johnston	30ecfef5a3	Bump version and changelog	2017-02-21 13:43:36 +00:00
Richard van der Hoff	0c4cf9372b	Fix a race in transaction queue ... It was theoretically possible for a PDU to get queued and not sent for ages. On closer inspection I think there were bigger problems elsewhere, but we might as well fix this since it's easy.	2017-02-20 16:46:25 +00:00
Erik Johnston	6226a27bf8	Remove unused param	2017-02-20 16:01:54 +00:00
Erik Johnston	efff39c030	Fix /context/ visibiltiy rules	2017-02-20 16:01:49 +00:00
Erik Johnston	17673404fb	Remove unused param	2017-02-20 15:02:01 +00:00
Erik Johnston	7f026792e1	Fix /context/ visibiltiy rules	2017-02-20 14:54:50 +00:00
Erik Johnston	699be7d1be	Fix up notif rotation	2017-02-18 14:42:39 +00:00
David Baker	ec8d7603e6	Merge pull request #1925 from matrix-org/dbkr/pushers_lang_lengthen ... Make the pushers lang field column longer	2017-02-17 11:29:06 +00:00
David Baker	8c87bb550e	Merge pull request #1922 from matrix-org/dbkr/allow_forget_for_ban ... Allow forgetting rooms you're banned from	2017-02-17 10:52:30 +00:00
David Baker	4aa29508af	Use TEXT rather than VARCHAR ... While we're changing anyway	2017-02-17 10:51:49 +00:00
David Baker	b4017539d4	Make the pushers lang field column longer ... To accommodate things like zh-Hans-CN Fixes https://github.com/vector-im/riot-ios/issues/1031	2017-02-17 10:42:57 +00:00
Erik Johnston	138e030cfe	Comment	2017-02-16 15:03:36 +00:00
Erik Johnston	502ae6c663	Comment	2017-02-16 14:47:11 +00:00
Erik Johnston	e6acf0c399	Store the default push actions in a more efficient manner	2017-02-16 14:40:24 +00:00
Erik Johnston	04eca2589d	Merge pull request #1916 from matrix-org/erikj/push_actions_delete ... Aggregate event push actions	2017-02-16 15:28:58 +01:00
David Baker	474c9aadbe	Allow forgetting rooms you're banned from	2017-02-15 19:32:20 +00:00
Richard van der Hoff	7dcbcca68c	Merge pull request #1921 from matrix-org/rav/fix_key_changes ... Fix bugs in the /keys/changes api	2017-02-15 11:25:16 +00:00
David Baker	fa467e62a9	Merge pull request #1917 from matrix-org/dbkr/make_ban_reasons_work ... Make kick & ban reasons work	2017-02-14 16:10:06 +00:00
David Baker	355d62c499	Make kick & ban reasons work ... We somehow specced APIs with reason strings, preserve the content in the events and even have the clients display them, but failed to actually pass the parameter through to the event content.	2017-02-14 15:10:55 +00:00
David Baker	ce3e583d94	WIP support for msisdn 3pid proxy methods	2017-02-14 15:05:55 +00:00
Richard van der Hoff	fc2f29c1d0	Fix bugs in the /keys/changes api ... * `get_forward_extremeties_for_room` takes a numeric `stream_ordering`. We were passing a `RoomStreamToken`, which meant that it returned the *current* extremities, rather than those corresponding to the `from_token`. However: * `get_state_ids_for_events` required a second ('types') parameter; this meant that a `TypeError` was thrown and we ended up acting as though there was *no* prev state. * `get_state_ids_for_events` actually returns a map from event_id to state dictionary - just looking up the state keys in it again meant that we acted as though there was no prev state. We now check if each member's state has changed since *any* of the extremities. Also add/fix some comments.	2017-02-14 13:59:50 +00:00
Erik Johnston	ce3c8df6df	Less aggressive timers	2017-02-14 13:41:24 +00:00
Erik Johnston	095b45c165	Aggregate event push actions	2017-02-14 13:39:41 +00:00
Erik Johnston	795f8e3fe7	Merge pull request #1873 from matrix-org/erikj/delete_push_actions ... Be more agressive about purging old room event_push_actions	2017-02-14 14:29:04 +01:00
Erik Johnston	d7457c7661	Merge pull request #1914 from matrix-org/erikj/cache_presence ... Cache get_presence storage	2017-02-13 16:59:19 +01:00
Erik Johnston	9e617cd4c2	Cache get_presence storage	2017-02-13 13:50:03 +00:00
Kegan Dougal	d0497425f8	Ordering is important on errbacks so add the cleanup func before creating an ObservableDeferred	2017-02-13 13:49:44 +00:00
Kegan Dougal	808ddf0ae7	Pop the txn from the map in case it has already been deleted somehow	2017-02-13 13:36:15 +00:00
Kegan Dougal	feb15dc99f	Don't cache errors at all	2017-02-13 13:33:12 +00:00
Kegan Dougal	ecd7e36047	http txns: Do not cache error responses ... Previously we did. This meant that, amongst other errors, rate-limiting errors would be cached and prevent messages with that txn ID being sent.	2017-02-13 13:16:48 +00:00
Erik Johnston	6bba80241c	Merge pull request #1912 from matrix-org/markjh/roominitialsync ... Add db functions needed for room initial sync to slave	2017-02-13 12:20:21 +01:00
Mark Haines	3a46280ca3	Add db functions needed for room initial sync to slave	2017-02-13 11:16:53 +00:00
Tyler Smith	df4407d665	Fix typo in config comments. ... Signed-off-by: Tyler Smith <tylersmith.me@gmail.com>	2017-02-11 23:02:57 -08:00
Erik Johnston	1beda9c8a7	Merge branch 'master' of github.com:matrix-org/synapse into develop	2017-02-09 10:46:58 +00:00
Erik Johnston	505bfd82bb	Update version and changelog	2017-02-09 10:41:02 +00:00
Erik Johnston	52cd019a54	Make None check explicit	2017-02-08 16:04:29 +00:00
Erik Johnston	f20cd34858	Merge pull request #1892 from matrix-org/erikj/rejection_fwd_extrem ... Ignore new rejected events when working out forward extremities.	2017-02-08 16:59:06 +01:00
Erik Johnston	7723b4caa4	Ignore new rejected events when working out forward extremeties.	2017-02-08 14:48:06 +00:00
David Baker	063a1251a9	Remove a few aspirational but unused constants ... from the Kegan era	2017-02-08 11:36:08 +00:00
Erik Johnston	af6da6db2d	Merge pull request #1784 from morteza-araby/user-admin ... Administration functionalities	2017-02-06 16:21:10 +01:00
Erik Johnston	38434a7fbb	Bump changelog and version	2017-02-04 08:27:51 +00:00
Erik Johnston	84f600b2ee	Bump changelog and version	2017-02-02 18:58:33 +00:00
Erik Johnston	aec1708c53	Merge branch 'develop' of github.com:matrix-org/synapse into release-v0.19.0	2017-02-02 18:57:05 +00:00
Erik Johnston	f3c8658217	Merge pull request #1879 from matrix-org/erikj/bump_cache_factors ... Bump cache sizes for common membership queries	2017-02-02 18:56:04 +00:00
Erik Johnston	38258a0976	Bump cache sizes for common membership queries	2017-02-02 18:45:55 +00:00
Erik Johnston	a597994fb6	Measure new device list stuff	2017-02-02 18:36:17 +00:00
Erik Johnston	82b3e0851c	Bump version and changelog	2017-02-02 17:15:17 +00:00
Erik Johnston	f8c407a13b	Merge branch 'develop' of github.com:matrix-org/synapse into release-v0.19.0	2017-02-02 16:50:28 +00:00
Erik Johnston	1232ae41cf	Use new get_users_who_share_room_with_user	2017-02-02 15:25:00 +00:00
Erik Johnston	99fa03e8b5	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/device_list_fixes	2017-02-02 15:23:45 +00:00
Erik Johnston	a8331897aa	Merge pull request #1876 from matrix-org/erikj/shared_member_store ... Make presence.get_new_events a bit faster	2017-02-02 15:20:14 +00:00
Erik Johnston	0f3e296cb7	Fix replication	2017-02-02 15:02:03 +00:00
Erik Johnston	6826593b81	sets aren't JSON serializable	2017-02-02 14:55:54 +00:00
Erik Johnston	6b61060b51	Comment	2017-02-02 14:47:15 +00:00
Erik Johnston	46ecd9fd6d	Use stream_ordering_to_exterm for /keys/changes	2017-02-02 14:27:19 +00:00
Erik Johnston	9efcc3f3be	Comment	2017-02-02 13:50:22 +00:00
Erik Johnston	832e9c52ca	Comment	2017-02-02 13:09:56 +00:00
Erik Johnston	54a79c1d37	Make presence.get_new_events a bit faster ... We do this by caching the set of users a user shares rooms with.	2017-02-02 13:07:18 +00:00
Morteza Araby	2849d3f29d	admin,storage: added more administrator functionalities ... administrators can now: - Set displayname of users - Update user avatars - Search for users by user_id - Browse all users in a paginated API - Reset user passwords - Deactivate users Helpers for doing paginated queries has also been added to storage Signed-off-by: Morteza Araby <morteza.araby@ericsson.com>	2017-02-02 14:02:26 +01:00
Erik Johnston	bfe3f5815f	Update changelog	2017-02-02 11:10:41 +00:00
Erik Johnston	cc01eae332	Merge branch 'develop' of github.com:matrix-org/synapse into release-v0.19.0	2017-02-02 11:05:57 +00:00
Erik Johnston	51adaac953	Fix email push in pusher worker ... This was broken when device list updates were implemented, as Mailer could no longer instantiate an AuthHandler due to a dependency on federation sending.	2017-02-02 10:53:36 +00:00
Erik Johnston	10e0737569	Bump version and changelog	2017-02-02 10:02:28 +00:00
Erik Johnston	fac3c03087	Be more agressive about purging old room event_push_actions	2017-02-01 18:27:24 +00:00
Erik Johnston	fbfe44bb4d	Doc args	2017-02-01 17:52:57 +00:00
Erik Johnston	d61a04583e	Comment	2017-02-01 17:35:23 +00:00
Erik Johnston	7e919bdbd0	Include newly joined users in /keys/changes API	2017-02-01 17:33:16 +00:00
Erik Johnston	df4ecff5a9	Correctly raise exceptions for ratelimitng. Ratelimit on 401	2017-02-01 15:42:19 +00:00
Erik Johnston	6d6591880e	Wake sync up for device changes	2017-02-01 15:15:16 +00:00
Erik Johnston	bd84387ac6	Merge pull request #1869 from matrix-org/erikj/device_list_stream ... Implement /keys/changes	2017-02-01 13:25:26 +00:00
Erik Johnston	ebfaff84c9	Merge pull request #1870 from matrix-org/erikj/cache_get_all_new_events ... Add a small cache get_all_new_events	2017-02-01 13:22:02 +00:00
Erik Johnston	73d676dc8b	Comment	2017-02-01 13:17:17 +00:00
Erik Johnston	62f6b86ba7	Merge pull request #1868 from matrix-org/erikj/replication_cache ... Only invalidate membership caches based on the cache stream	2017-02-01 13:12:30 +00:00
Erik Johnston	f6124311fd	Add m.room.member type to query	2017-02-01 11:59:17 +00:00
Erik Johnston	88a4d54883	Merge pull request #1867 from matrix-org/erikj/member_index ... Add an index to make membership queries faster	2017-02-01 11:44:27 +00:00
Erik Johnston	368c88c487	Add a small cache get_all_new_events	2017-02-01 10:50:44 +00:00
Erik Johnston	5deaf9e30b	Up get_latest_event_ids_in_room cache	2017-02-01 10:39:41 +00:00
Erik Johnston	acb501c46d	Comment	2017-02-01 10:32:49 +00:00
Erik Johnston	97479d0c54	Implement /keys/changes	2017-02-01 10:30:03 +00:00
Erik Johnston	06567ec513	Merge pull request #1866 from matrix-org/erikj/device_list_fixes ... Better handle 404 response for federation /send/	2017-02-01 09:44:14 +00:00
Erik Johnston	458b6f4733	Only invalidate membership caches based on the cache stream ... Before we completely invalidated get_users_in_room whenever we updated any current_state_events table. This was way too aggressive.	2017-01-31 16:09:03 +00:00
Erik Johnston	fe08db2713	Remove explicit < 400 check as apparently this is confusing	2017-01-31 15:21:32 +00:00
Erik Johnston	21b7375778	Add an index to make membership queries faster	2017-01-31 15:15:57 +00:00
Erik Johnston	4c0ec15bdc	Comment	2017-01-31 13:53:46 +00:00
Erik Johnston	85c590105f	Comment	2017-01-31 13:46:38 +00:00
Erik Johnston	ae7a132f38	Better handle 404 response for federation /send/	2017-01-31 13:40:09 +00:00
Erik Johnston	ac001dabdc	Merge pull request #1864 from matrix-org/erikj/device_list_fixes ... Fix clearing out old device list outbound pokes	2017-01-31 13:35:35 +00:00
Erik Johnston	bfb3d255b1	Merge pull request #1862 from matrix-org/erikj/presence_update ... Use DB cache of joined users for presence	2017-01-31 13:23:24 +00:00
Erik Johnston	ab55794b6f	Fix deletion of old sent devices correctly	2017-01-31 13:22:41 +00:00
Erik Johnston	d3169e8d28	Only fetch with row ts and count > 1	2017-01-31 11:20:03 +00:00
Erik Johnston	05b9f48ee5	Fix clearing out old device list outbound pokes	2017-01-31 10:08:55 +00:00
Erik Johnston	4b3403ca9b	Stream cache invalidations for room membership storage functions	2017-01-30 17:28:22 +00:00
Erik Johnston	1c13c9f6b6	Don't have such a large cache	2017-01-30 17:12:14 +00:00
Erik Johnston	c7a26b7c32	Fix unit tests	2017-01-30 17:11:24 +00:00
Erik Johnston	fd1c18c088	Use DB cache of joined users for presence	2017-01-30 17:00:24 +00:00
Erik Johnston	c2c9a78db9	Noop device key changes if they're the same	2017-01-30 16:55:04 +00:00
Erik Johnston	e75a779d9e	Fix query	2017-01-30 16:38:20 +00:00
Erik Johnston	828db669ec	Use get_users_in_room and declare it iterable	2017-01-30 16:37:22 +00:00
Erik Johnston	9636b2407d	Merge pull request #1857 from matrix-org/erikj/device_list_stream ... Implement device lists updates over federation	2017-01-30 14:35:21 +00:00
Erik Johnston	3670025e64	Rename func	2017-01-30 14:11:31 +00:00
Erik Johnston	4ac363a168	Remove debug logging	2017-01-30 14:10:12 +00:00
Erik Johnston	d360c97ae1	Clear out old destination pokes.	2017-01-30 10:14:37 +00:00
Erik Johnston	76100203ab	Always use the latest stream_id, sent or unsent	2017-01-30 10:14:25 +00:00
Erik Johnston	d1e1fd6210	Add ts column to device_lists_outbound_pokes	2017-01-27 15:23:48 +00:00
Erik Johnston	252b503fc8	Hook device list updates to replication	2017-01-27 14:31:35 +00:00
Erik Johnston	84a35f32c7	Comment	2017-01-27 10:35:12 +00:00
Erik Johnston	c517a19c2d	Comment	2017-01-27 10:33:26 +00:00
Erik Johnston	738a2867c8	SQL param ordering	2017-01-27 10:31:29 +00:00
Erik Johnston	755adff0e4	User if rather than for	2017-01-27 10:31:06 +00:00
Erik Johnston	888c59c955	Better name	2017-01-27 10:29:47 +00:00
Erik Johnston	f25a4a4692	Remove unused param	2017-01-27 10:27:39 +00:00
Erik Johnston	31aca5589c	Fix on sqlite: use left rather than outer join	2017-01-26 16:55:50 +00:00
Erik Johnston	76d40f4904	Handle users leaving rooms	2017-01-26 16:39:33 +00:00
Erik Johnston	fbfad76c03	Add comments	2017-01-26 16:33:21 +00:00
Erik Johnston	c974116f19	Implement device key caching over federation	2017-01-26 16:07:24 +00:00
Erik Johnston	51e9fe36e4	Fix up sending of m.device_list_update edus	2017-01-25 16:55:21 +00:00
Erik Johnston	2367c5568c	Add basic implementation of local device list changes	2017-01-25 14:27:27 +00:00
Paul "LeoNerd" Evans	10e48d8310	Don't clobber a displayname or avatar_url if provided by an m.room.member event	2017-01-24 18:06:07 +00:00
Erik Johnston	ba8e144554	Merge branch 'erikj/current_state_fix' into develop	2017-01-23 16:15:10 +00:00
Erik Johnston	f5b46482f4	Merge pull request #1840 from matrix-org/erikj/current_state_fix ... Insert delta of current_state_events to be more efficient	2017-01-23 16:14:34 +00:00
Erik Johnston	fdf2a31a51	Typo	2017-01-23 16:14:14 +00:00
Erik Johnston	41dab8a222	Fix bug where current_state_events renamed to current_state_ids	2017-01-23 15:22:48 +00:00
Erik Johnston	c77b24c092	Refactor to calculate state delta outside transaction	2017-01-23 14:51:33 +00:00
Erik Johnston	5d2134d485	Comments	2017-01-20 17:13:24 +00:00
Erik Johnston	a55fa2047f	Insert delta of current_state_events to be more efficient	2017-01-20 17:10:18 +00:00
Erik Johnston	3d9d48fffb	Merge pull request #1836 from matrix-org/erikj/current_state_fix ... Derive current_state_events from state groups	2017-01-20 15:14:05 +00:00
Erik Johnston	d0897dead5	Spelling	2017-01-20 15:05:11 +00:00
Erik Johnston	567aa35b67	Update all call sites after rename	2017-01-20 14:40:31 +00:00
Erik Johnston	f2f40e64a9	Comments	2017-01-20 14:38:13 +00:00
Erik Johnston	4c6a31cd6e	Calculate the forward extremeties once	2017-01-20 14:28:53 +00:00
Erik Johnston	86063d4321	Merge pull request #1835 from matrix-org/erikj/fix_workers ... Make worker listener config backwards compat	2017-01-20 11:55:56 +00:00
Erik Johnston	09eb08f910	Derive current_state_events from state groups	2017-01-20 11:52:51 +00:00
Erik Johnston	97efe99ae9	Make worker listener config backwards compat	2017-01-20 11:45:29 +00:00
Marvin Steadfast	86e6165687	Added default config for turn username and password	2017-01-19 14:35:55 +01:00
Marvin Steadfast	1e38be3a7a	Added username and password for turn server ... It makes it possible to use a turn server that needs a username and password instead of a token.	2017-01-19 14:08:20 +01:00
Erik Johnston	841c228533	Merge pull request #1828 from matrix-org/erikj/iterable_cache_size ... Update LruCache size estimate on clear	2017-01-18 14:57:54 +00:00
Erik Johnston	c430111d0e	Update LruCache size estimate on clear	2017-01-18 14:55:23 +00:00
David Baker	97d3918377	Merge pull request #1811 from aperezdc/unhardcode-riot-urls ... Allow configuring the Riot URL used in notification emails	2017-01-18 14:38:49 +00:00
David Baker	8c5009b628	Lowercase all email addresses before querying db ... Since we store all emails in the DB in lowercase (https://github.com/matrix-org/synapse/pull/1170)	2017-01-18 13:25:56 +00:00
Erik Johnston	ae7b4da4cc	Merge pull request #1823 from matrix-org/erikj/load_events_logs ... Remove loading events logs	2017-01-18 11:07:58 +00:00
Erik Johnston	fc7cae8aa3	Merge pull request #1824 from matrix-org/erikj/retry_host_log ... Lower the not retrying host log line to debug	2017-01-18 11:07:51 +00:00
Erik Johnston	f9058ca785	Merge pull request #1822 from matrix-org/erikj/statE_logging ... Change resolve_state_groups call site logging to DEBUG	2017-01-18 11:02:03 +00:00
Erik Johnston	f648313f98	Merge pull request #1821 from matrix-org/erikj/cache_metrics_string_intern ... Measure metrics of string_cache	2017-01-18 10:57:39 +00:00
Erik Johnston	15f012032c	Merge pull request #1818 from matrix-org/erikj/state_auth_splitout_split ... Optimise state resolution	2017-01-18 10:53:00 +00:00
Erik Johnston	4ec1cf49e2	Lower loading events log to DEBUG	2017-01-17 17:28:32 +00:00
Erik Johnston	f878f64f43	Lower the not retrying host log line to debug	2017-01-17 17:20:39 +00:00
Erik Johnston	5f027d1fc5	Change resolve_state_groups call site logging to DEBUG	2017-01-17 17:07:15 +00:00
Erik Johnston	380dba1020	Measure metrics of string_cache	2017-01-17 17:04:46 +00:00
Erik Johnston	ed4d176152	PEP8	2017-01-17 15:27:28 +00:00
Mark Haines	c6064a7ba6	Only construct sets when necessary	2017-01-17 15:23:07 +00:00
Erik Johnston	a8594fd19f	Use better names	2017-01-17 14:59:03 +00:00
Erik Johnston	7fae460402	Merge pull request #1820 from matrix-org/erikj/push_tools ... Get state at event rather than for room in push	2017-01-17 14:58:26 +00:00
Erik Johnston	37b4c7d8a9	Fix typo in return type	2017-01-17 14:43:32 +00:00
Erik Johnston	e5d2df9c34	Use better variable name	2017-01-17 14:32:53 +00:00
Erik Johnston	04006bb7f0	Get state at event rather than for room in push	2017-01-17 14:31:21 +00:00
Erik Johnston	ce59a2faad	Correctly handle case of rejected events in state res	2017-01-17 14:18:53 +00:00
Erik Johnston	633f97151c	Check event is in state_map	2017-01-17 13:33:54 +00:00
Erik Johnston	e6153e1bd1	Fix couple of federation state bugs	2017-01-17 13:22:34 +00:00
Erik Johnston	5d6bad1b3c	Optimise state resolution	2017-01-17 13:22:19 +00:00
Erik Johnston	e8ecbb6f20	Merge pull request #1812 from matrix-org/erikj/state_auth_splitout_split ... Split out static state methods from StateHandler	2017-01-17 11:55:18 +00:00
Erik Johnston	d6c75cb7c2	Rename and comment tree_to_leaves_iterator	2017-01-17 11:47:03 +00:00
Erik Johnston	1ccd5676e3	Remove needless call to evict()	2017-01-17 11:42:26 +00:00
Erik Johnston	d906206049	Increase state_group_cache_size	2017-01-17 11:31:08 +00:00
Erik Johnston	f85b6ca494	Speed up cache size calculation ... Instead of calculating the size of the cache repeatedly, which can take a long time now that it can use a callback, instead cache the size and update that on insertion and deletion. This requires changing the cache descriptors to have two caches, one for pending deferreds and the other for the actual values. There's no reason to evict from the pending deferreds as they won't take up any more memory.	2017-01-17 11:18:13 +00:00
Erik Johnston	6d00213e80	Use OrderedDict in ExpiringCache	2017-01-16 15:33:22 +00:00
Erik Johnston	897f8752da	Up cache max entries for state	2017-01-16 15:08:17 +00:00
Erik Johnston	beda469bc6	Put staticmethods at module level	2017-01-16 15:05:24 +00:00
Erik Johnston	46aebbbcbf	Add support for 'iterable' to ExpiringCache	2017-01-16 14:57:23 +00:00
Erik Johnston	01521299c7	Increase cache size limit	2017-01-16 11:56:51 +00:00
Erik Johnston	2fae34bd2c	Optionally measure size of cache by sum of length of values	2017-01-13 17:46:17 +00:00
Erik Johnston	95a22ae194	Merge pull request #1810 from matrix-org/erikj/state_auth_splitout_split ... Split out static auth methods from Auth object	2017-01-13 16:32:27 +00:00
Erik Johnston	ec0a523ac3	Split out static state methods from StateHandler	2017-01-13 15:25:06 +00:00
Erik Johnston	e178feca3f	Remove unused function	2017-01-13 15:16:45 +00:00
Erik Johnston	f0325a9ccc	Merge pull request #1793 from matrix-org/erikj/change_device_inbox_index ... Change device_inbox stream index to include user	2017-01-13 15:14:51 +00:00
Erik Johnston	c050f493dd	Add comment	2017-01-13 15:14:41 +00:00
Adrian Perez de Castro	a3e4a198e3	Allow configuring the Riot URL used in notification emails ... The URLs used for notification emails were hardcoded to use either matrix.to or vector.im; but for self-hosted setups where Riot is also self-hosted it may be desirable to allow configuring an alternative Riot URL. Fixes #1809. Signed-off-by: Adrian Perez de Castro <aperez@igalia.com>	2017-01-13 17:12:04 +02:00
Erik Johnston	8b2fa38256	Split event auth code into seperate module	2017-01-13 15:07:32 +00:00
Erik Johnston	641ccdbb14	Merge pull request #1795 from matrix-org/erikj/port_defaults ... Restore default bind address	2017-01-13 13:02:59 +00:00
Erik Johnston	ebf94aff8d	Fix spurious Unhandled Error log lines	2017-01-12 17:19:47 +00:00
Erik Johnston	bf5c9706d9	Remove full_twisted_stacktraces option ... The debug 'full_twisted_stacktraces' flag caused synapse to rewrite twisted deferreds to always fire the callback on the next reactor tick. This was to force the deferred to always store the stacktraces on exceptions, and thus be more likely to have a full stacktrace when it reaches the final error handlers and gets printed to the logs. Dynamically rewriting things is generally bad, and in particular this change violates assumptions of various bits of Twisted. This wouldn't necessarily be so bad, but it turns out this option has been turned on on some production servers. Turning the option can cause e.g. #1778. For now, lets just entirely nuke this option.	2017-01-12 10:32:52 +00:00
Erik Johnston	7b62d0bc70	Add missing None check	2017-01-11 10:57:03 +00:00
Erik Johnston	7e6c2937c3	Split out static auth methods from Auth object	2017-01-10 18:16:54 +00:00
Erik Johnston	b1dfd20292	Pop bind_address	2017-01-10 17:23:18 +00:00
Erik Johnston	edd6cdfc9a	Restore default bind address	2017-01-10 17:21:41 +00:00
Erik Johnston	8a0fddfd73	Remove spurious for..else..	2017-01-10 16:30:53 +00:00
Erik Johnston	d524bc9110	Merge pull request #1792 from matrix-org/erikj/limit_cache_prefill_device ... Limit number of entries to prefill from cache	2017-01-10 15:42:00 +00:00
Erik Johnston	d2b00d0866	Merge pull request #1790 from matrix-org/erikj/linearizer ... Add paranoia exception catch in Linearizer	2017-01-10 15:38:30 +00:00
Erik Johnston	ab655dca33	Explicitly close the cursor	2017-01-10 15:15:25 +00:00
Erik Johnston	5a32e9273e	Don't disable autocommit	2017-01-10 15:11:27 +00:00
Erik Johnston	caddadfc5a	Change device_inbox stream index to include user ... This makes fetching the nost recently changed users much tricker, and brings it in line with e.g. presence_stream indices.	2017-01-10 15:04:57 +00:00
Erik Johnston	dd52d4de4c	Limit number of entries to prefill from cache ... Some tables, like device_inbox, take a long time to query at startup for the stream change cache prefills. This is likely because they are slower growing streams and so are more fragmented on disk. For now, lets pull fewer entries out to make startup quicker. In future, we should add a better index to make it even faster.	2017-01-10 14:34:50 +00:00
Mark Haines	32019c9897	Log which files we saved attachments to in the media_repository	2017-01-10 14:19:50 +00:00
Erik Johnston	657488113e	Merge pull request #1789 from matrix-org/erikj/decouple_presence ... Don't block messages sending on bumping presence	2017-01-10 14:06:05 +00:00
Erik Johnston	3b4de17d2b	Comment	2017-01-10 14:05:53 +00:00
Erik Johnston	7d0981b312	Merge pull request #1787 from matrix-org/erikj/linearize_member ... Linearize updates to membership via PUT /state/	2017-01-10 14:04:54 +00:00
Erik Johnston	f477370c0c	Add paranoia exception catch in Linearizer	2017-01-10 14:04:13 +00:00
Erik Johnston	586f474a44	Don't block messages sending on bumping presence	2017-01-10 12:46:00 +00:00
Erik Johnston	6823fe5241	Linearize updates to membership via PUT /state/	2017-01-09 18:25:13 +00:00
Erik Johnston	f7085ac84f	Name linearizer's for better logs	2017-01-09 17:17:10 +00:00
Erik Johnston	9898bbd9dc	Merge branch 'master' of github.com:matrix-org/synapse into develop	2017-01-09 14:51:17 +00:00
Erik Johnston	9a8ae6f1bf	Bump version and changelog	2017-01-09 14:47:56 +00:00
Matthew Hodgson	2f4b2f4783	gah, fix mangled merge of 0.18.7 into develop	2017-01-07 04:00:42 +00:00
Matthew	6d363cea9d	Merge branch 'release-v0.18.7' into develop	2017-01-07 03:46:16 +00:00
Matthew	f0e4bac64e	bump changelog & version	2017-01-07 03:45:38 +00:00
Matthew	4304e7e593	do the discard check in the right place to avoid grabbing dependent events	2017-01-07 03:44:18 +00:00
Matthew Hodgson	8c48971b51	Merge branch 'release-v0.18.7' into develop	2017-01-07 02:23:37 +00:00
Matthew	e10c527930	Discard PDUs from invalid origins due to #1753 in 0.18.[56]	2017-01-07 02:13:14 +00:00
Matthew Hodgson	2f5be2d8dc	oops, this should have been rc1	2017-01-07 01:11:56 +00:00
Matthew Hodgson	4086026524	move logging to right place	2017-01-07 00:41:46 +00:00
Matthew	19e2fb4386	bump version	2017-01-06 23:38:22 +00:00
Matthew	b2850e62db	fix typo breaking the fix to #1753	2017-01-06 23:23:37 +00:00
Mark Haines	06c00bd19b	Merge branch 'release-v0.18.6' into develop	2017-01-06 14:46:27 +00:00
Mark Haines	b42a972b71	Bump version and changelog	2017-01-06 14:44:28 +00:00
Patrik Oldsberg	1ef6084b75	handlers/room_member: fix guest access check when joining rooms ... Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>	2017-01-06 14:36:56 +00:00
Matthew Hodgson	bd85434cb3	Merge branch 'release-v0.18.6' into develop	2017-01-05 13:58:19 +00:00
Mark Haines	c18f7fc410	Fix flake8 and update changelog	2017-01-05 13:50:22 +00:00
Matthew Hodgson	dafd50d178	Merge pull request #1767 from matrix-org/matthew/resolve_state_group_logging ... log call paths for resolve_state_group	2017-01-05 13:47:42 +00:00
Matthew Hodgson	883ff92a7f	Fix case	2017-01-05 13:45:02 +00:00
Matthew Hodgson	d79d165761	add logging for all the places we call resolve_state_groups. my kingdom for a backtrace that actually works.	2017-01-05 13:40:39 +00:00
Matthew Hodgson	8cfc0165e9	fix annoying typos	2017-01-05 13:39:43 +00:00
Mark Haines	62451800e7	Bump version and changelog to v0.18.6-rc3	2017-01-05 13:36:10 +00:00
Matthew Hodgson	b31ed22738	Merge branch 'release-v0.18.6' into develop	2017-01-05 13:03:02 +00:00
Mark Haines	dd3df11c55	More logging for the linearizer and for get_events	2017-01-05 12:32:47 +00:00
Matthew Hodgson	468749c9fc	fix comment	2017-01-05 12:00:11 +00:00
Matthew Hodgson	eedf400d05	limit total timeout for get_missing_events to 10s	2017-01-05 11:58:15 +00:00
Matthew Hodgson	8e82611f37	fix comment	2017-01-05 11:44:44 +00:00
Mark Haines	f784980d2b	Only send events that originate on this server. ... Or events that are sent via the federation "send_join" API. This should match the behaviour from before v0.18.5 and #1635 landed.	2017-01-05 11:26:30 +00:00
Mark Haines	e02bdaf08b	Get the destinations from the state from before the event ... Rather than the state after then event.	2017-01-04 15:17:15 +00:00
Mark Haines	b6b67715ed	Send ALL membership events to the server that was affected. ... Send all membership changes to the server that was affected. This ensures that if the last member of a room on a server was kicked or banned they get told about it.	2017-01-04 13:56:20 +00:00
Matthew Hodgson	555d702e34	limit total timeout for get_missing_events to 10s	2016-12-31 15:21:37 +00:00
Matthew Hodgson	899a3a1268	Merge branch 'release-v0.18.6' into develop	2016-12-31 02:38:26 +00:00
Mark Haines	f3de4f8cb7	Bump version and changelog	2016-12-30 20:21:04 +00:00
Mark Haines	62ce3034f3	s/aquire/acquire/g	2016-12-30 20:04:44 +00:00
Mark Haines	0aff09f6c9	Add more useful logging when we block fetching events	2016-12-30 20:00:44 +00:00
Mark Haines	cc50b1ae53	Remove fallback from get_missing_events. ... get_missing_events used to fallback to fetching the missing events individually requesting from every server in the room, one by one.e This could be unacceptably slow, possibly causing #1732	2016-12-30 18:13:15 +00:00
Mark Haines	f576c34594	Merge remote-tracking branch 'origin/release-v0.18.6' into develop	2016-12-30 15:13:49 +00:00
Mark Haines	822cb39dfa	Use the new twisted logging framework. ... Hopefully adding an observer to the new framework will avoid a memory leak https://twistedmatrix.com/trac/ticket/8164	2016-12-30 11:09:24 +00:00
Mark Haines	342fb8dae9	Merge branch 'release-v0.18.6' into develop	2016-12-29 17:33:46 +00:00
Mark Haines	f023be9293	Bump changelog and version	2016-12-29 16:18:04 +00:00
Mark Haines	97ffc5690b	Manually abort the underlying TLS connection. ... The abort() method calls loseConnection() which tries to shutdown the TLS connection cleanly. We now call abortConnection() directly which should promptly close both the TLS connection and the underlying TCP connection. I also added some TODO markers to consider cancelling the old previous timeout rather than checking time.time(). But given how urgently we want to get this code released I'd rather leave the existing code with the duplicate timeouts and the time.time() check.	2016-12-29 15:51:04 +00:00
Erik Johnston	b4bc6fef5b	Respect long_retries param and default to off	2016-12-29 00:58:34 +00:00
Erik Johnston	68030fd37b	Spelling and comments	2016-12-29 00:10:49 +00:00
Erik Johnston	b7336ff32d	Clean up	2016-12-29 00:09:33 +00:00
Erik Johnston	5b6672c66d	Wrap connections in an N minute timeout to ensure they get reaped correctly	2016-12-29 00:06:53 +00:00
David Baker	84cf00c645	Fix another comment typo	2016-12-21 09:51:43 +00:00
David Baker	0c88ab1844	Add /account/3pid/delete endpoint ... Also fix a typo in a comment	2016-12-20 18:27:30 +00:00
Johannes Löthberg	702c020e58	Fix check for bind_address ... The empty string is a valid setting for the bind_address option, so explicitly check for None here instead. Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>	2016-12-20 01:37:50 +01:00
Matthew Hodgson	09f15918be	Merge pull request #1711 from matrix-org/matthew/utf8-password-change ... fix ability to change password to a non-ascii one	2016-12-20 00:02:13 +00:00
Matthew Hodgson	da2c8f3c94	Merge pull request #1709 from kyrias/bind_addresses ... Add support for specifying multiple bind addresses	2016-12-19 23:49:34 +00:00
Matthew Hodgson	a58e4e0d48	Merge pull request #1696 from kyrias/ipv6 ... IPv6 support	2016-12-19 23:49:07 +00:00
Matthew Hodgson	f2a5aebf98	fix ability to change password to a non-ascii one ... https://github.com/vector-im/riot-web/issues/2658	2016-12-18 22:25:21 +00:00
Johannes Löthberg	a9c1b419a9	Bump twisted dependency ... At least 16.0.0 is needed for wrapClientTLS support. Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>	2016-12-18 23:16:43 +01:00
Johannes Löthberg	f5cd5ebd7b	Add IPv6 comment to default config ... Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>	2016-12-18 23:14:32 +01:00
Johannes Löthberg	c95e9fff99	Make default homeserver config use bind_addresses ... Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>	2016-12-18 21:51:56 +01:00
Johannes Löthberg	7dfd70fc83	Add support for specifying multiple bind addresses ... Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>	2016-12-18 21:51:56 +01:00
Erik Johnston	b2f8642d3d	Cache network room list queries.	2016-12-16 16:11:43 +00:00
Erik Johnston	b9b6d17ab1	Bump version and changelog	2016-12-16 10:18:02 +00:00
Richard van der Hoff	edc6a1e4f9	Add some logging for syncing to_device events ... Attempt to track down the loss of to_device events (https://github.com/vector-im/riot-web/issues/2711 etc).	2016-12-15 18:16:10 +00:00
Erik Johnston	35129ac998	Merge pull request #1698 from matrix-org/erikj/room_list ... Fix caching on public room list	2016-12-15 15:40:28 +00:00
Erik Johnston	aa1336c00a	Merge pull request #1700 from matrix-org/erikj/backfill_filter ... Fix /backfill returning events it shouldn't	2016-12-15 14:21:30 +00:00
Marcin Bachry	24c16fc349	Fix crash in url preview when html tag has no text ... Signed-off-by: Marcin Bachry <hegel666@gmail.com>	2016-12-14 22:38:18 +01:00
Erik Johnston	b8255eba26	Comment	2016-12-14 13:49:54 +00:00
Erik Johnston	b2999a7055	Fix /backfill returning events it shouldn't	2016-12-14 13:41:45 +00:00
Erik Johnston	c3208e45c9	Fixup membership query	2016-12-14 10:46:58 +00:00
Erik Johnston	9d95351cad	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/room_list	2016-12-13 17:57:08 +00:00
Erik Johnston	1de53a7a1a	Fix caching on public room list	2016-12-13 17:33:24 +00:00
Erik Johnston	b3d398343e	Bump changelog and version	2016-12-13 11:07:27 +00:00
Johannes Löthberg	0648e76979	Remove spurious newline ... Apparently I just removed the spaces instead... Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>	2016-12-12 18:41:30 +01:00
Erik Johnston	8588d0eb3d	Merge pull request #1697 from matrix-org/erikj/fix_bg_member ... Fix background update that prematurely stopped	2016-12-12 17:20:17 +00:00
Erik Johnston	1574b839e0	Merge pull request #1676 from matrix-org/erikj/room_list ... Add new API appservice specific public room list	2016-12-12 17:00:10 +00:00
Erik Johnston	7ec2bf9b77	Fix background update that prematurely stopped	2016-12-12 16:54:58 +00:00
Erik Johnston	2bf5a47b3e	Rename network_id to instance_id on client side	2016-12-12 16:05:45 +00:00
Johannes Löthberg	d3bd94805f	Fixup for #1689 and #1690 ... Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>	2016-12-12 16:32:47 +01:00
Erik Johnston	09cbcb78d3	Add cache to get_public_room_ids_at_stream_id	2016-12-12 14:41:51 +00:00
Erik Johnston	631376e2ac	Notify replication. Use correct network_id	2016-12-12 14:28:15 +00:00
Richard van der Hoff	abed247182	Remove unspecced GET endpoints for e2e keys ... GET /keys/claim is a terrible idea, since it isn't idempotent; also it throws 500 errors if you call it without all the right params. GET /keys/query is arguable, but it's unspecced, so let's get rid of it too to stop people relying on unspecced APIs.	2016-12-12 12:31:40 +00:00
Erik Johnston	d45c984653	Docstring	2016-12-12 11:00:27 +00:00
Erik Johnston	d53a80af25	Merge pull request #1620 from matrix-org/erikj/concurrent_room_access ... Limit the number of events that can be created on a given room concurrently	2016-12-12 10:30:23 +00:00
Richard van der Hoff	85cd30b1fd	Merge pull request #1686 from matrix-org/rav/fix_federation_key_fails ... E2E key query: handle federation fails	2016-12-12 09:33:39 +00:00
Richard van der Hoff	deca951241	Remove unused import	2016-12-12 09:24:35 +00:00
Glyph	9f07f4c559	IPv6 support for endpoint.py ... Similar to https://github.com/matrix-org/synapse/pull/1689, but for endpoint.py	2016-12-11 11:10:32 +01:00
Glyph	6e18805ac2	IPv6 support for client.py ... This is an (untested) general sketch of how to use wrapClientTLS to implement TLS over IPv6, as well as faster connections over IPv4.	2016-12-11 11:10:32 +01:00
Richard van der Hoff	efa4ccfaee	E2E key query: handle federation fails ... Don't fail the whole request if we can't connect to a particular server.	2016-12-09 18:31:01 +00:00
Erik Johnston	fbaf868f62	Correctly handle timeout errors	2016-12-09 16:30:29 +00:00
Erik Johnston	be14c24cea	Fix rare notifier bug where listeners dont timeout ... There was a race condition that caused the notifier to 'miss' the timeout notification, since there were no other checks for the timeout this caused listeners to get stuck in a loop until something happened.	2016-12-09 15:43:18 +00:00
Erik Johnston	1697f6a323	Merge pull request #1680 from matrix-org/erikj/joined_rooms ... Add new room membership APIs	2016-12-09 11:35:48 +00:00
Erik Johnston	52d12ca782	Add /room/<room_id>/joined_members API ... This returns the currently joined members in the room with their display names and avatar urls. This is more efficient than /members for large rooms where you don't need the full events.	2016-12-08 13:32:07 +00:00
Erik Johnston	c45d8e9ba2	Add profile data to the room_membership table for joins	2016-12-08 13:08:41 +00:00
Richard van der Hoff	b08f76bd23	Fix ignored read-receipts ... Don't ignore read-receipts which arrive in the same EDU as a read-receipt for an old event.	2016-12-08 12:13:01 +00:00
Richard van der Hoff	bd07a35c29	Fix result of insert_receipt ... This should fix the absence of notifications when new receipts arrive.	2016-12-08 12:11:34 +00:00
Erik Johnston	de796f27e6	Add joined_rooms servlet	2016-12-08 11:39:03 +00:00
Erik Johnston	2687af82d4	Comments	2016-12-07 09:58:33 +00:00
Erik Johnston	3727d66a0e	Don't include appservice id	2016-12-06 17:04:26 +00:00
Erik Johnston	f32fb65552	Add new API appservice specific public room list	2016-12-06 16:12:27 +00:00
Richard van der Hoff	1529c19675	Prevent user tokens being used as guest tokens (#1675) ... Make sure that a user cannot pretend to be a guest by adding 'guest = True' caveats.	2016-12-06 15:31:37 +00:00
Erik Johnston	9609c91e7d	Merge pull request #653 from matrix-org/erikj/preset_guest_join ... Enable guest access for private rooms by default	2016-12-05 17:47:14 +00:00
Erik Johnston	338df4f409	Merge pull request #1649 from matrix-org/dbkr/log_ui_auth_args ... Log the args that we have on UI auth completion	2016-12-05 16:40:58 +00:00
pik	c46e7a9c9b	Bugfix: Console logging handler missing default filter	2016-12-03 20:14:58 -03:00
Johannes Löthberg	32c8b5507c	preview_url_resource: Ellipsis must be in unicode string ... Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>	2016-12-01 13:12:13 +01:00
Richard van der Hoff	6841d8ff55	Fix doc-string ... Remove refresh_token reference	2016-12-01 11:42:17 +00:00
Richard van der Hoff	aa09d6b8f0	Rip out more refresh_token code ... We might as well treat all refresh_tokens as invalid. Just return a 403 from /tokenrefresh, so that we don't have a load of dead, untestable code hanging around. Still TODO: removing the table from the schema.	2016-11-30 17:40:18 +00:00
Richard van der Hoff	dc4b23e1a1	Merge branch 'develop' into rav/no_more_refresh_tokens	2016-11-30 17:10:04 +00:00
Richard van der Hoff	8379a741cc	Merge pull request #1660 from matrix-org/rav/better_content_type_validation ... More intelligent Content-Type parsing	2016-11-30 16:54:03 +00:00
Richard van der Hoff	321fe5c44c	Merge pull request #1656 from matrix-org/rav/remove_time_caveat ... Stop putting a time caveat on access tokens	2016-11-30 16:53:20 +00:00
Richard van der Hoff	b5b3a7e867	More intelligent Content-Type parsing ... Content-Type is allowed to contain options (`; charset=utf-8`, for instance). We should allow that.	2016-11-30 15:07:32 +00:00
Richard van der Hoff	4febfe47f0	Comments ... Update comments in verify_macaroon	2016-11-30 07:36:32 +00:00
Richard van der Hoff	77eca2487c	Merge pull request #1653 from matrix-org/rav/guest_e2e ... Implement E2E for guests	2016-11-29 17:41:35 +00:00
Richard van der Hoff	1c4f05db41	Stop putting a time caveat on access tokens ... The 'time' caveat on the access tokens was something of a lie, since we weren't enforcing it; more pertinently its presence stops us ever adding useful time caveats. Let's move in the right direction by not lying in our caveats.	2016-11-29 16:49:41 +00:00
Richard van der Hoff	5c4edc83b5	Stop generating refresh tokens ... Since we're not doing refresh tokens any more, we should start killing off the dead code paths. /tokenrefresh itself is a bit of a thornier subject, since there might be apps out there using it, but we can at least not generate refresh tokens on new logins.	2016-11-28 10:13:01 +00:00
Richard van der Hoff	b6146537d2	Merge pull request #1655 from matrix-org/rav/remove_redundant_macaroon_checks ... Remove redundant list of known caveat prefixes	2016-11-25 16:57:19 +00:00
Richard van der Hoff	f62b69e32a	Allow guest access to endpoints for E2E ... Expose /devices, /keys, and /sendToDevice to guest users, so that they can use E2E.	2016-11-25 15:26:34 +00:00
Richard van der Hoff	7f02e4d008	Give guest users a device_id ... We need to create devices for guests so that they can use e2e, but we don't have anywhere to store it, so just use a fixed one.	2016-11-25 15:25:30 +00:00
Erik Johnston	11bfe438a2	Use correct var	2016-11-24 15:26:53 +00:00
Erik Johnston	aaecffba3a	Correctly handle 500's and 429 on federation	2016-11-24 15:04:49 +00:00
Richard van der Hoff	e1d7c96814	Remove redundant list of known caveat prefixes ... Also add some comments.	2016-11-24 12:38:17 +00:00
Erik Johnston	7e03f9a484	Bump version and changelog	2016-11-24 12:29:58 +00:00
Erik Johnston	46ca345b06	Don't send old events as federation	2016-11-24 12:29:02 +00:00
Erik Johnston	f36ea03741	Bump changelog and version	2016-11-24 11:08:01 +00:00
David Baker	c9d4e7b716	Clarify that creds doesn not contain passwords.	2016-11-24 10:54:59 +00:00
David Baker	f681aab895	Log the args that we have on UI auth completion ... This will be super helpful for debugging if we have more registration woes.	2016-11-24 10:11:45 +00:00
Erik Johnston	1985860c6e	Comment	2016-11-23 15:59:59 +00:00
Erik Johnston	2ac516850b	More efficient notif count queries	2016-11-23 15:57:04 +00:00
Erik Johnston	302fbd218d	Merge pull request #1635 from matrix-org/erikj/split_out_fed_txn ... Split out federation transaction sending to a worker	2016-11-23 15:39:12 +00:00
Erik Johnston	b2d6e63b79	Merge pull request #1641 from matrix-org/erikj/as_pushers ... Ignore AS users when fetching push rules	2016-11-23 15:21:52 +00:00
Erik Johnston	feec718265	Shuffle receipt handler around so that worker apps don't need to load it	2016-11-23 15:14:24 +00:00
Erik Johnston	26072df6af	Ensure only main or federation_sender process can send federation traffic	2016-11-23 14:09:47 +00:00
Erik Johnston	b69f76c106	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/split_out_fed_txn	2016-11-23 11:31:53 +00:00
Erik Johnston	4d9b5c60f9	Comment	2016-11-23 11:11:41 +00:00
Erik Johnston	0163466d72	Ignore AS users when fetching push rules ... By ignoring AS users early on when fetching push rules for a room we can avoid needlessly hitting the DB and filling up the caches.	2016-11-23 11:01:01 +00:00
Erik Johnston	4c79a63fd7	Explicit federation ack	2016-11-23 10:40:44 +00:00
Erik Johnston	54fed21c04	Fix tests and flake8	2016-11-22 18:18:31 +00:00
Erik Johnston	90565d015e	Invalidate retry cache in both directions	2016-11-22 17:45:44 +00:00
Kegan Dougal	83bcdcee61	Return early on /sync code paths if a '*' filter is used ... This is currently very conservative in that it only does this if there is no `since` token. This limits the risk to clients likely to be doing one-off syncs (like bridges), but does mean that normal human clients won't benefit from the time savings here. If the savings are large enough, I would consider generalising this to just check the filter.	2016-11-22 16:38:35 +00:00
Kegsay	d4a459f7cb	Merge pull request #1638 from matrix-org/kegan/sync-event-fields ... Implement "event_fields" in filters	2016-11-22 14:02:38 +00:00
Kegan Dougal	c3d963ac24	Review comments	2016-11-22 13:42:11 +00:00
Kegan Dougal	6d4e6d4cba	Also check for dict since sometimes they aren't frozen	2016-11-22 10:39:41 +00:00
Erik Johnston	baf9e74a73	Merge branch 'master' of github.com:matrix-org/synapse into develop	2016-11-22 10:31:48 +00:00
Kegan Dougal	cea4e4e7b2	Glue only_event_fields into the sync rest servlet	2016-11-22 10:14:05 +00:00
Kegan Dougal	0a8b0eeca1	More tests	2016-11-22 09:59:27 +00:00
Erik Johnston	51e89709aa	Comments	2016-11-21 17:59:39 +00:00
Kegan Dougal	70a2157b64	Start adding some tests	2016-11-21 17:52:45 +00:00
Kegan Dougal	f97511a1f3	Move event_fields filtering to serialize_event ... Also make it an inclusive not exclusive filter, as the spec demands.	2016-11-21 17:42:16 +00:00
Erik Johnston	88d85ebae1	Add some metrics	2016-11-21 17:36:05 +00:00
Erik Johnston	50934ce460	Comments	2016-11-21 16:55:23 +00:00
Kegan Dougal	e90fcd9edd	Add filter_event_fields and filter_field to FilterCollection	2016-11-21 15:18:18 +00:00
Erik Johnston	9687e039e7	Remove explicit calls to send_pdu	2016-11-21 14:48:51 +00:00
Kegan Dougal	a2a6c1c22f	Fail with a coherent error message if /sync?filter= is invalid	2016-11-21 13:15:25 +00:00
Erik Johnston	524d61bf7e	Fix tests	2016-11-21 11:53:02 +00:00
Erik Johnston	7c9cdb2245	Store federation stream positions in the database	2016-11-21 11:33:08 +00:00
Mark Haines	a289150943	Fix flake8	2016-11-18 17:15:02 +00:00
David Baker	544722bad2	Work around client replacing reg params ... Works around https://github.com/vector-im/vector-android/issues/715 and equivalent for iOS	2016-11-18 17:07:35 +00:00
Erik Johnston	f8ee66250a	Handle sending events and device messages over federation	2016-11-17 15:48:04 +00:00
Erik Johnston	ed787cf09e	Hook up the send queue and create a federation sender worker	2016-11-16 17:34:44 +00:00
Erik Johnston	1587b5a033	Add initial cut of federation send queue	2016-11-16 14:47:52 +00:00
Erik Johnston	59ef517e6b	Use new federation_sender DI	2016-11-16 14:47:52 +00:00
Erik Johnston	847d5db1d1	Add transaction queue and transport layer to DI	2016-11-16 14:47:52 +00:00
Erik Johnston	daec6fc355	Move logic into transaction_queue	2016-11-16 14:47:52 +00:00
Erik Johnston	0e830d3770	Rename transaction queue functions to send_*	2016-11-16 14:47:52 +00:00
Erik Johnston	dc6cede78e	Merge pull request #1628 from matrix-org/erikj/ldap_split_out ... Use external ldap auth pacakge	2016-11-15 16:53:34 +00:00
Erik Johnston	c7546b3cdb	Merge pull request #1617 from matrix-org/erikj/intern_state_dict ... Correctly intern keys in state cache	2016-11-15 16:45:55 +00:00
Erik Johnston	d56c39cf24	Use external ldap auth pacakge	2016-11-15 13:03:19 +00:00
Erik Johnston	f9d156d270	New Flake8 fixes	2016-11-15 11:22:29 +00:00
Erik Johnston	9d58ccc547	Bump changelog and version	2016-11-14 15:05:04 +00:00
Kegsay	9355a5c42b	Merge pull request #1624 from matrix-org/kegan/idempotent-requests ... Store Promise<Response> instead of Response for HTTP API transactions	2016-11-14 12:45:30 +00:00
Kegan Dougal	3991b4cbdb	Clean transactions based on time. Add HttpTransactionCache tests.	2016-11-14 11:19:24 +00:00
Kegan Dougal	af4a1bac50	Move .observe() up to the cache to make things neater	2016-11-14 09:52:41 +00:00
Erik Johnston	0964005d84	Merge pull request #1625 from DanielDent/patch-1 ... Add support for durations in minutes	2016-11-12 11:20:46 +00:00
Daniel Dent	1c93cd9f9f	Add support for durations in minutes	2016-11-12 00:10:23 -08:00
Kegan Dougal	8ecaff51a1	Review comments	2016-11-11 17:47:03 +00:00
Kegan Dougal	f6c48802f5	More flake8	2016-11-11 15:08:24 +00:00
Kegan Dougal	a88bc67f88	Flake8 and fix whoopsie	2016-11-11 15:02:29 +00:00
Kegan Dougal	42c43cfafd	Use ObservableDeferreds instead of Deferreds as they behave as intended	2016-11-11 14:54:10 +00:00
Kegan Dougal	c7daf3136c	Use observable deferreds because they are sane	2016-11-11 14:13:32 +00:00
Erik Johnston	64038b806c	Comments	2016-11-11 10:42:08 +00:00
Erik Johnston	2bd4513a4d	Limit the number of events that can be created on a given room concurretnly	2016-11-10 16:44:35 +00:00
Erik Johnston	d073cb7ead	Add Limiter: limit concurrent access to resource	2016-11-10 16:29:51 +00:00
Kegan Dougal	8a8ad46f48	Flake8	2016-11-10 15:22:11 +00:00
Kegan Dougal	2771447c29	Store Promise<Response> instead of Response for HTTP API transactions ... This fixes a race whereby: - User hits an endpoint. - No cached transaction so executes main code. - User hits same endpoint. - No cache transaction so executes main code. - Main code finishes executing and caches response and returns. - Main code finishes executing and caches response and returns. This race is common in the wild when Synapse is struggling under load. This commit fixes the race by: - User hits an endpoint. - Caches the promise to execute the main code and executes main code. - User hits same endpoint. - Yields on the same promise as the first request. - Main code finishes executing and returns, unblocking both requests.	2016-11-10 14:49:26 +00:00
Erik Johnston	ac507e7ab8	Don't assume providers raise ConfigError's	2016-11-08 17:23:28 +00:00
Erik Johnston	e6651e8046	Merge branch 'master' of github.com:matrix-org/synapse into develop	2016-11-08 14:43:49 +00:00
Erik Johnston	3c09818d91	Bump version and changelog	2016-11-08 14:39:55 +00:00
Erik Johnston	27d3f2e7ab	Explicitly set authentication mode in ldap3 ... This only makes a difference for versions of ldap3 before 1.0, but a) its best to be explicit and b) there are distributions that package ancient versions for ldap3 (e.g. debian).	2016-11-08 14:35:25 +00:00
Erik Johnston	17e0a58020	Merge pull request #1615 from matrix-org/erikj/limit_prev_events ... Limit the number of prev_events of new events	2016-11-08 12:06:15 +00:00
Erik Johnston	587d8ac60f	Correctly intern keys in state cache	2016-11-08 11:53:25 +00:00
Erik Johnston	a4632783fb	Sample correctly	2016-11-08 11:20:26 +00:00
Erik Johnston	24772ba56e	Respect use_frozen_dicts option in workers	2016-11-08 11:07:18 +00:00
Erik Johnston	eeda4e618c	Limit the number of prev_events of new events	2016-11-08 11:02:29 +00:00
Erik Johnston	d24197bead	Merge pull request #1198 from euank/more-ip-blacklist ... default config: blacklist more internal ips	2016-11-07 09:41:34 +00:00
Euan Kemp	c6bbad109b	default config: blacklist more internal ips	2016-11-06 17:02:25 -08:00
Erik Johnston	63772443e6	Comment	2016-11-04 10:53:42 +00:00
Erik Johnston	a3f6576084	Remove unused but buggy function	2016-11-04 10:48:20 +00:00
Paul "LeoNerd" Evans	2938a00825	Rename the python-specific metrics now the docs claim that we have done	2016-11-03 17:03:52 +00:00
Paul "LeoNerd" Evans	5219f7e060	Since we don't export per-filetype fd counts any more, delete all the code related to that too	2016-11-03 16:41:32 +00:00
Paul "LeoNerd" Evans	93ebeb2aa8	Remove now-unused 'resource' import	2016-11-03 16:37:09 +00:00
Paul "LeoNerd" Evans	c1b077cd19	Now we have new-style metrics don't bother exporting legacy-named process ones	2016-11-03 16:34:16 +00:00
Erik Johnston	64c6566980	Remove spurious comment	2016-11-03 15:04:32 +00:00
Erik Johnston	8fd4d9129f	Replace postgres GIN with GIST ... This is because GIN can be slow to write too, especially when the table gets large.	2016-11-03 15:00:03 +00:00
David Baker	9084720993	Don't error on non-ascii passwords	2016-11-03 10:42:14 +00:00
Mark Haines	b1c27975d0	Set CORs headers on responses from the media repo	2016-11-02 11:29:25 +00:00
Erik Johnston	2746e805fe	Merge pull request #1188 from matrix-org/erikj/sent_transactions ... Remove sent_transactions table.	2016-11-01 14:52:55 +00:00
Erik Johnston	3c91c5b216	Bump version and changelog	2016-11-01 13:06:36 +00:00
Erik Johnston	760469c812	Continue to clean up received_transactions	2016-11-01 11:42:08 +00:00
Erik Johnston	f09d2b692f	Removed unused stuff	2016-10-31 17:10:56 +00:00
Erik Johnston	4c3eb14d68	Increase batching of sent transaction inserts ... This should further reduce the number of individual inserts, transactions and updates that are required for keeping sent_transactions up to date.	2016-10-31 16:07:45 +00:00
Mark Haines	82e71a259c	Bump changelog and version	2016-10-28 11:16:05 +01:00
Mark Haines	490b97d3e7	Merge branch 'develop' into release-v0.18.2	2016-10-28 11:12:31 +01:00
Paul "LeoNerd" Evans	1cc22da600	Set up the process collector during metrics __init__; that way all split-process workers have it	2016-10-27 18:09:34 +01:00
Paul "LeoNerd" Evans	aac13b1f9a	Pass the Metrics group into the process collector instead of having it find its own one; this avoids it needing to import from synapse.metrics	2016-10-27 18:08:15 +01:00
Paul "LeoNerd" Evans	ccc1a3d54d	Allow creation of a 'subspace' within a Metrics object, returning another one	2016-10-27 18:07:34 +01:00
Erik Johnston	665e53524e	Bump changelog and version	2016-10-27 14:52:47 +01:00
Erik Johnston	a9111786f9	Use most recently added binding, not most recently seen user.	2016-10-27 14:32:45 +01:00
Erik Johnston	1fc1bc2a51	Fix user_threepids schema delta ... The delta `37/user_threepids.sql` aimed to update all the email addresses to be lower case, however duplicate emails may exist in the table already. This commit adds a step where the delta moves the duplicate emails to a new `medium` `email_old`. Only the most recently used account keeps the binding intact. We move rather than delete so that we retain some record of which emails were associated with which account.	2016-10-27 14:14:44 +01:00
Erik Johnston	ab731d8f8e	Bump changelog and version	2016-10-27 11:07:02 +01:00
Mark Haines	177f104432	Merge pull request #1098 from matrix-org/markjh/bearer_token ... Allow clients to supply access_tokens as headers	2016-10-25 17:33:15 +01:00
Erik Johnston	22fbf86e4f	Merge branch 'release-v0.18.2' of github.com:matrix-org/synapse into develop	2016-10-25 15:09:46 +01:00
Erik Johnston	855645c719	Bump version and changelog	2016-10-25 11:16:16 +01:00
Erik Johnston	2ef617bc06	Fix infinite typing bug ... There's a bug somewhere that causes typing notifications to not be timed out properly. By adding a paranoia timer and using correct inequalities notifications should stop being stuck, even if it the root cause hasn't been fixed.	2016-10-24 15:51:22 +01:00
Erik Johnston	b6800a8ecd	Actually use the new function	2016-10-24 13:39:49 +01:00
Erik Johnston	d04e2ff3a4	Fix incredubly slow back pagination query ... If a client didn't specify a from token when paginating backwards synapse would attempt to query the (global) maximum topological token. This a) doesn't make much sense since they're room specific and b) there are no indices that lets postgres do this efficiently.	2016-10-24 13:35:51 +01:00
Paul Evans	a842fed418	Merge pull request #1177 from matrix-org/paul/standard-metric-names ... Standardise prometheus metrics	2016-10-21 13:06:19 +01:00
Luke Barnard	e01a1bc92d	Merge pull request #1175 from matrix-org/luke/feature-configurable-as-rate-limiting ... Allow Configurable Rate Limiting Per AS	2016-10-20 16:21:10 +01:00
Luke Barnard	6fdd31915b	Style	2016-10-20 13:53:15 +01:00
Luke Barnard	07caa749bf	Closing brace on following line	2016-10-20 12:07:16 +01:00
Luke Barnard	f09db236b1	as_user->app_service, less redundant comments, better positioned comments	2016-10-20 12:04:54 +01:00
Luke Barnard	8bfd01f619	flake8	2016-10-20 11:52:46 +01:00
Luke Barnard	1b17d1a106	Use real AS object by passing it through the requester ... This means synapse does not have to check if the AS is interested, but instead it effectively re-uses what it already knew about the requesting user	2016-10-20 11:43:05 +01:00
Paul "LeoNerd" Evans	b01aaadd48	Split callback metric lambda functions down onto their own lines to keep line lengths under 90	2016-10-19 18:26:13 +01:00
Paul "LeoNerd" Evans	1071c7d963	Adjust code for <100 char line limit	2016-10-19 18:23:25 +01:00
Paul "LeoNerd" Evans	6453d03edd	Cut the raw /proc/self/stat line up into named fields at collection time	2016-10-19 18:21:40 +01:00
Paul "LeoNerd" Evans	3ae48a1f99	Move the process metrics collector code into its own file	2016-10-19 18:10:24 +01:00
Paul "LeoNerd" Evans	4cedd53224	A slightly neater way to manage metric collector functions	2016-10-19 17:54:09 +01:00
Paul "LeoNerd" Evans	5663137e03	appease pep8	2016-10-19 16:09:42 +01:00
Paul "LeoNerd" Evans	b202531be6	Also guard /proc/self/fds-related code with a suitable psuedoconstant	2016-10-19 15:37:41 +01:00
Paul "LeoNerd" Evans	1b179455fc	Guard registration of process-wide metrics by existence of the requisite /proc entries	2016-10-19 15:34:38 +01:00
Paul "LeoNerd" Evans	981f852d54	Add standard process_start_time_seconds metric	2016-10-19 15:05:22 +01:00
Paul "LeoNerd" Evans	def63649df	Add standard process_max_fds metric	2016-10-19 15:05:21 +01:00
Paul "LeoNerd" Evans	06f1ad1625	Add standard process_open_fds metric	2016-10-19 15:05:21 +01:00
Paul "LeoNerd" Evans	95fc70216d	Add standard process_*_memory_bytes metrics	2016-10-19 15:05:21 +01:00
Paul "LeoNerd" Evans	9b0316c75a	Use /proc/self/stat to generate the new process_cpu_*_seconds_total metrics	2016-10-19 15:05:21 +01:00
Paul "LeoNerd" Evans	03c2720940	Export CPU usage metrics also under prometheus-standard metric name	2016-10-19 15:05:21 +01:00
Paul "LeoNerd" Evans	b21b9dbc37	Callback metric values might not just be integers - allow floats	2016-10-19 15:05:15 +01:00
Erik Johnston	78c083f159	Merge pull request #1164 from pik/error-codes ... Clarify Error codes for GET /filter/	2016-10-19 14:26:17 +01:00
Erik Johnston	3aa8925091	Merge pull request #1176 from matrix-org/erikj/eager_ratelimit_check ... Check whether to ratelimit sooner to avoid work	2016-10-19 14:25:52 +01:00
Erik Johnston	f2f74ffce6	Comment	2016-10-19 14:21:28 +01:00
David Baker	0108ed8ae6	Latest delta is now 37	2016-10-19 11:40:35 +01:00
David Baker	a7f48320b1	Merge remote-tracking branch 'origin/develop' into dbkr/password_reset_case_insensitive	2016-10-19 11:28:56 +01:00
David Baker	df2a616c7b	Convert emails to lowercase when storing ... And db migration sql to convert existing addresses.	2016-10-19 11:13:55 +01:00
Erik Johnston	550308c7a1	Check whether to ratelimit sooner to avoid work	2016-10-19 10:45:24 +01:00
pik	e8b1d2a452	Refactor test_filter to use real DataStore ... * add tests for filter api errors	2016-10-18 12:17:38 -05:00
Luke Barnard	5b54d51d1e	Allow Configurable Rate Limiting Per AS ... This adds a flag loaded from the registration file of an AS that will determine whether or not its users are rate limited (by ratelimit in _base.py). Needed for IRC bridge reasons - see https://github.com/matrix-org/matrix-appservice-irc/issues/240.	2016-10-18 17:04:09 +01:00
Erik Johnston	8ca05b5755	Fix push notifications for a single unread message	2016-10-18 10:57:33 +01:00
Erik Johnston	f0ca088280	Reduce redundant database work in email pusher ... Update the last stream ordering if the `get_unread_push_actions_for_user_in_range_for_email` returns no new push actions. This reduces the range that it needs to check next iteration.	2016-10-18 10:52:47 +01:00
Erik Johnston	b95dbdcba4	Bump version and changelog	2016-10-17 15:59:12 +01:00
Erik Johnston	6942d68247	Bump schema version	2016-10-17 11:17:45 +01:00
Erik Johnston	b59994b454	Remove TODO	2016-10-17 11:17:02 +01:00
Erik Johnston	816988baaa	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/remove_auth	2016-10-17 11:10:37 +01:00
Erik Johnston	2869a29fd7	Drop some unused indices	2016-10-17 11:08:19 +01:00
Erik Johnston	a68ade6ed3	Merge pull request #1162 from larroy/master ... Use sys.executable instead of hardcoded python. fixes #1161	2016-10-14 21:42:55 +01:00
David Baker	29c5922021	Revert part of 6207399 ... older sqlite doesn't support indexes on expressions, lets just store things lowercase in the db	2016-10-14 16:20:24 +01:00
Alexander Maznev	d9350b0db8	Error codes for filters ... * add tests Signed-off-by: Alexander Maznev <alexander.maznev@gmail.com>	2016-10-14 10:18:28 -05:00
David Baker	bcb1245a2d	Merge remote-tracking branch 'origin/develop' into dbkr/password_reset_case_insensitive	2016-10-14 15:10:38 +01:00
David Baker	62073992c5	Make password reset email field case insensitive	2016-10-14 13:56:53 +01:00
Erik Johnston	6f7540ada4	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/fix_email_notifs	2016-10-14 10:22:43 +01:00
Erik Johnston	1d107d8484	Fix email push notifs being dropped ... A lot of email push notifications were failing to be sent due to an exception being thrown along one of the (many) paths. This was due to a change where we moved from pulling out the full state for each room, but rather pulled out the event ids for the state and separately loaded the full events when needed.	2016-10-13 13:40:38 +01:00
Richard van der Hoff	f7aed3d7a2	Merge pull request #1168 from matrix-org/rav/ui_auth_on_device_delete ... User-interactive auth on delete device	2016-10-13 09:38:41 +01:00
Richard van der Hoff	9009143fb9	Handle delete device requests with no body ... We should probably return a 401 rather than a 400 for existing clients that don't know they have to do the UIA dance to delete a device.	2016-10-12 18:47:28 +01:00
Richard van der Hoff	fbd3866bc6	User-interactive auth on delete device	2016-10-12 16:16:31 +01:00
Mark Haines	9e18e0b1cb	Merge pull request #1167 from matrix-org/markjh/fingerprints ... Add config option for adding additional TLS fingerprints	2016-10-12 15:27:44 +01:00
Mark Haines	c61ddeedac	Explain how long the servers can cache the TLS fingerprints for	2016-10-12 14:48:24 +01:00
Mark Haines	0af6213019	Improve comment formatting	2016-10-12 14:45:13 +01:00
Erik Johnston	35e2cc8b52	Merge pull request #1155 from matrix-org/erikj/pluggable_pwd_auth ... Implement pluggable password auth	2016-10-12 11:41:20 +01:00
Mark Haines	6e9f3ab415	Add config option for adding additional TLS fingerprints	2016-10-11 19:14:46 +01:00
Erik Johnston	3061dac53e	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/replication_noop	2016-10-11 14:08:29 +01:00
Erik Johnston	668f91d707	Fix check of wrong variable	2016-10-11 13:57:22 +01:00
Richard van der Hoff	0061e8744f	Merge pull request #1166 from matrix-org/rav/grandfather_broken_riot_signup ... Work around email-spamming Riot bug	2016-10-11 11:58:58 +01:00
Richard van der Hoff	fa74fcf512	Work around email-spamming Riot bug ... 5d9546f9 introduced a change to synapse behaviour, in that failures in the interactive-auth process would return the flows and params data as well as an error code (as specced in https://github.com/matrix-org/matrix-doc/pull/397). That change exposed a bug in Riot which would make it request a new validation token (and send a new email) each time it got a 401 with a `flows` parameter (see https://github.com/vector-im/vector-web/issues/2447 and the fix at https://github.com/matrix-org/matrix-react-sdk/pull/510). To preserve compatibility with broken versions of Riot, grandfather in the old behaviour for the email validation stage.	2016-10-11 11:34:40 +01:00
Erik Johnston	a2f2516199	Merge pull request #1157 from Rugvip/nolimit ... Remove rate limiting from app service senders and fix get_or_create_user requester	2016-10-11 11:20:54 +01:00
Erik Johnston	a940618c94	Merge pull request #1150 from Rugvip/state_key ... api/auth: fix for not being allowed to set your own state_key	2016-10-11 11:19:55 +01:00
Pedro Larroy	c57f871184	Use sys.executable instead of hardcoded python. fixes #1161	2016-10-08 23:55:20 +02:00
Richard van der Hoff	8681aff4f1	Merge pull request #1160 from matrix-org/rav/401_on_password_fail ... Interactive Auth: Return 401 from for incorrect password	2016-10-07 10:57:43 +01:00
Richard van der Hoff	5d9546f9f4	Interactive Auth: Return 401 from for incorrect password ... This requires a bit of fettling, because I want to return a helpful error message too but we don't want to distinguish between unknown user and invalid password. To avoid hardcoding the error message into 15 places in the code, I've had to refactor a few methods to return None instead of throwing. Fixes https://matrix.org/jira/browse/SYN-744	2016-10-07 00:00:00 +01:00
Patrik Oldsberg	7b5546d077	rest/client/v1/register: use the correct requester in createUser ... Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>	2016-10-06 22:12:32 +02:00
Richard van der Hoff	5d34e32d42	Merge pull request #1159 from matrix-org/rav/uia_fallback_postmessage ... window.postmessage for Interactive Auth fallback	2016-10-06 19:56:43 +01:00
Richard van der Hoff	f382117852	window.postmessage for Interactive Auth fallback ... If you're a webapp running the fallback in an iframe, you can't set set a window.onAuthDone function. Let's post a message back to window.opener instead.	2016-10-06 18:16:59 +01:00
Patrik Oldsberg	3de7c8a4d0	handlers/profile: added admin override for set_displayname and set_avatar_url ... Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>	2016-10-06 15:24:59 +02:00
Patrik Oldsberg	2ff2d36b80	handers: do not ratelimit app service senders ... Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>	2016-10-06 15:24:59 +02:00
Patrik Oldsberg	9bfc617791	storage/appservice: make appservice methods only relying on the cache synchronous	2016-10-06 15:24:59 +02:00
Erik Johnston	4285be791d	Bump changelog and version	2016-10-05 14:40:38 +01:00
Erik Johnston	850b103b36	Implement pluggable password auth ... Allows delegating the password auth to an external module. This also moves the LDAP auth to using this system, allowing it to be removed from the synapse tree entirely in the future.	2016-10-03 10:36:40 +01:00
Patrik Oldsberg	24a70e19c7	api/auth: fix for not being allowed to set your own state_key ... Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>	2016-09-30 13:08:25 +02:00
Erik Johnston	04aa2f2863	Bump version and changelog	2016-09-30 10:34:57 +01:00
Martin Weinelt	3027ea22b0	Restructure ldap authentication ... - properly parse return values of ldap bind() calls - externalize authentication methods - change control flow to be more error-resilient - unbind ldap connections in many places - improve log messages and loglevels	2016-09-29 15:30:08 +01:00
Erik Johnston	5875a65253	Merge pull request #1145 from matrix-org/erikj/fix_reindex ... Fix background reindex of origin_server_ts	2016-09-29 13:53:48 +01:00
Erik Johnston	9040c9ffa1	Fix background reindex of origin_server_ts ... The storage function `_get_events_txn` was removed everywhere except from this background reindex. The function was removed due to it being (almost) completely unused while also being large and complex. Therefore, instead of resurrecting `_get_events_txn` we manually reimplement the bits that are needed directly.	2016-09-27 11:23:49 +01:00
Erik Johnston	4974147aa3	Remove duplication	2016-09-27 09:27:54 +01:00
Erik Johnston	13122e5e24	Remove unused variable	2016-09-27 09:21:51 +01:00
Erik Johnston	cf3e1cc200	Fix perf of fetching state in SQLite	2016-09-26 17:16:24 +01:00
Erik Johnston	a38d46249e	Merge pull request #1140 from matrix-org/erikj/typing_fed_timeout ... Time out typing over federation	2016-09-26 11:24:14 +01:00
Erik Johnston	748d8fdc7b	Reduce DB hits for replication ... Some streams will occaisonally advance their positions without actually having any new rows to send over federation. Currently this means that the token will not advance on the workers, leading to them repeatedly sending a slightly out of date token. This in turns requires the master to hit the DB to check if there are any new rows, rather than hitting the no op logic where we check if the given token matches the current token. This commit changes the API to always return an entry if the position for a stream has changed, allowing workers to advance their tokens correctly.	2016-09-23 16:49:21 +01:00
Erik Johnston	655891d179	Move FEDERATION_PING_INTERVAL timer. Update log line	2016-09-23 15:43:34 +01:00
Erik Johnston	4225a97f4e	Merge branch 'master' of github.com:matrix-org/synapse into develop	2016-09-23 15:36:59 +01:00
Erik Johnston	22578545a0	Time out typing over federation	2016-09-23 14:00:52 +01:00
Erik Johnston	667fcd54e8	Merge pull request #1136 from matrix-org/erikj/fix_signed_3pid ... Allow invites via 3pid to bypass sender sig check	2016-09-22 13:41:49 +01:00
Erik Johnston	f96020550f	Update comments	2016-09-22 12:54:22 +01:00
Erik Johnston	81964aeb90	Merge pull request #1132 from matrix-org/erikj/initial_sync_split ... Support /initialSync in synchrotron worker	2016-09-22 12:45:02 +01:00
Erik Johnston	2e9ee30969	Add comments	2016-09-22 11:59:46 +01:00
Erik Johnston	a61e4522b5	Shuffle things around to make unit tests work	2016-09-22 11:08:12 +01:00
Erik Johnston	1168cbd54d	Allow invites via 3pid to bypass sender sig check ... When a server sends a third party invite another server may be the one that the inviting user registers with. In this case it is that remote server that will issue an actual invitation, and wants to do it "in the name of" the original invitee. However, the new proper invite will not be signed by the original server, and thus other servers would reject the invite if it was seen as coming from the original user. To fix this, a special case has been added to the auth rules whereby another server can send an invite "in the name of" another server's user, so long as that user had previously issued a third party invite that is now being accepted.	2016-09-22 10:56:53 +01:00
Erik Johnston	bbc0d9617f	Merge pull request #1134 from matrix-org/erikj/fix_stream_public_deletion ... Fix _delete_old_forward_extrem_cache query	2016-09-21 17:04:04 +01:00
Erik Johnston	8009d84364	Match against event_id, rather than room_id	2016-09-21 16:46:59 +01:00
Erik Johnston	dc692556d6	Remove spurious AS clause	2016-09-21 16:28:47 +01:00
Erik Johnston	dc78db8c56	Update correct table	2016-09-21 15:52:44 +01:00
Erik Johnston	4f78108d8c	Readd entries to public_room_list_stream that were deleted	2016-09-21 15:24:22 +01:00
Erik Johnston	0b78d8adf2	Fix _delete_old_forward_extrem_cache query	2016-09-21 15:20:56 +01:00
Erik Johnston	90c070c850	Add total_room_count_estimate to /publicRooms	2016-09-21 13:30:05 +01:00
Erik Johnston	87528f0756	Support /initialSync in synchrotron worker	2016-09-21 11:46:28 +01:00
Erik Johnston	2b8ff4659f	Bump version and changelog	2016-09-19 17:16:56 +01:00
Erik Johnston	ddfcdd4778	Merge branch 'develop' of github.com:matrix-org/synapse into release-v0.18.0	2016-09-19 17:15:24 +01:00
Erik Johnston	6f0c5e5d9b	Merge pull request #1131 from matrix-org/matthew/e2e-notifs ... Notify on e2e events	2016-09-19 10:50:26 +01:00
Erik Johnston	49cf205dc7	_id field must uniquely identify different conditions	2016-09-19 10:34:01 +01:00
Erik Johnston	39af634dd2	Merge pull request #1130 from matrix-org/erikj/fix_pubroom_pag ... Handle fact that _generate_room_entry may not return a room entry	2016-09-19 10:13:59 +01:00
Matthew Hodgson	3f6ec271ba	proposal for notifying on e2e events	2016-09-17 22:05:06 +01:00
Erik Johnston	4d49e0bdfd	PEP8	2016-09-17 18:09:22 +01:00
Erik Johnston	81570abfb2	Handle fact that _generate_room_entry may not return a room entry	2016-09-17 18:01:54 +01:00
Erik Johnston	ddc89df89d	Enable guest access to POST /publicRooms	2016-09-17 15:55:24 +01:00
Erik Johnston	eb24aecf8c	Merge pull request #1129 from matrix-org/erikj/fix_pubroom_pag ... Fix and clean up publicRooms pagination	2016-09-17 15:30:34 +01:00
Erik Johnston	e1ba98d724	Merge pull request #1127 from matrix-org/dbkr/publicroom_search_case_insensitive ... Make public room search case insensitive	2016-09-17 15:01:17 +01:00
Erik Johnston	a298331de4	Spelling	2016-09-17 14:59:40 +01:00
Erik Johnston	71edaae981	Fix and clean up publicRooms pagination	2016-09-17 14:46:19 +01:00
Matthew Hodgson	883df2e983	fix logger for client_reader worker	2016-09-17 14:12:04 +01:00
David Baker	5336acd46f	Make public room search case insensitive	2016-09-16 19:02:42 +01:00
Erik Johnston	19fe990476	Update changelog and bump version	2016-09-16 17:30:59 +01:00
Erik Johnston	995f2f032f	Fix public room pagination for client_reader app	2016-09-16 14:48:21 +01:00
Erik Johnston	a68807d426	Comment	2016-09-16 11:36:20 +01:00
Erik Johnston	2e67cabd7f	Make POST /publicRooms require auth	2016-09-16 11:32:51 +01:00
Erik Johnston	b7b62bf9ea	Comment	2016-09-16 11:00:29 +01:00
Erik Johnston	d84319ae10	Add remote reoom cache	2016-09-16 10:31:59 +01:00
Erik Johnston	23b6701a28	Support filtering remote room lists	2016-09-16 10:24:15 +01:00
Erik Johnston	e58a9d781c	Filter remote rooms lists locally	2016-09-16 10:19:32 +01:00
Erik Johnston	74d4cdee25	Don't cache searched in /publicRooms	2016-09-16 09:05:11 +01:00
Erik Johnston	418bcd4309	Add new storage function to slave store	2016-09-16 08:37:39 +01:00
Erik Johnston	098db4aa52	Add very basic filter API to /publicRooms	2016-09-15 17:50:16 +01:00
Erik Johnston	c33b25fd8d	Change the way we calculate new_limit in /publicRooms and add POST API	2016-09-15 17:35:20 +01:00
Erik Johnston	de4f798f01	Handling expiring stream extrems correctly.	2016-09-15 17:34:59 +01:00
Erik Johnston	ea6dc356b0	Merge pull request #1125 from matrix-org/erikj/public_room_cache ... Change get_pos_of_last_change to return upper bound	2016-09-15 15:48:53 +01:00
Erik Johnston	955f34d23e	Change get_pos_of_last_change to return upper bound	2016-09-15 15:12:07 +01:00
Erik Johnston	241d7d2d62	Merge pull request #1124 from matrix-org/erikj/enable_state_caching_workers ... Enable state caches on workers	2016-09-15 15:01:47 +01:00
Erik Johnston	4be85281f9	Enable state caches on workers	2016-09-15 14:31:22 +01:00
Erik Johnston	cb3edec6af	Use stream_change cache to make get_forward_extremeties_for_room cache more effective	2016-09-15 14:28:13 +01:00
Erik Johnston	55e6fc917c	Add cache to get_forward_extremeties_for_room	2016-09-15 14:04:28 +01:00
Erik Johnston	68c1ed4d1a	Remove default public rooms limit	2016-09-15 13:56:20 +01:00
Erik Johnston	b82fa849c8	Merge pull request #1120 from matrix-org/erikj/push_invite_cache ... Ensure we don't mutate the cache of push rules	2016-09-15 13:27:18 +01:00
Erik Johnston	e457034e99	Merge pull request #1121 from matrix-org/erikj/public_room_paginate ... Add pagination support to publicRooms	2016-09-15 13:27:09 +01:00
Erik Johnston	1d98cf26be	By default limit /publicRooms to 100 entries	2016-09-15 13:18:35 +01:00
Erik Johnston	211786ecd6	Stream public room changes down replication	2016-09-15 11:47:23 +01:00
Erik Johnston	4fb65a1091	Base public room list off of public_rooms stream	2016-09-15 11:27:04 +01:00
Erik Johnston	5810cffd33	Pass since/from parameters over federation	2016-09-15 10:36:19 +01:00
Erik Johnston	f3eead0660	Allow paginating both forwards and backwards	2016-09-15 10:15:37 +01:00
Erik Johnston	4131381123	Remove support for aggregate room lists	2016-09-15 09:28:15 +01:00
Erik Johnston	6a5ded5988	Ensure we don't mutate the cache of push rules	2016-09-15 09:16:13 +01:00
Erik Johnston	4f181f361d	Accept optional token to public room list	2016-09-15 09:08:57 +01:00
Erik Johnston	c566f0ee17	Calculate the public room list from a stream_ordering	2016-09-14 17:42:47 +01:00
Erik Johnston	772c6067a3	Refactor public rooms to not pull out the full state for each room	2016-09-14 17:29:25 +01:00
Erik Johnston	baffe96d95	Add a room visibility stream	2016-09-14 17:29:19 +01:00
Erik Johnston	264a48aedf	Merge pull request #1117 from matrix-org/erikj/fix_state ... Ensure we don't mutate state cache entries	2016-09-14 16:50:37 +01:00
Erik Johnston	ed992ae6ba	Add a DB index to figure out past state at a stream ordering in a room	2016-09-14 16:20:27 +01:00
Erik Johnston	e0b6db29ed	Split out public room list into a worker process	2016-09-14 14:42:51 +01:00
Erik Johnston	a70a43bc51	Move RoomListHandler into a separate file	2016-09-14 14:07:37 +01:00
Erik Johnston	f2b2cd8eb4	Amalgamate two identical consecutive if statements	2016-09-14 11:16:22 +01:00
Erik Johnston	00f51493f5	Fix reindex	2016-09-14 10:18:30 +01:00
Erik Johnston	d5ae1f1291	Ensure we don't mutate state cache entries	2016-09-14 10:03:48 +01:00
Matthew Hodgson	1b01488d27	Merge pull request #1111 from matrix-org/matthew/device-ids ... make device IDs more useful for human disambiguation	2016-09-14 01:18:02 +01:00
Paul Evans	0f73f0e70e	Merge pull request #1116 from matrix-org/paul/tiny-fixes ... Fix typo "persiting"	2016-09-13 13:27:42 +01:00
Paul "LeoNerd" Evans	ca35e54d6b	Fix typo "persiting"	2016-09-13 13:26:33 +01:00
Mark Haines	ad816b0add	Limit how often we ask for keys from dead servers	2016-09-13 11:53:50 +01:00
Erik Johnston	43253c10b8	Remove redundant event_auth index	2016-09-13 11:47:48 +01:00
Mark Haines	18ab019a4a	Move the E2E key handling into the e2e handler	2016-09-13 11:35:35 +01:00
Mark Haines	76b09c29b0	Merge pull request #1110 from matrix-org/markjh/e2e_timeout ... Add a timeout parameter for end2end key queries.	2016-09-13 10:50:45 +01:00
Erik Johnston	ba6bc2faa0	Merge pull request #1109 from matrix-org/erikj/partial_indices ... Add WHERE clause support to index creation	2016-09-13 09:06:16 +01:00
Matthew Hodgson	edbcb4152b	make device IDs more useful for human disambiguation	2016-09-13 00:02:39 +01:00
Mark Haines	949c2c5435	Add a timeout parameter for end2end key queries. ... Add a timeout parameter for controlling how long synapse will wait for responses from remote servers. For servers that fail include how they failed to make it easier to debug. Fetch keys from different servers in parallel rather than in series. Set the default timeout to 10s.	2016-09-12 18:17:09 +01:00
Erik Johnston	b17af156c7	Remove where clause	2016-09-12 17:05:54 +01:00
Erik Johnston	1c9da43a95	Merge pull request #1108 from matrix-org/erikj/create_dm ... Add is_direct param to /createRoom	2016-09-12 16:57:16 +01:00
Erik Johnston	0b32bb20bb	Index contains_url for file search queries	2016-09-12 16:57:05 +01:00
Erik Johnston	c94de0ab60	Add WHERE clause support to index creation	2016-09-12 16:55:01 +01:00
Erik Johnston	502c901e11	Merge pull request #1107 from matrix-org/erikj/backfill_none ... Fix backfill when cannot find an event.	2016-09-12 16:48:01 +01:00
Erik Johnston	48a5a7552d	Add is_direct param to /createRoom	2016-09-12 16:34:20 +01:00
Erik Johnston	706b5d76ed	Fix backfill when cannot find an event. ... `get_pdu` can succeed but return None.	2016-09-12 14:59:51 +01:00
Erik Johnston	7c679b1118	Merge pull request #1106 from matrix-org/erikj/state_reindex_concurrent ... Create new index concurrently	2016-09-12 14:38:40 +01:00
Erik Johnston	d080b3425c	Merge pull request #1105 from matrix-org/erikj/make_notif_highlight_query_fast ... Optimise /notifications query	2016-09-12 14:34:12 +01:00
Erik Johnston	03a98aff3c	Create new index concurrently	2016-09-12 14:27:01 +01:00
Erik Johnston	fa20c9ce94	Change the index to be stream_ordering, highlight	2016-09-12 14:04:08 +01:00
Erik Johnston	5ef5435529	Remove unused import	2016-09-12 13:32:58 +01:00
Mark Haines	aa7b890cfe	Merge pull request #1104 from matrix-org/markjh/direct_to_device_federation_sync ... Fix direct to device messages recieved over federation to notify sync	2016-09-12 13:25:23 +01:00
Erik Johnston	7cd6edb947	Use register_background_index_update	2016-09-12 12:54:48 +01:00
Erik Johnston	0294c14ec4	Add back in query change	2016-09-12 12:43:56 +01:00
Erik Johnston	7fe42cf949	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/make_notif_highlight_query_fast	2016-09-12 12:37:09 +01:00
Erik Johnston	15ca0c6a4d	Make reindex happen in bg	2016-09-12 12:36:36 +01:00
Mark Haines	a232e06100	Fix direct to device messages recieved over federation to notify sync	2016-09-12 12:30:46 +01:00
Mark Haines	4a32d25d4c	Merge branch 'develop' into markjh/bearer_token	2016-09-12 11:14:56 +01:00
Mark Haines	31f85f9db9	Add comments to existing schema deltas that used "CREATE INDEX" directly	2016-09-12 11:00:26 +01:00
Mark Haines	ec609f8094	Fix unit tests	2016-09-12 10:46:02 +01:00
Erik Johnston	54417999b6	Revert "Add index to event_push_actions"	2016-09-12 10:39:55 +01:00
Erik Johnston	45dc260060	Merge pull request #1101 from matrix-org/erikj/state_types_idx ... Change state fetch query for postgres to be faster	2016-09-12 10:20:38 +01:00
Erik Johnston	d1c217c823	Merge pull request #1097 from matrix-org/erikj/replication_typing_rest ... Correctly handle typing stream id resetting	2016-09-12 10:10:15 +01:00
Erik Johnston	897d57bc58	Change state fetch query for postgres to be faster ... It turns out that postgres doesn't like doing a list of OR's and is about 1000x slower, so we just issue a query for each specific type seperately.	2016-09-12 10:05:07 +01:00
Erik Johnston	555460ae1b	Merge pull request #1095 from matrix-org/erikj/batch_edus ... Clobber EDUs in send queue	2016-09-12 08:04:15 +01:00
Richard van der Hoff	4162f820ff	Merge pull request #1100 from VShell/fix-cas ... Conform better to the CAS protocol specification	2016-09-09 21:57:42 +01:00
Shell Turner	29205e9596	Conform better to the CAS protocol specification ... Redirect to CAS's /login endpoint properly, and don't require an <attributes> element. Signed-off-by: Shell Turner <cam.turn@gmail.com>	2016-09-09 21:20:14 +01:00
David Baker	b91e2833b3	Merge remote-tracking branch 'origin/develop' into dbkr/make_notif_highlight_query_fast	2016-09-09 19:11:34 +01:00
David Baker	f2acc3dcf9	Add index to event_push_actions ... and remove room_id caluse so it uses it Mostly from @negativemjark	2016-09-09 18:54:54 +01:00
Mark Haines	3ddec016ff	Merge branch 'develop' into markjh/bearer_token	2016-09-09 18:51:22 +01:00
Mark Haines	8e01263587	Allow clients to supply access_tokens as headers ... Clients can continue to supply access tokens as query parameters or can supply the token as a header: Authorization: Bearer <access_token_goes_here> This matches the ouath2 format of https://tools.ietf.org/html/rfc6750#section-2.1	2016-09-09 18:17:42 +01:00
Erik Johnston	3265def8c7	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/batch_edus	2016-09-09 18:06:01 +01:00
Erik Johnston	af4701b311	Fix incorrect attribute name	2016-09-09 17:36:56 +01:00
Erik Johnston	44330a21e9	Comment	2016-09-09 17:22:07 +01:00
Erik Johnston	464ffd1b5e	Comment	2016-09-09 17:17:23 +01:00
Erik Johnston	327425764e	Add edu.type as part of key. Remove debug logging	2016-09-09 17:13:30 +01:00
Mark Haines	dbff7e9436	Merge pull request #1096 from matrix-org/markjh/get_access_token ... Add helper function for getting access_tokens from requests	2016-09-09 17:09:27 +01:00
Erik Johnston	a4339de9de	Correctly handle typing stream id resetting	2016-09-09 16:44:26 +01:00
Mark Haines	8aee5aa068	Add helper function for getting access_tokens from requests ... Rather than reimplementing the token parsing in the various places. This will make it easier to change the token parsing to allow access_tokens in HTTP headers.	2016-09-09 16:33:15 +01:00
Erik Johnston	52b2318777	Clobber EDUs in send queue	2016-09-09 15:59:08 +01:00
Paul Evans	56f38d1776	Merge pull request #1091 from matrix-org/paul/third-party-lookup ... Improvements to 3PE lookup API	2016-09-09 15:43:11 +01:00
Paul Evans	8cb252d00c	Merge pull request #1094 from matrix-org/paul/get-state-whole-event ... Allow clients to ask for the whole of a single state event	2016-09-09 15:26:17 +01:00
Paul "LeoNerd" Evans	776594f99d	Log if rejecting 3PE query metadata result due to type check	2016-09-09 15:11:31 +01:00
Paul "LeoNerd" Evans	ed44c475d8	Reject malformed 3PE query metadata results earlier in AS API handling code	2016-09-09 15:07:04 +01:00
Erik Johnston	ab80d5e0a9	Drop replication log levels	2016-09-09 14:56:50 +01:00
Paul "LeoNerd" Evans	f25d74f69c	Minor fixes from PR comments	2016-09-09 14:54:16 +01:00
Paul "LeoNerd" Evans	d271383e63	Filter returned events for client-facing format	2016-09-09 14:40:15 +01:00
Paul "LeoNerd" Evans	0fc0a3bdff	Allow clients to specify the format a room state event is returned in	2016-09-09 14:34:29 +01:00
Erik Johnston	6c4d582144	Deduplicate presence in _update_states	2016-09-09 14:28:22 +01:00
Erik Johnston	685da5a3b0	Merge pull request #1092 from matrix-org/erikj/transaction_queue_check ... Check if destination is ready for retry earlier	2016-09-09 13:49:56 +01:00
Erik Johnston	a6c6750166	Check if destination is ready for retry earlier	2016-09-09 13:46:05 +01:00
Paul "LeoNerd" Evans	bdbcfc2a80	appease pep8	2016-09-09 13:31:39 +01:00
Paul "LeoNerd" Evans	6eb0c8a2e4	Python isn't JavaScript; have to quote dict keys	2016-09-09 13:31:17 +01:00
Mark Haines	6b54fa81de	Merge pull request #1089 from matrix-org/markjh/direct_to_device_stream ... Track the max device stream_id in a separate table,	2016-09-09 13:31:07 +01:00
Paul "LeoNerd" Evans	25eb769b26	Efficiency fix for lookups of a single protocol	2016-09-09 13:25:02 +01:00
Paul "LeoNerd" Evans	3328428d05	Allow lookup of a single 3PE protocol query metadata	2016-09-09 13:19:04 +01:00
Erik Johnston	4598682b43	Fix tightloop on sending transaction	2016-09-09 13:12:53 +01:00
Paul "LeoNerd" Evans	033d43e419	Don't corrupt shared cache on subsequent protocol requests	2016-09-09 13:10:36 +01:00
Mark Haines	647c724573	Use the previous MAX value if any to set the stream_id	2016-09-09 11:52:44 +01:00
Erik Johnston	a15ba15e64	Merge pull request #1088 from matrix-org/erikj/transaction_queue_check ... Correctly guard against multiple concurrent transactions	2016-09-09 11:49:06 +01:00
Mark Haines	6a6cbfcf1e	Track the max_stream_device_id in a separate table, since we delete from the inbox table	2016-09-09 11:48:23 +01:00
Erik Johnston	d2688d7f03	Correctly guard against multiple concurrent transactions	2016-09-09 11:44:36 +01:00
Mark Haines	303b6f29f0	Merge pull request #1087 from matrix-org/markjh/reapply_delta ... Reapply 34/device_outbox in 35/device_outbox_again.py since the schem…	2016-09-09 11:24:27 +01:00
Erik Johnston	1fe7ca1362	Merge branch 'release-v0.17.3' of github.com:matrix-org/synapse into develop	2016-09-09 11:15:40 +01:00
Erik Johnston	66efcbbff1	Bump changelog and version	2016-09-09 11:07:39 +01:00
Mark Haines	0877157353	Just move the schema and add some DROPs	2016-09-09 11:04:47 +01:00
Erik Johnston	2ffec928e2	Reduce batch size to be under SQL limit	2016-09-09 11:03:31 +01:00
Erik Johnston	b390756150	Update last_device_stream_id_by_dest if there is nothing to send	2016-09-09 11:00:15 +01:00
Matthew Hodgson	b8f84f99ff	Merge pull request #1081 from matrix-org/dbkr/notifications_only_highlight ... Implement `only=highlight` on `/notifications`	2016-09-09 00:09:51 +01:00
Mark Haines	43b77c5d97	Only catch databas errors	2016-09-08 17:44:21 +01:00
Paul "LeoNerd" Evans	2f267ee160	Collect up all the "instances" lists of individual AS protocol results into one combined answer to the client	2016-09-08 17:43:53 +01:00
Mark Haines	7d5b142547	Add a stub run_upgrade	2016-09-08 17:39:11 +01:00
David Baker	c3276aef25	Merge pull request #1080 from matrix-org/dbkr/fix_notifications_api_with_from ... Fix /notifications API when used with `from` param	2016-09-08 17:35:35 +01:00
Mark Haines	fa722a699c	Reapply 34/device_outbox in 35/device_outbox_again.py since the schema was bumped before it landed on develop	2016-09-08 17:35:16 +01:00
Erik Johnston	5c688739d6	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/reindex_state_groups	2016-09-08 16:52:09 +01:00
Erik Johnston	ebb46497ba	Add delta file	2016-09-08 16:38:54 +01:00
Mark Haines	91ec972277	Merge pull request #1084 from matrix-org/markjh/direct_to_device_wildcard ... Support wildcard device_ids for direct to device messages	2016-09-08 16:28:08 +01:00
Erik Johnston	5beda10bbd	Reindex state_groups_state after pruning	2016-09-08 16:18:01 +01:00
Erik Johnston	257025ac89	Merge pull request #1082 from matrix-org/erikj/remote_public_rooms ... Add server param to /publicRooms	2016-09-08 16:04:22 +01:00
Erik Johnston	3f9889bfd6	Use parse_string	2016-09-08 15:51:10 +01:00
Erik Johnston	caa22334b3	Merge branch 'master' of github.com:matrix-org/synapse into develop	2016-09-08 15:33:00 +01:00
Erik Johnston	b152ee71fe	Bump version and changelog	2016-09-08 15:23:44 +01:00
Mark Haines	a1c8f268e5	Support wildcard device_ids for direct to device messages	2016-09-08 15:13:05 +01:00
Erik Johnston	8b93af662d	Check the user_id for presence/typing matches origin	2016-09-08 15:07:38 +01:00
Mark Haines	fa9d36e050	Merge branch 'develop' into markjh/direct_to_device_federation	2016-09-08 13:43:43 +01:00
David Baker	4ef222ab61	Implement only=highlight on /notifications	2016-09-08 13:43:35 +01:00
Erik Johnston	61cd9af09b	Log delta files we're applying	2016-09-08 13:40:46 +01:00
Erik Johnston	791658b576	Add server param to /publicRooms	2016-09-08 11:53:05 +01:00
David Baker	c5b49eb7ca	Fix /notifications API when used with from param	2016-09-08 09:40:10 +01:00
Erik Johnston	b568ca309c	Temporarily disable sequential scans for state fetching	2016-09-08 09:38:54 +01:00
Mark Haines	3c320c006c	Merge pull request #1077 from matrix-org/markjh/device_logging ... Log the types and values when failing to store devices	2016-09-07 18:24:24 +01:00
Mark Haines	85b51fdd6b	Log the types and values when failing to store devices	2016-09-07 17:19:18 +01:00
Mark Haines	43954d000e	Add a new method to enqueue the device messages rather than sending a dummy EDU	2016-09-07 16:10:51 +01:00
Mark Haines	2a0159b8ae	Fix the stream change cache to work over replication	2016-09-07 15:58:00 +01:00
Mark Haines	cb98ac261b	Move the check for federated device_messages. ... Move the check into _attempt_new_transaction. Only delete messages if there were messages to delete.	2016-09-07 15:39:13 +01:00
Mark Haines	31a07d2335	Add stream change caches for device messages	2016-09-07 15:27:07 +01:00
Erik Johnston	513188aa56	Comment	2016-09-07 14:53:23 +01:00
Erik Johnston	fadb01551a	Add appopriate framing clause	2016-09-07 14:39:01 +01:00
Erik Johnston	d25c20ccbe	Use windowing function to make use of index	2016-09-07 14:22:22 +01:00
Mark Haines	7d893beebe	Comment the add_messages storage functions	2016-09-07 12:03:37 +01:00
Erik Johnston	94a83b534f	Merge pull request #1065 from matrix-org/erikj/state_storage ... Move to storing state_groups_state as deltas	2016-09-07 09:39:58 +01:00
Mark Haines	d4a35ada28	Send device messages over federation	2016-09-06 18:16:20 +01:00
Mark Haines	e020834e4f	Add storage methods for federated device messages	2016-09-06 15:12:13 +01:00
Mark Haines	2ad72da931	Add tables for federated device messages ... Adds tables for storing the messages that need to be sent to a remote device and for deduplicating messages received.	2016-09-06 15:10:40 +01:00
Erik Johnston	3c4208a057	Record counts of state changes	2016-09-06 11:31:01 +01:00
Mark Haines	f4164edb70	Move _add_messages_to_device_inbox_txn into a separate method	2016-09-06 11:26:37 +01:00
Erik Johnston	2eed4d7af4	Merge pull request #1072 from matrix-org/erikj/presence_fiddle ... Fiddle should_notify to better report stats	2016-09-06 10:57:48 +01:00
Erik Johnston	438ef47637	Short circuit if presence is the same	2016-09-06 10:28:35 +01:00
Erik Johnston	74a3b4a650	Fiddle should_notify to better report stats	2016-09-06 10:23:38 +01:00
Erik Johnston	d51b8a1674	Add quotes and be explicity about script-src	2016-09-05 17:35:01 +01:00
Erik Johnston	662b031a30	Allow PDF to be rendered from media repo	2016-09-05 17:25:26 +01:00
Erik Johnston	0595413c0f	Scale the batch size so that we're not bitten by the minimum	2016-09-05 15:49:57 +01:00
Erik Johnston	a7032abb2e	Correctly handle reindexing state groups that already have an edge	2016-09-05 15:07:23 +01:00
Erik Johnston	9e6d88f4e2	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/state_storage	2016-09-05 15:01:33 +01:00
Erik Johnston	8c93e0bae7	Merge pull request #1070 from matrix-org/erikj/presence_stats ... Record why we have chosen to notify	2016-09-05 15:01:30 +01:00
Erik Johnston	70332a12dd	Take value in a better way	2016-09-05 14:57:14 +01:00
Erik Johnston	373654c635	Comment about sqlite and WITH RECURSIVE	2016-09-05 14:50:36 +01:00
Erik Johnston	485d999c8a	Correctly delete old state groups in purge history API	2016-09-05 14:49:08 +01:00
Erik Johnston	69054e3d4c	Record why we have chosen to notify	2016-09-05 14:12:11 +01:00
Erik Johnston	69a2d4e38c	Use get_joined_users_from_context instead of manually looking up hosts	2016-09-05 13:44:40 +01:00
Erik Johnston	f12993ec16	Bump changelog and version	2016-09-05 13:01:14 +01:00
Erik Johnston	bb36b93f71	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/state_storage	2016-09-05 11:53:11 +01:00
Erik Johnston	d87b87adf7	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/bulk_push	2016-09-05 11:52:49 +01:00
Erik Johnston	caed150363	Remove unused imports	2016-09-05 10:52:01 +01:00
Erik Johnston	80a6a445fa	Only fetch local pushers	2016-09-05 10:43:32 +01:00
Erik Johnston	628e65721b	Add comments	2016-09-05 10:41:27 +01:00
Mark Haines	274c2f50a5	Merge pull request #1067 from matrix-org/markjh/idempotent ... Fix membership changes to be idempotent	2016-09-05 10:21:25 +01:00
Erik Johnston	a99e933550	Add upgrade script that will slowly prune state_groups_state entries	2016-09-05 10:05:36 +01:00
Erik Johnston	3847fa38c4	Make bulk_get_push_rules_for_room use get_joined_users_from_context cache	2016-09-05 10:02:38 +01:00
Mark Haines	f2690c6423	Fix membership changes to be idempotent	2016-09-02 19:23:22 +01:00
Mark Haines	81b94c5750	Merge pull request #1066 from matrix-org/markjh/direct_to_device_lowerbound ... Only return new device messages in /sync	2016-09-02 16:18:34 +01:00
Mark Haines	65fa37ac5e	Only return new device messages in /sync	2016-09-02 15:50:37 +01:00
Erik Johnston	3baf641a48	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/state_storage	2016-09-02 14:54:07 +01:00
Erik Johnston	c0238ecbed	Explicitly specify state_key for history_visibility fetching	2016-09-02 14:53:46 +01:00
Erik Johnston	273b6bcf22	Merge pull request #1064 from matrix-org/erikj/on_receive_check ... Only check if host is in room if we have state and auth_chain	2016-09-02 14:43:35 +01:00
Erik Johnston	f7f1027d3d	Comment on when auth chain and state are None	2016-09-02 14:42:38 +01:00
Erik Johnston	34e5e17f91	Comment	2016-09-02 14:26:07 +01:00
Erik Johnston	b96c6c3185	Docstrings	2016-09-02 14:19:22 +01:00
Erik Johnston	1ffe9578d1	Merge pull request #1063 from matrix-org/erikj/pull_out_ids_only ... Only pull out IDs from DB for /state_ids/ request	2016-09-02 14:16:28 +01:00
Erik Johnston	cce957e254	Bump max_entries on get_destination_retry_timings	2016-09-02 14:08:33 +01:00
Erik Johnston	bd9b8d87ae	Only check if host is in room if we have state and auth_chain	2016-09-02 13:40:28 +01:00
Erik Johnston	657847e4c6	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/state_storage	2016-09-02 11:04:48 +01:00
Mark Haines	965168a842	Merge branch 'develop' into markjh/direct_to_device_synchrotron	2016-09-02 10:59:24 +01:00
Erik Johnston	2854ee2a52	Only pull out IDs from DB for /state_ids/ request	2016-09-02 10:53:36 +01:00
Erik Johnston	598317927c	Limit the length of state chains	2016-09-02 10:41:38 +01:00
Mark Haines	7ed5acacf4	Fix up the calls to the notifier for device messages	2016-09-01 18:08:40 +01:00
Erik Johnston	051a9ea921	Linearize state resolution to help caches	2016-09-01 14:55:03 +01:00
Erik Johnston	265d847ffd	Fix typo in log line	2016-09-01 14:50:06 +01:00
Erik Johnston	f4778d4cd9	Merge branch 'erikj/pdu_check' of github.com:matrix-org/synapse into develop	2016-09-01 14:40:36 +01:00
Erik Johnston	9e25443db8	Move to storing state_groups_state as deltas	2016-09-01 14:31:26 +01:00
Erik Johnston	44982606ee	Merge pull request #1060 from matrix-org/erikj/state_ids ... Assign state groups in state handler.	2016-09-01 14:20:42 +01:00
Erik Johnston	516a272aca	Ensure we only return a validated pdu in get_pdu	2016-09-01 10:55:02 +01:00
Erik Johnston	0cfd6c3161	Use state_groups table to test existence	2016-08-31 16:25:57 +01:00
Erik Johnston	5405351b14	Lower get_linearized_receipts_for_room cache size	2016-08-31 16:19:44 +01:00
Erik Johnston	ed7a703d4c	Handle the fact that workers can't generate state groups	2016-08-31 15:53:19 +01:00
Erik Johnston	1671913287	Merge pull request #1059 from matrix-org/erikj/sent_transaction_delete ... Clean up old sent transactions	2016-08-31 15:07:33 +01:00
Erik Johnston	f51888530d	Always specify state_group so that its in the cache	2016-08-31 14:58:11 +01:00
Erik Johnston	826ca61745	Add storage function to SlaveStore	2016-08-31 14:45:04 +01:00
Erik Johnston	fbd2615de4	Merge pull request #1057 from matrix-org/erikj/fix_email_name ... Fix email notifs by adding missing param	2016-08-31 14:33:45 +01:00
Erik Johnston	c10cb581c6	Correctly handle the difference between prev and current state	2016-08-31 14:26:22 +01:00
Erik Johnston	ef0cc648cf	Clean up old sent transactions	2016-08-31 11:12:02 +01:00
Mark Haines	761f9fccff	Merge pull request #1058 from matrix-org/markjh/direct_to_device_synchrotron ... Add a replication stream for direct to device messages	2016-08-31 11:08:18 +01:00
Mark Haines	a662252758	Return the current stream position from add_messages_to_device_inbox	2016-08-31 10:42:52 +01:00
Mark Haines	1aa3e1d287	Add a replication stream for direct to device messages	2016-08-31 10:38:58 +01:00
Erik Johnston	1bb8ec296d	Generate state group ids in state layer	2016-08-31 10:09:46 +01:00
Erik Johnston	d80f64d370	Fix email notifs by adding missing param	2016-08-30 21:46:39 +01:00
Kegan Dougal	c882783535	flake8	2016-08-30 17:20:31 +01:00
Kegan Dougal	572acde483	Use None instead of the empty string ... Change how we validate the 'url' field as a result.	2016-08-30 17:16:00 +01:00
Erik Johnston	5dc2a702cf	Make _state_groups_id_gen a normal IdGenerator	2016-08-30 16:55:11 +01:00
Erik Johnston	3e784eff74	Remove state replication stream	2016-08-30 16:51:36 +01:00
Kegan Dougal	16b652f0a3	Flake8	2016-08-30 16:30:12 +01:00
Kegan Dougal	e82247f990	Allow application services to have an optional 'url' ... If 'url' is not specified, they will not be pushed for events or queries. This is useful for bots who simply wish to reserve large chunks of user/alias namespace, and don't care about being pushed for events.	2016-08-30 16:21:16 +01:00
Erik Johnston	c7f665d700	Merge pull request #1055 from matrix-org/erikj/occaisonally_persist ... Occaisonally persist unpersisted presence updates	2016-08-30 15:59:00 +01:00
Erik Johnston	097330bae8	Check correct variable	2016-08-30 15:50:20 +01:00
Erik Johnston	21b977ccfe	Occaisonally persist unpersisted presence updates	2016-08-30 15:39:50 +01:00
Erik Johnston	bc1a8b1f7a	Don't notify for online -> online transitions. ... Specifically, if currently_active remains true then we should not notify if only the last active time changes.	2016-08-30 15:05:32 +01:00
Erik Johnston	67f0c990f8	Merge pull request #1051 from matrix-org/erikj/fix_push_names ... Fix push room names for rooms with only an alias	2016-08-30 11:50:34 +01:00
Erik Johnston	c8cd87b21b	Comment about message deletion	2016-08-30 11:23:26 +01:00
Erik Johnston	55e17d3697	Fix push room names for rooms with only an alias	2016-08-30 11:19:59 +01:00
Erik Johnston	1ee6285905	Fix check	2016-08-30 11:17:46 +01:00
Erik Johnston	68e1a872fd	Noop get_new_messages_for_device if token hasn't changed	2016-08-30 10:58:46 +01:00
Erik Johnston	55fc17cf4b	Merge pull request #1049 from matrix-org/erikj/presence_users_in_room ... Use state handler instead of get_users_in_room/get_joined_hosts	2016-08-30 10:50:37 +01:00
Erik Johnston	41788bba50	Add to slave store	2016-08-30 09:55:17 +01:00
Erik Johnston	873f870e5a	Add new direct message storage functions to slave store	2016-08-30 09:40:32 +01:00
Erik Johnston	bed10f9880	Use state handler instead of get_users_in_room/get_joined_hosts	2016-08-26 14:54:30 +01:00
Mark Haines	4bbef62124	Merge remote-tracking branch 'origin/develop' into markjh/direct_to_device	2016-08-26 14:35:31 +01:00
Erik Johnston	a234e895cf	Fix room name in email notifs	2016-08-26 14:10:21 +01:00
Erik Johnston	4daa397a00	Add is_host_joined to slave storage	2016-08-26 13:10:56 +01:00
Erik Johnston	c7cd35d682	Typo	2016-08-26 11:23:58 +01:00
Erik Johnston	54cc69154e	Make None optional	2016-08-26 11:20:59 +01:00
Erik Johnston	11faa4296d	Measure _filter_events_for_server	2016-08-26 11:15:40 +01:00
Erik Johnston	f6338d6a3e	Don't pull out full state for _filter_events_for_server	2016-08-26 11:13:16 +01:00
Erik Johnston	1ccdc1e93a	Cache check_host_in_room	2016-08-26 10:59:40 +01:00
Erik Johnston	25414b44a2	Add measure on check_host_in_room	2016-08-26 10:47:00 +01:00
Erik Johnston	50943ab942	Add new state storage funcs to replication	2016-08-26 09:57:32 +01:00
Erik Johnston	30961182f2	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/state_ids	2016-08-26 09:48:13 +01:00
Erik Johnston	c1a133a6b6	Merge pull request #1043 from matrix-org/erikj/backfill_fix ... Fix None check in backfill	2016-08-26 09:07:05 +01:00
Erik Johnston	778fa85f47	Make sync not pull out full state	2016-08-25 18:59:44 +01:00
Erik Johnston	7356d52e73	Fix up push to use get_current_state_ids	2016-08-25 18:35:49 +01:00
Paul "LeoNerd" Evans	9459137f1e	Just sprintf the 'kind' argument into uri directly	2016-08-25 18:35:38 +01:00
Paul "LeoNerd" Evans	1294d4a329	Move ThirdPartyEntityKind into api.constants so the expectation becomes that the value is significant	2016-08-25 18:34:47 +01:00
Mark Haines	ab34fdecb7	Merge branch 'develop' into markjh/direct_to_device	2016-08-25 18:34:46 +01:00
Mark Haines	b162cb2e41	Add some TODOs	2016-08-25 18:18:53 +01:00
Erik Johnston	0e1900d819	Pull out full state less	2016-08-25 18:15:51 +01:00
Mark Haines	641efb6a39	Fix the deduplication of incoming direct-to-device messages	2016-08-25 18:14:02 +01:00
Paul "LeoNerd" Evans	e7af8be5ae	Root the 3PE lookup API within /_matrix/app/unstable instead of at toplevel	2016-08-25 18:06:29 +01:00
Paul "LeoNerd" Evans	142983b4ea	APP_SERVICE_PREFIX is never used; don't bother	2016-08-25 18:06:05 +01:00
Erik Johnston	721414d98a	Add desc	2016-08-25 17:49:05 +01:00
Mark Haines	e993925279	Add store-and-forward direct-to-device messaging	2016-08-25 17:35:37 +01:00
Erik Johnston	a3dc1e9cbe	Replace context.current_state with context.current_state_ids	2016-08-25 17:32:22 +01:00
Paul Evans	d9dcb2ba3a	Merge pull request #1041 from matrix-org/paul/third-party-lookup ... Extend 3PE lookup APIs for metadata query	2016-08-25 17:06:53 +01:00
Paul "LeoNerd" Evans	adf53f04ce	appease pep8	2016-08-25 16:00:31 +01:00
Paul "LeoNerd" Evans	c435bfee9c	Don't need toplevel cache on 3PE lookup metadata any more	2016-08-25 15:57:07 +01:00
Paul "LeoNerd" Evans	db7283cc6b	Implement a ResponseCache around 3PE lookup metadata lookups	2016-08-25 15:56:27 +01:00
Paul "LeoNerd" Evans	d0b8d49f71	Kill PROTOCOL_META since I'm not using it any more	2016-08-25 15:45:28 +01:00
Paul "LeoNerd" Evans	5474824975	Actually query over AS API for 3PE lookup metadata	2016-08-25 15:29:36 +01:00
Erik Johnston	17f4f14df7	Pull out event ids rather than full events for state	2016-08-25 13:42:44 +01:00
Erik Johnston	cd5b264b03	Fix None check in backfill	2016-08-25 10:39:19 +01:00
Erik Johnston	60a015550a	Bump changelog and version	2016-08-24 14:31:13 +01:00
Erik Johnston	90d5983d7a	Merge branch 'develop' of github.com:matrix-org/synapse into release-v0.17.1	2016-08-24 14:28:57 +01:00
Erik Johnston	c20cb5160d	Remove tracer	2016-08-24 13:22:47 +01:00
Erik Johnston	fda97dd58a	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/preserve_log_contexts	2016-08-24 13:22:02 +01:00
Paul "LeoNerd" Evans	8e1ed09dff	Move static knowledge of protocol metadata into AS handler; cache the result	2016-08-24 13:01:53 +01:00
Paul "LeoNerd" Evans	965f33c901	Declare 'gitter' known protocol, with user lookup	2016-08-24 12:34:03 +01:00
Paul "LeoNerd" Evans	9899824b85	Initial hack at the 3PN protocols metadata lookup API	2016-08-24 12:33:01 +01:00
Erik Johnston	9219139351	Preserve some logcontexts	2016-08-24 11:58:40 +01:00
Paul "LeoNerd" Evans	63c19e1df9	Move 3PU/3PL lookup APIs into /thirdparty containing entity	2016-08-24 11:55:57 +01:00
Erik Johnston	86bcf4d6a7	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/pagination	2016-08-24 10:38:21 +01:00
Erik Johnston	ba07d4a70e	Add None checks to backfill	2016-08-24 10:31:05 +01:00
Kegan Dougal	4b31426a02	Pass through user-supplied content in /join/$room_id ... It was always intended to allow custom keys on the join event, but this has at some point been lost. Restore it. If the user specifies keys like "avatar_url" then they will be clobbered.	2016-08-23 16:32:04 +01:00
Erik Johnston	122c7a43c9	Merge pull request #1038 from matrix-org/erikj/receved_txn_purge ... Delete old received_transactions rows	2016-08-23 11:02:40 +01:00
Erik Johnston	14047126d8	Bump changelog and version	2016-08-22 18:26:07 +01:00
Erik Johnston	d143f211c8	Merge pull request #1028 from matrix-org/dbkr/notifications_api ... Add the Notifications API	2016-08-22 18:23:24 +01:00
Erik Johnston	aad8a1a825	Delete old received_transactions	2016-08-22 16:29:46 +01:00
Mark Haines	689f4cb914	Update comment	2016-08-22 16:17:31 +01:00
Mark Haines	c8f9b45bc2	Add usage stats to prometheus monitoring	2016-08-22 15:34:38 +01:00
Erik Johnston	e65bc7d315	Merge pull request #1031 from matrix-org/erikj/measure_notifier ... Add more Measure blocks	2016-08-22 12:13:07 +01:00
Erik Johnston	33f3624ff7	Add exception logging. Fix typo	2016-08-22 10:49:31 +01:00
Erik Johnston	8c52160b07	Allow request handlers to override metric name	2016-08-22 10:44:45 +01:00
Erik Johnston	a093fab253	Use top level measure	2016-08-22 10:18:12 +01:00
Matthew Hodgson	6e80c03d45	Merge branch 'develop' into dbkr/notifications_api	2016-08-20 00:16:18 +01:00
Erik Johnston	27e0178da9	Add a top level measure	2016-08-19 18:49:37 +01:00
Erik Johnston	6d1a94d218	Remove redundant measure	2016-08-19 18:40:31 +01:00
Erik Johnston	8731197e54	Only abort Measure on Exceptions	2016-08-19 18:23:45 +01:00
Erik Johnston	afbf6b33fc	defer.returnValue must not be called within Measure	2016-08-19 18:23:44 +01:00
Erik Johnston	37adde32dc	Move defer.returnValue out of Measure	2016-08-19 18:23:44 +01:00
Erik Johnston	04fc8bbcb0	Update keyring Measure	2016-08-19 18:23:44 +01:00
Erik Johnston	39b900b316	Measure http.server render	2016-08-19 18:23:44 +01:00
Erik Johnston	47dd8f02a1	Measure _get_event_from_row	2016-08-19 18:23:44 +01:00
Erik Johnston	2426c2f21a	Measure keyrings	2016-08-19 18:23:44 +01:00
Erik Johnston	39242090e3	Add measure blocks to notifier	2016-08-19 18:23:44 +01:00
Erik Johnston	e6784daf07	Merge pull request #1030 from matrix-org/erikj/cache_contexts ... Add concept of cache contexts	2016-08-19 16:29:58 +01:00
Erik Johnston	45fd2c8942	Ensure invalidation list does not grow unboundedly	2016-08-19 16:09:16 +01:00
Erik Johnston	c0d7d9d642	Rename to on_invalidate	2016-08-19 15:13:58 +01:00
Erik Johnston	dc76a3e909	Make cache_context an explicit option	2016-08-19 15:02:38 +01:00
Erik Johnston	f164fd9220	Move _bulk_get_push_rules_for_room to storage layer	2016-08-19 14:29:20 +01:00
Erik Johnston	ba214a5e32	Remove lru option	2016-08-19 14:17:11 +01:00
Erik Johnston	4161ff2fc4	Add concept of cache contexts	2016-08-19 14:17:07 +01:00
Erik Johnston	b770435389	Make get_new_events_for_appservice use indices	2016-08-19 10:28:42 +01:00
Paul Evans	5674ea3e6c	Merge pull request #1026 from matrix-org/paul/thirdpartylookup ... 3rd party entity lookup	2016-08-18 20:52:50 +01:00
David Baker	1e4217c90c	Explicit join	2016-08-18 17:53:44 +01:00
David Baker	0acdd0f1ea	Use tuple comparison ... Hopefully easier to read	2016-08-18 17:51:08 +01:00
Paul "LeoNerd" Evans	65201631a4	Move validation logic for AS 3PE query response into ApplicationServiceApi class, to keep the handler logic neater	2016-08-18 17:33:56 +01:00
Paul "LeoNerd" Evans	697872cf08	More warnings about invalid results from AS 3PE query	2016-08-18 17:24:39 +01:00
Paul "LeoNerd" Evans	b515f844ee	Avoid so much copypasta between 3PU and 3PL query by unifying around a ThirdPartyEntityKind enumeration	2016-08-18 17:19:55 +01:00
David Baker	602c84cd9c	Merge remote-tracking branch 'origin/develop' into dbkr/notifications_api	2016-08-18 17:15:26 +01:00
Paul "LeoNerd" Evans	2a91799fcc	Minor syntax neatenings	2016-08-18 16:58:25 +01:00
Paul "LeoNerd" Evans	fcf1dec809	Appease pep8	2016-08-18 16:26:19 +01:00
Paul "LeoNerd" Evans	105ff162d4	Authenticate 3PE lookup requests	2016-08-18 16:19:23 +01:00
Paul "LeoNerd" Evans	06964c4a0a	Copypasta the 3PU support code to also do 3PL	2016-08-18 16:09:50 +01:00
Paul "LeoNerd" Evans	f3afd6ef1a	Remove TODO note about request fields being strings - they're always strings	2016-08-18 15:53:01 +01:00
Erik Johnston	bcbd74dc5b	Remove log lines	2016-08-18 15:52:10 +01:00
Paul "LeoNerd" Evans	d7b42afc74	Log a warning if an AS yields an invalid 3PU lookup result	2016-08-18 15:49:55 +01:00
Paul "LeoNerd" Evans	80f4740c8f	Scattergather the call out to ASes; validate received results	2016-08-18 15:40:41 +01:00
Erik Johnston	07229bbdae	Add appservice worker	2016-08-18 14:59:55 +01:00
Paul "LeoNerd" Evans	434bbf2cb5	Filter 3PU lookups by only ASes that declare knowledge of that protocol	2016-08-18 14:56:02 +01:00
Paul "LeoNerd" Evans	d5bf7a4a99	Merge remote-tracking branch 'origin/develop' into paul/thirdpartylookup	2016-08-18 14:21:01 +01:00
Paul "LeoNerd" Evans	718ffcf8bb	Since empty lookups now return 200/empty list not 404, we can safely log failures as exceptions	2016-08-18 14:18:37 +01:00
Paul "LeoNerd" Evans	3856582741	Ensure that 3PU lookup request fields actually get passed in	2016-08-18 14:06:02 +01:00
Paul "LeoNerd" Evans	f0c73a1e7a	Extend individual list results into the main return list, don't append	2016-08-18 13:53:54 +01:00
Paul "LeoNerd" Evans	b3511adb92	Don't catch the return-value-as-exception that @defer.inlineCallbacks will use	2016-08-18 13:45:18 +01:00
Erik Johnston	9da84a9a1e	Make AppserviceHandler stream events from database ... This is for two reasons: 1. Suppresses duplicates correctly, as the notifier doesn't do any duplicate suppression. 2. Makes it easier to connect the AppserviceHandler to the replication stream.	2016-08-18 11:54:41 +01:00
Mark Haines	403ecd8a2c	Missed a s/federation reader/media repository/ in a log message	2016-08-18 10:26:15 +01:00
Mark Haines	47fbff7a05	Merge pull request #1024 from matrix-org/markjh/media_repository ... Add a media repository worker	2016-08-18 10:12:17 +01:00
Mark Haines	396624864a	Add a media repository worker	2016-08-18 09:38:42 +01:00
Erik Johnston	e73dcb66da	Merge pull request #1022 from matrix-org/erikj/as_notify_perf ... Make notify_interested_services faster	2016-08-18 09:35:19 +01:00
Paul "LeoNerd" Evans	3ec10dffd6	Actually make 3PU lookup calls out to ASes	2016-08-18 00:39:09 +01:00
Erik Johnston	732cf72b86	Fix push_display_name_rename schema update	2016-08-17 18:12:21 +01:00
Erik Johnston	abcb9aee5b	Make push Measure finer grained	2016-08-17 18:00:18 +01:00
Erik Johnston	2a0d8f8219	Merge pull request #1021 from matrix-org/erikj/mediasecurity_policy ... Set `Content-Security-Policy` on media repo	2016-08-17 17:22:43 +01:00
Erik Johnston	320dfe523c	Make notify_interested_services faster	2016-08-17 17:20:50 +01:00
Erik Johnston	0af9e1a637	Set Content-Security-Policy on media repo ... This is to inform browsers that they should sandbox the returned media. This is particularly cruical for javascript/HTML files.	2016-08-17 16:27:39 +01:00
Paul "LeoNerd" Evans	fa87c981e1	Thread 3PU lookup through as far as the AS API object; which currently noöps it	2016-08-17 16:17:28 +01:00
David Baker	0d7cef0943	Merge pull request #1011 from matrix-org/dbkr/contains_display_name_override ... Move display name rule	2016-08-17 16:15:45 +01:00
Erik Johnston	f90b3d83a3	Add None check to _iterate_over_text	2016-08-17 15:17:17 +01:00
Matrix	f743471380	Change name of metric	2016-08-17 15:05:50 +01:00
Erik Johnston	b9e888858c	Move Measure block inside loop	2016-08-17 14:52:26 +01:00
Erik Johnston	973d67a033	Merge pull request #1019 from matrix-org/erikj/appservice_clean ... Clean up _ServiceQueuer	2016-08-17 14:37:21 +01:00
Paul "LeoNerd" Evans	e3e3fbc23a	Initial empty implementation that just registers an API endpoint handler	2016-08-17 12:46:49 +01:00
Erik Johnston	7321f45457	Clean up _ServiceQueuer	2016-08-17 12:03:04 +01:00
Erik Johnston	b9abf3e4e3	Remove dead appservice code	2016-08-17 11:48:23 +01:00
Erik Johnston	62c5245c87	Measure notify_interested_services	2016-08-17 11:12:29 +01:00
Erik Johnston	949629291c	Do it in storage function	2016-08-16 17:05:34 +01:00
David Baker	ad42322257	Add migration script ... To port existing rule actions & enable entries to the new name	2016-08-16 16:56:30 +01:00
David Baker	0bba2799b6	Merge remote-tracking branch 'origin/develop' into dbkr/contains_display_name_override	2016-08-16 16:46:37 +01:00
Erik Johnston	64a2acb161	Don't update caches replication stream if tokens haven't advanced	2016-08-16 16:44:19 +01:00
David Baker	1594eba29e	s/underride/override/ in the rule_id too	2016-08-16 16:44:07 +01:00
Erik Johnston	109a560905	Flake8	2016-08-16 14:57:21 +01:00
Erik Johnston	48b5829aea	Fix up preview URL API. Add tests. ... This includes: - Splitting out methods of a class into stand alone functions, to make them easier to test. - Adding unit tests to split out functions, testing HTML -> preview. - Handle the fact that elements in lxml may have tail text.	2016-08-16 14:53:24 +01:00
Erik Johnston	7c6f4f9427	Merge pull request #1012 from matrix-org/erikj/limit_backfill_uri ... Limit number of extremeties in backfill request	2016-08-16 12:55:42 +01:00
Erik Johnston	25c2332071	Merge pull request #1010 from matrix-org/erikj/refactor_deletions ... Refactor user_delete_access_tokens. Invalidate get_user_by_access_token to slaves.	2016-08-16 11:37:53 +01:00
Erik Johnston	2ee1bd124c	Limit number of extremeties in backfill request ... This works around a bug where if we make a backfill request with too many extremeties it causes the request URI to be too long.	2016-08-16 11:34:36 +01:00
Erik Johnston	a2427981b7	Use cached get_user_by_access_token in slaves	2016-08-16 11:24:32 +01:00
Erik Johnston	6cbd1b495e	Merge branch 'fix_integrity_retry' of https://github.com/Ralith/synapse into Ralith-fix_integrity_retry	2016-08-16 10:50:24 +01:00
David Baker	1c7c317df1	Move display name rule ... As per https://github.com/matrix-org/matrix-doc/pull/373 and comment	2016-08-15 18:34:53 +01:00
Erik Johnston	dc3a00f24f	Refactor user_delete_access_tokens. Invalidate get_user_by_access_token to slaves.	2016-08-15 17:04:39 +01:00
Erik Johnston	75299af4fc	Merge pull request #1009 from matrix-org/erikj/event_split ... Split out /events to synchrotron	2016-08-15 15:39:05 +01:00
Erik Johnston	89e786bd85	Doc get_next() context manager usage	2016-08-15 13:45:26 +01:00
Erik Johnston	d9664344ec	Rename table. Add docs.	2016-08-15 11:45:57 +01:00
Erik Johnston	784a2d4f2c	Remove broken cache stuff	2016-08-15 11:25:48 +01:00
Erik Johnston	0be963472b	Use cached version of get_aliases_for_room	2016-08-15 11:24:12 +01:00
Erik Johnston	64e7e11853	Implement cache replication stream	2016-08-15 11:16:45 +01:00
Erik Johnston	4d70d1f80e	Add some invalidations to a cache_stream	2016-08-15 11:15:17 +01:00
Erik Johnston	99bbd90b0d	Always run txn.after_callbacks	2016-08-15 09:45:44 +01:00
Benjamin Saunders	8a57cc3123	Add missing database corruption recovery case ... Signed-off-by: Benjamin Saunders <ben.e.saunders@gmail.com>	2016-08-14 11:50:22 -07:00
Daniel Ehlers	dfaf0fee31	Log the value which is observed in the first place. ... The name 'result' is of bool type and has no len property, resulting in a TypeError. Futhermore in the flow control conn.response is observed and hence should be reported. Signed-off-by: Daniel Ehlers <sargon@toppoint.de>	2016-08-14 16:49:05 +02:00
Daniel Ehlers	e380538b59	Fix AttributeError when bind_dn is not defined. ... In case one does not define bind_dn in ldap configuration, filter attribute is not declared. Since auth code only uses ldap_filter attribute when according LDAP mode is selected, it is safe to only declare the attribute in that case. Signed-off-by: Daniel Ehlers <sargon@toppoint.de>	2016-08-14 16:48:33 +02:00
Erik Johnston	4e1cebd56f	Make synchrotron accept /events	2016-08-12 15:31:44 +01:00
Erik Johnston	866a5320de	Dont invoke get_handlers fromClientV1RestServlet ... hs.get_handlers() can not be invoked from split out processes. Moving the invocations down a level means that we can slowly split out individual servlets.	2016-08-12 10:03:19 +01:00
Erik Johnston	448ac6cf0d	Only process one local membership event per room at a time	2016-08-12 09:32:19 +01:00
Erik Johnston	832799dbff	Merge pull request #997 from Half-Shot/develop ... Don't change status_msg on /sync	2016-08-11 14:10:55 +01:00
David Baker	b4ecf0b886	Merge remote-tracking branch 'origin/develop' into dbkr/notifications_api	2016-08-11 14:09:13 +01:00
Will Hunt	5b5148b7ec	Synced up synchrotron set_state with PresenceHandler set_state	2016-08-11 11:48:30 +01:00
Erik Johnston	1d40373c9d	Include prev_content in redacted state events	2016-08-11 10:24:41 +01:00
Erik Johnston	c315922b5f	PEP8	2016-08-10 16:34:10 +01:00
Erik Johnston	ca8abfbf30	Clean up TransactionQueue	2016-08-10 16:24:16 +01:00
Erik Johnston	5aeadb7414	Merge pull request #999 from matrix-org/erikj/measure_more ... Measure federation send transaction resources	2016-08-10 14:16:14 +01:00
Erik Johnston	487bc49bf8	Don't stop on 4xx series errors	2016-08-10 13:39:12 +01:00
Erik Johnston	739ea29d1e	Also check if server is in the room	2016-08-10 13:32:23 +01:00
Erik Johnston	ea8c4094db	Also pull out rejected events	2016-08-10 13:26:13 +01:00
Erik Johnston	7f41bcbeec	Correctly auth /event/ requests	2016-08-10 13:22:20 +01:00
Erik Johnston	11fdfaf03b	Only resign our own events	2016-08-10 13:16:58 +01:00
Will Hunt	2510db3e76	Don't change status_msg on /sync	2016-08-10 12:59:59 +01:00
Erik Johnston	f91df1f761	Store if we fail to fetch an event from a destination	2016-08-10 11:31:46 +01:00
Erik Johnston	3bc9629be5	Measure federation send transaction resources	2016-08-10 10:56:38 +01:00
Erik Johnston	d45489474d	Merge pull request #996 from matrix-org/erikj/tls_error ... Don't print stack traces when failing to get remote keys	2016-08-10 10:56:08 +01:00
Erik Johnston	fa1ce4d8ad	Don't print stack traces when failing to get remote keys	2016-08-10 10:44:37 +01:00
Richard van der Hoff	79ebfbe7c6	/login: Respond with a 403 when we get an invalid m.login.token	2016-08-09 16:29:28 +01:00
David Baker	cd41c6ece2	Merge pull request #995 from matrix-org/rav/clean_up_cas_login ... Clean up CAS login code	2016-08-09 10:21:56 +01:00
David Baker	27771b2495	Merge pull request #994 from matrix-org/rav/fix_cas_login ... Fix CAS login	2016-08-08 17:47:45 +01:00
Richard van der Hoff	d3250499c1	Merge pull request #993 from matrix-org/rav/fix_token_login ... Fix token login	2016-08-08 17:43:02 +01:00
Richard van der Hoff	65666fedd5	Clean up CAS login code ... Remove some apparently unused code. Clean up parse_cas_response, mostly to catch the exception if the CAS response isn't valid XML.	2016-08-08 17:17:25 +01:00
Richard van der Hoff	0682ca04b3	Fix CAS login ... Attempting to log in with CAS was giving a 500 error.	2016-08-08 17:01:30 +01:00
Richard van der Hoff	6fe6a6f029	Fix login with m.login.token ... login with token (as used by CAS auth) was broken by 067596d, such that it always returned a 401.	2016-08-08 16:40:39 +01:00
Erik Johnston	27fe3e2d4f	Bump changelog and version	2016-08-08 11:31:12 +01:00
Erik Johnston	3410142741	Merge branch 'develop' of github.com:matrix-org/synapse into release-v0.17.0	2016-08-08 11:24:53 +01:00
Erik Johnston	7c1a92274c	Make psutil optional	2016-08-08 11:12:21 +01:00
Erik Johnston	f5deaff424	Merge pull request #991 from matrix-org/erikj/retry_make ... Retry joining via other servers if first one failed. Fix some other bugs.	2016-08-05 18:21:27 +01:00
Erik Johnston	5f360182c6	Fix a couple of python bugs	2016-08-05 18:08:32 +01:00
Erik Johnston	46453bfc2f	Retry joining via other servers if first one failed	2016-08-05 18:02:03 +01:00
Erik Johnston	6bf6bc1d1d	Merge pull request #990 from matrix-org/erikj/fed_vers ... Add federation /version API	2016-08-05 16:59:16 +01:00
Erik Johnston	24f36469bc	Add federation /version API	2016-08-05 16:36:07 +01:00
Erik Johnston	597c79be10	Change the way we specify if we require auth or not	2016-08-05 16:17:04 +01:00
Erik Johnston	e6021c370e	Merge pull request #989 from matrix-org/erikj/raise_404 ... Raise 404 when couldn't find event	2016-08-05 15:54:16 +01:00
Erik Johnston	a8b946decb	Raise 404 when couldn't find event	2016-08-05 15:31:02 +01:00
Erik Johnston	5bcccfde6c	Don't include html comments in description	2016-08-05 14:45:11 +01:00
Erik Johnston	4d87d3659a	Merge branch 'develop' of github.com:matrix-org/synapse into release-v0.17.0	2016-08-05 13:19:22 +01:00
Erik Johnston	93acf49e9b	Fix backfill auth events	2016-08-05 12:59:04 +01:00
Erik Johnston	b2c290a6e5	Bump version and changelog	2016-08-05 11:16:54 +01:00
Erik Johnston	499dc1b349	Merge branch 'develop' of github.com:matrix-org/synapse into release-v0.17.0	2016-08-05 11:15:48 +01:00
Erik Johnston	2d4de61fb7	Fix typo	2016-08-05 10:48:56 +01:00
Erik Johnston	fccadb7719	Check if we already have the events returned by /state/	2016-08-05 10:43:47 +01:00
Erik Johnston	f0fa66f495	Delete more tables	2016-08-05 10:40:08 +01:00
Erik Johnston	1515d1b581	Fallback to /state/ on both 400 and 404	2016-08-05 10:24:23 +01:00
Benjamin Saunders	a2b7102eea	Tweak integrity error recovery to work as intended	2016-08-04 20:38:08 -07:00
Erik Johnston	a5d7968b3e	Merge pull request #973 from matrix-org/erikj/xpath_fix ... Change the way we summarize URLs	2016-08-04 16:37:25 +01:00
Erik Johnston	b5525c76d1	Typo	2016-08-04 16:10:08 +01:00
Erik Johnston	e97648c4e2	Test summarization	2016-08-04 16:09:09 +01:00
Erik Johnston	835ceeee76	Merge pull request #983 from matrix-org/erikj/retry_on_integrity_error ... Retry event persistence on IntegrityError	2016-08-04 15:40:03 +01:00
Erik Johnston	b3682df2ca	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/xpath_fix	2016-08-04 15:29:45 +01:00
Erik Johnston	8ad8490cff	Fix typo	2016-08-04 15:21:29 +01:00
Erik Johnston	59fa91fe88	Retry event persistence on IntegrityError ... Due to a bug in the porting script some backfilled events were not correctly persisted, causing irrecoverable IntegrityErrors on future attempts to persist those events. This commit adds a retry mechanism invoked upon IntegrityError, where when retried the tables are purged for all references to the events being persisted.	2016-08-04 15:02:15 +01:00
Erik Johnston	257c41cc2e	Fix typos.	2016-08-04 14:05:45 +01:00
Erik Johnston	b4e2290d89	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/state_ids_api	2016-08-04 14:04:35 +01:00
Erik Johnston	e3ee63578f	Tidy up get_events	2016-08-04 14:01:18 +01:00
Richard van der Hoff	f131cd9e53	keys/query: Omit device displayname if null ... ... which makes it more consistent with user displaynames.	2016-08-04 10:59:51 +01:00
Erik Johnston	edb33eb163	Rename fields to _ids	2016-08-03 17:19:15 +01:00
Erik Johnston	bcc9cda8ca	Fix copy + paste fails	2016-08-03 17:17:26 +01:00
Richard van der Hoff	98385888b8	PEP8	2016-08-03 15:42:08 +01:00
Richard van der Hoff	68264d7404	Include device name in /keys/query response ... Add an 'unsigned' section which includes the device display name.	2016-08-03 15:42:08 +01:00
Richard van der Hoff	91fa69e029	keys/query: return all users which were asked for ... In the situation where all of a user's devices get deleted, we want to indicate this to a client, so we want to return an empty dictionary, rather than nothing at all.	2016-08-03 15:41:44 +01:00
Erik Johnston	4c56bedee3	Actually call get_room_state	2016-08-03 15:04:29 +01:00
Erik Johnston	520ee9bd2c	Fix syntax error	2016-08-03 15:03:15 +01:00
Erik Johnston	a60a2eaa02	Comment	2016-08-03 14:52:43 +01:00
Erik Johnston	e3a720217a	Add /state_ids federation API ... The new API only returns the event_ids for the state, as most requesters will already have the vast majority of the events already.	2016-08-03 14:47:37 +01:00
Richard van der Hoff	530bc862dc	Merge branch 'rav/null_default_device_displayname' into develop	2016-08-03 14:30:32 +01:00
Richard van der Hoff	a6f5cc65d9	PEP8	2016-08-03 14:30:06 +01:00
Richard van der Hoff	e555bc6551	Merge branch 'rav/refactor_device_query' into develop	2016-08-03 14:26:01 +01:00
Richard van der Hoff	a843868fe9	E2eKeysHandler: minor tweaks ... PR feedback	2016-08-03 14:24:33 +01:00
Erik Johnston	80ad710217	Remove other bit of deduplication	2016-08-03 13:25:59 +01:00
Richard van der Hoff	4fec5e57be	Default device_display_name to null ... It turns out that it's more useful to return a null device display name (and let clients decide how to handle it: eg, falling back to device_id) than using a constant string like "unknown device".	2016-08-03 11:53:00 +01:00
Erik Johnston	a8a32d2714	Ensure we only persist an event once at a time	2016-08-03 11:23:39 +01:00
Mark Haines	921f17f938	Merge branch 'develop' into rav/refactor_device_query	2016-08-03 11:12:47 +01:00
Erik Johnston	58c9653c6b	Don't infer paragrahs from newlines	2016-08-02 18:50:24 +01:00
Erik Johnston	6b58ade2f0	Comment on why we clone	2016-08-02 18:41:22 +01:00
Erik Johnston	9e66c58ceb	Spelling.	2016-08-02 18:37:31 +01:00
Erik Johnston	f83f5fbce8	Make it actually compile	2016-08-02 18:32:42 +01:00
Erik Johnston	aecaec3e10	Change the way we summarize URLs ... Using XPath is slow on some machines (for unknown reasons), so use a different approach to get a list of text nodes. Try to generate a summary that respect paragraph and then word boundaries, adding ellipses when appropriate.	2016-08-02 18:25:53 +01:00
Richard van der Hoff	1efee2f52b	E2E keys: Make federation query share code with client query ... Refactor the e2e query handler to separate out the local query, and then make the federation handler use it.	2016-08-02 18:12:00 +01:00
Erik Johnston	49e047c55e	Bump version and changelog	2016-08-02 17:10:26 +01:00
Erik Johnston	59a2c6d60e	Merge branch 'develop' of github.com:matrix-org/synapse into release-v0.17.0	2016-08-02 17:07:54 +01:00
Erik Johnston	c9154b970c	Don't double wrap 200	2016-08-02 16:45:53 +01:00
Erik Johnston	b3d5c4ad9d	Fix response cache	2016-08-02 16:42:21 +01:00
Erik Johnston	8f650bd338	Bump changeog and version	2016-08-02 15:43:52 +01:00
Erik Johnston	7b0f6293f2	Merge pull request #940 from matrix-org/erikj/fed_state_cache ... Cache federation state responses	2016-08-02 15:21:37 +01:00
Erik Johnston	342e072024	Merge pull request #967 from matrix-org/erikj/fed_reader ... Split out the federation reading portions into a separate.	2016-08-02 13:59:47 +01:00
Mark Haines	54de6a812a	Merge branch 'develop' into dbkr/fix_add_email_on_register	2016-08-02 10:18:15 +01:00
Erik Johnston	733bf44290	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/fed_reader	2016-08-02 09:56:31 +01:00
Richard van der Hoff	986615b0b2	Move e2e query logic into a handler	2016-08-01 18:02:07 +01:00
Erik Johnston	b260f92936	Ignore AlreadyCalled errors on timer cancel	2016-07-31 16:00:12 +01:00
David Baker	271d3e7865	Fix adding emails on registration ... Synapse was not adding email addresses to accounts registered with an email address, due to too many different variables called 'result'. Rename both of them. Also remove the defer.returnValue() with no params because that's not a thing.	2016-07-29 15:25:24 +01:00
Paul Evans	18b7eb830b	Merge pull request #958 from matrix-org/paul/SYN-738 ... Forbid non-ASes from registering users whose names begin with '_'	2016-07-29 14:10:45 +01:00
Erik Johnston	2a9ce8c422	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/fed_reader	2016-07-29 11:33:46 +01:00
Erik Johnston	cbea0c7044	Merge pull request #964 from matrix-org/erikj/fed_join_fix ... Handle the case of missing auth events when joining a room	2016-07-29 11:33:16 +01:00
Erik Johnston	5aa024e501	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/fed_reader	2016-07-29 11:24:56 +01:00
Erik Johnston	c51a52f300	Mention that func will fetch auth events	2016-07-29 11:17:04 +01:00
Erik Johnston	3d13c3a295	Update docstring	2016-07-29 10:45:05 +01:00
Mark Haines	8dad08a950	Fix SQL to supply arguments in the same order	2016-07-29 09:57:13 +01:00
Mark Haines	0a7d3cd00f	Create separate methods for getting messages to push ... for the email and http pushers rather than trying to make a single method that will work with their conflicting requirements. The http pusher needs to get the messages in ascending stream order, and doesn't want to miss a message. The email pusher needs to get the messages in descending timestamp order, and doesn't mind if it misses messages.	2016-07-28 20:24:24 +01:00
Erik Johnston	ec8b217722	Add destination retry to slave store	2016-07-28 17:35:53 +01:00
Erik Johnston	76b89d0edb	Add slace storage functions for public room list	2016-07-28 17:03:40 +01:00
Kegan Dougal	370135ad0b	Comment get_unread_push_actions_for_user_in_range function	2016-07-28 16:47:37 +01:00
Erik Johnston	0fcbca531f	Add get_auth_chain to slave store	2016-07-28 16:36:28 +01:00
Erik Johnston	1e2740caab	Handle the case of missing auth events when joining a room	2016-07-28 16:08:33 +01:00
Erik Johnston	6ede23ff1b	Add more key storage funcs into slave store	2016-07-28 15:41:26 +01:00
Erik Johnston	5fb41a955c	Merge branch 'release-v0.17.0' of github.com:matrix-org/synapse into develop	2016-07-28 14:56:32 +01:00
Mark Haines	4329f20e3f	Merge pull request #962 from matrix-org/markjh/retry ... Fix retry utils to check if the exception is a subclass of CME	2016-07-28 11:12:19 +01:00
Erik Johnston	a285194021	Merge branch 'erikj/key_client_fix' of github.com:matrix-org/synapse into release-v0.17.0	2016-07-28 10:47:06 +01:00
Mark Haines	bf81e38d36	Fix retry utils to check if the exception is a subclass of CME	2016-07-28 10:47:02 +01:00
Erik Johnston	78cac3e594	Merge pull request #941 from matrix-org/erikj/key_client_fix ... Send the correct host header when fetching keys	2016-07-28 10:46:36 +01:00
Erik Johnston	7871790db1	Bump version and changelog	2016-07-28 10:38:56 +01:00
Erik Johnston	18e044628e	Merge branch 'develop' of github.com:matrix-org/synapse into release-v0.17.0	2016-07-28 10:37:43 +01:00
Erik Johnston	b557b682d9	Merge pull request #961 from matrix-org/dbkr/fix_push_invite_name ... Don't include name of room for invites in push	2016-07-28 10:36:41 +01:00
David Baker	389c890f14	Don't include name of room for invites in push ... Avoids insane pushes like, "Bob invited you to invite from Bob"	2016-07-28 10:20:47 +01:00
Richard van der Hoff	cd8738ab63	Merge pull request #960 from matrix-org/rav/support_r0.2 ... Add r0.2.0 to the "supported versions" list	2016-07-28 10:14:33 +01:00
Richard van der Hoff	f6f8f81a48	Add r0.1.0 to the "supported versions" list	2016-07-28 10:14:07 +01:00
David Baker	ecd5e6bfa4	Typo	2016-07-28 10:04:52 +01:00
Richard van der Hoff	fda078f995	Add r0.2.0 to the "supported versions" list	2016-07-28 09:14:21 +01:00
Richard van der Hoff	40e539683c	Merge pull request #956 from matrix-org/rav/check_device_id_on_key_upload ... Make the device id on e2e key upload optional	2016-07-27 18:16:57 +01:00
Paul "LeoNerd" Evans	05f6447301	Forbid non-ASes from registering users whose names begin with '_' (SYN-738)	2016-07-27 17:54:26 +01:00
Erik Johnston	5238960850	Bump CHANGES and version	2016-07-27 17:33:09 +01:00
Richard van der Hoff	ccec25e2c6	key upload tweaks ... 1. Add v2_alpha URL back in, since things seem to be using it. 2. Don't reject the request if the device_id in the upload request fails to match that in the access_token.	2016-07-27 16:41:06 +01:00
Mark Haines	29b25d59c6	Merge branch 'develop' into markjh/verify ... Conflicts: synapse/crypto/keyring.py	2016-07-27 15:11:02 +01:00
Mark Haines	884b800899	Merge pull request #955 from matrix-org/markjh/only_from2 ... Add a couple more checks to the keyring	2016-07-27 15:08:22 +01:00
Mark Haines	09d31815b4	Merge pull request #954 from matrix-org/markjh/even_more_fixes ... Fix a couple of bugs in the transaction and keyring code	2016-07-27 15:08:14 +01:00
Mark Haines	fe1b369946	Clean up verify_json_objects_for_server	2016-07-27 14:10:43 +01:00
Richard van der Hoff	26cb0efa88	SQL syntax fix	2016-07-27 12:30:22 +01:00
Richard van der Hoff	d47115ff8b	Delete e2e keys on device delete	2016-07-27 12:24:52 +01:00
Richard van der Hoff	2e3d90d67c	Make the device id on e2e key upload optional ... We should now be able to get our device_id from the access_token, so the device_id on the upload request is optional. Where it is supplied, we should check that it matches. For active access_tokens without an associated device_id, we ought to register the device in the devices table. Also update the table on upgrade so that all of the existing e2e keys are associated with real devices.	2016-07-26 23:38:12 +01:00
Mark Haines	a4b06b619c	Add a couple more checks to the keyring	2016-07-26 19:50:11 +01:00
Mark Haines	c63b1697f4	Merge pull request #952 from matrix-org/markjh/more_fixes ... Check if the user is banned when handling 3pid invites	2016-07-26 19:20:56 +01:00
Mark Haines	87ffd21b29	Fix a couple of bugs in the transaction and keyring code	2016-07-26 19:19:08 +01:00
Richard van der Hoff	eb359eced4	Add create_requester function ... Wrap the `Requester` constructor with a function which provides sensible defaults, and use it throughout	2016-07-26 16:46:53 +01:00
Mark Haines	c824b29e77	Check if the user is banned when handling 3pid invites	2016-07-26 16:39:14 +01:00
Richard van der Hoff	33d7776473	Fix typo	2016-07-26 13:32:15 +01:00
Richard van der Hoff	9ad8d9b17c	Merge branch 'develop' into rav/delete_refreshtoken_on_delete_device	2016-07-26 13:29:46 +01:00
Richard van der Hoff	5b1825ba5b	Merge pull request #951 from matrix-org/rav/flake8 ... Fix flake8 noise	2016-07-26 13:27:54 +01:00
Mark Haines	9c4cf83259	Merge pull request #948 from matrix-org/markjh/auth_fixes ... Don't add rejections to the state_group, persist all rejections	2016-07-26 13:22:57 +01:00
Richard van der Hoff	05e7e5e972	Fix flake8 violation ... Apparently flake8 v3 puts the error on a different line to v2. Easiest way to make sure that happens is by putting the whole statement on one line :)	2016-07-26 11:59:08 +01:00
Richard van der Hoff	8e02494166	Delete refresh tokens when deleting devices	2016-07-26 11:10:37 +01:00
Mark Haines	a6f06ce3e2	Fix how push_actions are redacted.	2016-07-26 11:05:39 +01:00
David Baker	d34e9f93b7	Merge pull request #949 from matrix-org/rav/update_devices ... Implement updates and deletes for devices	2016-07-26 10:49:55 +01:00
Mark Haines	efeb6176c1	Don't add rejected events if we've seen them befrore. Add some comments to explain what the code is doing mechanically	2016-07-26 10:49:52 +01:00
Matthew Hodgson	242c52d607	typo	2016-07-26 10:09:25 +02:00
Richard van der Hoff	012b4c1913	Implement updating devices ... You can update the displayname of devices now.	2016-07-26 07:35:48 +01:00
Richard van der Hoff	436bffd15f	Implement deleting devices	2016-07-26 07:35:48 +01:00
Mark Haines	1b3c3e6d68	Only update the events and event_json tables for rejected events	2016-07-25 18:44:30 +01:00
Richard van der Hoff	33d08e8433	Log when adding listeners	2016-07-25 17:22:15 +01:00
Mark Haines	8f7f4cb92b	Don't add the events to forward extremities if the event is rejected	2016-07-25 17:13:37 +01:00
Mark Haines	2623cec874	Don't add rejections to the state_group, persist all rejections	2016-07-25 16:12:16 +01:00
David Baker	4fcdf7b4b2	Merge pull request #946 from matrix-org/dbkr/log_recaptcha_hostname ... Log the hostname the reCAPTCHA was completed on	2016-07-25 16:10:39 +01:00
Mark Haines	955ef1f06c	fix: defer.returnValue takes one argument	2016-07-25 16:04:45 +01:00
Richard van der Hoff	2ee4c9ee02	background updates: fix assert again	2016-07-25 16:01:46 +01:00
Richard van der Hoff	9dbd903f41	background updates: Fix assertion to do something	2016-07-25 14:05:23 +01:00
Richard van der Hoff	465117d7ca	Fix background_update tests ... A bit of a cleanup for background_updates, and make sure that the real background updates have run before we start the unit tests, so that they don't interfere with the tests.	2016-07-25 12:10:42 +01:00
David Baker	7ed58bb347	Use get to avoid KeyErrors	2016-07-22 17:18:50 +01:00
David Baker	dad2da7e54	Log the hostname the reCAPTCHA was completed on ... This could be useful information to have in the logs. Also comment about how & why we don't verify the hostname.	2016-07-22 17:00:56 +01:00
Richard van der Hoff	363786845b	PEP8	2016-07-22 13:21:07 +01:00
Richard van der Hoff	ec5717caf5	Create index on user_ips in the background ... user_ips is kinda big, so really we want to add the index in the background once we're running. Replace the schema delta with one which will do that. I've done this in a way that's reasonably easy to reuse as there a few other indexes I need, and I don't suppose they will be the last.	2016-07-22 13:16:39 +01:00
Erik Johnston	d26b660aa6	Cache getPeer	2016-07-21 17:38:51 +01:00
Erik Johnston	aede7248ab	Split out a FederationReader process	2016-07-21 17:37:44 +01:00
Richard van der Hoff	55abbe1850	make /devices return a list ... Turns out I specced this to return a list of devices rather than a dict of them	2016-07-21 15:57:28 +01:00
David Baker	2c28e25bda	Merge pull request #943 from matrix-org/rav/get_device_api ... Implement GET /device/{deviceId}	2016-07-21 13:41:42 +01:00
David Baker	1e6e370b76	Merge pull request #942 from matrix-org/rav/fix_register_deviceid ... Preserve device_id from first call to /register	2016-07-21 13:16:31 +01:00
Richard van der Hoff	1c3c202b96	Fix PEP8 errors	2016-07-21 13:15:15 +01:00
Richard van der Hoff	406f7aa0f6	Implement GET /device/{deviceId}	2016-07-21 12:00:29 +01:00
Richard van der Hoff	c445f5fec7	storage/client_ips: remove some dead code	2016-07-21 11:58:47 +01:00
Erik Johnston	cf94a78872	Set host not path	2016-07-21 11:45:53 +01:00
Richard van der Hoff	1a64dffb00	Preserve device_id from first call to /register ... device_id may only be passed in the first call to /register, so make sure we fish it out of the register `params` rather than the body of the final call.	2016-07-21 11:34:16 +01:00
Erik Johnston	081e5d55e6	Send the correct host header when fetching keys	2016-07-21 11:14:54 +01:00
Erik Johnston	248e6770ca	Cache federation state responses	2016-07-21 10:30:12 +01:00
Richard van der Hoff	7314bf4682	Merge branch 'develop' into rav/get_devices_api ... (pick up PR #938 in the hope of fixing the UTs)	2016-07-20 17:40:00 +01:00
Richard van der Hoff	e9e3eaa67d	Merge pull request #938 from matrix-org/rav/add_device_id_to_client_ips ... Record device_id in client_ips	2016-07-20 17:38:45 +01:00
Erik Johnston	d36b1d849d	Don't explode if we have no snapshots yet	2016-07-20 16:59:52 +01:00
David Baker	742056be0d	Merge pull request #937 from matrix-org/rav/register_device_on_register ... Register a device_id in the /v2/register flow.	2016-07-20 16:51:27 +01:00
Richard van der Hoff	bc8f265f0a	GET /devices endpoint ... implement a GET /devices endpoint which lists all of the user's devices. It also returns the last IP where we saw that device, so there is some dancing to fish that out of the user_ips table.	2016-07-20 16:42:32 +01:00
Richard van der Hoff	ec041b335e	Record device_id in client_ips ... Record the device_id when we add a client ip; it's somewhat redundant as we could get it via the access_token, but it will make querying rather easier.	2016-07-20 16:41:03 +01:00
Richard van der Hoff	053e83dafb	More doc-comments ... Fix some more comments on some things	2016-07-20 16:40:28 +01:00
Richard van der Hoff	b97a1356b1	Register a device_id in the /v2/register flow. ... This doesn't cover *all* of the registration flows, but it does cover the most common ones: in particular: shared_secret registration, appservice registration, and normal user/pass registration. Pull device_id from the registration parameters. Register the device in the devices table. Associate the device with the returned access and refresh tokens. Profit.	2016-07-20 16:38:27 +01:00
Erik Johnston	b73dc0ef4d	Merge pull request #936 from matrix-org/erikj/log_rss ... Add metrics for psutil derived memory usage	2016-07-20 16:32:38 +01:00
Erik Johnston	66868119dc	Add metrics for psutil derived memory usage	2016-07-20 16:00:21 +01:00
Erik Johnston	aba0b2a39b	Merge pull request #935 from matrix-org/erikj/backfill_notifs ... Don't notify pusher pool for backfilled events	2016-07-20 13:39:16 +01:00
Erik Johnston	57dca35692	Don't notify pusher pool for backfilled events	2016-07-20 13:25:06 +01:00
Richard van der Hoff	c68518dfbb	Merge pull request #933 from matrix-org/rav/type_annotations ... Type annotations	2016-07-20 12:26:32 +01:00
David Baker	e967bc86e7	Merge pull request #932 from matrix-org/rav/register_refactor ... Further registration refactoring	2016-07-20 11:03:33 +01:00
Erik Johnston	1e2a7f18a1	Merge pull request #922 from matrix-org/erikj/file_api2 ... Feature: Add filter to /messages. Add 'contains_url' to filter.	2016-07-20 10:40:48 +01:00
Erik Johnston	f91faf09b3	Comment	2016-07-20 10:18:09 +01:00
Richard van der Hoff	3413f1e284	Type annotations ... Add some type annotations to help PyCharm (in particular) to figure out the types of a bunch of things.	2016-07-19 18:56:16 +01:00
Richard van der Hoff	40cbffb2d2	Further registration refactoring ... * `RegistrationHandler.appservice_register` no longer issues an access token: instead it is left for the caller to do it. (There are two of these, one in `synapse/rest/client/v1/register.py`, which now simply calls `AuthHandler.issue_access_token`, and the other in `synapse/rest/client/v2_alpha/register.py`, which is covered below). * In `synapse/rest/client/v2_alpha/register.py`, move the generation of access_tokens into `_create_registration_details`. This means that the normal flow no longer needs to call `AuthHandler.issue_access_token`; the shared-secret flow can tell `RegistrationHandler.register` not to generate a token; and the appservice flow continues to work despite the above change.	2016-07-19 18:46:19 +01:00
David Baker	b9e997f561	Merge pull request #931 from matrix-org/rav/refactor_register ... rest/client/v2_alpha/register.py: Refactor flow somewhat.	2016-07-19 16:13:45 +01:00
Richard van der Hoff	9a7a77a22a	Merge pull request #929 from matrix-org/rav/support_deviceid_in_login ... Add device_id support to /login	2016-07-19 15:53:04 +01:00
Richard van der Hoff	8f6281ab0c	Don't bind email unless threepid contains expected fields	2016-07-19 15:50:01 +01:00
Richard van der Hoff	0da0d0a29d	rest/client/v2_alpha/register.py: Refactor flow somewhat. ... This is meant to be an *almost* non-functional change, with the exception that it fixes what looks a lot like a bug in that it only calls `auth_handler.add_threepid` and `add_pusher` once instead of three times. The idea is to move the generation of the `access_token` out of `registration_handler.register`, because `access_token`s now require a device_id, and we only want to generate a device_id once registration has been successful.	2016-07-19 13:12:22 +01:00
Richard van der Hoff	022b9176fe	schema fix ... device_id should be text, not bigint.	2016-07-19 11:44:05 +01:00
Mark Haines	c41d52a042	Summary line	2016-07-19 10:28:27 +01:00
Mark Haines	7e554aac86	Update docstring on Handlers. ... To indicate it is deprecated.	2016-07-19 10:20:58 +01:00
Richard van der Hoff	f863a52cea	Add device_id support to /login ... Add a 'devices' table to the storage, as well as a 'device_id' column to refresh_tokens. Allow the client to pass a device_id, and initial_device_display_name, to /login. If login is successful, then register the device in the devices table if it wasn't known already. If no device_id was supplied, make one up. Associate the device_id with the access token and refresh token, so that we can get at it again later. Ensure that the device_id is copied from the refresh token to the access_token when the token is refreshed.	2016-07-18 16:39:44 +01:00
Richard van der Hoff	93efcb8526	Merge pull request #928 from matrix-org/rav/refactor_login ... Refactor login flow	2016-07-18 16:12:35 +01:00
Richard van der Hoff	dcfd71aa4c	Refactor login flow ... Make sure that we have the canonical user_id *before* calling get_login_tuple_for_user_id. Replace login_with_password with a method which just validates the password, and have the caller call get_login_tuple_for_user_id. This brings the password flow into line with the other flows, and will give us a place to register the device_id if necessary.	2016-07-18 15:23:54 +01:00
Erik Johnston	fca90b3445	Merge pull request #924 from matrix-org/erikj/purge_history ... Fix /purge_history bug	2016-07-18 15:11:11 +01:00
Mark Haines	a292454aa1	Merge pull request #925 from matrix-org/markjh/auth_fix ... Fix 500 ISE when sending alias event without a state_key	2016-07-18 15:04:47 +01:00
Erik Johnston	4f81edbd4f	Merge pull request #927 from Half-Shot/develop ... Fall back to 'username' if 'user' is not given for appservice registration.	2016-07-18 10:44:56 +01:00
Richard van der Hoff	6344db659f	Fix a doc-comment ... The `store` in a handler is a generic DataStore, not just an events.StateStore.	2016-07-18 09:48:10 +01:00
Will Hunt	511a52afc8	Use body.get to check for 'user'	2016-07-16 18:44:08 +01:00
Will Hunt	e885e2a623	Fall back to 'username' if 'user' is not given for appservice reg.	2016-07-16 18:33:48 +01:00
Mark Haines	d137e03231	Fix 500 ISE when sending alias event without a state_key	2016-07-15 18:58:25 +01:00
Erik Johnston	f52565de50	Fix /purge_history bug ... This was caused by trying to insert duplicate backward extremeties	2016-07-15 14:23:15 +01:00
Erik Johnston	a2d288c6a9	Merge pull request #923 from matrix-org/erikj/purge_history ... Various purge_history fixes	2016-07-15 13:23:29 +01:00
Erik Johnston	bd7c51921d	Merge pull request #919 from matrix-org/erikj/auth_fix ... Various auth.py fixes.	2016-07-15 11:38:33 +01:00
Erik Johnston	978fa53cc2	Pull out min stream_ordering from ex_outlier_stream	2016-07-15 10:22:30 +01:00
Erik Johnston	eec9609e96	event_backwards_extremeties may not be empty	2016-07-15 10:22:09 +01:00
Erik Johnston	9e1b43bcbf	Comment	2016-07-15 09:29:54 +01:00
Erik Johnston	a3036ac37e	Merge pull request #921 from matrix-org/erikj/account_deactivate ... Feature: Add an /account/deactivate endpoint	2016-07-14 17:25:15 +01:00
Erik Johnston	ebdafd8114	Check sender signed event	2016-07-14 17:03:24 +01:00
Erik Johnston	a98d215204	Add filter param to /messages API	2016-07-14 16:30:56 +01:00
Erik Johnston	d554ca5e1d	Add support for filters in paginate_room_events	2016-07-14 15:59:04 +01:00
Erik Johnston	209e04fa11	Merge pull request #918 from negzi/bugfix_for_token_expiry ... Bug fix: expire invalid access tokens	2016-07-14 15:51:52 +01:00
Erik Johnston	e5142f65a6	Add 'contains_url' to filter	2016-07-14 15:35:48 +01:00
Erik Johnston	b64aa6d687	Add sender and contains_url field to events table	2016-07-14 15:35:43 +01:00
Erik Johnston	848d3bf2e1	Add hs object	2016-07-14 10:25:52 +01:00
Erik Johnston	b55c770271	Only accept password auth	2016-07-14 10:00:38 +01:00
Erik Johnston	d543b72562	Add an /account/deactivate endpoint	2016-07-14 09:56:53 +01:00
Negar Fazeli	0136a522b1	Bug fix: expire invalid access tokens	2016-07-13 15:00:37 +02:00
Erik Johnston	2cb758ac75	Check if alias event's state_key matches sender's domain	2016-07-13 13:12:25 +01:00
Erik Johnston	560c71c735	Check creation event's room_id domain matches sender's	2016-07-13 13:07:19 +01:00
David Baker	a37ee2293c	Merge pull request #915 from matrix-org/dbkr/more_requesttokens ... Add requestToken endpoints	2016-07-13 11:51:46 +01:00
David Baker	c55ad2e375	be more pythonic	2016-07-12 14:15:10 +01:00
David Baker	aaa9d9f0e1	on_OPTIONS isn't neccessary	2016-07-12 14:13:14 +01:00
David Baker	75fa7f6b3c	Remove other debug logging	2016-07-12 14:08:57 +01:00
David Baker	a5db0026ed	Separate out requestTokens to separate handlers	2016-07-11 09:57:07 +01:00
David Baker	9c491366c5	Oops, remove debug logging	2016-07-11 09:07:40 +01:00
David Baker	385aec4010	Implement https://github.com/matrix-org/matrix-doc/pull/346/files	2016-07-08 17:42:48 +01:00
Mark Haines	dfde67a6fe	Add a comment explaining allow_none	2016-07-08 15:57:06 +01:00
Mark Haines	10c843fcfb	Ensure that the guest user is in the database when upgrading accounts	2016-07-08 15:15:55 +01:00
Erik Johnston	58930da52b	Merge branch 'master' of github.com:matrix-org/synapse into develop	2016-07-08 14:11:37 +01:00
Erik Johnston	0870588c20	Merge branch 'hotfixes-v0.16.1'	2016-07-08 13:22:32 +01:00
Erik Johnston	f90cf150e2	Bump version and changelog	2016-07-07 16:33:00 +01:00
Erik Johnston	067596d341	Fix bug where we did not correctly explode when multiple user_ids were set in macaroon	2016-07-07 16:22:24 +01:00
Erik Johnston	b92e7955be	Comment	2016-07-07 11:42:15 +01:00
Erik Johnston	c98e1479bd	Return 400 rather than 500	2016-07-07 11:41:07 +01:00
Erik Johnston	67f2c901ea	Add rest servlet. Fix SQL.	2016-07-06 15:56:59 +01:00
Erik Johnston	eef7778af9	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/test2	2016-07-06 14:50:22 +01:00
Erik Johnston	a17e7caeb7	Merge branch 'erikj/shared_secret' into erikj/test2	2016-07-06 14:46:31 +01:00
Erik Johnston	f0c06ac65c	Merge pull request #909 from matrix-org/erikj/shared_secret ... Add an admin option to shared secret registration (breaks backwards compat)	2016-07-06 14:08:51 +01:00
Erik Johnston	76b18df3d9	Check that there are no null bytes in user and passsword	2016-07-06 11:17:53 +01:00
Erik Johnston	0da24cac8b	Add null separator to hmac	2016-07-06 11:05:16 +01:00
Kent Shikama	8d9a884cee	Update password config comment ... Signed-off-by: Kent Shikama <kent@kentshikama.com>	2016-07-06 12:18:19 +09:00
Erik Johnston	be3548f7e1	Remove spurious txn	2016-07-05 17:46:51 +01:00
Erik Johnston	4adf93e0f7	Fix for postgres	2016-07-05 17:34:25 +01:00
Erik Johnston	651faee698	Add an admin option to shared secret registration	2016-07-05 17:30:22 +01:00
Erik Johnston	caf33b2d9b	Protect password when registering using shared secret	2016-07-05 17:18:19 +01:00
Erik Johnston	8f8798bc0d	Add ReadWriteLock for pagination and history prune	2016-07-05 15:30:25 +01:00
Erik Johnston	7335f0adda	Add ReadWriteLock	2016-07-05 15:23:17 +01:00
David Baker	ef535178ff	Merge pull request #904 from matrix-org/dbkr/register_email_no_untrusted_id_server ... requestToken update	2016-07-05 15:13:34 +01:00
Mark Haines	04dee11e97	Merge pull request #906 from matrix-org/markjh/faster_events_around ... Use a query that postgresql optimises better for get_events_around	2016-07-05 14:48:34 +01:00
Mark Haines	dd2ccee27d	Fix typo	2016-07-05 14:06:07 +01:00
Mark Haines	b6b0132ac7	Make get_events_around more efficient on sqlite3	2016-07-05 13:55:18 +01:00
Kent Shikama	252ee2d979	Remove default password pepper string	2016-07-05 19:15:51 +09:00
Kent Shikama	14362bf359	Fix password config	2016-07-05 19:12:53 +09:00
Kent Shikama	1ee2584307	Fix pep8	2016-07-05 19:01:00 +09:00
Kent Shikama	507b8bb091	Add comment to prompt changing of pepper	2016-07-05 18:42:35 +09:00
Mark Haines	d44d11d864	Use true/false for boolean parameter inclusive to avoid potential for sqli, and possibly make the code clearer	2016-07-05 10:39:13 +01:00
Erik Johnston	2d21d43c34	Add purge_history API	2016-07-05 10:28:51 +01:00
Mark Haines	0fb76c71ac	Use different SQL for postgres and sqlite3 for when using multicolumn indexes	2016-07-04 19:44:55 +01:00
Kent Shikama	8bdaf5f7af	Add pepper to password hashing ... Signed-off-by: Kent Shikama <kent@kentshikama.com>	2016-07-05 02:13:52 +09:00
Erik Johnston	a67bf0b074	Add storage function to purge history for a room	2016-07-04 16:02:50 +01:00
Mark Haines	f18d7546c6	Use a query that postgresql optimises better for get_events_around	2016-07-04 15:48:25 +01:00
Kent Shikama	bb069079bb	Fix style violations ... Signed-off-by: Kent Shikama <kent@kentshikama.com>	2016-07-04 22:07:11 +09:00
Kent Shikama	2e5a31f197	Use .get() instead of [] to access password_hash	2016-07-04 22:00:13 +09:00
Kent Shikama	fc8007dbec	Optionally include password hash in createUser endpoint ... Signed-off-by: Kent Shikama <kent@kentshikama.com>	2016-07-03 15:08:15 +09:00
David Baker	be8be535f7	requestToken update ... Don't send requestToken request to untrusted ID servers Also correct the THREEPID_IN_USE error to add the M_ prefix. This is a backwards incomaptible change, but the only thing using this is the angular client which is now unmaintained, so it's probably better to just do this now.	2016-06-30 17:51:28 +01:00
Erik Johnston	f328d95cef	Feature: Add deactivate account admin API ... Allows server admins to "deactivate" accounts, which: - Revokes all access tokens - Removes all threepids - Removes password The API is a POST to `/admin/deactivate/<user_id>`	2016-06-30 15:40:58 +01:00
Erik Johnston	f52cb4cd78	Remove race	2016-06-29 15:24:50 +01:00
Mark Haines	6783534a0f	Merge pull request #886 from matrix-org/markjh/async_commit ... Optionally make committing to postgres asynchronous.	2016-06-29 15:21:58 +01:00
Erik Johnston	a70688445d	Implement purge_media_cache admin API	2016-06-29 14:57:59 +01:00
Erik Johnston	314b146b2e	Track approximate last access time for remote media	2016-06-29 11:41:20 +01:00
David Baker	db74dcda5b	Merge pull request #894 from matrix-org/dbkr/push_room_naming ... Use similar naming we use in email notifs for push	2016-06-28 10:12:24 +01:00
Matthew Hodgson	63bb8f0df9	remove vector.im from default secondary DS list	2016-06-27 13:13:33 +04:00
David Baker	0b640aa56b	even more pep8	2016-06-24 11:47:11 +01:00
David Baker	aa3a4944d5	more pep8	2016-06-24 11:45:23 +01:00
David Baker	46b7362304	pep8	2016-06-24 11:44:57 +01:00
David Baker	870c45913e	Use similar naming we use in email notifs for push ... Fixes https://github.com/vector-im/vector-web/issues/1654	2016-06-24 11:41:11 +01:00
Mark Haines	05f1a4596a	Merge branch 'master' into develop	2016-06-23 11:17:48 +01:00
David Baker	b5fb7458d5	Actually we need to order these properly ... otherwise we'll end up returning the wrong 20	2016-06-22 18:07:14 +01:00
David Baker	f73fdb04a6	Style	2016-06-22 17:51:40 +01:00
David Baker	3a4120e49a	Put most recent 20 messages in notif ... Fixes https://github.com/vector-im/vector-web/issues/1648	2016-06-22 17:47:18 +01:00
Martin Weinelt	0a32208e5d	Rework ldap integration with ldap3 ... Use the pure-python ldap3 library, which eliminates the need for a system dependency. Offer both a `search` and `simple_bind` mode, for more sophisticated ldap scenarios. - `search` tries to find a matching DN within the `user_base` while employing the `user_filter`, then tries the bind when a single matching DN was found. - `simple_bind` tries the bind against a specific DN by combining the localpart and `user_base` Offer support for STARTTLS on a plain connection. The configuration was changed to reflect these new possibilities. Signed-off-by: Martin Weinelt <hexa@darmstadt.ccc.de>	2016-06-22 17:51:59 +02:00
Mark Haines	774f3a692c	Merge pull request #889 from matrix-org/markjh/synctl_workers ... Optionally start or stop workers in synctl.	2016-06-21 17:58:19 +01:00
Mark Haines	5cc7564c5c	Optionally start or stop workers in synctl. ... Optionally start or stop an individual worker by passing -w with the path to the worker config. Optionally start or stop every worker and the main synapse by passing -a with a path to a directory containing worker configs. The "-w" is intended to be used to bounce individual workers proceses. THe "-a" is intended for when you want to restart all the workers simultaneuously, for example when performing database upgrades.	2016-06-21 16:38:05 +01:00
Mark Haines	0fe0b0eeb6	Merge pull request #888 from matrix-org/markjh/content_repo ... Remove the legacy v0 content upload API.	2016-06-21 14:01:01 +01:00
Mark Haines	13e334506c	Remove the legacy v0 content upload API. ... The existing content can still be downloaded. The last upload to the matrix.org server was in January 2015, so it is probably safe to remove the upload API.	2016-06-21 11:47:39 +01:00
Mark Haines	d5fb561709	Optionally make committing to postgres asynchronous. ... Useful when running tests when you don't care whether the server will lose data that it claims that it has committed.	2016-06-20 17:53:38 +01:00
Erik Johnston	d8ec81cc31	Merge pull request #879 from matrix-org/erikj/linearize_fed_server ... Linearize some federation endpoints based on (origin, room_id)	2016-06-20 17:34:29 +01:00
Erik Johnston	bc72d381b2	Merge branch 'release-v0.16.1' of github.com:matrix-org/synapse	2016-06-20 14:18:04 +01:00
Erik Johnston	4d362a61ea	Bump version and changelog	2016-06-20 14:17:42 +01:00
Mark Haines	41e4b2efea	Add the create_profile method back since the tests use it	2016-06-17 19:20:47 +01:00
Mark Haines	0c13d45522	Add a comment on why we don't create a profile for upgrading users	2016-06-17 19:18:53 +01:00
Mark Haines	9f1800fba8	Remove registered_users from the distributor. ... The only place that was observed was to set the profile. I've made it so that the profile is set within store.register in the same transaction that creates the user. This required some slight changes to the registration code for upgrading guest users, since it previously relied on the distributor swallowing errors if the profile already existed.	2016-06-17 19:14:16 +01:00
Erik Johnston	8f4a9bbc16	Linearize some federation endpoints based on (origin, room_id)	2016-06-17 16:43:45 +01:00
Erik Johnston	9ba2bf1570	Merge pull request #878 from matrix-org/erikj/ujson ... Disable responding with canonical json for federation	2016-06-17 16:22:12 +01:00
Erik Johnston	120c238705	Disable responding with canonical json for federation	2016-06-17 16:10:37 +01:00
Erik Johnston	1c1f633b13	Merge pull request #877 from matrix-org/erikj/frozen_default ... Turn use_frozen_events off by default	2016-06-17 15:33:55 +01:00
Erik Johnston	6660f37558	Merge pull request #876 from matrix-org/erikj/sign_own ... Only re-sign our own events	2016-06-17 15:23:20 +01:00
Erik Johnston	3e41de05cc	Turn use_frozen_events off by default	2016-06-17 15:11:22 +01:00
Erik Johnston	2884712ca7	Only re-sign our own events	2016-06-17 14:47:33 +01:00
Mark Haines	ded01c3bf6	Fix `KeyError: 'msgtype'. Use .get` ... Fixes a key error where the mailer tried to get the ``msgtype`` of an event that was missing a ``msgtype``. ``` File "synapse/push/mailer.py", line 264, in get_notif_vars File "synapse/push/mailer.py", line 285, in get_message_vars File ".../frozendict/__init__.py", line 10, in __getitem__ return self.__dict[key] KeyError: 'msgtype' ```	2016-06-17 13:49:16 +01:00
Mark Haines	8c75040c25	Fix setting gc thresholds in the workers	2016-06-17 11:48:12 +01:00
Mark Haines	a352b68acf	Use worker_ prefixes for worker config, use existing support for multiple config files	2016-06-16 17:29:50 +01:00
Mark Haines	364d616792	Access the event_cache_size directly from the server object. ... This means that the workers can override the event_cache_size directly without clobbering the value in the main synapse config.	2016-06-16 12:53:15 +01:00
Mark Haines	bde13833cb	Access replication_url from the worker config directly	2016-06-16 12:44:40 +01:00
Mark Haines	80a1bc7db5	Comment on what's going on in clobber_with_worker_config	2016-06-16 11:29:45 +01:00
Mark Haines	f1f70bf4b5	Merge remote-tracking branch 'origin/develop' into markjh/worker_config	2016-06-16 11:20:17 +01:00
Mark Haines	dbb5a39b64	Add worker config module	2016-06-16 11:09:15 +01:00
Mark Haines	885ee861f7	Inline the synchrotron and pusher configs into the main config	2016-06-16 11:06:12 +01:00
Erik Johnston	486b9a6a2d	Merge pull request #873 from vt0r/bugfix/bcrypt-utf8-encode ... Fix TypeError in call to bcrypt.hashpw	2016-06-16 10:12:17 +01:00
Salvatore LaMendola	ed5f43a55a	Fix TypeError in call to bcrypt.hashpw ... - At the very least, this TypeError caused logins to fail on my own running instance of Synapse, and the simple (explicit) UTF-8 conversion resolved login errors for me. Signed-off-by: Salvatore LaMendola <salvatore.lamendola@gmail.com>	2016-06-16 00:43:42 -04:00
Erik Johnston	09a17f965c	Line lengths	2016-06-15 16:58:12 +01:00
Erik Johnston	1e9026e484	Handle floats as img widths	2016-06-15 16:58:05 +01:00
Erik Johnston	a60169ea09	Handle og props with not content	2016-06-15 16:57:48 +01:00
Erik Johnston	0ef0655b83	Bump version and changelog	2016-06-15 15:50:48 +01:00
Erik Johnston	d41a1a91d3	Linearize fetching of gaps on incoming events ... This potentially stops the server from doing multiple requests for the same data.	2016-06-15 15:16:14 +01:00
Richard van der Hoff	15bf3e3376	Merge pull request #870 from matrix-org/rav/work_around_tls_bug ... Work around TLS bug in twisted	2016-06-15 13:28:13 +01:00
Erik Johnston	b31c49d676	Correctly mark backfilled events as backfilled	2016-06-15 10:59:08 +01:00
Richard van der Hoff	255c229f23	Work around TLS bug in twisted ... Wrap up twisted's FileBodyProducer to work around https://twistedmatrix.com/trac/ticket/8473. Hopefully this fixes https://matrix.org/jira/browse/SYN-700.	2016-06-15 10:39:08 +01:00
Erik Johnston	d12134ce37	Merge pull request #868 from matrix-org/erikj/invalid_id ... Make get_domain_from_id throw SynapseError on invalid ID	2016-06-14 15:09:07 +01:00
Erik Johnston	36e2aade87	Make get_domain_from_id throw SynapseError on invalid ID	2016-06-14 13:25:29 +01:00
Matthew Hodgson	33546b58aa	point to the CAPTCHA docs	2016-06-12 23:11:29 +01:00
Mark Haines	41b35412bf	Merge pull request #863 from matrix-org/markjh/load_config ... Add function to load config without generating it	2016-06-10 11:12:32 +01:00
Mark Haines	7dbb473339	Add function to load config without generating it ... Renames ``load_config`` to ``load_or_generate_config`` Adds a method called ``load_config`` that just loads the config. The main synapse.app.homeserver will continue to use ``load_or_generate_config`` to retain backwards compat. However new worker processes can use ``load_config`` to load the config avoiding some of the cruft needed to generate the config. As the new ``load_config`` method is expected to be used by new configs it removes support for the legacy commandline overrides that ``load_or_generate_config`` supports	2016-06-09 18:50:38 +01:00
Erik Johnston	16a8884233	Merge branch 'master' of github.com:matrix-org/synapse into develop	2016-06-09 16:57:33 +01:00
Erik Johnston	a31befbcd0	Bump version and changelog	2016-06-09 13:23:41 +01:00
Erik Johnston	5ee5b655b2	Merge pull request #862 from matrix-org/erikj/media_remote_error ... 502 on /thumbnail when can't contact remote server	2016-06-09 13:11:02 +01:00
Erik Johnston	0a43219a27	Merge pull request #860 from negzi/bug_fix_get_or_create_user ... Fix a bug caused by a change in auth_handler function	2016-06-09 11:40:12 +01:00
Erik Johnston	eba4ff1bcb	502 on /thumbnail when can't contact remote server	2016-06-09 11:29:43 +01:00
Erik Johnston	95f305c35a	Remove redundant exception log in /events	2016-06-09 11:15:04 +01:00
Negar Fazeli	6e7dc7c7dd	Fix a bug caused by a change in auth_handler function ... Fix the relevant unit test cases	2016-06-08 23:22:39 +02:00
Erik Johnston	b7fbc9bd95	Merge pull request #859 from matrix-org/erikj/public_room_performance ... Pull full state for each room all at once	2016-06-08 16:36:06 +01:00
Erik Johnston	919a2c74f6	Merge branch 'release-v0.16.0' of github.com:matrix-org/synapse into develop	2016-06-08 16:26:26 +01:00
Erik Johnston	81c07a32fd	Pull full state for each room all at once	2016-06-08 15:51:49 +01:00
Erik Johnston	690029d1a3	Don't make rooms visibile by default	2016-06-08 14:47:42 +01:00
Erik Johnston	d88faf92d1	Fix up federation PublicRoomList	2016-06-08 14:39:31 +01:00
Erik Johnston	958c968d02	Merge pull request #856 from matrix-org/erikj/fed_pub_rooms ... Enable auth on /publicRoom endpoints	2016-06-08 14:36:09 +01:00
Erik Johnston	746b2f5657	Merge pull request #854 from matrix-org/erikj/federation_logging ... Add some logging for when servers ask for missing events	2016-06-08 14:31:41 +01:00
Erik Johnston	efeabd3180	Log user that is making /publicRooms calls	2016-06-08 14:23:15 +01:00
Erik Johnston	1fd6eb695d	Enable auth on federation PublicRoomList	2016-06-08 14:15:18 +01:00
Erik Johnston	17aab5827a	Add some logging for when servers ask for missing events	2016-06-08 11:55:31 +01:00
Erik Johnston	1a815fb04f	Don't hit DB for noop replications queries	2016-06-08 11:33:30 +01:00
Erik Johnston	bab916bccc	Bump version and changelog to v0.16.0-rc2	2016-06-08 11:05:45 +01:00
Erik Johnston	e0fda29f94	Merge pull request #850 from matrix-org/erikj/gc_threshold ... Add gc_threshold to pusher and synchrotron	2016-06-08 10:05:56 +01:00
Erik Johnston	0f2165ccf4	Don't track total objects as its too expensive to calculate	2016-06-07 17:00:45 +01:00
Erik Johnston	18f0cc7d99	Record some more GC metrics	2016-06-07 16:55:49 +01:00
Erik Johnston	2d1d1025fa	Add gc_threshold to pusher and synchrotron	2016-06-07 16:26:25 +01:00
Erik Johnston	b01e71e719	Merge pull request #849 from matrix-org/erikj/gc_threshold ... Allow setting of gc.set_thresholds	2016-06-07 16:07:54 +01:00
Erik Johnston	dded389ac1	Allow setting of gc.set_thresholds	2016-06-07 15:45:56 +01:00
Mark Haines	0b2158719c	Remove dead code. ... Loading push rules now happens in the datastore, so we can remove the methods that loaded them outside the datastore. The ``waiting_for_join_list`` in federation handler is populated by anything, so can be removed. The ``_get_members_events_txn`` method isn't called from anywhere so can be removed.	2016-06-07 15:07:11 +01:00
Erik Johnston	48e65099b5	Also record number of unreachable objects	2016-06-07 13:40:22 +01:00
Erik Johnston	75331c5fca	Change the way we do stats	2016-06-07 13:33:13 +01:00
Erik Johnston	8c966fbd51	Merge pull request #771 from matrix-org/erikj/gc_tick ... Manually run GC on reactor tick.	2016-06-07 13:18:36 +01:00
Mark Haines	88625db05f	Notify users for events in rooms they join. ... Change how the notifier updates the map from room_id to user streams on receiving a join event. Make it update the map when it notifies for the join event, rather than using the "user_joined_room" distributor signal	2016-06-07 11:33:36 +01:00
Erik Johnston	84379062f9	Fix AS retries, but with correct ordering	2016-06-07 10:24:50 +01:00
Erik Johnston	310197bab5	Fix AS retries	2016-06-07 09:34:50 +01:00
Mark Haines	b0932b34cb	Merge pull request #845 from matrix-org/markjh/synchrotron_presence ... Fix a KeyError in the synchrotron presence	2016-06-06 16:52:27 +01:00
Mark Haines	4a5bbb1941	Fix a KeyError in the synchrotron presence	2016-06-06 16:37:12 +01:00
Mark Haines	87f60e7053	Merge pull request #844 from matrix-org/markjh/yield_on_sleep ... Yield on the sleeps intended to backoff replication	2016-06-06 16:20:54 +01:00
Mark Haines	5ef84da4f1	Yield on the sleeps intended to backoff replication	2016-06-06 16:05:28 +01:00
Erik Johnston	216a05b3e3	.values() returns list of sets	2016-06-06 16:00:09 +01:00
Erik Johnston	96dc600579	Fix typos	2016-06-06 15:44:41 +01:00
Erik Johnston	377eb480ca	Fire after 30s not 8h	2016-06-06 15:14:21 +01:00
Erik Johnston	e4134c5e13	Merge pull request #841 from matrix-org/erikj/event_counter ... Add metric counter for number of persisted events	2016-06-06 14:17:40 +01:00
Erik Johnston	7aa778fba9	Add metric counter for number of persisted events	2016-06-06 11:58:09 +01:00
Erik Johnston	70aee0717c	Add events to cache when we persist them	2016-06-06 11:34:53 +01:00
Erik Johnston	3210f4c385	Merge pull request #836 from matrix-org/erikj/change_event_cache ... Change the way we cache events	2016-06-03 18:47:46 +01:00
Erik Johnston	cffe46408f	Don't rely on options when inserting event into cache	2016-06-03 18:25:21 +01:00
Mark Haines	ac9716f154	Fix spelling	2016-06-03 18:10:00 +01:00
Mark Haines	8f79084bd4	Add get_presence_list_accepted to the broken caches in synchrotron	2016-06-03 18:03:40 +01:00
Erik Johnston	10ea3f46ba	Change the way we cache events	2016-06-03 17:57:50 +01:00
Erik Johnston	f6be734be9	Merge pull request #835 from matrix-org/erikj/get_event_txn ... Remove event fetching from DB threads	2016-06-03 17:30:00 +01:00
Erik Johnston	05e01f21d7	Remove event fetching from DB threads	2016-06-03 17:22:13 +01:00
David Baker	72c4d482e9	3rd time lucky: we'd already calculated it above	2016-06-03 16:39:50 +01:00
David Baker	fbf608decb	Oops, we're using the dict form	2016-06-03 16:38:39 +01:00
David Baker	06d40c8b98	Add substitutions to email notif From	2016-06-03 16:31:23 +01:00
Erik Johnston	5f88549f4a	Merge branch 'release-v0.16.0' of github.com:matrix-org/synapse into develop	2016-06-03 16:27:24 +01:00
Mark Haines	ca457f594e	Merge pull request #831 from matrix-org/markjh/synchrotronII ... Add a separate process that can handle /sync requests	2016-06-03 16:06:25 +01:00
Erik Johnston	21961c93c7	Bump changelog and version	2016-06-03 15:33:14 +01:00
Mark Haines	48340e4f13	Clear the list of ongoing syncs on shutdown	2016-06-03 15:02:27 +01:00
Erik Johnston	fcbc282f56	Merge branch 'release-v0.15.0' of github.com:matrix-org/synapse into release-v0.16.0	2016-06-03 15:02:04 +01:00
Erik Johnston	51773bcbaf	Merge pull request #832 from matrix-org/erikj/presence_coount ... Change def of small delta in presence stream. Add metrics.	2016-06-03 14:57:00 +01:00
Mark Haines	da491e75b2	Appease flake8	2016-06-03 14:56:36 +01:00
Mark Haines	0b3c80a234	Use ClientIpStore to record client ips	2016-06-03 14:55:01 +01:00
Mark Haines	dd6f62ed99	Merge branch 'develop' into markjh/synchrotronII	2016-06-03 14:51:33 +01:00
Mark Haines	eef541a291	Move insert_client_ip to a separate class	2016-06-03 14:42:35 +01:00
Mark Haines	80aade3805	Send updates to the syncing users every ten seconds or immediately if they've just come online	2016-06-03 14:24:19 +01:00
Erik Johnston	ab116bdb0c	Fix typo	2016-06-03 14:03:42 +01:00
Erik Johnston	4ce84a1acd	Change metric style	2016-06-03 13:49:16 +01:00
Erik Johnston	a7ff5a1770	Presence metrics. Change def of small delta	2016-06-03 13:40:55 +01:00
Matthew Hodgson	8d740132f4	Merge branch 'develop' into matthew/brand-from-header	2016-06-03 12:14:18 +01:00
Erik Johnston	3b096c5f5c	Merge branch 'erikj/cache_perf' of github.com:matrix-org/synapse into develop	2016-06-03 12:00:33 +01:00
Erik Johnston	43b7f371f5	Merge pull request #830 from matrix-org/erikj/metrics_perf ... Change CacheMetrics to be quicker	2016-06-03 11:57:29 +01:00
Mark Haines	abb151f3c9	Add a separate process that can handle /sync requests	2016-06-03 11:57:26 +01:00
Erik Johnston	4982b28868	Merge pull request #829 from matrix-org/erikj/poke_notifier ... Poke notifier on next reactor tick	2016-06-03 11:52:10 +01:00
Erik Johnston	d06f2a229e	Merge pull request #828 from matrix-org/erikj/joined_hosts_for_room ... Make get_joined_hosts_for_room use get_users_in_room	2016-06-03 11:50:30 +01:00
Erik Johnston	58a224a651	Pull out update_results_dict	2016-06-03 11:47:07 +01:00
Mark Haines	20eccd84d4	Merge pull request #827 from matrix-org/markjh/more_slaved_methods ... Add methods to events, account data and receipt slaves	2016-06-03 11:46:21 +01:00
Erik Johnston	722472b48c	Merge pull request #825 from matrix-org/erikj/cache_push_rules ... Load push rules in storage layer so that they get cached	2016-06-03 11:44:32 +01:00
Erik Johnston	73c7112433	Change CacheMetrics to be quicker ... We change it so that each cache has an individual CacheMetric, instead of having one global CacheMetric. This means that when a cache tries to increment a counter it does not need to go through so many indirections.	2016-06-03 11:26:52 +01:00
Mark Haines	b09f348530	Merge pull request #824 from matrix-org/markjh/slaved_presence_store ... Add a slaved store for presence	2016-06-03 11:26:33 +01:00
Erik Johnston	4c04222fa5	Poke notifier on next reactor tick	2016-06-03 11:24:16 +01:00
Erik Johnston	ccb56fc24b	Make get_joined_hosts_for_room use get_users_in_room	2016-06-03 11:20:23 +01:00
Mark Haines	81cf449daa	Add methods to events, account data and receipt slaves ... Adds the methods needed by /sync to the slaved events, account data and receipt stores.	2016-06-03 11:19:27 +01:00
Erik Johnston	e043ede4a2	Small optimisation to CacheListDescriptor	2016-06-03 11:19:22 +01:00
Mark Haines	b821c839dc	Merge pull request #823 from matrix-org/markjh/more_slaved_stores ... Add slaved stores for filters, tokens, and push rules	2016-06-03 11:17:43 +01:00
Erik Johnston	597013caa5	Make cachedList go a bit faster	2016-06-03 11:13:29 +01:00
Erik Johnston	6a0afa582a	Load push rules in storage layer, so that they get cached	2016-06-03 11:10:00 +01:00
Mark Haines	3ae915b27e	Add a slaved store for presence	2016-06-03 11:05:53 +01:00
Erik Johnston	59f2d73522	Remove unnecessary sets	2016-06-03 11:05:45 +01:00
Erik Johnston	9c26b390a2	Only get local users	2016-06-03 11:04:31 +01:00
Mark Haines	f88d747f79	Add a comment explaining why the filter cache doesn't need exipiring	2016-06-03 11:03:10 +01:00
Erik Johnston	065e739d6e	Merge pull request #811 from matrix-org/erikj/state_users_in_room ... Use state to calculate get_users_in_room	2016-06-03 10:58:27 +01:00
Erik Johnston	696d7c5937	Merge pull request #809 from matrix-org/erikj/cache_receipts_in_room ... Add get_users_with_read_receipts_in_room cache	2016-06-03 10:58:24 +01:00
Mark Haines	0eae075723	Add slaved stores for filters, tokens, and push rules	2016-06-03 10:58:03 +01:00
Matthew Hodgson	79d1f072f4	brand the email from header	2016-06-02 21:34:40 +01:00
David Baker	6bb9aacf9d	Merge pull request #821 from matrix-org/dbkr/email_unsubscribe ... Email unsubscribe links that don't require logging in	2016-06-02 17:44:55 +01:00
David Baker	745ddb4dd0	peppate	2016-06-02 17:38:41 +01:00
David Baker	7a5a5f2df2	Merge pull request #820 from matrix-org/dbkr/email_notif_string_fmt_error ... Fix error in email notification string formatting	2016-06-02 17:26:06 +01:00
David Baker	1f31cc37f8	Working unsubscribe links going straight to the HS ... and authed by macaroons that let you delete pushers and nothing else	2016-06-02 17:21:31 +01:00
Matthew Hodgson	2675c1e40e	add some branding debugging	2016-06-02 17:21:12 +01:00
David Baker	c71177f285	Merge remote-tracking branch 'origin/dbkr/email_notif_string_fmt_error' into dbkr/email_unsubscribe	2016-06-02 17:20:56 +01:00
David Baker	07a5559916	Fix error in email notification string formatting	2016-06-02 17:17:16 +01:00
Mark Haines	56d15a0530	Store the typing users as user_id strings. (#819) ... Rather than storing them as UserID objects.	2016-06-02 16:28:54 +01:00
David Baker	812b5de0fe	Merge remote-tracking branch 'origin/develop' into dbkr/email_unsubscribe	2016-06-02 15:33:28 +01:00
Mark Haines	80f34d7b57	Fix setting the _clock in SQLBaseStore	2016-06-02 15:23:56 +01:00
Mark Haines	661a540dd1	Deduplicate presence entries in sync (#818)	2016-06-02 15:20:28 +01:00
Mark Haines	70599ce925	Allow external processes to mark a user as syncing. (#812) ... * Add infrastructure to the presence handler to track sync requests in external processes * Expire stale entries for dead external processes * Add an http endpoint for making users as syncing Add some docstrings and comments. * Fixes	2016-06-02 15:20:15 +01:00
David Baker	fb2193cc63	Merge pull request #817 from matrix-org/dbkr/split_out_auth_handler ... Split out the auth handler	2016-06-02 14:31:35 +01:00
Erik Johnston	356f13c069	Disable INCLUDE_ALL_UNREAD_NOTIFS	2016-06-02 14:07:38 +01:00
Erik Johnston	02ac463dbf	Merge pull request #800 from matrix-org/erikj/sync_refactor ... Refactor SyncHandler	2016-06-02 14:02:13 +01:00
Matthew Hodgson	c5af1b6b00	Merge pull request #814 from matrix-org/matthew/3pid_invite_auth ... special case m.room.third_party_invite event auth to match invites,	2016-06-02 13:47:40 +01:00
David Baker	3a3fb2f6f9	Merge branch 'dbkr/split_out_auth_handler' into dbkr/email_unsubscribe	2016-06-02 13:35:25 +01:00
David Baker	4a10510cd5	Split out the auth handler	2016-06-02 13:31:45 +01:00
Matthew Hodgson	f84b89f0c6	if an email pusher specifies a brand param, use it	2016-06-02 13:29:48 +01:00
David Baker	a15ad60849	Email unsubscribing that may in theory, work ... Were it not for that fact that you can't use the base handler in the pusher because it pulls in the world. Comitting while I fix that on a different branch.	2016-06-02 11:44:15 +01:00
David Baker	e793866398	Use user_id in email greeting if display name is null	2016-06-02 09:41:13 +01:00
Matthew Hodgson	aaa70e26a2	special case m.room.third_party_invite event auth to match invites, otherwise they get out of sync and you get https://github.com/vector-im/vector-web/issues/1208	2016-06-01 22:13:47 +01:00
Erik Johnston	a04a2d043c	Merge pull request #807 from matrix-org/erikj/push_rules_cache ... Ensure we always return boolean in push rules	2016-06-01 18:07:48 +01:00
Erik Johnston	0f06b496d1	Merge pull request #806 from matrix-org/erikj/hash_cache ... Cache get_event_reference_hashes	2016-06-01 18:07:42 +01:00
David Baker	e0deeff23e	Fix room list spidering	2016-06-01 17:58:58 +01:00
David Baker	991af8b0d6	WIP on unsubscribing email notifs without logging in	2016-06-01 17:40:52 +01:00
David Baker	00c487a8db	Merge pull request #808 from matrix-org/dbkr/room_list_spider ... Add secondary_directory_servers option to fetch room list from other servers	2016-06-01 15:32:52 +01:00
Erik Johnston	c8285564a3	Use state to calculate get_users_in_room	2016-06-01 15:25:25 +01:00
David Baker	d60eed0710	Limit number of notifications in an email notification	2016-06-01 11:45:43 +01:00
Erik Johnston	43db0d9f6a	Add get_users_with_read_receipts_in_room cache	2016-06-01 10:54:32 +01:00
David Baker	8e539f13c0	Merge remote-tracking branch 'origin/develop' into dbkr/room_list_spider	2016-06-01 09:54:36 +01:00
David Baker	6ecb2ca4ec	pep8	2016-06-01 09:48:55 +01:00
David Baker	2a449fec4d	Add cache to remote room lists ... Poll for updates from remote servers, waiting for the poll if there's no cache entry.	2016-05-31 18:27:23 +01:00
David Baker	6ca4d3ae9a	Add vector.im to default secondary_directory_servers and add comment explaining it's not a permanent solution	2016-05-31 17:24:50 +01:00
Erik Johnston	dea9f20f8c	Force boolean	2016-05-31 17:24:30 +01:00
David Baker	963e3ed282	Apparently I am not permitted to have two blank lines here	2016-05-31 17:22:53 +01:00
David Baker	d240796ded	Basic, un-cached support for secondary_directory_servers	2016-05-31 17:20:07 +01:00
Mark Haines	c8c5bf950a	Fix synapse/storage/schema/delta/30/as_users.py	2016-05-31 17:10:40 +01:00
Erik Johnston	c9ca285d33	Merge pull request #805 from matrix-org/erikj/push_rules_cache ... Fix GET /push_rules	2016-05-31 16:42:21 +01:00
Erik Johnston	1d4ee854e2	Fix typo	2016-05-31 15:45:53 +01:00
Erik Johnston	cca0093fa9	Change fix	2016-05-31 15:44:08 +01:00
Erik Johnston	4efa389299	Fix GET /push_rules	2016-05-31 15:37:53 +01:00
Erik Johnston	aefd2d1cbc	Cache get_event_reference_hashes	2016-05-31 15:32:32 +01:00
Erik Johnston	10de8c2631	Merge pull request #804 from matrix-org/erikj/push_rules_cache ... Add caches to bulk_get_push_rules*	2016-05-31 15:04:40 +01:00
Mark Haines	c626fc576a	Move the AS handler out of the Handlers object. ... Access it directly from the homeserver itself. It already wasn't inheriting from BaseHandler storing it on the Handlers object was already somewhat dubious.	2016-05-31 13:53:48 +01:00
Erik Johnston	e5b0bbcd33	Add caches to bulk_get_push_rules*	2016-05-31 13:46:58 +01:00
David Baker	70ecb415f5	Fix c+p fail	2016-05-31 12:00:54 +01:00
David Baker	e1625d62a8	Add federation room list servlet	2016-05-31 11:55:57 +01:00
David Baker	887c6e6f05	Split out the room list handler ... So I can use it from federation bits without pulling in all the handlers.	2016-05-31 11:05:16 +01:00
Erik Johnston	85b992f621	Fix to allow start with postgres	2016-05-27 10:44:44 +01:00
Erik Johnston	cc84f7cb8e	Send down correct error response if user not found	2016-05-27 10:35:15 +01:00
David Baker	209ba4d024	Merge pull request #795 from matrix-org/dbkr/delete_push_actions_after_a_month ... Only delete push actions after 30 days	2016-05-24 16:22:13 +01:00
David Baker	b007ee4606	Check for presence of 'avatar_url' key	2016-05-24 15:12:05 +01:00
David Baker	cbf8d146ac	Merge pull request #799 from matrix-org/matthew/quieter-email-notifs ... Tune email notifs to make them quieter:	2016-05-24 14:30:51 +01:00
Erik Johnston	faad233ea6	Change short circuit path	2016-05-24 14:27:19 +01:00
Erik Johnston	6900303997	Don't send down all ephemeral events	2016-05-24 11:44:55 +01:00
David Baker	37b7e84620	Include the ts the notif was received at	2016-05-24 11:33:32 +01:00
Erik Johnston	1c5ed2a19b	Only work out newly_joined_users for incremental sync	2016-05-24 11:21:34 +01:00
Erik Johnston	b08ad0389e	Only include non-offline presence in initial sync	2016-05-24 11:15:05 +01:00
Erik Johnston	be2c677386	Spell builder correctly	2016-05-24 10:53:03 +01:00
Erik Johnston	79bea8ab9a	Inline function. Make load_filtered_recents private	2016-05-24 10:22:24 +01:00
Erik Johnston	84f94e4cbb	Add comments	2016-05-24 10:14:53 +01:00
Erik Johnston	137e6a4557	Shuffle things room	2016-05-24 09:50:55 +01:00
Matthew Hodgson	680f1d9387	catch thinko in presentable names	2016-05-23 22:55:11 +01:00
Matthew Hodgson	cb8a321bdd	fix NPE in room ordering	2016-05-23 22:54:56 +01:00
Matthew Hodgson	88ea5ab2c3	consistency is the better part of valour	2016-05-23 19:33:45 +01:00
Matthew Hodgson	989bdc9e56	Tune email notifs to make them quieter: ... * After initial 10 minute window, only alert every 24h for room notifs * Reset room state after 6h of idleness * Synchronise throttles for messages sent in the same notif, so the 24 hourly notifs 'line up' * Fix the email subjects to say what triggered the notification * Order the rooms in reverse activity order in the email, so the 'reason' room should always come first	2016-05-23 19:24:11 +01:00
Negi Fazeli	6fe04ffef2	Fix set profile error with Requester. ... Replace flush_user with delete access token due to function removal Add a new test case for if the user is already registered	2016-05-23 19:50:28 +02:00
David Baker	b791a530da	Actually make the 'read' flag correct	2016-05-23 18:48:02 +01:00
David Baker	a24bc5b2dc	Add GET /notifications API	2016-05-23 18:33:51 +01:00
Erik Johnston	c0c79ef444	Add back concurrently_execute	2016-05-23 18:21:27 +01:00
Erik Johnston	b5605dfecc	Refactor SyncHandler	2016-05-23 18:08:18 +01:00
David Baker	31b5395ab6	Remove debug logging	2016-05-23 16:32:01 +01:00
David Baker	c2da3406fc	Oops, missing comma	2016-05-20 18:03:31 +01:00
David Baker	ccffb0965d	Remove stale line	2016-05-20 17:59:10 +01:00
David Baker	18d68bfee4	Handle empty events table	2016-05-20 17:58:09 +01:00
David Baker	d4503e25ed	Make deleting push actions more efficient ... There's no index on received_ts, so manually binary search using the stream_ordering index, and only update it once an hour.	2016-05-20 17:56:10 +01:00
David Baker	149fa411e2	Only delete push actions after 30 days	2016-05-20 15:25:12 +01:00
Kegan Dougal	332d7e9b97	Allow clients to specify a server_name to avoid 'No known servers' ... Multiple server_names are supported via ?server_name=foo&server_name=bar	2016-05-19 13:50:52 +01:00
Matthew Hodgson	6fb51eaf7b	Merge pull request #793 from matrix-org/matthew/one-push-badge-per-convo ... increment badge count per missed convo, not per msg	2016-05-18 13:56:38 +01:00
Matthew Hodgson	e837df6adb	increment badge count per missed convo, not per msg	2016-05-18 11:53:25 +01:00
Erik Johnston	42368ea8db	Add desc to get_presence_for_users	2016-05-18 11:38:10 +01:00
Mark Haines	0cb441fedd	Move typing handler out of the Handlers object	2016-05-17 15:58:46 +01:00
Mark Haines	6a30a0bfd3	Move the functions for parsing app service config	2016-05-17 11:28:58 +01:00
Mark Haines	523d5bcd0b	Merge remote-tracking branch 'origin/develop' into markjh/liberate_sync_handler	2016-05-17 10:43:58 +01:00
Matthew Hodgson	43e1e0489c	Merge pull request #786 from matrix-org/matthew/email_notifs_tuning ... tune email notifs, fix CSS a bit, and add debugging details	2016-05-17 10:43:24 +01:00
Mark Haines	526bf8126f	Remove unused get_joined_rooms_for_user	2016-05-17 10:20:51 +01:00
Mark Haines	425e6b4983	Merge branch 'develop' into markjh/member_cleanup	2016-05-17 10:13:16 +01:00
Mark Haines	b153f5b150	Merge pull request #787 from matrix-org/markjh/liberate_presence_handler ... Move the presence handler out of the Handlers object	2016-05-17 10:09:43 +01:00
Mark Haines	3fd8a07fca	Merge pull request #788 from matrix-org/markjh/domian ... Spell "domain" correctly	2016-05-17 10:09:34 +01:00
Mark Haines	f68eea808a	Move SyncHandler out of the Handlers object	2016-05-16 20:19:26 +01:00
Mark Haines	53e171f345	Merge branch 'markjh/liberate_presence_handler' into markjh/liberate_sync_handler	2016-05-16 20:08:32 +01:00
Mark Haines	80cb9becd8	Remove get_joined_rooms_for_user from RoomMemberHandler	2016-05-16 20:06:55 +01:00
Mark Haines	816df9f267	get_room_members is unused now	2016-05-16 19:51:43 +01:00
Mark Haines	821306120a	Replaces calls to fetch_room_distributions_into with get_joined_hosts_for_room	2016-05-16 19:48:07 +01:00
Mark Haines	1a3a2002ff	Spell "domain" correctly ... s/domian/domain/g	2016-05-16 19:17:23 +01:00
Mark Haines	e168abbcff	Don't inherit PresenceHandler from BaseHandler, remove references to self.hs from presence handler	2016-05-16 19:08:40 +01:00
Matthew Hodgson	cbd2adc95e	tune email notifs, fix CSS a bit, and add debugging details	2016-05-16 18:58:38 +01:00
Mark Haines	3b86ecfa79	Move the presence handler out of the Handlers object	2016-05-16 18:56:37 +01:00
David Baker	647781ca56	Fix emailpusher import ... Try importing at the root level rather than conditionally importing, as per comment	2016-05-16 18:41:32 +01:00
Erik Johnston	c39f305067	os.environ requires a string	2016-05-16 17:21:30 +01:00
Erik Johnston	678c8a7f1e	Merge pull request #785 from matrix-org/erikj/cache_factor_synctly ... Make synctl read a cache factor from config file	2016-05-16 17:07:35 +01:00
Erik Johnston	c5c5a7403b	Make synctl read a cache factor from config file	2016-05-16 17:01:57 +01:00
Mark Haines	eb79110beb	Clean up the blacklist/whitelist handling. ... Always set the config key with an empty list, even if a list isn't specified. This means that the codepaths are the same for both the empty list and for a missing key. Since the behaviour is the same for both cases this makes the code somewhat easier to reason about.	2016-05-16 13:03:59 +01:00
Mark Haines	dd95eb4cb5	Merge branch 'develop' into matthew/preview_url_ip_whitelist	2016-05-16 12:59:41 +01:00
Erik Johnston	60d53f9e95	Count number of GC collects	2016-05-16 09:34:42 +01:00
Matthew Hodgson	782471b7e1	fix matrix.to URLs	2016-05-13 17:50:16 +01:00
Mark Haines	0466454b00	Assert that stream replicated stream positions are ints	2016-05-13 17:33:44 +01:00
Mark Haines	b3f29dc1e5	Manually expire broken caches like the who_forgot_in_room	2016-05-13 17:16:27 +01:00
Mark Haines	f03ddc98ec	Use the SlavedAccountDataStore	2016-05-13 17:01:28 +01:00
Mark Haines	1f71f386f6	Merge branch 'develop' into dbkr/email_notifs_on_pusher	2016-05-13 16:59:56 +01:00
Mark Haines	206eb9fd94	Shift some of the state_group methods into the SlavedEventStore	2016-05-13 16:58:14 +01:00
Erik Johnston	7d6e89ed22	Add a comment	2016-05-13 16:31:08 +01:00
Mark Haines	21018c2c13	Merge pull request #783 from matrix-org/markjh/slave_account_data ... Add a slaved datastore for account data	2016-05-13 15:56:04 +01:00
Mark Haines	b7381d5338	Allow receipts for events we haven't seen in the db	2016-05-13 15:46:41 +01:00
Mark Haines	3abab26458	Add a slaved datastore for account data	2016-05-13 15:34:06 +01:00
Erik Johnston	99b5a2e560	Merge pull request #741 from negzi/create_user_with_expiry ... Create user with expiry	2016-05-13 14:46:53 +01:00
Erik Johnston	ba5c616ff4	Merge pull request #778 from matrix-org/erikj/add_pusher ... Fixup add_pusher	2016-05-13 14:43:23 +01:00
Erik Johnston	0c11c1be88	Spelling	2016-05-13 14:42:25 +01:00
Erik Johnston	e00e8f2166	Merge pull request #769 from matrix-org/erikj/push_actions_delete ... Delete old pushers	2016-05-13 14:41:36 +01:00
Erik Johnston	fd8e921b6e	Merge pull request #779 from matrix-org/erikj/receipts ... Use tree cache for get_linearized_receipts_for_room	2016-05-13 14:41:21 +01:00
Erik Johnston	c9aff0736c	Remove topics table	2016-05-13 14:40:38 +01:00
Negi Fazeli	40aa6e8349	Create user with expiry ... - Add unittests for client, api and handler Signed-off-by: Negar Fazeli <negar.fazeli@ericsson.com>	2016-05-13 15:34:15 +02:00
Mark Haines	9295fa30a8	Annotate the removed indicies with why they were removed.	2016-05-13 14:16:57 +01:00
Erik Johnston	5e50058473	Remove unused indices ... This includes removing both unused indices and indices that are subsets of other indices.	2016-05-13 13:28:07 +01:00
Mark Haines	cdda850ce1	Merge pull request #781 from matrix-org/markjh/replication_problems ... Fix a bug in replication that was causing the pusher to tight loop	2016-05-13 12:05:19 +01:00
Mark Haines	0e792e7903	Log the stream IDs in an order that makes sense	2016-05-13 11:54:44 +01:00
Mark Haines	3547e66bc6	Make sure we advance our stream position	2016-05-13 11:53:00 +01:00
Erik Johnston	6da7f39d95	Use tree cache for get_linearized_receipts_for_room	2016-05-13 11:41:23 +01:00
David Baker	b5e646a18c	Make email notifs work on the pusher synapse ... Plus general bugfix to email notif code	2016-05-13 11:36:50 +01:00
Erik Johnston	13d37c3c56	Fixup add_pusher	2016-05-13 11:25:02 +01:00
Mark Haines	a458a40337	missed a spot	2016-05-12 18:19:58 +01:00
Mark Haines	7e23476814	move filter_events_for_client out of base handler	2016-05-11 13:42:37 +01:00
Mark Haines	1620578b13	Shuffle when we get the signing_key attribute. ... Wait until we sign a message to get the signing key from the homeserver config. This means that the message handler can be created without having a signing key in the config which means that separate processes like the pusher that don't send messages and don't need to sign them can still access the handlers.	2016-05-11 12:20:57 +01:00
Erik Johnston	108434e53d	Merge pull request #775 from matrix-org/erikj/password_hash ... Correctly handle NULL password hashes from the database	2016-05-11 12:18:13 +01:00
Erik Johnston	1400bb1663	Correctly handle NULL password hashes from the database	2016-05-11 12:06:02 +01:00
Mark Haines	458a435114	Fix typo	2016-05-11 10:35:33 +01:00
Mark Haines	30057b1e15	Move _create_new_client_event and handle_new_client_event out of base handler	2016-05-11 09:09:20 +01:00
David Baker	ae1af262f6	Pass through _get_state_group_for_events	2016-05-10 19:18:03 +02:00
David Baker	90afc07f39	StateStore, not EventsStore	2016-05-10 19:10:46 +02:00
David Baker	89b5ef7c4b	Cached functions must be accessed through the dict	2016-05-10 19:05:22 +02:00
David Baker	35b6e6d2a8	Pass though _get_state_group_for_events	2016-05-10 18:56:40 +02:00
David Baker	3367e65476	Pass through get_state_groups	2016-05-10 18:53:15 +02:00
David Baker	0c4ccdcb83	Also pass through get_profile_displayname	2016-05-10 18:51:14 +02:00
David Baker	f28643cea9	Uncommit accidentally commited edit to cipher list	2016-05-10 18:44:32 +02:00
David Baker	5f46be19a7	Pass through get_events to pusher too	2016-05-10 18:43:40 +02:00
David Baker	d46b18a00f	Pass through _get_event_txn	2016-05-10 18:27:06 +02:00
Matrix	3b1930e8ec	unbreak schema	2016-05-10 16:42:37 +01:00
Matthew Hodgson	fe97b81c09	Merge pull request #759 from matrix-org/dbkr/email_notifs ... Send email notifications for missed messages	2016-05-10 16:30:05 +02:00
David Baker	997db04648	Merge remote-tracking branch 'origin/develop' into dbkr/email_notifs	2016-05-10 14:40:19 +02:00
David Baker	c00b484eff	More consistent config naming	2016-05-10 14:39:16 +02:00
David Baker	94040b0798	Add config option to not send email notifs for new users	2016-05-10 14:34:53 +02:00
David Baker	e04b1d6b0a	Make pep8 happy	2016-05-10 14:23:16 +02:00
Matthew Hodgson	5599608887	Switch from CSS to Table layout for HTML mails so they work in Outlook aka Word ... Remove templates-vector and theme templates with variables instead Switch to matrix.to URLs by default for links	2016-05-10 00:14:48 +02:00
Erik Johnston	c9eb6dfc1b	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/ignore_user	2016-05-09 13:21:06 +01:00
Erik Johnston	3f84da139c	Merge pull request #773 from matrix-org/erikj/get_domian_from_id ... Add and use get_domain_from_id	2016-05-09 13:21:00 +01:00
Erik Johnston	def64d6ef3	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/ignore_user	2016-05-09 13:05:09 +01:00
Erik Johnston	100e2c42f6	Merge pull request #764 from matrix-org/erikj/replication_logging ... Add some log information at returned replication streams	2016-05-09 11:18:00 +01:00
Erik Johnston	8715731559	Merge pull request #772 from matrix-org/erikj/get_user_cache ... Add cache to get_user_by_id	2016-05-09 11:12:11 +01:00
Erik Johnston	34b3af3363	Merge pull request #770 from matrix-org/erikj/transaction_reactor ... Run transaction queue on reactor	2016-05-09 11:12:06 +01:00
Erik Johnston	08dfa8eee2	Add and use get_domian_from_id	2016-05-09 10:36:03 +01:00
Erik Johnston	1f1dee94f6	Manually run GC on reactor tick. ... This also adds a metric for amount of time spent in GC.	2016-05-09 10:13:25 +01:00
Erik Johnston	f6ebaf4a32	Run transaction queue on reactor ... This ensures that any CPU work that happens doesn't block message sending.	2016-05-09 10:10:06 +01:00
Erik Johnston	4ea762c1a2	Add cache to get_user_by_id	2016-05-09 10:08:21 +01:00
Erik Johnston	012cb5416c	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/push_actions_delete	2016-05-06 15:59:20 +01:00
Erik Johnston	fcb2c3f0db	Remove unused import	2016-05-06 15:47:40 +01:00
Erik Johnston	fd85b167ec	Pull loop one level up	2016-05-06 15:38:42 +01:00
Erik Johnston	b6e0be701e	Queue events for persistence	2016-05-06 14:31:44 +01:00
Erik Johnston	d13459636f	Pull prev txn from in memory	2016-05-06 11:30:55 +01:00
Erik Johnston	1d275dba69	Don't needlessly enter transaction	2016-05-06 11:25:58 +01:00
Erik Johnston	56b5e83e36	Reduce database inserts when sending transactions	2016-05-06 11:20:18 +01:00
Matthew Hodgson	53ca739f1f	better mail subject lines	2016-05-05 15:55:44 +01:00
Matthew Hodgson	81c2176cba	fix layout; handle app naming in synapse, not jinja	2016-05-05 15:54:29 +01:00
Mark Haines	573ef3f1c9	Rename openid/token to openid/request_token	2016-05-05 15:15:00 +01:00
Erik Johnston	8940281d1b	Don't warn	2016-05-05 15:10:03 +01:00
Mark Haines	9c272da05f	Add an openidish mechanism for proving to third parties that you own a given user_id	2016-05-05 13:42:44 +01:00
Erik Johnston	5d8a93a10e	Add some log information at returned replication streams	2016-05-05 10:29:21 +01:00
Erik Johnston	1f0f5ffa1e	Add bulk fetch storage API	2016-05-05 10:03:15 +01:00
Matthew Hodgson	634efb65f1	pep8	2016-05-05 02:10:57 +01:00
Matthew Hodgson	ce81ccb063	handle fragments correctly on mxc URLs. ... switch to vector.im permalinks as matrix.to isn't ready yet. merge overlapping notifications together. give one message of context after a notification (in the unlikely event it exists, but it's possible thanks to throttling). include name of app in mail templates	2016-05-05 02:00:33 +01:00
Matthew Hodgson	1cf5c379cb	spell out emailpusher full path	2016-05-05 01:59:39 +01:00
Erik Johnston	fee1118a20	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/ignore_user	2016-05-04 19:08:27 +01:00
Erik Johnston	97b9141245	Merge pull request #762 from matrix-org/erikj/report_event ... Add /report endpoint	2016-05-04 19:05:49 +01:00
Erik Johnston	fcd1eb642d	Add primary key	2016-05-04 16:51:51 +01:00
Erik Johnston	8e6a163f27	Add timestamp and auto incrementing ID	2016-05-04 15:19:12 +01:00
David Baker	39d0a99972	Include no context ... until we can de-dup between the context and other notifs	2016-05-04 14:52:49 +01:00
David Baker	9ef05a12c3	Add date header & message id	2016-05-04 14:52:10 +01:00
David Baker	80be396464	Correct SQL statement for postgres ... In standard sql, join binds tighter than comma, so we were joining on the wrong table. Postgres follows the standard (apparently).	2016-05-04 13:19:59 +01:00
Erik Johnston	5650e38e7d	Move event_id to path	2016-05-04 13:19:39 +01:00
David Baker	8cc82aad87	Add db functions used for email to the pusher app	2016-05-04 11:47:59 +01:00
David Baker	de22001ab5	pep8	2016-05-04 11:41:35 +01:00
Matthew Hodgson	f1026418ea	copyright	2016-05-04 11:38:01 +01:00
Matthew Hodgson	17cbf773b9	fix assorted typos in default config	2016-05-04 11:38:01 +01:00
Erik Johnston	984d4a2c0f	Add /report endpoint	2016-05-04 11:28:10 +01:00
David Baker	e6bffa4475	Unused import	2016-05-04 11:26:58 +01:00
David Baker	92f0f3d21d	Catch all exceptions when creating a pusher	2016-05-04 11:24:07 +01:00
Erik Johnston	a438a6d2bc	Implement basic ignore user	2016-05-04 10:16:46 +01:00
Erik Johnston	7ea3b4118d	Merge pull request #757 from matrix-org/erikj/event_auth_typo ... Fix typo in event_auth servlet path	2016-05-03 14:23:15 +01:00
Erik Johnston	183f23f10d	Delete old pushers	2016-05-03 14:22:33 +01:00
Matthew Hodgson	792def4928	add a url_preview_ip_range_whitelist config param so we can whitelist the matrix.org IP space	2016-05-01 12:44:24 +01:00
David Baker	2df75de505	Merge remote-tracking branch 'origin/develop' into dbkr/email_notifs	2016-04-29 20:28:47 +01:00
David Baker	b084e4d963	Add constant for throttle multiplier	2016-04-29 20:14:55 +01:00
David Baker	35b7b8e4bc	Remove unused function	2016-04-29 20:10:34 +01:00
David Baker	6b9b6a9169	Remove unused arg	2016-04-29 20:02:52 +01:00
David Baker	c7c75e87dc	Docstring	2016-04-29 19:47:35 +01:00
David Baker	b0a1036d93	Use explicit join	2016-04-29 19:28:56 +01:00
David Baker	8f99cd5996	Oops, actually specify the user id	2016-04-29 19:27:03 +01:00
David Baker	60f44c098d	Remove unnecessary if	2016-04-29 19:17:10 +01:00
David Baker	50ad8005e4	Put spaces at start of line	2016-04-29 19:16:15 +01:00
David Baker	83618d719a	Try imports in config	2016-04-29 19:13:52 +01:00
David Baker	e7a76b5123	Use the constant	2016-04-29 19:10:45 +01:00
David Baker	29c8cf8db8	Avoid vars builtin	2016-04-29 19:09:28 +01:00
David Baker	d3da5294e8	Use named parameter format	2016-04-29 19:04:40 +01:00
David Baker	765f2b8446	Default enable email notifs to False	2016-04-29 14:46:18 +01:00
David Baker	311b5ce051	pep8	2016-04-29 14:37:30 +01:00
David Baker	3facde2536	Remove rather pointless get function	2016-04-29 14:36:45 +01:00
David Baker	4364ea1272	Stop processing notifs once we've sent a mail	2016-04-29 14:31:27 +01:00
David Baker	4b0c3a3270	Correct public_baseurl default	2016-04-29 14:30:15 +01:00
David Baker	5048455965	Nicer get() shorthand	2016-04-29 14:27:40 +01:00
David Baker	6c8957be7f	Remove redundant docstring	2016-04-29 14:25:28 +01:00
David Baker	18ce88bd2d	Correct default template and add text template	2016-04-29 14:24:25 +01:00
David Baker	40d40e470d	Send mail notifs with a plaintext part too	2016-04-29 13:56:21 +01:00
David Baker	dc2c527ce9	Fix password reset ... Default requester to None, otherwise it isn't defined when resetting using email auth	2016-04-29 12:07:54 +01:00
Erik Johnston	62b51b8452	Fix typo in event_auth servlet path	2016-04-29 12:00:51 +01:00
David Baker	b2c04da8dc	Add an email pusher for new users ... If they registered with an email address and email notifs are enabled on the HS	2016-04-29 11:43:57 +01:00
David Baker	ec9cbe847d	pep8 newline	2016-04-29 10:07:30 +01:00
David Baker	acded821c4	Merge remote-tracking branch 'origin/develop' into dbkr/email_notifs	2016-04-29 10:05:20 +01:00
David Baker	e800ee2f63	May as well always include room link	2016-04-28 17:28:27 +01:00
David Baker	cc0874cf71	Put back real delay before mailing	2016-04-28 17:00:40 +01:00
David Baker	68f8fc2f14	Support file messages & fix plain text	2016-04-28 16:59:57 +01:00
David Baker	4845c7359d	Support image notifs	2016-04-28 15:55:53 +01:00
Mark Haines	351b50a887	Fix more typos in per-request metrics	2016-04-28 15:29:46 +01:00
David Baker	60f86fc876	pep8	2016-04-28 15:16:30 +01:00
David Baker	937c407eef	Only import email pusher if email notifs are on	2016-04-28 15:12:14 +01:00
Mark Haines	dcfc10b129	Fix typo in request metrics	2016-04-28 15:11:06 +01:00
Matthew Hodgson	aebd0c9717	fix typo	2016-04-28 15:09:25 +01:00
Mark Haines	1a12766e3b	Add a comment explaining why automatic metric reporting is disabled for JsonResource	2016-04-28 12:31:26 +01:00
Mark Haines	6037349512	Check if report_metrics is True	2016-04-28 12:26:07 +01:00
David Baker	ebbabc4986	Handle room invites in email notifs	2016-04-28 11:49:36 +01:00
Mark Haines	8d7ad44331	Report per request metrics for all of the things using request_handler	2016-04-28 10:57:49 +01:00
David Baker	9dba1b668c	Linkify plain text messages too	2016-04-28 10:55:08 +01:00
David Baker	424a7f48f8	Run filter_events_for_client ... so we don't accidentally mail out events people shouldn't see	2016-04-27 17:50:49 +01:00
David Baker	4ed1e45869	Make html messages work	2016-04-27 17:18:51 +01:00
Mark Haines	21d188bf95	Merge pull request #755 from matrix-org/markjh/right_direction ... Fix backfill replication to advance the stream correctly	2016-04-27 15:54:48 +01:00
Mark Haines	8a65666454	Fix backfill replication to advance the stream correctly	2016-04-27 15:38:43 +01:00
David Baker	8781083960	Better grammar for multiple messages in a room ... Say who the messages are from if there's no room name, otherwise it's a bit nonsensical	2016-04-27 15:30:41 +01:00
David Baker	fa12209c1b	Hopefully all remaining bits for email notifs ... Add public facing base url to the server so synapse knows what URL to use when converting mxc to http urls for use in emails	2016-04-27 15:09:55 +01:00
Mark Haines	871357d539	Check that somethign has happend before running the selects	2016-04-27 11:54:13 +01:00
Mark Haines	71df327190	Actually start the pusher daemon	2016-04-26 17:07:09 +01:00
Mark Haines	c9eab73f2a	Fix typo in default pusher config	2016-04-26 17:06:18 +01:00
Mark Haines	b80b93ea0f	Add a log context to the daemonized pusher	2016-04-26 15:57:28 +01:00
Mark Haines	6df5a6a833	Optionally daemonize the pusher	2016-04-26 15:37:41 +01:00
Erik Johnston	5164ccc3e5	Bump changelog and version	2016-04-26 11:26:32 +01:00
Erik Johnston	3306cf45ca	Merge pull request #750 from matrix-org/erikj/jwt_optional ... Make pyjwt dependency optional	2016-04-26 11:07:22 +01:00
Mark Haines	9c417c54d4	Add a couple of update methods to the PusherSlaveStore	2016-04-26 10:45:02 +01:00
David Baker	7b4715bad7	More variable calculation for email notifs ... Include name of the person we're sending to and add summary text at the top giving an overview of what's happened.	2016-04-25 18:27:04 +01:00
Mark Haines	f15e9e8de4	Remove the uncomments from the comments	2016-04-25 17:56:24 +01:00
Mark Haines	72e2fafa20	Add a metrics listener and a ssh listener to the pusher	2016-04-25 17:34:25 +01:00
Mark Haines	f22f46f4f9	Move the listenTCP call outside the manhole function	2016-04-25 14:59:21 +01:00
David Baker	290f125a13	Typo	2016-04-25 14:42:59 +01:00
Erik Johnston	52ecbc2843	Make pyjwt dependency optional	2016-04-25 14:30:15 +01:00
David Baker	05e49ffbdf	No we don't: it's just the display name	2016-04-22 18:44:17 +01:00
David Baker	bd0f9c2065	Actually do UTF8 correctly	2016-04-22 18:42:00 +01:00
David Baker	c5b3c6e101	Sort member events ... So names of people in a room are given in order	2016-04-22 18:33:36 +01:00
David Baker	83bf65297a	Mime part is binary so encode it first. ... Doesn't get character enocind right yet but makes it not error.	2016-04-22 18:31:47 +01:00
David Baker	e8701e64b9	Implement group-of-people names	2016-04-22 17:28:42 +01:00
David Baker	c553797c4f	No inlineCallbacks necessary on this	2016-04-22 17:27:54 +01:00
Mark Haines	5905f36f05	Split out setting up the manhole to a separate file	2016-04-22 17:09:15 +01:00
Mark Haines	62607d5452	Merge branch 'develop' into markjh/split_out_site.py ... Conflicts: synapse/app/homeserver.py	2016-04-22 16:26:57 +01:00
Mark Haines	e856036f4c	Move SynapseSite to its own file	2016-04-22 16:09:55 +01:00
Mark Haines	9e7aa98c22	Split out create_resource_tree to a separate file	2016-04-22 15:40:51 +01:00
Mark Haines	2022ae0fb9	Merge pull request #746 from matrix-org/markjh/split_out_pusher ... Optionally split out the pushers into a separate process	2016-04-22 11:34:08 +01:00
Erik Johnston	64ec3493c1	Merge pull request #745 from matrix-org/erikj/search-index ... Optimise event_search in postgres	2016-04-22 11:23:49 +01:00
Erik Johnston	183cacac90	Simplify query and handle finishing correctly	2016-04-22 10:01:57 +01:00
David Baker	c10ed26c30	Flesh out email templating ... Mostly WIP porting the room name calculation logic from the web client so our room names in the email mirror the clients.	2016-04-21 19:19:07 +01:00
Erik Johnston	ae571810f2	Order NULLs first	2016-04-21 18:14:18 +01:00
Erik Johnston	3ddbb1687c	Fix query	2016-04-21 18:02:36 +01:00
Erik Johnston	8fae3d7b1e	Use special UPDATE syntax	2016-04-21 18:01:49 +01:00
Erik Johnston	b57dcb4b51	Typo	2016-04-21 17:49:00 +01:00
Erik Johnston	26db18bc90	Need to do _background_update_progress_txn in actual transaction	2016-04-21 17:45:56 +01:00
Erik Johnston	b9675ef6e6	Merge pull request #687 from nikriek/jwt-fix ... Fix issues with JWT login	2016-04-21 17:42:25 +01:00
Erik Johnston	e395eb1108	Update progress when creating index	2016-04-21 17:39:24 +01:00
Erik Johnston	3b0fa77f50	Fix SQL statement	2016-04-21 17:37:42 +01:00
Erik Johnston	129e403487	Create index must be on a conn	2016-04-21 17:35:51 +01:00
Mark Haines	a3ac837599	Optionally split out the pushers into a separate process	2016-04-21 17:22:37 +01:00
Mark Haines	78741cf025	Merge pull request #743 from matrix-org/markjh/slave_pushers ... Replicate the pushers	2016-04-21 17:21:29 +01:00
Erik Johnston	51bb339ab2	Create index concurrently	2016-04-21 17:16:11 +01:00
Erik Johnston	b743c1237e	Add missing run_upgrade	2016-04-21 17:12:04 +01:00
Mark Haines	31719ad124	Merge pull request #744 from matrix-org/markjh/replication_remove_pusher ... Add a replication endpoint for deleting pushers	2016-04-21 17:10:49 +01:00
Niklas Riekenbrauck	565c2edb0a	Fix issues with JWT login	2016-04-21 18:10:48 +02:00
Erik Johnston	c877f0f034	Optimise event_search in postgres	2016-04-21 16:56:14 +01:00
Mark Haines	cfe1ff4bdb	Add a replication endpoint for deleting pushers	2016-04-21 16:33:05 +01:00
Mark Haines	d4823efad9	Replicate the pushers	2016-04-21 16:18:00 +01:00
Mark Haines	c0d8e0eb63	Replicate push actions	2016-04-21 15:25:58 +01:00
David Baker	2ed0adb075	Generate mails from a template	2016-04-20 18:35:29 +01:00
David Baker	05adc6c2de	more pep8	2016-04-20 13:02:45 +01:00
David Baker	f63bd4ff47	Send a rather basic email notif ... Also pep8 fixes	2016-04-20 13:02:01 +01:00
Erik Johnston	5bbc321588	Always use state cache entry if it exists ... Also check if the resolved state matches an existing state group.	2016-04-20 11:49:10 +01:00
Erik Johnston	4cf4320593	Add some logging to state resolve_events	2016-04-20 11:06:02 +01:00
Erik Johnston	eab47ea1e5	Merge pull request #739 from matrix-org/erikj/cache_get_state_groups_for_groups ... Add cache to _get_state_groups_from_groups	2016-04-19 17:37:19 +01:00
Mark Haines	f52dd35ac3	Merge pull request #738 from matrix-org/markjh/slaved_receipts ... Add a slaved receipts store	2016-04-19 17:31:59 +01:00
Erik Johnston	61c7edfd34	Add cache to _get_state_groups_from_groups	2016-04-19 17:22:03 +01:00
Mark Haines	5bbd424ee0	Add a slaved receipts store	2016-04-19 17:14:08 +01:00
Erik Johnston	6ac40f7b65	Merge pull request #737 from matrix-org/erikj/spider_ssl_factory ... Use tls_server_context_factory for SpiderEndpoint	2016-04-19 16:22:05 +01:00
Erik Johnston	f505575f69	Make InsecureInterceptableContextFactory work with SpiderEndpoint	2016-04-19 16:08:14 +01:00
Mark Haines	e99365f601	Replicate get_invited_rooms_for_user	2016-04-19 15:22:14 +01:00
David Baker	e2a01455af	Add single instance & logging stuff ... Copy the stuff over from http pusher that prevents multiple instances of process running at once and sets up logging and measure blocks.	2016-04-19 14:52:58 +01:00
Erik Johnston	e8884e5e9c	Add self.media_repo to PreviewUrlResource	2016-04-19 14:51:34 +01:00
Erik Johnston	a7001c311b	_make_dirs was moved to MediaRepository	2016-04-19 14:49:31 +01:00
Erik Johnston	9181e2f4c7	Add store to PreviewUrlResource	2016-04-19 14:48:24 +01:00
Erik Johnston	fb76a81ff7	Reorder imports	2016-04-19 14:45:05 +01:00
David Baker	07d765209d	First bits of emailpusher ... Mostly logic of when to send an email	2016-04-19 14:24:36 +01:00
Erik Johnston	0c93df89b6	Move MediaRepository to media_repository module	2016-04-19 11:31:43 +01:00
Erik Johnston	43f0941e8f	Split out BaseMediaResource into MediaRepository ... This is so that a single MediaRepository can be shared across all resources, rather than having a "copy" per resource. In particular this allows us to guard against both the thumbnail and download resource triggering a download of remote content at the same time.	2016-04-19 11:24:59 +01:00
Erik Johnston	eb8619e256	Create log context in Measure if one doesn't exist	2016-04-18 16:08:32 +01:00
Erik Johnston	4ef7a25c10	Merge pull request #733 from matrix-org/erikj/make_member_timeout ... Lower timeout for make_membership_event	2016-04-18 15:08:05 +01:00
Erik Johnston	3727a15764	Merge pull request #732 from matrix-org/erikj/login ... Simplify _check_password	2016-04-18 15:07:57 +01:00
Matthew Hodgson	aaabbd3e9e	explicitly pass in the charset from Content-Type to lxml to fix cyrillic woes better	2016-04-15 14:32:25 +01:00
Matthew Hodgson	84f9cac4d0	fix cyrillic URL previews by hardcoding all page decoding to UTF-8 for now, rather than relying on lxml's heuristics which seem to get it wrong	2016-04-15 13:20:08 +01:00
Erik Johnston	914f1eafac	Lower timeout for make_membership_event ... Calls to make_membership_event are done in response to client requests, and so should not be retried over long timeframes.	2016-04-15 11:22:23 +01:00
Erik Johnston	6fd2f685fe	Simplify _check_password	2016-04-15 11:17:18 +01:00
Erik Johnston	cb9c465707	Use SynapseError 504 for Timeout errors	2016-04-15 10:21:32 +01:00
Mark Haines	3c79bdd7a0	Fix check_password rather than inverting the meaning of _check_local_password (#730)	2016-04-14 19:00:21 +01:00
David Baker	a4c56bf67b	Merge pull request #729 from matrix-org/dbkr/fix_login_nonexistent_user ... Fix login to error for nonexistent users	2016-04-14 18:46:45 +01:00
David Baker	4c1b32d7e2	Fix login to error for nonexistent users ... Fixes SYN-680	2016-04-14 18:28:42 +01:00
Matthew Hodgson	f78b479118	fix urlparse import thinko breaking tiny URLs	2016-04-14 15:23:55 +01:00
Kegan Dougal	83776d6219	Make v2_alpha reg follow the AS API specification ... The spec is clear the key should be 'user' not 'username' and this is indeed the case for v1. This is not true for v2_alpha though, which is what this commit is fixing.	2016-04-14 14:52:26 +01:00
Matthew Hodgson	bd77216d06	comment out 2c838f6459 due to risk of https://en.wikipedia.org/wiki/Billion_laughs attacks - thanks @torhve	2016-04-14 14:39:24 +01:00
Erik Johnston	9ae64c9910	Measure push action generator	2016-04-14 13:42:22 +01:00
Erik Johnston	b42ad359e9	Merge pull request #725 from matrix-org/dbkr/push_only_joined ... Don't push for everyone who ever sent an RR to the room	2016-04-14 12:05:13 +01:00
David Baker	757e2c79b4	Don't push for everyone who ever sent an RR to the room	2016-04-14 12:02:50 +01:00
Erik Johnston	86e9bbc74e	Add missing yield	2016-04-14 11:56:52 +01:00
Erik Johnston	e40f25ebe1	Rename log context	2016-04-14 11:54:14 +01:00
Erik Johnston	2ae91a9e2f	Make send_badge private	2016-04-14 11:37:50 +01:00
Erik Johnston	d213d69fe3	Add desc arg	2016-04-14 11:36:23 +01:00
Erik Johnston	56da835eaf	Add necessary logging contexts	2016-04-14 11:33:50 +01:00
Erik Johnston	96bcfb29c7	Add index	2016-04-14 11:26:33 +01:00
Erik Johnston	7be1065b8f	Add extra Measure	2016-04-14 11:26:15 +01:00
Erik Johnston	a2546b9082	Fix query for get_unread_push_actions_for_user_in_range	2016-04-14 11:08:31 +01:00
Erik Johnston	ceeb5b909f	Merge pull request #721 from matrix-org/erikj/spider ... Sanitize the optional dependencies for spider API	2016-04-14 09:59:29 +01:00
David Baker	43a89cca8e	Merge pull request #722 from matrix-org/dbkr/only_unread_event_actions ... Only return unread notifications	2016-04-13 14:54:26 +01:00
Erik Johnston	f338bf9257	Give install requirements	2016-04-13 14:33:48 +01:00
David Baker	767fc0b739	pep8	2016-04-13 14:23:27 +01:00
David Baker	54d08c8868	Only return unread notifications ... Make get_unread_push_actions_for_user_in_range only return unread event actions, being more true to its name. Done in two separate sql queries to get actions after a read receipt and those in a room wiht no receipt at all. SQL queries by Erik.	2016-04-13 14:16:45 +01:00
Erik Johnston	5880bc5417	Merge pull request #718 from matrix-org/erikj/public_room_list ... Don't return empty public rooms	2016-04-13 14:07:26 +01:00
Erik Johnston	f613a3e332	Merge pull request #720 from matrix-org/erikj/auth_chec ... Don't auto log failed auth checks	2016-04-13 14:07:23 +01:00
Erik Johnston	bfe586843f	Add back in helpful description for missing url_preview_ip_range_blacklist	2016-04-13 13:52:57 +01:00
Erik Johnston	d0633e6dbe	Sanitize the optional dependencies for spider API	2016-04-13 13:38:09 +01:00
Erik Johnston	0f2ca8cde1	Measure Auth.check	2016-04-13 11:15:59 +01:00
Erik Johnston	c53f9d561e	Don't auto log failed auth checks	2016-04-13 11:11:46 +01:00
David Baker	65141161f6	Unused member variable	2016-04-12 16:25:26 +01:00
Erik Johnston	72f454b752	Don't return empty public rooms	2016-04-12 16:06:18 +01:00
David Baker	d33d623f0d	Merge pull request #716 from matrix-org/dbkr/get_pushers ... Add get endpoint for pushers	2016-04-12 14:40:37 +01:00
David Baker	7984ffdc6a	Unneccessarywhitespaceisunnecessary	2016-04-12 13:55:57 +01:00
David Baker	c1267d04c5	Oops, forgot the desc.	2016-04-12 13:55:32 +01:00
David Baker	a04c076b7f	Make the /set part mandatory	2016-04-12 13:54:41 +01:00
David Baker	44891b4a0a	Tidy up get_pusher functions ... Decodes pushers rows on the main thread rather than the db thread and uses _simple_select_list. Also do the same to the function I copied and factor out the duplication into a helper function.	2016-04-12 13:47:17 +01:00
David Baker	7b39bcdaae	Mis-named function	2016-04-12 13:35:08 +01:00
David Baker	d937f342bb	Split into separate servlet classes	2016-04-12 13:33:30 +01:00
Erik Johnston	c48465dbaa	More comments	2016-04-12 12:48:30 +01:00
Erik Johnston	8be1a37909	More comments	2016-04-12 12:04:19 +01:00
Erik Johnston	d3d0be4167	Don't append to unused list	2016-04-12 11:59:00 +01:00
Erik Johnston	762ada1e07	Add back backfilled parameter that was removed	2016-04-12 11:58:04 +01:00
Erik Johnston	0d3da210f0	Add comment	2016-04-12 11:54:41 +01:00
Erik Johnston	cccf86dd05	Check if we've already backfilled events	2016-04-12 11:19:32 +01:00
David Baker	8a76094965	Add get endpoint for pushers ... As per https://github.com/matrix-org/matrix-doc/pull/308	2016-04-11 18:00:03 +01:00
Mark Haines	790f5848b2	Fix the rule_id for .m.rule.invite_for_me (#715)	2016-04-11 16:10:39 +01:00
Mark Haines	82d7eea7e3	Move the versionstring code out of app.homeserver into util	2016-04-11 14:57:09 +01:00
David Baker	2547dffccc	Merge pull request #705 from matrix-org/dbkr/pushers_use_event_actions ... Change pushers to use the event_actions table	2016-04-11 12:58:55 +01:00
David Baker	9bb041791c	Run unsafe proces in a loop until we've caught up ... and wrap unsafe process in a try block	2016-04-11 12:48:30 +01:00
Erik Johnston	17515bae14	PEP8	2016-04-11 11:02:50 +01:00
Matthew Hodgson	4bd3d25218	Merge pull request #688 from matrix-org/matthew/preview_urls ... URL previewing support	2016-04-11 10:40:29 +01:00
Matthew Hodgson	5ffacc5e84	fix typos and needless try/except from PR review	2016-04-11 10:39:16 +01:00
Matthew Hodgson	83b2f83da0	actually throw meaningful errors	2016-04-08 21:36:59 +01:00
Mark Haines	b36270b5e1	Fix pep8 warning	2016-04-08 19:52:23 +01:00
Matthew Hodgson	6ff7a79308	move local_media_repository_url_cache.sql to schema v31	2016-04-08 19:09:02 +01:00
Matthew Hodgson	af582b66bb	fix typo	2016-04-08 19:08:47 +01:00
Matthew Hodgson	2460d904bd	fix error checking for new SQL	2016-04-08 19:04:29 +01:00
Matthew Hodgson	1ccabe2965	more PR feedback	2016-04-08 18:58:08 +01:00
Matthew Hodgson	fb83f6a1fc	fix SQL based on PR feedback	2016-04-08 18:55:38 +01:00
Matthew Hodgson	dafef5a688	Add url_preview_enabled config option to turn on/off preview_url endpoint. defaults to off. ... Add url_preview_ip_range_blacklist to let admins specify internal IP ranges that must not be spidered. Add url_preview_url_blacklist to let admins specify URL patterns that must not be spidered. Implement a custom SpiderEndpoint and associated support classes to implement url_preview_ip_range_blacklist Add commentary and generally address PR feedback	2016-04-08 18:37:15 +01:00
David Baker	d96a070a3a	Actually check if we;re processing	2016-04-08 16:49:39 +01:00
David Baker	ed3979df5f	Fix invite pushes ... * If the event is an invite event, add the invitee to list of user we run push rules for (if they have a pusher etc) * Move invite_for_me to be higher prio than member events otherwise member events matches them * Spell override right	2016-04-08 15:29:59 +01:00
Erik Johnston	79fc4ff6f9	Merge pull request #677 from matrix-org/erikj/dns_cache ... Read from DNS cache if within TTL	2016-04-08 14:09:56 +01:00
David Baker	7b6d519482	Make sure max stream ordering only increases	2016-04-08 14:08:16 +01:00