6943 Commits

Author	SHA1	Message	Date
Mark Haines	e6e2627636	Fix code for reporting old verify keys in synapse	2017-04-24 18:51:25 +01:00
Richard van der Hoff	30f7bfa121	Merge pull request #2145 from matrix-org/rav/reject_invite_to_unreachable_server ... Fix rejection of invites to unreachable servers	2017-04-24 15:20:52 +01:00
Erik Johnston	7af825bae4	Merge pull request #2155 from matrix-org/erikj/string_intern ... Only intern ascii strings	2017-04-24 14:28:41 +01:00
Erik Johnston	d134d0935e	Only intern ascii strings	2017-04-24 14:07:48 +01:00
Erik Johnston	e4f3431116	Remove unused cache	2017-04-24 13:27:38 +01:00
David Baker	a46982cee9	Need the HTTP status code	2017-04-21 16:20:12 +01:00
David Baker	70caf49914	Do the same for get_json	2017-04-21 16:09:03 +01:00
David Baker	a1595cec78	Don't error for 3xx responses	2017-04-21 11:51:17 +01:00
David Baker	2e165295b7	Merge remote-tracking branch 'origin/develop' into dbkr/http_request_propagate_error	2017-04-21 11:35:52 +01:00
David Baker	a90a0f5c8a	Propagate errors sensibly from proxied IS requests ... When we're proxying Matrix endpoints, parse out Matrix error responses and turn them into SynapseErrors so they can be propagated sensibly upstream.	2017-04-21 11:32:48 +01:00
Richard van der Hoff	91b3981800	Try harder when sending leave events ... When we're rejecting invites, ignore the backoff data, so that we have a better chance of not getting the room out of sync.	2017-04-21 01:50:36 +01:00
Richard van der Hoff	0cdb32fc43	Remove redundant try/except clauses ... The `except SynapseError` clauses were pointless because the wrapped functions would never throw a `SynapseError` (they either throw a `CodeMessageException` or a `RuntimeError`). The `except CodeMessageException` is now also pointless because the caller treats all exceptions equally, so we may as well just throw the `CodeMessageException`.	2017-04-21 01:32:01 +01:00
Richard van der Hoff	838810b76a	Broaden the conditions for locally_rejecting invites ... The logic for marking invites as locally rejected was all well and good, but didn't happen when the remote server returned a 500, or wasn't reachable, or had no DNS, or whatever. Just expand the except clause to catch everything. Fixes https://github.com/matrix-org/synapse/issues/761.	2017-04-21 01:31:37 +01:00
Richard van der Hoff	736b9a4784	Remove redundant function ... inline `reject_remote_invite`, which only existed to make tracing the callflow more difficult.	2017-04-21 01:31:09 +01:00
Richard van der Hoff	4903ccf159	Fix some lies, and other clarifications, in docstrings ... The documentation on get_json has been wrong ever since the very first commit to synapse...	2017-04-21 01:31:09 +01:00
Matthew Wolff	d4040e9e28	Queried CONDITIONAL_REQUIREMENTS	2017-04-18 16:19:48 -05:00
Luke Barnard	3fb8784c92	m.read_marker -> m.fully_read (#2128) ... Also: - change the REST endpoint to have a "S" on the end (so it's now /read_markers) - change the content of the m.read_up_to event to have the key "event_id" instead of "marker".	2017-04-18 17:46:15 +01:00
Matthew Wolff	54f9a4cb59	Fixed travis build failure ... Signed-off-by: Matthew Wolff <matthewjwolff@gmail.com>	2017-04-17 01:38:27 -05:00
Matthew Wolff	8e780b113d	web_server_root documentation fix ... Signed-off-by: Matthew Wolff <matthewjwolff@gmail.com>	2017-04-17 00:49:11 -05:00
Anant Prakash	574d573ac2	Check that requested room_id exists	2017-04-14 23:50:59 +05:30
Luke Barnard	78f0ddbfad	Merge pull request #2120 from matrix-org/luke/read-markers ... Implement Read Marker API	2017-04-13 14:21:31 +01:00
Luke Barnard	6a70647d45	Correct logic in is_event_after	2017-04-13 13:46:17 +01:00
Anant Prakash	c1f52a321d	synctl.py: Check if synapse is already running	2017-04-13 18:00:02 +05:30
Luke Barnard	b9557064bf	Simplify is_event_after logic	2017-04-12 14:36:20 +01:00
Luke Barnard	cf6121e3da	More null-guard changes	2017-04-12 14:02:03 +01:00
Erik Johnston	247c736b9b	Merge pull request #2115 from matrix-org/erikj/dedupe_federation_repl ... Reduce federation replication traffic	2017-04-12 11:07:13 +01:00
Paul Evans	8fbc0d29ee	Merge pull request #2121 from matrix-org/paul/sent-transactions-metric ... Add a counter metric for successfully-sent transactions	2017-04-12 11:04:31 +01:00
Erik Johnston	c06c00190f	Merge pull request #2116 from matrix-org/erikj/dedupe_federation_repl2 ... Dedupe KeyedEdu and Devices federation repl traffic	2017-04-12 10:57:24 +01:00
Luke Barnard	c0aba0a23e	Remove Unused ref to hs	2017-04-12 10:52:11 +01:00
Luke Barnard	b9676a75f6	Move a space	2017-04-12 10:51:17 +01:00
Luke Barnard	69a18514e9	Only notify user, not entire room	2017-04-12 10:50:37 +01:00
Luke Barnard	122cd52ce4	Remove comment, simplify null-guard	2017-04-12 10:48:32 +01:00
Erik Johnston	26ae5178a4	Add some comments	2017-04-12 10:36:29 +01:00
Erik Johnston	bf9060156a	Merge pull request #2117 from matrix-org/erikj/remove_http_replication ... Remove HTTP replication APIs	2017-04-12 10:21:42 +01:00
Erik Johnston	1745069543	Comment	2017-04-12 10:17:10 +01:00
Erik Johnston	c7ddb5ef7a	Reuse get_interested_parties	2017-04-12 10:16:26 +01:00
Erik Johnston	7b41013102	Merge pull request #2118 from matrix-org/erikj/no_devices ... Fix getting latest device IP for user with no devices	2017-04-12 10:14:32 +01:00
Luke Barnard	77fb2b72ae	Handle no previous RM	2017-04-12 09:47:29 +01:00
Luke Barnard	7f94709066	travis flake8..	2017-04-11 18:35:45 +01:00
Luke Barnard	867822fa1e	flake8	2017-04-11 17:36:04 +01:00
Luke Barnard	73880268ef	Refactor event ordering check to events store	2017-04-11 17:34:09 +01:00
Luke Barnard	131485ef66	Copyright	2017-04-11 17:33:51 +01:00
Paul "LeoNerd" Evans	11dbceb761	Add a counter metric for successfully-sent transactions	2017-04-11 17:16:12 +01:00
Luke Barnard	0127423027	flake8	2017-04-11 17:07:07 +01:00
Erik Johnston	85657eedf8	Bail on where clause instead	2017-04-11 16:24:31 +01:00
Erik Johnston	b48045a8f5	Don't bother with outer check for now	2017-04-11 16:23:24 +01:00
Erik Johnston	9c712a366f	Move get_presence_list_* to SlaveStore	2017-04-11 16:07:33 +01:00
Erik Johnston	a8c8e4efd4	Comment	2017-04-11 15:35:49 +01:00
Erik Johnston	414522aed5	Move get_interested_parties	2017-04-11 15:33:26 +01:00
Erik Johnston	2be8a281d2	Comments	2017-04-11 15:28:24 +01:00
Erik Johnston	6308ac45b0	Move get_interested_remotes back to presence handler	2017-04-11 15:19:26 +01:00
Erik Johnston	b9b72bc6e2	Comments	2017-04-11 15:15:34 +01:00
Luke Barnard	d892079844	Finish implementing RM endpoint ... - This change causes a 405 to be sent if "m.read_marker" is set via /account_data - This also fixes-up the RM endpoint so that it actually Works.	2017-04-11 15:01:39 +01:00
lukebarnard	e263c26690	Initial commit of RM server-side impl ... (See https://docs.google.com/document/d/1UWqdS-e1sdwkLDUY0wA4gZyIkRp-ekjsLZ8k6g_Zvso/edit#heading=h.lndohpg8at5u)	2017-04-11 11:55:30 +01:00
Erik Johnston	f3cf3ff8b6	Merge branch 'master' of github.com:matrix-org/synapse into develop	2017-04-11 11:13:32 +01:00
Erik Johnston	34840cdcef	Fix getting latest device IP for user with no devices	2017-04-11 09:56:54 +01:00
Erik Johnston	28a4649785	Remove HTTP replication APIs	2017-04-11 09:52:11 +01:00
Matthew Hodgson	7c551ec445	trust a hypothetical future riot.im IS	2017-04-10 17:58:36 +01:00
Erik Johnston	84fbb80c8f	Use generators	2017-04-10 16:55:56 +01:00
Erik Johnston	40453b3f84	Dedupe KeyedEdu and Devices federation repl traffic	2017-04-10 16:49:51 +01:00
Erik Johnston	29574fd5b3	Reduce federation presence replication traffic ... This is mainly done by moving the calculation of where to send presence updates from the presence handler to the transaction queue, so we only need to send the presence event (and not the destinations) across the replication connection. Before we were duplicating by sending the full state across once per destination.	2017-04-10 16:48:30 +01:00
Erik Johnston	2e6f5a4910	Typo	2017-04-10 16:17:40 +01:00
David Baker	405ba4178a	Merge pull request #2102 from DanielDent/add-auth-email ... Support authenticated SMTP	2017-04-10 15:42:16 +01:00
Erik Johnston	efcb6db688	Merge pull request #2109 from matrix-org/erikj/send_queue_fix ... Fix up federation SendQueue and document types	2017-04-10 13:09:25 +01:00
Erik Johnston	0018491af2	Rename variable	2017-04-10 12:44:43 +01:00
Erik Johnston	0364d23210	Up replication ping timeout	2017-04-10 11:32:05 +01:00
Erik Johnston	8c5f03cec7	Revert to sending the same data type as before	2017-04-10 10:07:18 +01:00
Erik Johnston	f8434db549	Change name	2017-04-10 10:03:07 +01:00
Erik Johnston	ab904caf33	Comments	2017-04-10 10:02:17 +01:00
Richard van der Hoff	64765e5199	When we do an invite rejection, save the signed leave event to the db ... During a rejection of an invite received over federation, we ask a remote server to make us a `leave` event, then sign it, then send that with `send_leave`. We were saving the *unsigned* version of the event (which has a different event id to the signed version) to our db (and sending it to the clients), whereas other servers in the room will have seen the *signed* version. We're not aware of any actual problems that caused, except that it makes the database confusing to look at and generally leaves the room in a weird state.	2017-04-07 14:39:32 +01:00
Erik Johnston	2a3e822f44	Comment	2017-04-07 13:47:04 +01:00
Erik Johnston	a828a64b75	Comment	2017-04-07 11:54:03 +01:00
Erik Johnston	d4d176e5d0	Add logging	2017-04-07 11:51:28 +01:00
Erik Johnston	449d1297ca	Fix up federation SendQueue and document types	2017-04-07 11:48:33 +01:00
Erik Johnston	d72667fcce	Speed up get_current_state_ids ... Using _simple_select_list is fairly expensive for functions that return a lot of rows and/or get called a lot. (This is because it carefully constructs a list of dicts). get_current_state_ids gets called a lot on startup and e.g. when the IRC bridge decided to send tonnes of joins/leaves (as it invalidates the cache). We therefore replace it with a custon txn function that builds up the final result dict without building up and intermediate representation.	2017-04-07 10:10:49 +01:00
Erik Johnston	a41fe500d6	Bump version and changelog	2017-04-07 10:03:48 +01:00
Erik Johnston	54f59bd7d4	Merge pull request #2107 from HarHarLinks/patch-1 ... fix typo in synctl help	2017-04-07 09:54:37 +01:00
Erik Johnston	98ce212093	Merge pull request #2103 from matrix-org/erikj/no-double-encode ... Don't double encode replication data	2017-04-07 09:39:52 +01:00
Kim Brose	8a1137ceab	fix typo in synctl help	2017-04-06 17:10:20 +02:00
Erik Johnston	877c029c16	Use iteritems	2017-04-06 15:51:22 +01:00
Erik Johnston	944692ef69	Merge pull request #2106 from matrix-org/erikj/reduce_user_sync ... Reduce rate of USER_SYNC repl commands	2017-04-06 13:35:31 +01:00
Erik Johnston	391712a4f9	Comment	2017-04-06 13:35:00 +01:00
Erik Johnston	ad544c803a	Document types of the replication streams	2017-04-06 13:28:52 +01:00
Erik Johnston	dbf87282d3	Docs	2017-04-06 13:11:21 +01:00
Erik Johnston	69b3fd485d	Fix incorrect type when using InvalidateCacheCommand	2017-04-06 09:36:38 +01:00
Daniel Dent	5058292537	Support authenticated SMTP ... Closes (SYN-714) #1385 Signed-off-by: Daniel Dent <matrixcontrib@contactdaniel.net>	2017-04-05 21:01:08 -07:00
Erik Johnston	fcc803b2bf	Add log lines	2017-04-05 17:13:44 +01:00
Erik Johnston	3f213d908d	Rearrange metrics	2017-04-05 14:15:09 +01:00
Erik Johnston	1ca0e78ca1	Fix typo	2017-04-05 13:43:39 +01:00
Erik Johnston	b43d3267e2	Fixup some metrics for tcp repl	2017-04-05 13:34:54 +01:00
Erik Johnston	b5cb6347a4	Don't immediately notify the master about users whose syncs have gone away	2017-04-05 13:25:40 +01:00
Erik Johnston	96b9b6c127	Don't double json encode typing replication data	2017-04-05 11:34:20 +01:00
Erik Johnston	f10ce8944b	Don't double json encode federation replication data	2017-04-05 11:10:28 +01:00
Erik Johnston	a5c401bd12	Merge pull request #2097 from matrix-org/erikj/repl_tcp_client ... Move to using TCP replication	2017-04-05 09:36:21 +01:00
Erik Johnston	d1d5362267	Add comment	2017-04-04 16:41:03 +01:00
Erik Johnston	9f26d3b75b	Deduplicate new deviceinbox rows for replication	2017-04-04 16:21:21 +01:00
Erik Johnston	a76886726b	Merge pull request #2098 from matrix-org/erikj/repl_tcp_fix ... Advance replication streams even if nothing is listening	2017-04-04 15:40:51 +01:00
Erik Johnston	ac66e11f2b	Add the appropriate amount of preserve_fn	2017-04-04 15:22:54 +01:00
Erik Johnston	4264ceb31c	Fiddle tcp replication logging	2017-04-04 14:14:03 +01:00
Erik Johnston	023ee197be	Advance replication streams even if nothing is listening ... Otherwise the streams don't advance and steadily fall behind, so when a worker does connect either a) they'll be streamed lots of old updates or b) the connection will fail as the streams are too far behind.	2017-04-04 13:19:26 +01:00
Erik Johnston	d1605794ad	Remove unused worker config option	2017-04-04 11:17:00 +01:00
Erik Johnston	3376f16012	Shuffle and comment synchrotron presence	2017-04-04 11:14:16 +01:00
Erik Johnston	6ce6bbedcb	Move where we ack federation	2017-04-04 11:02:44 +01:00
Erik Johnston	62b89daac6	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/repl_tcp_server	2017-04-04 09:46:16 +01:00
Richard van der Hoff	773e64cc1a	Merge pull request #2095 from matrix-org/rav/cull_log_preserves ... Cull spurious PreserveLoggingContexts	2017-04-03 17:02:25 +01:00
Richard van der Hoff	2d05eb3cf5	Merge remote-tracking branch 'origin/release-v0.20.0' into develop	2017-04-03 16:23:23 +01:00
Richard van der Hoff	30bcbf775a	Accept join events from all servers ... Make sure that we accept join events from any server, rather than just the origin server, to make the federation join dance work correctly. (Fixes #1893).	2017-04-03 15:58:07 +01:00
Richard van der Hoff	7eb9f34cc3	Remove spurious yield ... In `MessageHandler`, remove `yield` on call to `Notifier.on_new_room_event`: it doesn't return anything anyway.	2017-04-03 15:44:19 +01:00
Richard van der Hoff	0b08c48fc5	Remove more spurious PreserveLoggingContexts ... Remove `PreserveLoggingContext` around calls to `Notifier.on_new_room_event`; there is no problem if the logcontext is set when calling it.	2017-04-03 15:43:37 +01:00
Richard van der Hoff	65e1683680	Remove spurious PreserveLoggingContext ... In `on_new_room_event`, remove `PreserveLoggingContext` - we can call its subroutines with the logcontext set.	2017-04-03 15:42:38 +01:00
Richard van der Hoff	feb496056e	preserve_fn some deferred-returning things ... In `Notifier._on_new_room_event`, `preserve_fn` around its subroutines which return deferreds, so that it is safe to call it with an active logcontext.	2017-04-03 15:41:17 +01:00
Richard van der Hoff	e2eebf1696	Fix fixme in preserve_fn ... `preserve_fn` is no longer used as a decorator anywhere, so we can safely fix a fixme therein.	2017-04-03 15:38:02 +01:00
Erik Johnston	36c28bc467	Update all the workers and master to use TCP replication	2017-04-03 15:35:52 +01:00
Erik Johnston	3a1f3f8388	Change slave storage to use new replication interface ... As the TCP replication uses a slightly different API and streams than the HTTP replication. This breaks HTTP replication.	2017-04-03 15:34:19 +01:00
Erik Johnston	52bfa604e1	Add basic replication client handler and factory	2017-04-03 15:34:13 +01:00
Erik Johnston	0a6a966e2b	Always advance stream tokens	2017-04-03 15:22:56 +01:00
Richard van der Hoff	773e1c6d68	Remove spurious @preserve_fn decorators ... Remove `@preserve_fn` decorators on `on_new_room_event`, `_notify_pending_new_room_events`, `_on_new_room_event`, `on_new_event`, and `on_new_replication_data` - none of these functions return a deferred, and the decorator does nothing unless the wrapped function returns a deferred, so the decorator was a no-op.	2017-04-03 15:14:11 +01:00
Erik Johnston	0d1c85e643	Merge branch 'release-v0.20.0' of github.com:matrix-org/synapse into develop	2017-04-03 14:58:14 +01:00
Erik Johnston	1df7c28661	Use callbacks to notify tcp replication rather than deferreds	2017-03-31 15:42:51 +01:00
Erik Johnston	36d2b66f90	Add a timestamp to USER_SYNC command ... This timestamp is used to indicate when the user last sync'd	2017-03-31 15:42:22 +01:00
Erik Johnston	8a240e4f9c	Merge pull request #2078 from APwhitehat/assertuserfriendly ... add user friendly report of assertion error in synctl.py	2017-03-31 14:41:49 +01:00
Erik Johnston	142b6b4abf	Merge pull request #2011 from matrix-org/matthew/turn_allow_guests ... add setting (on by default) to support TURN for guests	2017-03-31 14:37:09 +01:00
Erik Johnston	2a06b44be2	Merge pull request #1986 from matrix-org/matthew/enable_guest_3p ... enable guest access for the 3pl/3pid APIs	2017-03-31 14:36:03 +01:00
Matthew Hodgson	9ee397b440	switch to allow_guest=True for authing 3Ps as per PR feedback	2017-03-31 13:54:26 +01:00
Erik Johnston	9d0170ac6c	Fix up presence	2017-03-31 11:36:32 +01:00
Erik Johnston	bfcf016714	Fix up docs	2017-03-31 11:19:24 +01:00
Erik Johnston	9cee0ce7db	Merge pull request #2075 from matrix-org/erikj/cache_speed ... Speed up cached function access	2017-03-31 10:10:56 +01:00
Erik Johnston	350333a09a	Merge pull request #2076 from matrix-org/erikj/as_perf ... Make AS's faster	2017-03-31 09:43:10 +01:00
Erik Johnston	4d17add8de	Remove unused instance variable	2017-03-31 09:38:27 +01:00
Erik Johnston	27b1b4a2c9	Speed up copy_and_replace	2017-03-30 17:50:31 +01:00
Erik Johnston	5b5b171f3e	Docs	2017-03-30 17:05:53 +01:00
Erik Johnston	b282fe7170	Revert log context change	2017-03-30 17:03:59 +01:00
Erik Johnston	9ff4e0e91b	Bump version and changelog	2017-03-30 16:37:40 +01:00
Erik Johnston	0834d1a70c	Merge pull request #2079 from matrix-org/erikj/push_regex_cache ... Cache glob to regex at a higher level for push	2017-03-30 16:28:18 +01:00
Erik Johnston	63fcc42990	Remove user from process_presence when stops syncing	2017-03-30 14:26:08 +01:00
Erik Johnston	6194a64ae9	Doc new instance variables	2017-03-30 14:19:10 +01:00
Erik Johnston	014fee93b3	Manually calculate cache key as getcallargs is expensive ... This is because getcallargs recomputes the getargspec, amongst other things, which we don't need to do as its already been done	2017-03-30 14:14:46 +01:00
Erik Johnston	86780a8bc3	Don't convert to deferreds when not necessary	2017-03-30 14:14:36 +01:00
Erik Johnston	3ba2859e0c	Add tcp replication listener type and hook it up	2017-03-30 13:31:10 +01:00
Erik Johnston	e9dd8370b0	Add functions to presence to support remote syncs ... The TCP replication protocol streams deltas of who has started or stopped syncing. This is different from the HTTP API which periodically sends the full list of users who are syncing. This commit adds support for the new TCP style of sending deltas.	2017-03-30 13:25:14 +01:00
Erik Johnston	4d7fc7f977	Add server side resource for tcp replication	2017-03-30 13:24:45 +01:00
Richard van der Hoff	f9b4bb05e0	Fix the logcontext handling in the cache wrappers (#2077) ... The cache wrappers had a habit of leaking the logcontext into the reactor while the lookup function was running, and then not restoring it correctly when the lookup function had completed. It's all the fault of `preserve_context_over_{fn,deferred}` which are basically a bit broken.	2017-03-30 13:22:24 +01:00
Erik Johnston	7450693435	Initial TCP protocol implementation ... This defines the low level TCP replication protocol	2017-03-30 12:54:46 +01:00
Erik Johnston	8da6f0be48	Define the various streams we will replicate	2017-03-30 12:54:46 +01:00
Erik Johnston	11880103b1	Make federation send queue take the current position	2017-03-30 12:54:36 +01:00
Erik Johnston	7984708a55	Add a simple hook to wait for replication traffic	2017-03-30 11:57:52 +01:00
Erik Johnston	24d35ab47b	Add new storage functions for new replication ... The new replication protocol will keep all the streams separate, rather than muxing multiple streams into one.	2017-03-30 11:48:35 +01:00
Erik Johnston	30348c924c	Use txn.fetchall() so we can reuse txn	2017-03-30 10:30:05 +01:00
Anant Prakash	6cdca71079	synctl.py: wait for synapse to stop before restarting (#2020)	2017-03-29 19:20:13 +01:00
Anant Prakash	305d16d612	add user friendly report of assertion error in synctl.py ... Signed-off-by: Anant Prakash <anantprakashjsr@gmail.com>	2017-03-29 20:41:39 +05:30
Erik Johnston	a3810136fe	Cache glob to regex at a higher level for push	2017-03-29 15:53:14 +01:00
Erik Johnston	3ce8d59176	Increase cache size for _get_state_group_for_event	2017-03-29 14:31:46 +01:00
Erik Johnston	b9c2ae6788	Merge pull request #1849 from matrix-org/erikj/state_typo ... Fix bug where current_state_events renamed to current_state_ids	2017-03-29 11:53:42 +01:00
Erik Johnston	85be3dde81	Bail early if remote wouldn't be retried (#2064) ... * Bail early if remote wouldn't be retried * Don't always return true * Just use get_retry_limiter * Spelling	2017-03-29 11:48:27 +01:00
Erik Johnston	2f8b580b64	Merge pull request #2053 from matrix-org/erikj/e2e_one_time_upsert ... Don't user upsert to persist new one time keys	2017-03-29 11:44:23 +01:00
Erik Johnston	c5b0bdd542	Merge pull request #2067 from matrix-org/erikj/notify_on_fed ... Notify on new federation traffic	2017-03-29 11:41:37 +01:00
Erik Johnston	e4df0e189d	Decrank last commit	2017-03-29 11:02:35 +01:00
Erik Johnston	4ad613f6be	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/e2e_one_time_upsert	2017-03-29 10:57:19 +01:00
Erik Johnston	ac6bc55512	Correctly look up key	2017-03-29 10:56:26 +01:00
Erik Johnston	69efd77749	Add comment	2017-03-29 09:50:05 +01:00
Erik Johnston	51b156d48a	Cache whether an AS is interested based on members	2017-03-28 13:27:21 +01:00
Erik Johnston	30f5ffdca2	Remove param and cast at call site	2017-03-28 13:27:21 +01:00
Erik Johnston	650f0e69f2	Compile the regex's used in ASes	2017-03-28 13:27:21 +01:00
Erik Johnston	d28db583da	Merge pull request #2063 from matrix-org/erikj/device_list_batch ... Batch sending of device list pokes	2017-03-28 11:35:41 +01:00
Erik Johnston	58a35366be	The algorithm is part of the key id	2017-03-28 11:34:37 +01:00
Erik Johnston	bac9bf1b12	Typo	2017-03-27 18:02:17 +01:00
Erik Johnston	d82c42837f	Short circuit if all new events have same state group	2017-03-27 18:00:47 +01:00
Erik Johnston	35b4aa04be	Notify on new federation traffic	2017-03-27 14:07:47 +01:00
Erik Johnston	2a28b79e04	Batch sending of device list pokes	2017-03-24 14:44:49 +00:00
Erik Johnston	281553afe6	Merge pull request #2062 from matrix-org/erikj/presence_replication ... Use presence replication stream to invalidate cache	2017-03-24 13:57:45 +00:00
Erik Johnston	987f4945b4	Actually call invalidate	2017-03-24 13:28:20 +00:00
Erik Johnston	31d56c3fb5	Merge pull request #2061 from matrix-org/erikj/add_transaction_store ... Add slave transaction store to workers who send federation requests	2017-03-24 13:24:27 +00:00
Erik Johnston	09f79aaad0	Use presence replication stream to invalidate cache ... Instead of using the cache invalidation replication stream to invalidate the _get_presence_cache, we can instead rely on the presence replication stream. This reduces the amount of replication traffic considerably.	2017-03-24 13:21:08 +00:00
Erik Johnston	23e0ff840a	Merge pull request #2060 from matrix-org/erikj/cache_hosts_in_room ... Cache hosts in room	2017-03-24 13:07:22 +00:00
Erik Johnston	48e7697911	Add slave transaction store	2017-03-24 13:05:30 +00:00
Richard van der Hoff	f136c89d5e	Merge pull request #2058 from matrix-org/rav/logcontext_leaks_2 ... try not to drop context after federation requests	2017-03-24 12:47:26 +00:00
Richard van der Hoff	01fc847f7f	Merge pull request #2057 from matrix-org/rav/missing_yield_2 ... Add another missing yield on check_device_registered	2017-03-24 12:46:43 +00:00
Erik Johnston	7fc1f1e2b6	Cache hosts in room	2017-03-24 11:46:24 +00:00
Erik Johnston	57cfa513f5	Merge pull request #2054 from matrix-org/erikj/user_iter_cursor ... Reduce some CPU work on DB threads	2017-03-24 11:40:08 +00:00
Erik Johnston	d58b1ffe94	Replace some calls to cursor_to_dict ... cursor_to_dict can be surprisinglh expensive for large result sets, so lets only call it when we need to.	2017-03-24 11:07:02 +00:00
Erik Johnston	e71940aa64	Use iter(items|values)	2017-03-24 10:57:02 +00:00
David Baker	e36950dec5	Merge pull request #2055 from matrix-org/dbkr/fix_add_msisdn_requestToken ... Fix token request for addition of phone numbers	2017-03-24 10:32:57 +00:00
David Baker	f902e89d4b	Merge pull request #2056 from matrix-org/dbkr/fix_invite_reject ... Fix rejection of invites not reaching sync	2017-03-24 10:32:51 +00:00
Richard van der Hoff	a380f041c2	try not to drop context after federation requests ... preserve_context_over_fn uses a ContextPreservingDeferred, which only restores context for the duration of its callbacks, which isn't really correct, and means that subsequent operations in the same request can end up without their logcontexts.	2017-03-23 22:36:21 +00:00
Richard van der Hoff	9397edb28b	Merge pull request #2050 from matrix-org/rav/federation_backoff ... push federation retry limiter down to matrixfederationclient	2017-03-23 22:27:01 +00:00
Richard van der Hoff	06ce7335e9	Merge pull request #2052 from matrix-org/rav/time_bound_deferred ... Fix time_bound_deferred to throw the right exception	2017-03-23 22:22:54 +00:00
Richard van der Hoff	13c8749ac9	Add another missing yield on check_device_registered	2017-03-23 22:18:53 +00:00
David Baker	e1f1784f99	Fix rejection of invites not reaching sync ... Always allow the user to see their own leave events, otherwise they won't see the event if they reject an invite for a room whose history visibility is set such that they cannot see events before joining.	2017-03-23 18:50:31 +00:00
David Baker	86e865d7d2	Oops, remove unintentional change	2017-03-23 18:48:30 +00:00
David Baker	a2dfab12c5	Fix token request for addition of phone numbers	2017-03-23 18:46:17 +00:00
Erik Johnston	00957d1aa4	User Cursor.__iter__ instead of fetchall ... This prevents unnecessary construction of lists	2017-03-23 17:53:49 +00:00
pik	250ce11ab9	Add jsonschema to python_dependencies.py ... Signed-off-by: pik <alexander.maznev@gmail.com>	2017-03-23 11:42:47 -03:00
pik	566641a0b5	use jsonschema.FormatChecker for RoomID and UserID strings ... * use a valid filter in rest/client/v2_alpha test Signed-off-by: pik <alexander.maznev@gmail.com>	2017-03-23 11:42:41 -03:00
pik	acafcf1c5b	Add valid filter tests, flake8, fix typo ... Signed-off-by: pik <alexander.maznev@gmail.com>	2017-03-23 11:42:10 -03:00
pik	e56c79c114	check_valid_filter using JSONSchema ... * add invalid filter tests Signed-off-by: pik <alexander.maznev@gmail.com>	2017-03-23 11:42:07 -03:00
Erik Johnston	6ebe2d23b1	Raise a more helpful exception	2017-03-23 13:48:30 +00:00
Erik Johnston	e64655c25d	Don't user upsert to persist new one time keys ... Instead we no-op duplicate one time key uploads, an error if the key_id already exists but encodes a different key.	2017-03-23 13:17:00 +00:00
Richard van der Hoff	5a16cb4bf0	Ignore backoff history for invites, aliases, and roomdirs ... Add a param to the federation client which lets us ignore historical backoff data for federation queries, and set it for a handful of operations.	2017-03-23 12:23:22 +00:00
Richard van der Hoff	b88a323ffb	Fix time_bound_deferred to throw the right exception ... Due to a failure to instantiate DeferredTimedOutError, time_bound_deferred would throw a CancelledError when the deferred timed out, which was rather confusing.	2017-03-23 12:07:11 +00:00
Richard van der Hoff	55366814a6	Merge pull request #2048 from matrix-org/rav/missing_yield ... Add a missing yield in device key upload	2017-03-23 09:29:44 +00:00
Richard van der Hoff	4bd597d9fc	push federation retry limiter down to matrixfederationclient ... rather than having to instrument everywhere we make a federation call, make the MatrixFederationHttpClient manage the retry limiter.	2017-03-23 09:28:46 +00:00
Richard van der Hoff	ad8a26e361	MatrixFederationHttpClient: clean up ... rename _create_request to _request, and push ascii-encoding of `destination` and `path` down into it	2017-03-23 00:27:04 +00:00
Richard van der Hoff	19b9366d73	Fix a couple of logcontext leaks ... Use preserve_fn to correctly manage the logcontexts around things we don't want to yield on.	2017-03-23 00:17:46 +00:00
Richard van der Hoff	e08f81d96a	Add a missing yield in device key upload ... (this would only very very rarely actually be a useful thing, so the main problem was the logcontext leak...)	2017-03-23 00:16:43 +00:00
Richard van der Hoff	35e0cfb54d	Merge pull request #2044 from matrix-org/rav/crypto_docs ... fix up some key verif docstrings	2017-03-22 17:12:55 +00:00
Richard van der Hoff	7b67848042	Merge pull request #2042 from matrix-org/rav/fix_key_caching ... Fix caching of remote servers' signature keys	2017-03-22 17:11:55 +00:00
Richard van der Hoff	95f21c7a66	Fix caching of remote servers' signature keys ... The `@cached` decorator on `KeyStore._get_server_verify_key` was missing its `num_args` parameter, which meant that it was returning the wrong key for any server which had more than one recorded key. By way of a fix, change the default for `num_args` to be *all* arguments. To implement that, factor out a common base class for `CacheDescriptor` and `CacheListDescriptor`.	2017-03-22 15:11:30 +00:00
Matthew Hodgson	d101488c5f	Merge branch 'master' into develop	2017-03-21 22:48:21 +01:00
Richard van der Hoff	64778693be	fix up some key verif docstrings	2017-03-21 13:27:50 +00:00
Erik Johnston	37a187bfab	Merge pull request #2033 from matrix-org/erikj/repl_speed ... Don't send the full event json over replication	2017-03-21 13:11:15 +00:00
Erik Johnston	633dcc316c	Bump changelog and version	2017-03-20 16:37:14 +00:00
Richard van der Hoff	c36d15d2de	Add some debug to help diagnose weird federation issue	2017-03-20 15:36:14 +00:00
Erik Johnston	aac6d1fc9b	PEP8	2017-03-20 13:47:56 +00:00
Richard van der Hoff	bd08ee7a46	Merge pull request #2026 from matrix-org/rav/logcontext_docs ... Logcontext docs	2017-03-20 12:05:21 +00:00
Richard van der Hoff	eddce9d74a	Merge pull request #2027 from matrix-org/rav/logcontext_leaks ... A few fixes to logcontext things	2017-03-20 11:53:36 +00:00
Richard van der Hoff	f40c2db05a	Stop preserve_fn leaking context into the reactor ... Fix a bug in ``logcontext.preserve_fn`` which made it leak context into the reactor, and add a test for it. Also, get rid of ``logcontext.reset_context_after_deferred``, which tried to do the same thing but had its own, different, set of bugs.	2017-03-18 00:07:43 +00:00
Richard van der Hoff	067b00d49d	Run the reactor with the sentinel logcontext ... This fixes a class of 'Unexpected logcontext' messages, which were happening because the logcontext was somewhat arbitrarily swapping between the sentinel and the `run` logcontext.	2017-03-18 00:07:43 +00:00
Richard van der Hoff	994d7ae7c5	Remove broken use of clock.call_later ... background_updates was using `call_later` in a way that leaked the logcontext into the reactor. We could have rewritten it to do it properly, but given that we weren't using the fancier facilities provided by `call_later`, we might as well just use `async.sleep`, which does the logcontext stuff properly.	2017-03-18 00:01:37 +00:00
Richard van der Hoff	d2d146a314	Logcontext docs	2017-03-17 23:59:28 +00:00
Erik Johnston	61f471f779	Don't send the full event json over replication	2017-03-17 15:50:01 +00:00
Richard van der Hoff	0c01f829ae	Avoid resetting state on rejected events ... When we get a rejected event, give it the same state_group as its prev_event, rather than no state_group at all. This should fix https://github.com/matrix-org/synapse/issues/1935.	2017-03-17 15:06:08 +00:00
Richard van der Hoff	5068fb16a5	Refactoring and cleanups ... A few non-functional changes: * A bunch of docstrings to document types * Split `EventsStore._persist_events_txn` up a bit. Hopefully it's a bit more readable. * Rephrase `EventFederationStore._update_min_depth_for_room_txn` to avoid mind-bending conditional. * Rephrase rejected/outlier conditional in `_update_outliers_txn` to avoid mind-bending conditional.	2017-03-17 15:06:07 +00:00
Richard van der Hoff	2abe85d50e	Merge pull request #2016 from matrix-org/rav/queue_pdus_during_join ... Queue up federation PDUs while a room join is in progress	2017-03-17 11:32:44 +00:00
Erik Johnston	da146657c9	Comments	2017-03-16 13:04:07 +00:00
Erik Johnston	a158c36a8a	Comment	2017-03-16 11:57:45 +00:00
Erik Johnston	6957bfdca6	Don't recreate so many sets	2017-03-16 11:54:26 +00:00
Erik Johnston	2ccf3b241c	Implement no op for room stream in sync	2017-03-16 11:06:41 +00:00
Richard van der Hoff	9ce53a3861	Queue up federation PDUs while a room join is in progress ... This just takes the existing `room_queues` logic and moves it out to `on_receive_pdu` instead of `_process_received_pdu`, which ensures that we don't start trying to fetch prev_events and whathaveyou until the join has completed.	2017-03-15 18:01:11 +00:00
Erik Johnston	54d2b7e596	Merge pull request #2014 from Half-Shot/hs/fix-appservice-presence ... Add fallback to last_active_ts if it beats the last sync time on a presence timeout.	2017-03-15 17:37:15 +00:00
Erik Johnston	a8f96c63aa	Comment	2017-03-15 16:01:01 +00:00
Erik Johnston	f83ac78201	Cache set of users whose presence the other user should see	2017-03-15 15:29:19 +00:00
Will Hunt	e6032054bf	Add a great comment to handle_timeout for active vs sync times.	2017-03-15 15:24:48 +00:00
Will Hunt	ebf5a6b14c	Add fallback to last_active_ts if it beats the last sync time.	2017-03-15 15:17:16 +00:00
Erik Johnston	e892457a03	Comment	2017-03-15 15:01:39 +00:00
Erik Johnston	a297155a97	Remove unused import	2017-03-15 14:49:25 +00:00
Erik Johnston	6c82de5100	Format presence events on the edges instead of reformatting them multiple times	2017-03-15 14:27:34 +00:00
David Baker	0ad44acb5a	Merge pull request #1997 from matrix-org/dbkr/cas_partialdownload ... Handle PartialDownloadError in CAS login	2017-03-15 13:52:34 +00:00
Matthew Hodgson	0970e0307e	typo	2017-03-15 12:40:42 +00:00
Matthew Hodgson	5aa42d4292	set default for turn_allow_guests correctly	2017-03-15 12:40:13 +00:00
Matthew Hodgson	e0ff66251f	add setting (on by default) to support TURN for guests	2017-03-15 12:22:18 +00:00
Richard van der Hoff	29ed09e80a	Fix assertion to stop transaction queue getting wedged ... ... and update some docstrings to correctly reflect the types being used. get_new_device_msgs_for_remote can return a long under some circumstances, which was being stored in last_device_list_stream_id_by_dest, and was then upsetting things on the next loop.	2017-03-15 12:16:55 +00:00
Erik Johnston	872e75a3d5	Add some metrics on notifier	2017-03-15 10:56:51 +00:00
Erik Johnston	7827251daf	Merge pull request #1994 from matrix-org/dbkr/msisdn_signin_2 ... Phone number registration / login support v2	2017-03-15 09:59:54 +00:00
Richard van der Hoff	b5d1c68beb	Implement reset_context_after_deferred ... to correctly reset the context when we fire off a deferred we aren't going to wait for.	2017-03-15 02:21:07 +00:00
Richard van der Hoff	f2ed64eaaf	Merge pull request #1992 from matrix-org/rav/fix_media_loop ... Fix routing loop when fetching remote media	2017-03-14 23:40:35 +00:00
Erik Johnston	bad72b0b8e	Merge pull request #2002 from matrix-org/erikj/dont_sync_by_default ... Reduce number of spurious sync result generations.	2017-03-14 16:58:45 +00:00
Erik Johnston	fd2eef49c8	Reduce spurious calls to generate sync	2017-03-14 16:33:45 +00:00
Richard van der Hoff	1d09586599	Address review comments ... - don't blindly proxy all HTTPRequestExceptions - log unexpected exceptions at error - avoid `isinstance` - improve docs on `from_http_response_exception`	2017-03-14 14:15:37 +00:00
Richard van der Hoff	7f237800e9	re-refactor exception heirarchy ... Give CodeMessageException back its `msg` attribute, and use that to hold the HTTP status message for HttpResponseException.	2017-03-14 14:15:37 +00:00
David Baker	1ece06273e	Handle PartialDownloadError in CAS login	2017-03-14 13:37:36 +00:00
Erik Johnston	bb256ac96f	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/public_list_speed	2017-03-14 11:35:05 +00:00
Erik Johnston	cc7a294e2e	Fix current_state_events table to not lie ... If we try and persist two state events that have the same ancestor we calculate the wrong current state when persisting those events.	2017-03-14 10:57:43 +00:00
David Baker	7b6ed9871e	Use extend instead of +=	2017-03-14 10:49:55 +00:00
David Baker	d79a687d85	Oops, remove print	2017-03-14 10:40:20 +00:00
Luke Barnard	f29d85d9e4	Merge pull request #1993 from matrix-org/luke/delete-devices ... Implement delete_devices API	2017-03-14 10:02:56 +00:00
Luke Barnard	bbeeb97f75	Implement _simple_delete_many_txn, use it to delete devices ... (But this doesn't implement the same for deleting access tokens or e2e keys. Also respond to code review.	2017-03-13 17:53:23 +00:00
David Baker	0a9945220e	Fix registration for broken clients ... Only offer msisdn flows if the x_show_msisdn option is given.	2017-03-13 17:29:38 +00:00
David Baker	73a5f06652	Support registration / login with phone number ... Changes from https://github.com/matrix-org/synapse/pull/1971	2017-03-13 17:27:51 +00:00
Luke Barnard	c077c3277b	Flake	2017-03-13 16:45:38 +00:00
Richard van der Hoff	31f3ca1b2b	Merge pull request #1990 from matrix-org/rav/log_config_comments ... Add helpful texts to logger config options	2017-03-13 16:42:12 +00:00
Luke Barnard	c81f33f73d	Implement delete_devices API ... This implements the proposal here https://docs.google.com/document/d/1C-25Gqz3TXy2jIAoeOKxpNtmme0jI4g3yFGqv5GlAAk for deleting multiple devices at once in a single request.	2017-03-13 16:33:51 +00:00
Richard van der Hoff	170ccc9de5	Fix routing loop when fetching remote media ... When we proxy a media request to a remote server, add a query-param, which will tell the remote server to 404 if it doesn't recognise the server_name. This should fix a routing loop where the server keeps forwarding back to itself. Also improves the error handling on remote media fetches, so that we don't always return a rather obscure 502.	2017-03-13 16:30:36 +00:00
Erik Johnston	45c7f12d2a	Add new storage function to slave store	2017-03-13 16:26:44 +00:00
Richard van der Hoff	6037a9804c	Add helpful texts to logger config options	2017-03-13 12:33:35 +00:00
Richard van der Hoff	3c69f32402	Merge remote-tracking branch 'origin/develop' into rav/refactor_received_pdu	2017-03-13 12:20:47 +00:00
Richard van der Hoff	6bfe8e32b5	Merge pull request #1983 from matrix-org/rav/no_redirect_stdio ... Add an option to disable stdio redirect	2017-03-13 12:20:07 +00:00
Richard van der Hoff	5fc9261929	Merge pull request #1982 from matrix-org/rav/sighup_for_logconfig ... Reread log config on SIGHUP	2017-03-13 12:19:54 +00:00
Erik Johnston	0162994983	Comments	2017-03-13 11:53:26 +00:00
Erik Johnston	254b7c5b15	Bump changelog and versions	2017-03-13 10:02:58 +00:00
Erik Johnston	672dcf59d3	Merge branch 'develop' of github.com:matrix-org/synapse into release-v0.19.3	2017-03-13 09:59:54 +00:00
Erik Johnston	7eae6eaa2f	Revert "Support registration & login with phone number"	2017-03-13 09:59:33 +00:00
Erik Johnston	79926e016e	Assume rooms likely haven't changed	2017-03-13 09:50:10 +00:00
Matthew Hodgson	a61dd408ed	enable guest access for the 3pl/3pid APIs	2017-03-12 19:30:45 +00:00
Erik Johnston	8ffbe43ba1	Get current state by using current_state_events table	2017-03-10 17:39:35 +00:00
Richard van der Hoff	bcfa5cd00c	Add an option to disable stdio redirect ... This makes it tractable to run synapse under pdb.	2017-03-10 15:38:29 +00:00
Richard van der Hoff	d84bd51e95	Refactor logger config for workers ... - to make it easier to add more config options.	2017-03-10 15:34:01 +00:00
Richard van der Hoff	9072a8c627	Reread log config on SIGHUP ... When we are using a log_config file, reread it on SIGHUP.	2017-03-10 15:29:55 +00:00
Erik Johnston	8f267fa8a8	Fix it for the workers	2017-03-10 11:22:25 +00:00
Erik Johnston	64d62e41b8	Noop repated delete device inbox calls from /sync	2017-03-10 10:36:43 +00:00
Erik Johnston	3545e17f43	Add setdefault key to ExpiringCache	2017-03-10 10:30:49 +00:00
Richard van der Hoff	29235901b8	Move FederationServer._handle_new_pdu to FederationHandler ... Unfortunately this significantly increases the size of the already-rather-big FederationHandler, but the code fits more naturally here, and it paves the way for the tighter integration that I need between handling incoming PDUs and doing the join dance. Other than renaming the existing `FederationHandler.on_receive_pdu` to `_process_received_pdu` to make way for it, this just consists of the move, and replacing `self.handler` with `self` and `self` with `self.replication_layer`.	2017-03-09 16:20:13 +00:00
Richard van der Hoff	e8b1721290	Move sig check out of _handle_new_pdu ... When we receive PDUs via `get_missing_events`, we have already checked their sigs, so there is no need to do it again.	2017-03-09 15:50:44 +00:00
Richard van der Hoff	3406333a58	Factor _get_missing_events_for_pdu out of _handle_new_pdu ... This should be functionally identical: it just seeks to improve readability by reducing indentation.	2017-03-09 15:50:44 +00:00
Richard van der Hoff	45d173a59a	Fix docstring	2017-03-09 15:50:29 +00:00
David Baker	663396e45d	Merge pull request #1971 from matrix-org/dbkr/msisdn_signin ... Support registration & login with phone number	2017-03-09 10:18:51 +00:00
David Baker	ece7e00048	Comment when our 3pids would be incomplete	2017-03-08 19:07:18 +00:00
David Baker	9d0d40fc15	Docs	2017-03-08 19:05:29 +00:00
David Baker	3edc57296d	Incorrectly copied copyright ... This file post-dates OM	2017-03-08 19:00:51 +00:00
David Baker	727124a762	Not any more, it doesn't	2017-03-08 19:00:23 +00:00
Richard van der Hoff	6ad71cc29d	Remove spurious SQL logging (#1972) ... looks like the upsert function was accidentally sending sql logging to the general logger. We already log the sql in `txn.execute`.	2017-03-08 18:00:44 +00:00
David Baker	d4d3629aaf	Better error message	2017-03-08 17:01:26 +00:00
Erik Johnston	3170c56e07	Bump changelog and version	2017-03-08 13:00:47 +00:00
David Baker	1c99934b28	pep8	2017-03-08 11:58:20 +00:00
David Baker	a9e2b9ec16	Add msisdn util file	2017-03-08 11:53:36 +00:00
David Baker	85bb322333	Pull out datastore in initialiser	2017-03-08 11:51:25 +00:00
David Baker	65d43f3ca5	Minor fixes from PR feedback	2017-03-08 11:48:43 +00:00
David Baker	0e0aee25c4	Fix log line	2017-03-08 11:46:22 +00:00
David Baker	82c5e7de25	Typos	2017-03-08 11:42:44 +00:00
David Baker	2e27339add	Refector out assert_params_in_request ... and replace requestEmailToken where we meant requestMsisdnToken	2017-03-08 11:37:34 +00:00
David Baker	88df6c0c9a	Factor out msisdn canonicalisation ... Plus a couple of other minor fixes	2017-03-08 11:03:39 +00:00
David Baker	402a7bf63d	Fix pep8	2017-03-08 09:33:40 +00:00
David Baker	00466e2feb	Support new login format ... https://docs.google.com/document/d/1-6ZSSW5YvCGhVFDyD2QExAUAdpCWjccvJT5xiyTTG2Y/edit#	2017-03-07 16:37:23 +00:00
Erik Johnston	ac5491f563	Select distinct devices from DB ... Otherwise we might pull out tonnes of duplicate user_ids and this can make synapse sad.	2017-03-06 11:10:14 +00:00
David Baker	b0effa2160	Add msisdns as 3pids during registration ... and support binding them with the bind_msisdn param	2017-03-03 18:34:39 +00:00
Erik Johnston	82f7f1543b	Merge pull request #1964 from matrix-org/erikj/device_list_update_fix ... Fix device list update to not constantly resync	2017-03-03 16:18:16 +00:00
Erik Johnston	f2581ee8b8	Don't keep around old stream IDs forever	2017-03-03 16:02:53 +00:00
Erik Johnston	9834367eea	Spelling	2017-03-03 15:31:57 +00:00
Erik Johnston	da52d3af31	Fix up	2017-03-03 15:29:13 +00:00
David Baker	ad882cd54d	Just return the deferred straight off ... defer.returnValue doth not maketh a generator: it would need a yield to be a generator, and this doesn't need a yield.	2017-03-01 18:08:51 +00:00
David Baker	3557cf34dc	Merge remote-tracking branch 'origin/develop' into dbkr/msisdn_signin	2017-03-01 17:20:37 +00:00
Erik Johnston	d766343668	Add index to device_lists_stream	2017-03-01 15:56:30 +00:00
Erik Johnston	36be39b8b3	Fix device list update to not constantly resync	2017-03-01 14:12:11 +00:00
Erik Johnston	3365117151	Clobber old device list stream entries	2017-03-01 10:21:30 +00:00
Erik Johnston	e933a2712d	Don't log unknown cache warnings in workers	2017-02-28 16:22:41 +00:00
Erik Johnston	d638a7484b	Merge pull request #1959 from matrix-org/erikj/intern_once ... Intern table column names once	2017-02-28 15:15:16 +00:00
Erik Johnston	b84907bdbb	Intern table column names once	2017-02-28 14:38:16 +00:00
Erik Johnston	e4919b9329	Add stream_id index to device_lists_outbound_pokes ... As this is used for replication streaming	2017-02-28 11:19:06 +00:00
Erik Johnston	8a12b6f1eb	Fix up txn name	2017-02-28 10:15:50 +00:00
Erik Johnston	848cf95ea0	Pop with default value to stop throwing	2017-02-28 10:02:54 +00:00
Erik Johnston	eda96586ca	Merge pull request #1955 from matrix-org/erikj/current_state_query_bypass ... Don't fetch current state in common case	2017-02-27 19:15:59 +00:00
Erik Johnston	64a2cef9bb	Pop rather than del from dict	2017-02-27 19:15:36 +00:00
Erik Johnston	a41dce8f8a	Remove needless check	2017-02-27 18:54:43 +00:00
Erik Johnston	c0d6045776	It should be all	2017-02-27 18:45:24 +00:00
Erik Johnston	49f4bc4709	Don't fetch current state in common case ... Currently we fetch the list of current state events whenever we send something in a room. This is overkill for the common case of persisting a simple chain of non-state events, so lets handle that case specially.	2017-02-27 18:33:41 +00:00
Erik Johnston	fc5d876dba	Merge pull request #1954 from matrix-org/erikj/cache_device2 ... Cache get_user_devices_from_cache	2017-02-27 16:36:40 +00:00
Erik Johnston	f58dbb02a6	Cache get_user_devices_from_cache	2017-02-27 16:22:12 +00:00
Erik Johnston	acf6d4d2e3	Merge pull request #1945 from jkolo/fix_dynamic_thumbnails_aspect ... Fix #1677 (dynamic thumbnails aspect)	2017-02-27 09:51:52 +00:00
Jurek	aea5461488	Fix dynamic thumbnails aspect	2017-02-24 22:43:27 +01:00
Erik Johnston	1a4f8022e6	Strip newlines from SQL queries	2017-02-23 11:15:31 +00:00
Erik Johnston	b2d20e94fa	Remove lock from rotate notifs	2017-02-22 14:24:02 +00:00
Erik Johnston	7455ba436a	Ensure we pass positive ints to delay function	2017-02-22 12:08:14 +00:00
Erik Johnston	b7442c3e2b	Store looping call	2017-02-21 13:59:25 +00:00
Erik Johnston	a3708a1885	Merge branch 'master' of github.com:matrix-org/synapse into develop	2017-02-21 13:46:27 +00:00
Erik Johnston	30ecfef5a3	Bump version and changelog	2017-02-21 13:43:36 +00:00
Richard van der Hoff	0c4cf9372b	Fix a race in transaction queue ... It was theoretically possible for a PDU to get queued and not sent for ages. On closer inspection I think there were bigger problems elsewhere, but we might as well fix this since it's easy.	2017-02-20 16:46:25 +00:00
Erik Johnston	6226a27bf8	Remove unused param	2017-02-20 16:01:54 +00:00
Erik Johnston	efff39c030	Fix /context/ visibiltiy rules	2017-02-20 16:01:49 +00:00
Erik Johnston	17673404fb	Remove unused param	2017-02-20 15:02:01 +00:00
Erik Johnston	7f026792e1	Fix /context/ visibiltiy rules	2017-02-20 14:54:50 +00:00
Erik Johnston	699be7d1be	Fix up notif rotation	2017-02-18 14:42:39 +00:00
David Baker	ec8d7603e6	Merge pull request #1925 from matrix-org/dbkr/pushers_lang_lengthen ... Make the pushers lang field column longer	2017-02-17 11:29:06 +00:00
David Baker	8c87bb550e	Merge pull request #1922 from matrix-org/dbkr/allow_forget_for_ban ... Allow forgetting rooms you're banned from	2017-02-17 10:52:30 +00:00
David Baker	4aa29508af	Use TEXT rather than VARCHAR ... While we're changing anyway	2017-02-17 10:51:49 +00:00
David Baker	b4017539d4	Make the pushers lang field column longer ... To accommodate things like zh-Hans-CN Fixes https://github.com/vector-im/riot-ios/issues/1031	2017-02-17 10:42:57 +00:00
Erik Johnston	138e030cfe	Comment	2017-02-16 15:03:36 +00:00
Erik Johnston	502ae6c663	Comment	2017-02-16 14:47:11 +00:00
Erik Johnston	e6acf0c399	Store the default push actions in a more efficient manner	2017-02-16 14:40:24 +00:00
Erik Johnston	04eca2589d	Merge pull request #1916 from matrix-org/erikj/push_actions_delete ... Aggregate event push actions	2017-02-16 15:28:58 +01:00
David Baker	474c9aadbe	Allow forgetting rooms you're banned from	2017-02-15 19:32:20 +00:00
Richard van der Hoff	7dcbcca68c	Merge pull request #1921 from matrix-org/rav/fix_key_changes ... Fix bugs in the /keys/changes api	2017-02-15 11:25:16 +00:00
David Baker	fa467e62a9	Merge pull request #1917 from matrix-org/dbkr/make_ban_reasons_work ... Make kick & ban reasons work	2017-02-14 16:10:06 +00:00
David Baker	355d62c499	Make kick & ban reasons work ... We somehow specced APIs with reason strings, preserve the content in the events and even have the clients display them, but failed to actually pass the parameter through to the event content.	2017-02-14 15:10:55 +00:00
David Baker	ce3e583d94	WIP support for msisdn 3pid proxy methods	2017-02-14 15:05:55 +00:00
Richard van der Hoff	fc2f29c1d0	Fix bugs in the /keys/changes api ... * `get_forward_extremeties_for_room` takes a numeric `stream_ordering`. We were passing a `RoomStreamToken`, which meant that it returned the *current* extremities, rather than those corresponding to the `from_token`. However: * `get_state_ids_for_events` required a second ('types') parameter; this meant that a `TypeError` was thrown and we ended up acting as though there was *no* prev state. * `get_state_ids_for_events` actually returns a map from event_id to state dictionary - just looking up the state keys in it again meant that we acted as though there was no prev state. We now check if each member's state has changed since *any* of the extremities. Also add/fix some comments.	2017-02-14 13:59:50 +00:00
Erik Johnston	ce3c8df6df	Less aggressive timers	2017-02-14 13:41:24 +00:00
Erik Johnston	095b45c165	Aggregate event push actions	2017-02-14 13:39:41 +00:00
Erik Johnston	795f8e3fe7	Merge pull request #1873 from matrix-org/erikj/delete_push_actions ... Be more agressive about purging old room event_push_actions	2017-02-14 14:29:04 +01:00
Erik Johnston	d7457c7661	Merge pull request #1914 from matrix-org/erikj/cache_presence ... Cache get_presence storage	2017-02-13 16:59:19 +01:00
Erik Johnston	9e617cd4c2	Cache get_presence storage	2017-02-13 13:50:03 +00:00
Kegan Dougal	d0497425f8	Ordering is important on errbacks so add the cleanup func before creating an ObservableDeferred	2017-02-13 13:49:44 +00:00
Kegan Dougal	808ddf0ae7	Pop the txn from the map in case it has already been deleted somehow	2017-02-13 13:36:15 +00:00
Kegan Dougal	feb15dc99f	Don't cache errors at all	2017-02-13 13:33:12 +00:00
Kegan Dougal	ecd7e36047	http txns: Do not cache error responses ... Previously we did. This meant that, amongst other errors, rate-limiting errors would be cached and prevent messages with that txn ID being sent.	2017-02-13 13:16:48 +00:00
Erik Johnston	6bba80241c	Merge pull request #1912 from matrix-org/markjh/roominitialsync ... Add db functions needed for room initial sync to slave	2017-02-13 12:20:21 +01:00
Mark Haines	3a46280ca3	Add db functions needed for room initial sync to slave	2017-02-13 11:16:53 +00:00
Tyler Smith	df4407d665	Fix typo in config comments. ... Signed-off-by: Tyler Smith <tylersmith.me@gmail.com>	2017-02-11 23:02:57 -08:00
Erik Johnston	1beda9c8a7	Merge branch 'master' of github.com:matrix-org/synapse into develop	2017-02-09 10:46:58 +00:00
Erik Johnston	505bfd82bb	Update version and changelog	2017-02-09 10:41:02 +00:00
Erik Johnston	52cd019a54	Make None check explicit	2017-02-08 16:04:29 +00:00
Erik Johnston	f20cd34858	Merge pull request #1892 from matrix-org/erikj/rejection_fwd_extrem ... Ignore new rejected events when working out forward extremities.	2017-02-08 16:59:06 +01:00
Erik Johnston	7723b4caa4	Ignore new rejected events when working out forward extremeties.	2017-02-08 14:48:06 +00:00
David Baker	063a1251a9	Remove a few aspirational but unused constants ... from the Kegan era	2017-02-08 11:36:08 +00:00
Erik Johnston	af6da6db2d	Merge pull request #1784 from morteza-araby/user-admin ... Administration functionalities	2017-02-06 16:21:10 +01:00
Erik Johnston	38434a7fbb	Bump changelog and version	2017-02-04 08:27:51 +00:00
Erik Johnston	84f600b2ee	Bump changelog and version	2017-02-02 18:58:33 +00:00
Erik Johnston	aec1708c53	Merge branch 'develop' of github.com:matrix-org/synapse into release-v0.19.0	2017-02-02 18:57:05 +00:00
Erik Johnston	f3c8658217	Merge pull request #1879 from matrix-org/erikj/bump_cache_factors ... Bump cache sizes for common membership queries	2017-02-02 18:56:04 +00:00
Erik Johnston	38258a0976	Bump cache sizes for common membership queries	2017-02-02 18:45:55 +00:00
Erik Johnston	a597994fb6	Measure new device list stuff	2017-02-02 18:36:17 +00:00
Erik Johnston	82b3e0851c	Bump version and changelog	2017-02-02 17:15:17 +00:00
Erik Johnston	f8c407a13b	Merge branch 'develop' of github.com:matrix-org/synapse into release-v0.19.0	2017-02-02 16:50:28 +00:00
Erik Johnston	1232ae41cf	Use new get_users_who_share_room_with_user	2017-02-02 15:25:00 +00:00
Erik Johnston	99fa03e8b5	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/device_list_fixes	2017-02-02 15:23:45 +00:00
Erik Johnston	a8331897aa	Merge pull request #1876 from matrix-org/erikj/shared_member_store ... Make presence.get_new_events a bit faster	2017-02-02 15:20:14 +00:00
Erik Johnston	0f3e296cb7	Fix replication	2017-02-02 15:02:03 +00:00
Erik Johnston	6826593b81	sets aren't JSON serializable	2017-02-02 14:55:54 +00:00
Erik Johnston	6b61060b51	Comment	2017-02-02 14:47:15 +00:00
Erik Johnston	46ecd9fd6d	Use stream_ordering_to_exterm for /keys/changes	2017-02-02 14:27:19 +00:00
Erik Johnston	9efcc3f3be	Comment	2017-02-02 13:50:22 +00:00
Erik Johnston	832e9c52ca	Comment	2017-02-02 13:09:56 +00:00
Erik Johnston	54a79c1d37	Make presence.get_new_events a bit faster ... We do this by caching the set of users a user shares rooms with.	2017-02-02 13:07:18 +00:00
Morteza Araby	2849d3f29d	admin,storage: added more administrator functionalities ... administrators can now: - Set displayname of users - Update user avatars - Search for users by user_id - Browse all users in a paginated API - Reset user passwords - Deactivate users Helpers for doing paginated queries has also been added to storage Signed-off-by: Morteza Araby <morteza.araby@ericsson.com>	2017-02-02 14:02:26 +01:00
Erik Johnston	bfe3f5815f	Update changelog	2017-02-02 11:10:41 +00:00
Erik Johnston	cc01eae332	Merge branch 'develop' of github.com:matrix-org/synapse into release-v0.19.0	2017-02-02 11:05:57 +00:00
Erik Johnston	51adaac953	Fix email push in pusher worker ... This was broken when device list updates were implemented, as Mailer could no longer instantiate an AuthHandler due to a dependency on federation sending.	2017-02-02 10:53:36 +00:00
Erik Johnston	10e0737569	Bump version and changelog	2017-02-02 10:02:28 +00:00
Erik Johnston	fac3c03087	Be more agressive about purging old room event_push_actions	2017-02-01 18:27:24 +00:00
Erik Johnston	fbfe44bb4d	Doc args	2017-02-01 17:52:57 +00:00
Erik Johnston	d61a04583e	Comment	2017-02-01 17:35:23 +00:00
Erik Johnston	7e919bdbd0	Include newly joined users in /keys/changes API	2017-02-01 17:33:16 +00:00
Erik Johnston	df4ecff5a9	Correctly raise exceptions for ratelimitng. Ratelimit on 401	2017-02-01 15:42:19 +00:00
Erik Johnston	6d6591880e	Wake sync up for device changes	2017-02-01 15:15:16 +00:00
Erik Johnston	bd84387ac6	Merge pull request #1869 from matrix-org/erikj/device_list_stream ... Implement /keys/changes	2017-02-01 13:25:26 +00:00
Erik Johnston	ebfaff84c9	Merge pull request #1870 from matrix-org/erikj/cache_get_all_new_events ... Add a small cache get_all_new_events	2017-02-01 13:22:02 +00:00
Erik Johnston	73d676dc8b	Comment	2017-02-01 13:17:17 +00:00
Erik Johnston	62f6b86ba7	Merge pull request #1868 from matrix-org/erikj/replication_cache ... Only invalidate membership caches based on the cache stream	2017-02-01 13:12:30 +00:00
Erik Johnston	f6124311fd	Add m.room.member type to query	2017-02-01 11:59:17 +00:00
Erik Johnston	88a4d54883	Merge pull request #1867 from matrix-org/erikj/member_index ... Add an index to make membership queries faster	2017-02-01 11:44:27 +00:00
Erik Johnston	368c88c487	Add a small cache get_all_new_events	2017-02-01 10:50:44 +00:00
Erik Johnston	5deaf9e30b	Up get_latest_event_ids_in_room cache	2017-02-01 10:39:41 +00:00
Erik Johnston	acb501c46d	Comment	2017-02-01 10:32:49 +00:00
Erik Johnston	97479d0c54	Implement /keys/changes	2017-02-01 10:30:03 +00:00
Erik Johnston	06567ec513	Merge pull request #1866 from matrix-org/erikj/device_list_fixes ... Better handle 404 response for federation /send/	2017-02-01 09:44:14 +00:00
Erik Johnston	458b6f4733	Only invalidate membership caches based on the cache stream ... Before we completely invalidated get_users_in_room whenever we updated any current_state_events table. This was way too aggressive.	2017-01-31 16:09:03 +00:00
Erik Johnston	fe08db2713	Remove explicit < 400 check as apparently this is confusing	2017-01-31 15:21:32 +00:00
Erik Johnston	21b7375778	Add an index to make membership queries faster	2017-01-31 15:15:57 +00:00
Erik Johnston	4c0ec15bdc	Comment	2017-01-31 13:53:46 +00:00
Erik Johnston	85c590105f	Comment	2017-01-31 13:46:38 +00:00
Erik Johnston	ae7a132f38	Better handle 404 response for federation /send/	2017-01-31 13:40:09 +00:00
Erik Johnston	ac001dabdc	Merge pull request #1864 from matrix-org/erikj/device_list_fixes ... Fix clearing out old device list outbound pokes	2017-01-31 13:35:35 +00:00
Erik Johnston	bfb3d255b1	Merge pull request #1862 from matrix-org/erikj/presence_update ... Use DB cache of joined users for presence	2017-01-31 13:23:24 +00:00
Erik Johnston	ab55794b6f	Fix deletion of old sent devices correctly	2017-01-31 13:22:41 +00:00
Erik Johnston	d3169e8d28	Only fetch with row ts and count > 1	2017-01-31 11:20:03 +00:00
Erik Johnston	05b9f48ee5	Fix clearing out old device list outbound pokes	2017-01-31 10:08:55 +00:00
Erik Johnston	4b3403ca9b	Stream cache invalidations for room membership storage functions	2017-01-30 17:28:22 +00:00
Erik Johnston	1c13c9f6b6	Don't have such a large cache	2017-01-30 17:12:14 +00:00
Erik Johnston	c7a26b7c32	Fix unit tests	2017-01-30 17:11:24 +00:00
Erik Johnston	fd1c18c088	Use DB cache of joined users for presence	2017-01-30 17:00:24 +00:00
Erik Johnston	c2c9a78db9	Noop device key changes if they're the same	2017-01-30 16:55:04 +00:00
Erik Johnston	e75a779d9e	Fix query	2017-01-30 16:38:20 +00:00
Erik Johnston	828db669ec	Use get_users_in_room and declare it iterable	2017-01-30 16:37:22 +00:00
Erik Johnston	9636b2407d	Merge pull request #1857 from matrix-org/erikj/device_list_stream ... Implement device lists updates over federation	2017-01-30 14:35:21 +00:00
Erik Johnston	3670025e64	Rename func	2017-01-30 14:11:31 +00:00
Erik Johnston	4ac363a168	Remove debug logging	2017-01-30 14:10:12 +00:00
Erik Johnston	d360c97ae1	Clear out old destination pokes.	2017-01-30 10:14:37 +00:00
Erik Johnston	76100203ab	Always use the latest stream_id, sent or unsent	2017-01-30 10:14:25 +00:00
Erik Johnston	d1e1fd6210	Add ts column to device_lists_outbound_pokes	2017-01-27 15:23:48 +00:00
Erik Johnston	252b503fc8	Hook device list updates to replication	2017-01-27 14:31:35 +00:00
Erik Johnston	84a35f32c7	Comment	2017-01-27 10:35:12 +00:00
Erik Johnston	c517a19c2d	Comment	2017-01-27 10:33:26 +00:00
Erik Johnston	738a2867c8	SQL param ordering	2017-01-27 10:31:29 +00:00
Erik Johnston	755adff0e4	User if rather than for	2017-01-27 10:31:06 +00:00
Erik Johnston	888c59c955	Better name	2017-01-27 10:29:47 +00:00
Erik Johnston	f25a4a4692	Remove unused param	2017-01-27 10:27:39 +00:00
Erik Johnston	31aca5589c	Fix on sqlite: use left rather than outer join	2017-01-26 16:55:50 +00:00
Erik Johnston	76d40f4904	Handle users leaving rooms	2017-01-26 16:39:33 +00:00
Erik Johnston	fbfad76c03	Add comments	2017-01-26 16:33:21 +00:00
Erik Johnston	c974116f19	Implement device key caching over federation	2017-01-26 16:07:24 +00:00
Erik Johnston	51e9fe36e4	Fix up sending of m.device_list_update edus	2017-01-25 16:55:21 +00:00
Erik Johnston	2367c5568c	Add basic implementation of local device list changes	2017-01-25 14:27:27 +00:00
Paul "LeoNerd" Evans	10e48d8310	Don't clobber a displayname or avatar_url if provided by an m.room.member event	2017-01-24 18:06:07 +00:00
Erik Johnston	ba8e144554	Merge branch 'erikj/current_state_fix' into develop	2017-01-23 16:15:10 +00:00
Erik Johnston	f5b46482f4	Merge pull request #1840 from matrix-org/erikj/current_state_fix ... Insert delta of current_state_events to be more efficient	2017-01-23 16:14:34 +00:00
Erik Johnston	fdf2a31a51	Typo	2017-01-23 16:14:14 +00:00
Erik Johnston	41dab8a222	Fix bug where current_state_events renamed to current_state_ids	2017-01-23 15:22:48 +00:00
Erik Johnston	c77b24c092	Refactor to calculate state delta outside transaction	2017-01-23 14:51:33 +00:00
Erik Johnston	5d2134d485	Comments	2017-01-20 17:13:24 +00:00
Erik Johnston	a55fa2047f	Insert delta of current_state_events to be more efficient	2017-01-20 17:10:18 +00:00
Erik Johnston	3d9d48fffb	Merge pull request #1836 from matrix-org/erikj/current_state_fix ... Derive current_state_events from state groups	2017-01-20 15:14:05 +00:00
Erik Johnston	d0897dead5	Spelling	2017-01-20 15:05:11 +00:00
Erik Johnston	567aa35b67	Update all call sites after rename	2017-01-20 14:40:31 +00:00
Erik Johnston	f2f40e64a9	Comments	2017-01-20 14:38:13 +00:00
Erik Johnston	4c6a31cd6e	Calculate the forward extremeties once	2017-01-20 14:28:53 +00:00
Erik Johnston	86063d4321	Merge pull request #1835 from matrix-org/erikj/fix_workers ... Make worker listener config backwards compat	2017-01-20 11:55:56 +00:00
Erik Johnston	09eb08f910	Derive current_state_events from state groups	2017-01-20 11:52:51 +00:00
Erik Johnston	97efe99ae9	Make worker listener config backwards compat	2017-01-20 11:45:29 +00:00
Marvin Steadfast	86e6165687	Added default config for turn username and password	2017-01-19 14:35:55 +01:00
Marvin Steadfast	1e38be3a7a	Added username and password for turn server ... It makes it possible to use a turn server that needs a username and password instead of a token.	2017-01-19 14:08:20 +01:00
Erik Johnston	841c228533	Merge pull request #1828 from matrix-org/erikj/iterable_cache_size ... Update LruCache size estimate on clear	2017-01-18 14:57:54 +00:00
Erik Johnston	c430111d0e	Update LruCache size estimate on clear	2017-01-18 14:55:23 +00:00
David Baker	97d3918377	Merge pull request #1811 from aperezdc/unhardcode-riot-urls ... Allow configuring the Riot URL used in notification emails	2017-01-18 14:38:49 +00:00
David Baker	8c5009b628	Lowercase all email addresses before querying db ... Since we store all emails in the DB in lowercase (https://github.com/matrix-org/synapse/pull/1170)	2017-01-18 13:25:56 +00:00
Erik Johnston	ae7b4da4cc	Merge pull request #1823 from matrix-org/erikj/load_events_logs ... Remove loading events logs	2017-01-18 11:07:58 +00:00
Erik Johnston	fc7cae8aa3	Merge pull request #1824 from matrix-org/erikj/retry_host_log ... Lower the not retrying host log line to debug	2017-01-18 11:07:51 +00:00
Erik Johnston	f9058ca785	Merge pull request #1822 from matrix-org/erikj/statE_logging ... Change resolve_state_groups call site logging to DEBUG	2017-01-18 11:02:03 +00:00
Erik Johnston	f648313f98	Merge pull request #1821 from matrix-org/erikj/cache_metrics_string_intern ... Measure metrics of string_cache	2017-01-18 10:57:39 +00:00
Erik Johnston	15f012032c	Merge pull request #1818 from matrix-org/erikj/state_auth_splitout_split ... Optimise state resolution	2017-01-18 10:53:00 +00:00
Erik Johnston	4ec1cf49e2	Lower loading events log to DEBUG	2017-01-17 17:28:32 +00:00
Erik Johnston	f878f64f43	Lower the not retrying host log line to debug	2017-01-17 17:20:39 +00:00
Erik Johnston	5f027d1fc5	Change resolve_state_groups call site logging to DEBUG	2017-01-17 17:07:15 +00:00
Erik Johnston	380dba1020	Measure metrics of string_cache	2017-01-17 17:04:46 +00:00
Erik Johnston	ed4d176152	PEP8	2017-01-17 15:27:28 +00:00
Mark Haines	c6064a7ba6	Only construct sets when necessary	2017-01-17 15:23:07 +00:00
Erik Johnston	a8594fd19f	Use better names	2017-01-17 14:59:03 +00:00
Erik Johnston	7fae460402	Merge pull request #1820 from matrix-org/erikj/push_tools ... Get state at event rather than for room in push	2017-01-17 14:58:26 +00:00
Erik Johnston	37b4c7d8a9	Fix typo in return type	2017-01-17 14:43:32 +00:00
Erik Johnston	e5d2df9c34	Use better variable name	2017-01-17 14:32:53 +00:00
Erik Johnston	04006bb7f0	Get state at event rather than for room in push	2017-01-17 14:31:21 +00:00
Erik Johnston	ce59a2faad	Correctly handle case of rejected events in state res	2017-01-17 14:18:53 +00:00
Erik Johnston	633f97151c	Check event is in state_map	2017-01-17 13:33:54 +00:00
Erik Johnston	e6153e1bd1	Fix couple of federation state bugs	2017-01-17 13:22:34 +00:00
Erik Johnston	5d6bad1b3c	Optimise state resolution	2017-01-17 13:22:19 +00:00
Erik Johnston	e8ecbb6f20	Merge pull request #1812 from matrix-org/erikj/state_auth_splitout_split ... Split out static state methods from StateHandler	2017-01-17 11:55:18 +00:00
Erik Johnston	d6c75cb7c2	Rename and comment tree_to_leaves_iterator	2017-01-17 11:47:03 +00:00
Erik Johnston	1ccd5676e3	Remove needless call to evict()	2017-01-17 11:42:26 +00:00
Erik Johnston	d906206049	Increase state_group_cache_size	2017-01-17 11:31:08 +00:00
Erik Johnston	f85b6ca494	Speed up cache size calculation ... Instead of calculating the size of the cache repeatedly, which can take a long time now that it can use a callback, instead cache the size and update that on insertion and deletion. This requires changing the cache descriptors to have two caches, one for pending deferreds and the other for the actual values. There's no reason to evict from the pending deferreds as they won't take up any more memory.	2017-01-17 11:18:13 +00:00
Erik Johnston	6d00213e80	Use OrderedDict in ExpiringCache	2017-01-16 15:33:22 +00:00
Erik Johnston	897f8752da	Up cache max entries for state	2017-01-16 15:08:17 +00:00
Erik Johnston	beda469bc6	Put staticmethods at module level	2017-01-16 15:05:24 +00:00
Erik Johnston	46aebbbcbf	Add support for 'iterable' to ExpiringCache	2017-01-16 14:57:23 +00:00
Erik Johnston	01521299c7	Increase cache size limit	2017-01-16 11:56:51 +00:00
Erik Johnston	2fae34bd2c	Optionally measure size of cache by sum of length of values	2017-01-13 17:46:17 +00:00
Erik Johnston	95a22ae194	Merge pull request #1810 from matrix-org/erikj/state_auth_splitout_split ... Split out static auth methods from Auth object	2017-01-13 16:32:27 +00:00
Erik Johnston	ec0a523ac3	Split out static state methods from StateHandler	2017-01-13 15:25:06 +00:00
Erik Johnston	e178feca3f	Remove unused function	2017-01-13 15:16:45 +00:00
Erik Johnston	f0325a9ccc	Merge pull request #1793 from matrix-org/erikj/change_device_inbox_index ... Change device_inbox stream index to include user	2017-01-13 15:14:51 +00:00
Erik Johnston	c050f493dd	Add comment	2017-01-13 15:14:41 +00:00
Adrian Perez de Castro	a3e4a198e3	Allow configuring the Riot URL used in notification emails ... The URLs used for notification emails were hardcoded to use either matrix.to or vector.im; but for self-hosted setups where Riot is also self-hosted it may be desirable to allow configuring an alternative Riot URL. Fixes #1809. Signed-off-by: Adrian Perez de Castro <aperez@igalia.com>	2017-01-13 17:12:04 +02:00
Erik Johnston	8b2fa38256	Split event auth code into seperate module	2017-01-13 15:07:32 +00:00
Erik Johnston	641ccdbb14	Merge pull request #1795 from matrix-org/erikj/port_defaults ... Restore default bind address	2017-01-13 13:02:59 +00:00
Erik Johnston	ebf94aff8d	Fix spurious Unhandled Error log lines	2017-01-12 17:19:47 +00:00
Erik Johnston	bf5c9706d9	Remove full_twisted_stacktraces option ... The debug 'full_twisted_stacktraces' flag caused synapse to rewrite twisted deferreds to always fire the callback on the next reactor tick. This was to force the deferred to always store the stacktraces on exceptions, and thus be more likely to have a full stacktrace when it reaches the final error handlers and gets printed to the logs. Dynamically rewriting things is generally bad, and in particular this change violates assumptions of various bits of Twisted. This wouldn't necessarily be so bad, but it turns out this option has been turned on on some production servers. Turning the option can cause e.g. #1778. For now, lets just entirely nuke this option.	2017-01-12 10:32:52 +00:00
Erik Johnston	7b62d0bc70	Add missing None check	2017-01-11 10:57:03 +00:00
Erik Johnston	7e6c2937c3	Split out static auth methods from Auth object	2017-01-10 18:16:54 +00:00
Erik Johnston	b1dfd20292	Pop bind_address	2017-01-10 17:23:18 +00:00
Erik Johnston	edd6cdfc9a	Restore default bind address	2017-01-10 17:21:41 +00:00
Erik Johnston	8a0fddfd73	Remove spurious for..else..	2017-01-10 16:30:53 +00:00
Erik Johnston	d524bc9110	Merge pull request #1792 from matrix-org/erikj/limit_cache_prefill_device ... Limit number of entries to prefill from cache	2017-01-10 15:42:00 +00:00
Erik Johnston	d2b00d0866	Merge pull request #1790 from matrix-org/erikj/linearizer ... Add paranoia exception catch in Linearizer	2017-01-10 15:38:30 +00:00
Erik Johnston	ab655dca33	Explicitly close the cursor	2017-01-10 15:15:25 +00:00
Erik Johnston	5a32e9273e	Don't disable autocommit	2017-01-10 15:11:27 +00:00
Erik Johnston	caddadfc5a	Change device_inbox stream index to include user ... This makes fetching the nost recently changed users much tricker, and brings it in line with e.g. presence_stream indices.	2017-01-10 15:04:57 +00:00
Erik Johnston	dd52d4de4c	Limit number of entries to prefill from cache ... Some tables, like device_inbox, take a long time to query at startup for the stream change cache prefills. This is likely because they are slower growing streams and so are more fragmented on disk. For now, lets pull fewer entries out to make startup quicker. In future, we should add a better index to make it even faster.	2017-01-10 14:34:50 +00:00
Mark Haines	32019c9897	Log which files we saved attachments to in the media_repository	2017-01-10 14:19:50 +00:00
Erik Johnston	657488113e	Merge pull request #1789 from matrix-org/erikj/decouple_presence ... Don't block messages sending on bumping presence	2017-01-10 14:06:05 +00:00
Erik Johnston	3b4de17d2b	Comment	2017-01-10 14:05:53 +00:00
Erik Johnston	7d0981b312	Merge pull request #1787 from matrix-org/erikj/linearize_member ... Linearize updates to membership via PUT /state/	2017-01-10 14:04:54 +00:00
Erik Johnston	f477370c0c	Add paranoia exception catch in Linearizer	2017-01-10 14:04:13 +00:00
Erik Johnston	586f474a44	Don't block messages sending on bumping presence	2017-01-10 12:46:00 +00:00
Erik Johnston	6823fe5241	Linearize updates to membership via PUT /state/	2017-01-09 18:25:13 +00:00
Erik Johnston	f7085ac84f	Name linearizer's for better logs	2017-01-09 17:17:10 +00:00
Erik Johnston	9898bbd9dc	Merge branch 'master' of github.com:matrix-org/synapse into develop	2017-01-09 14:51:17 +00:00
Erik Johnston	9a8ae6f1bf	Bump version and changelog	2017-01-09 14:47:56 +00:00
Matthew Hodgson	2f4b2f4783	gah, fix mangled merge of 0.18.7 into develop	2017-01-07 04:00:42 +00:00
Matthew	6d363cea9d	Merge branch 'release-v0.18.7' into develop	2017-01-07 03:46:16 +00:00
Matthew	f0e4bac64e	bump changelog & version	2017-01-07 03:45:38 +00:00
Matthew	4304e7e593	do the discard check in the right place to avoid grabbing dependent events	2017-01-07 03:44:18 +00:00
Matthew Hodgson	8c48971b51	Merge branch 'release-v0.18.7' into develop	2017-01-07 02:23:37 +00:00
Matthew	e10c527930	Discard PDUs from invalid origins due to #1753 in 0.18.[56]	2017-01-07 02:13:14 +00:00
Matthew Hodgson	2f5be2d8dc	oops, this should have been rc1	2017-01-07 01:11:56 +00:00
Matthew Hodgson	4086026524	move logging to right place	2017-01-07 00:41:46 +00:00
Matthew	19e2fb4386	bump version	2017-01-06 23:38:22 +00:00
Matthew	b2850e62db	fix typo breaking the fix to #1753	2017-01-06 23:23:37 +00:00
Mark Haines	06c00bd19b	Merge branch 'release-v0.18.6' into develop	2017-01-06 14:46:27 +00:00
Mark Haines	b42a972b71	Bump version and changelog	2017-01-06 14:44:28 +00:00
Patrik Oldsberg	1ef6084b75	handlers/room_member: fix guest access check when joining rooms ... Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>	2017-01-06 14:36:56 +00:00
Matthew Hodgson	bd85434cb3	Merge branch 'release-v0.18.6' into develop	2017-01-05 13:58:19 +00:00
Mark Haines	c18f7fc410	Fix flake8 and update changelog	2017-01-05 13:50:22 +00:00
Matthew Hodgson	dafd50d178	Merge pull request #1767 from matrix-org/matthew/resolve_state_group_logging ... log call paths for resolve_state_group	2017-01-05 13:47:42 +00:00
Matthew Hodgson	883ff92a7f	Fix case	2017-01-05 13:45:02 +00:00
Matthew Hodgson	d79d165761	add logging for all the places we call resolve_state_groups. my kingdom for a backtrace that actually works.	2017-01-05 13:40:39 +00:00
Matthew Hodgson	8cfc0165e9	fix annoying typos	2017-01-05 13:39:43 +00:00
Mark Haines	62451800e7	Bump version and changelog to v0.18.6-rc3	2017-01-05 13:36:10 +00:00
Matthew Hodgson	b31ed22738	Merge branch 'release-v0.18.6' into develop	2017-01-05 13:03:02 +00:00
Mark Haines	dd3df11c55	More logging for the linearizer and for get_events	2017-01-05 12:32:47 +00:00
Matthew Hodgson	468749c9fc	fix comment	2017-01-05 12:00:11 +00:00
Matthew Hodgson	eedf400d05	limit total timeout for get_missing_events to 10s	2017-01-05 11:58:15 +00:00
Matthew Hodgson	8e82611f37	fix comment	2017-01-05 11:44:44 +00:00
Mark Haines	f784980d2b	Only send events that originate on this server. ... Or events that are sent via the federation "send_join" API. This should match the behaviour from before v0.18.5 and #1635 landed.	2017-01-05 11:26:30 +00:00
Mark Haines	e02bdaf08b	Get the destinations from the state from before the event ... Rather than the state after then event.	2017-01-04 15:17:15 +00:00
Mark Haines	b6b67715ed	Send ALL membership events to the server that was affected. ... Send all membership changes to the server that was affected. This ensures that if the last member of a room on a server was kicked or banned they get told about it.	2017-01-04 13:56:20 +00:00
Matthew Hodgson	555d702e34	limit total timeout for get_missing_events to 10s	2016-12-31 15:21:37 +00:00
Matthew Hodgson	899a3a1268	Merge branch 'release-v0.18.6' into develop	2016-12-31 02:38:26 +00:00
Mark Haines	f3de4f8cb7	Bump version and changelog	2016-12-30 20:21:04 +00:00
Mark Haines	62ce3034f3	s/aquire/acquire/g	2016-12-30 20:04:44 +00:00
Mark Haines	0aff09f6c9	Add more useful logging when we block fetching events	2016-12-30 20:00:44 +00:00
Mark Haines	cc50b1ae53	Remove fallback from get_missing_events. ... get_missing_events used to fallback to fetching the missing events individually requesting from every server in the room, one by one.e This could be unacceptably slow, possibly causing #1732	2016-12-30 18:13:15 +00:00
Mark Haines	f576c34594	Merge remote-tracking branch 'origin/release-v0.18.6' into develop	2016-12-30 15:13:49 +00:00
Mark Haines	822cb39dfa	Use the new twisted logging framework. ... Hopefully adding an observer to the new framework will avoid a memory leak https://twistedmatrix.com/trac/ticket/8164	2016-12-30 11:09:24 +00:00
Mark Haines	342fb8dae9	Merge branch 'release-v0.18.6' into develop	2016-12-29 17:33:46 +00:00
Mark Haines	f023be9293	Bump changelog and version	2016-12-29 16:18:04 +00:00
Mark Haines	97ffc5690b	Manually abort the underlying TLS connection. ... The abort() method calls loseConnection() which tries to shutdown the TLS connection cleanly. We now call abortConnection() directly which should promptly close both the TLS connection and the underlying TCP connection. I also added some TODO markers to consider cancelling the old previous timeout rather than checking time.time(). But given how urgently we want to get this code released I'd rather leave the existing code with the duplicate timeouts and the time.time() check.	2016-12-29 15:51:04 +00:00
Erik Johnston	b4bc6fef5b	Respect long_retries param and default to off	2016-12-29 00:58:34 +00:00
Erik Johnston	68030fd37b	Spelling and comments	2016-12-29 00:10:49 +00:00
Erik Johnston	b7336ff32d	Clean up	2016-12-29 00:09:33 +00:00
Erik Johnston	5b6672c66d	Wrap connections in an N minute timeout to ensure they get reaped correctly	2016-12-29 00:06:53 +00:00
David Baker	84cf00c645	Fix another comment typo	2016-12-21 09:51:43 +00:00
David Baker	0c88ab1844	Add /account/3pid/delete endpoint ... Also fix a typo in a comment	2016-12-20 18:27:30 +00:00
Johannes Löthberg	702c020e58	Fix check for bind_address ... The empty string is a valid setting for the bind_address option, so explicitly check for None here instead. Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>	2016-12-20 01:37:50 +01:00
Matthew Hodgson	09f15918be	Merge pull request #1711 from matrix-org/matthew/utf8-password-change ... fix ability to change password to a non-ascii one	2016-12-20 00:02:13 +00:00
Matthew Hodgson	da2c8f3c94	Merge pull request #1709 from kyrias/bind_addresses ... Add support for specifying multiple bind addresses	2016-12-19 23:49:34 +00:00
Matthew Hodgson	a58e4e0d48	Merge pull request #1696 from kyrias/ipv6 ... IPv6 support	2016-12-19 23:49:07 +00:00
Matthew Hodgson	f2a5aebf98	fix ability to change password to a non-ascii one ... https://github.com/vector-im/riot-web/issues/2658	2016-12-18 22:25:21 +00:00
Johannes Löthberg	a9c1b419a9	Bump twisted dependency ... At least 16.0.0 is needed for wrapClientTLS support. Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>	2016-12-18 23:16:43 +01:00
Johannes Löthberg	f5cd5ebd7b	Add IPv6 comment to default config ... Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>	2016-12-18 23:14:32 +01:00
Johannes Löthberg	c95e9fff99	Make default homeserver config use bind_addresses ... Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>	2016-12-18 21:51:56 +01:00
Johannes Löthberg	7dfd70fc83	Add support for specifying multiple bind addresses ... Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>	2016-12-18 21:51:56 +01:00
Erik Johnston	b2f8642d3d	Cache network room list queries.	2016-12-16 16:11:43 +00:00
Erik Johnston	b9b6d17ab1	Bump version and changelog	2016-12-16 10:18:02 +00:00
Richard van der Hoff	edc6a1e4f9	Add some logging for syncing to_device events ... Attempt to track down the loss of to_device events (https://github.com/vector-im/riot-web/issues/2711 etc).	2016-12-15 18:16:10 +00:00
Erik Johnston	35129ac998	Merge pull request #1698 from matrix-org/erikj/room_list ... Fix caching on public room list	2016-12-15 15:40:28 +00:00
Erik Johnston	aa1336c00a	Merge pull request #1700 from matrix-org/erikj/backfill_filter ... Fix /backfill returning events it shouldn't	2016-12-15 14:21:30 +00:00
Marcin Bachry	24c16fc349	Fix crash in url preview when html tag has no text ... Signed-off-by: Marcin Bachry <hegel666@gmail.com>	2016-12-14 22:38:18 +01:00
Erik Johnston	b8255eba26	Comment	2016-12-14 13:49:54 +00:00
Erik Johnston	b2999a7055	Fix /backfill returning events it shouldn't	2016-12-14 13:41:45 +00:00
Erik Johnston	c3208e45c9	Fixup membership query	2016-12-14 10:46:58 +00:00
Erik Johnston	9d95351cad	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/room_list	2016-12-13 17:57:08 +00:00
Erik Johnston	1de53a7a1a	Fix caching on public room list	2016-12-13 17:33:24 +00:00
Erik Johnston	b3d398343e	Bump changelog and version	2016-12-13 11:07:27 +00:00
Johannes Löthberg	0648e76979	Remove spurious newline ... Apparently I just removed the spaces instead... Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>	2016-12-12 18:41:30 +01:00
Erik Johnston	8588d0eb3d	Merge pull request #1697 from matrix-org/erikj/fix_bg_member ... Fix background update that prematurely stopped	2016-12-12 17:20:17 +00:00
Erik Johnston	1574b839e0	Merge pull request #1676 from matrix-org/erikj/room_list ... Add new API appservice specific public room list	2016-12-12 17:00:10 +00:00
Erik Johnston	7ec2bf9b77	Fix background update that prematurely stopped	2016-12-12 16:54:58 +00:00
Erik Johnston	2bf5a47b3e	Rename network_id to instance_id on client side	2016-12-12 16:05:45 +00:00
Johannes Löthberg	d3bd94805f	Fixup for #1689 and #1690 ... Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>	2016-12-12 16:32:47 +01:00
Erik Johnston	09cbcb78d3	Add cache to get_public_room_ids_at_stream_id	2016-12-12 14:41:51 +00:00
Erik Johnston	631376e2ac	Notify replication. Use correct network_id	2016-12-12 14:28:15 +00:00
Richard van der Hoff	abed247182	Remove unspecced GET endpoints for e2e keys ... GET /keys/claim is a terrible idea, since it isn't idempotent; also it throws 500 errors if you call it without all the right params. GET /keys/query is arguable, but it's unspecced, so let's get rid of it too to stop people relying on unspecced APIs.	2016-12-12 12:31:40 +00:00
Erik Johnston	d45c984653	Docstring	2016-12-12 11:00:27 +00:00
Erik Johnston	d53a80af25	Merge pull request #1620 from matrix-org/erikj/concurrent_room_access ... Limit the number of events that can be created on a given room concurrently	2016-12-12 10:30:23 +00:00
Richard van der Hoff	85cd30b1fd	Merge pull request #1686 from matrix-org/rav/fix_federation_key_fails ... E2E key query: handle federation fails	2016-12-12 09:33:39 +00:00
Richard van der Hoff	deca951241	Remove unused import	2016-12-12 09:24:35 +00:00
Glyph	9f07f4c559	IPv6 support for endpoint.py ... Similar to https://github.com/matrix-org/synapse/pull/1689, but for endpoint.py	2016-12-11 11:10:32 +01:00
Glyph	6e18805ac2	IPv6 support for client.py ... This is an (untested) general sketch of how to use wrapClientTLS to implement TLS over IPv6, as well as faster connections over IPv4.	2016-12-11 11:10:32 +01:00
Richard van der Hoff	efa4ccfaee	E2E key query: handle federation fails ... Don't fail the whole request if we can't connect to a particular server.	2016-12-09 18:31:01 +00:00
Erik Johnston	fbaf868f62	Correctly handle timeout errors	2016-12-09 16:30:29 +00:00
Erik Johnston	be14c24cea	Fix rare notifier bug where listeners dont timeout ... There was a race condition that caused the notifier to 'miss' the timeout notification, since there were no other checks for the timeout this caused listeners to get stuck in a loop until something happened.	2016-12-09 15:43:18 +00:00
Erik Johnston	1697f6a323	Merge pull request #1680 from matrix-org/erikj/joined_rooms ... Add new room membership APIs	2016-12-09 11:35:48 +00:00
Erik Johnston	52d12ca782	Add /room/<room_id>/joined_members API ... This returns the currently joined members in the room with their display names and avatar urls. This is more efficient than /members for large rooms where you don't need the full events.	2016-12-08 13:32:07 +00:00
Erik Johnston	c45d8e9ba2	Add profile data to the room_membership table for joins	2016-12-08 13:08:41 +00:00
Richard van der Hoff	b08f76bd23	Fix ignored read-receipts ... Don't ignore read-receipts which arrive in the same EDU as a read-receipt for an old event.	2016-12-08 12:13:01 +00:00
Richard van der Hoff	bd07a35c29	Fix result of insert_receipt ... This should fix the absence of notifications when new receipts arrive.	2016-12-08 12:11:34 +00:00
Erik Johnston	de796f27e6	Add joined_rooms servlet	2016-12-08 11:39:03 +00:00
Erik Johnston	2687af82d4	Comments	2016-12-07 09:58:33 +00:00
Erik Johnston	3727d66a0e	Don't include appservice id	2016-12-06 17:04:26 +00:00
Erik Johnston	f32fb65552	Add new API appservice specific public room list	2016-12-06 16:12:27 +00:00
Richard van der Hoff	1529c19675	Prevent user tokens being used as guest tokens (#1675) ... Make sure that a user cannot pretend to be a guest by adding 'guest = True' caveats.	2016-12-06 15:31:37 +00:00
Erik Johnston	9609c91e7d	Merge pull request #653 from matrix-org/erikj/preset_guest_join ... Enable guest access for private rooms by default	2016-12-05 17:47:14 +00:00
Erik Johnston	338df4f409	Merge pull request #1649 from matrix-org/dbkr/log_ui_auth_args ... Log the args that we have on UI auth completion	2016-12-05 16:40:58 +00:00
pik	c46e7a9c9b	Bugfix: Console logging handler missing default filter	2016-12-03 20:14:58 -03:00
Johannes Löthberg	32c8b5507c	preview_url_resource: Ellipsis must be in unicode string ... Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>	2016-12-01 13:12:13 +01:00
Richard van der Hoff	6841d8ff55	Fix doc-string ... Remove refresh_token reference	2016-12-01 11:42:17 +00:00
Richard van der Hoff	aa09d6b8f0	Rip out more refresh_token code ... We might as well treat all refresh_tokens as invalid. Just return a 403 from /tokenrefresh, so that we don't have a load of dead, untestable code hanging around. Still TODO: removing the table from the schema.	2016-11-30 17:40:18 +00:00
Richard van der Hoff	dc4b23e1a1	Merge branch 'develop' into rav/no_more_refresh_tokens	2016-11-30 17:10:04 +00:00
Richard van der Hoff	8379a741cc	Merge pull request #1660 from matrix-org/rav/better_content_type_validation ... More intelligent Content-Type parsing	2016-11-30 16:54:03 +00:00
Richard van der Hoff	321fe5c44c	Merge pull request #1656 from matrix-org/rav/remove_time_caveat ... Stop putting a time caveat on access tokens	2016-11-30 16:53:20 +00:00
Richard van der Hoff	b5b3a7e867	More intelligent Content-Type parsing ... Content-Type is allowed to contain options (`; charset=utf-8`, for instance). We should allow that.	2016-11-30 15:07:32 +00:00
Richard van der Hoff	4febfe47f0	Comments ... Update comments in verify_macaroon	2016-11-30 07:36:32 +00:00
Richard van der Hoff	77eca2487c	Merge pull request #1653 from matrix-org/rav/guest_e2e ... Implement E2E for guests	2016-11-29 17:41:35 +00:00
Richard van der Hoff	1c4f05db41	Stop putting a time caveat on access tokens ... The 'time' caveat on the access tokens was something of a lie, since we weren't enforcing it; more pertinently its presence stops us ever adding useful time caveats. Let's move in the right direction by not lying in our caveats.	2016-11-29 16:49:41 +00:00
Richard van der Hoff	5c4edc83b5	Stop generating refresh tokens ... Since we're not doing refresh tokens any more, we should start killing off the dead code paths. /tokenrefresh itself is a bit of a thornier subject, since there might be apps out there using it, but we can at least not generate refresh tokens on new logins.	2016-11-28 10:13:01 +00:00
Richard van der Hoff	b6146537d2	Merge pull request #1655 from matrix-org/rav/remove_redundant_macaroon_checks ... Remove redundant list of known caveat prefixes	2016-11-25 16:57:19 +00:00
Richard van der Hoff	f62b69e32a	Allow guest access to endpoints for E2E ... Expose /devices, /keys, and /sendToDevice to guest users, so that they can use E2E.	2016-11-25 15:26:34 +00:00
Richard van der Hoff	7f02e4d008	Give guest users a device_id ... We need to create devices for guests so that they can use e2e, but we don't have anywhere to store it, so just use a fixed one.	2016-11-25 15:25:30 +00:00
Erik Johnston	11bfe438a2	Use correct var	2016-11-24 15:26:53 +00:00
Erik Johnston	aaecffba3a	Correctly handle 500's and 429 on federation	2016-11-24 15:04:49 +00:00
Richard van der Hoff	e1d7c96814	Remove redundant list of known caveat prefixes ... Also add some comments.	2016-11-24 12:38:17 +00:00
Erik Johnston	7e03f9a484	Bump version and changelog	2016-11-24 12:29:58 +00:00
Erik Johnston	46ca345b06	Don't send old events as federation	2016-11-24 12:29:02 +00:00
Erik Johnston	f36ea03741	Bump changelog and version	2016-11-24 11:08:01 +00:00
David Baker	c9d4e7b716	Clarify that creds doesn not contain passwords.	2016-11-24 10:54:59 +00:00
David Baker	f681aab895	Log the args that we have on UI auth completion ... This will be super helpful for debugging if we have more registration woes.	2016-11-24 10:11:45 +00:00
Erik Johnston	1985860c6e	Comment	2016-11-23 15:59:59 +00:00
Erik Johnston	2ac516850b	More efficient notif count queries	2016-11-23 15:57:04 +00:00
Erik Johnston	302fbd218d	Merge pull request #1635 from matrix-org/erikj/split_out_fed_txn ... Split out federation transaction sending to a worker	2016-11-23 15:39:12 +00:00
Erik Johnston	b2d6e63b79	Merge pull request #1641 from matrix-org/erikj/as_pushers ... Ignore AS users when fetching push rules	2016-11-23 15:21:52 +00:00
Erik Johnston	feec718265	Shuffle receipt handler around so that worker apps don't need to load it	2016-11-23 15:14:24 +00:00
Erik Johnston	26072df6af	Ensure only main or federation_sender process can send federation traffic	2016-11-23 14:09:47 +00:00
Erik Johnston	b69f76c106	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/split_out_fed_txn	2016-11-23 11:31:53 +00:00
Erik Johnston	4d9b5c60f9	Comment	2016-11-23 11:11:41 +00:00
Erik Johnston	0163466d72	Ignore AS users when fetching push rules ... By ignoring AS users early on when fetching push rules for a room we can avoid needlessly hitting the DB and filling up the caches.	2016-11-23 11:01:01 +00:00
Erik Johnston	4c79a63fd7	Explicit federation ack	2016-11-23 10:40:44 +00:00
Erik Johnston	54fed21c04	Fix tests and flake8	2016-11-22 18:18:31 +00:00
Erik Johnston	90565d015e	Invalidate retry cache in both directions	2016-11-22 17:45:44 +00:00
Kegan Dougal	83bcdcee61	Return early on /sync code paths if a '*' filter is used ... This is currently very conservative in that it only does this if there is no `since` token. This limits the risk to clients likely to be doing one-off syncs (like bridges), but does mean that normal human clients won't benefit from the time savings here. If the savings are large enough, I would consider generalising this to just check the filter.	2016-11-22 16:38:35 +00:00
Kegsay	d4a459f7cb	Merge pull request #1638 from matrix-org/kegan/sync-event-fields ... Implement "event_fields" in filters	2016-11-22 14:02:38 +00:00
Kegan Dougal	c3d963ac24	Review comments	2016-11-22 13:42:11 +00:00
Kegan Dougal	6d4e6d4cba	Also check for dict since sometimes they aren't frozen	2016-11-22 10:39:41 +00:00
Erik Johnston	baf9e74a73	Merge branch 'master' of github.com:matrix-org/synapse into develop	2016-11-22 10:31:48 +00:00
Kegan Dougal	cea4e4e7b2	Glue only_event_fields into the sync rest servlet	2016-11-22 10:14:05 +00:00
Kegan Dougal	0a8b0eeca1	More tests	2016-11-22 09:59:27 +00:00
Erik Johnston	51e89709aa	Comments	2016-11-21 17:59:39 +00:00
Kegan Dougal	70a2157b64	Start adding some tests	2016-11-21 17:52:45 +00:00
Kegan Dougal	f97511a1f3	Move event_fields filtering to serialize_event ... Also make it an inclusive not exclusive filter, as the spec demands.	2016-11-21 17:42:16 +00:00
Erik Johnston	88d85ebae1	Add some metrics	2016-11-21 17:36:05 +00:00
Erik Johnston	50934ce460	Comments	2016-11-21 16:55:23 +00:00
Kegan Dougal	e90fcd9edd	Add filter_event_fields and filter_field to FilterCollection	2016-11-21 15:18:18 +00:00
Erik Johnston	9687e039e7	Remove explicit calls to send_pdu	2016-11-21 14:48:51 +00:00
Kegan Dougal	a2a6c1c22f	Fail with a coherent error message if /sync?filter= is invalid	2016-11-21 13:15:25 +00:00
Erik Johnston	524d61bf7e	Fix tests	2016-11-21 11:53:02 +00:00
Erik Johnston	7c9cdb2245	Store federation stream positions in the database	2016-11-21 11:33:08 +00:00
Mark Haines	a289150943	Fix flake8	2016-11-18 17:15:02 +00:00
David Baker	544722bad2	Work around client replacing reg params ... Works around https://github.com/vector-im/vector-android/issues/715 and equivalent for iOS	2016-11-18 17:07:35 +00:00
Erik Johnston	f8ee66250a	Handle sending events and device messages over federation	2016-11-17 15:48:04 +00:00
Erik Johnston	ed787cf09e	Hook up the send queue and create a federation sender worker	2016-11-16 17:34:44 +00:00
Erik Johnston	1587b5a033	Add initial cut of federation send queue	2016-11-16 14:47:52 +00:00
Erik Johnston	59ef517e6b	Use new federation_sender DI	2016-11-16 14:47:52 +00:00
Erik Johnston	847d5db1d1	Add transaction queue and transport layer to DI	2016-11-16 14:47:52 +00:00
Erik Johnston	daec6fc355	Move logic into transaction_queue	2016-11-16 14:47:52 +00:00
Erik Johnston	0e830d3770	Rename transaction queue functions to send_*	2016-11-16 14:47:52 +00:00
Erik Johnston	dc6cede78e	Merge pull request #1628 from matrix-org/erikj/ldap_split_out ... Use external ldap auth pacakge	2016-11-15 16:53:34 +00:00
Erik Johnston	c7546b3cdb	Merge pull request #1617 from matrix-org/erikj/intern_state_dict ... Correctly intern keys in state cache	2016-11-15 16:45:55 +00:00
Erik Johnston	d56c39cf24	Use external ldap auth pacakge	2016-11-15 13:03:19 +00:00
Erik Johnston	f9d156d270	New Flake8 fixes	2016-11-15 11:22:29 +00:00
Erik Johnston	9d58ccc547	Bump changelog and version	2016-11-14 15:05:04 +00:00
Kegsay	9355a5c42b	Merge pull request #1624 from matrix-org/kegan/idempotent-requests ... Store Promise<Response> instead of Response for HTTP API transactions	2016-11-14 12:45:30 +00:00
Kegan Dougal	3991b4cbdb	Clean transactions based on time. Add HttpTransactionCache tests.	2016-11-14 11:19:24 +00:00
Kegan Dougal	af4a1bac50	Move .observe() up to the cache to make things neater	2016-11-14 09:52:41 +00:00
Erik Johnston	0964005d84	Merge pull request #1625 from DanielDent/patch-1 ... Add support for durations in minutes	2016-11-12 11:20:46 +00:00
Daniel Dent	1c93cd9f9f	Add support for durations in minutes	2016-11-12 00:10:23 -08:00
Kegan Dougal	8ecaff51a1	Review comments	2016-11-11 17:47:03 +00:00
Kegan Dougal	f6c48802f5	More flake8	2016-11-11 15:08:24 +00:00
Kegan Dougal	a88bc67f88	Flake8 and fix whoopsie	2016-11-11 15:02:29 +00:00
Kegan Dougal	42c43cfafd	Use ObservableDeferreds instead of Deferreds as they behave as intended	2016-11-11 14:54:10 +00:00
Kegan Dougal	c7daf3136c	Use observable deferreds because they are sane	2016-11-11 14:13:32 +00:00
Erik Johnston	64038b806c	Comments	2016-11-11 10:42:08 +00:00
Erik Johnston	2bd4513a4d	Limit the number of events that can be created on a given room concurretnly	2016-11-10 16:44:35 +00:00
Erik Johnston	d073cb7ead	Add Limiter: limit concurrent access to resource	2016-11-10 16:29:51 +00:00
Kegan Dougal	8a8ad46f48	Flake8	2016-11-10 15:22:11 +00:00
Kegan Dougal	2771447c29	Store Promise<Response> instead of Response for HTTP API transactions ... This fixes a race whereby: - User hits an endpoint. - No cached transaction so executes main code. - User hits same endpoint. - No cache transaction so executes main code. - Main code finishes executing and caches response and returns. - Main code finishes executing and caches response and returns. This race is common in the wild when Synapse is struggling under load. This commit fixes the race by: - User hits an endpoint. - Caches the promise to execute the main code and executes main code. - User hits same endpoint. - Yields on the same promise as the first request. - Main code finishes executing and returns, unblocking both requests.	2016-11-10 14:49:26 +00:00
Erik Johnston	ac507e7ab8	Don't assume providers raise ConfigError's	2016-11-08 17:23:28 +00:00
Erik Johnston	e6651e8046	Merge branch 'master' of github.com:matrix-org/synapse into develop	2016-11-08 14:43:49 +00:00
Erik Johnston	3c09818d91	Bump version and changelog	2016-11-08 14:39:55 +00:00
Erik Johnston	27d3f2e7ab	Explicitly set authentication mode in ldap3 ... This only makes a difference for versions of ldap3 before 1.0, but a) its best to be explicit and b) there are distributions that package ancient versions for ldap3 (e.g. debian).	2016-11-08 14:35:25 +00:00
Erik Johnston	17e0a58020	Merge pull request #1615 from matrix-org/erikj/limit_prev_events ... Limit the number of prev_events of new events	2016-11-08 12:06:15 +00:00
Erik Johnston	587d8ac60f	Correctly intern keys in state cache	2016-11-08 11:53:25 +00:00
Erik Johnston	a4632783fb	Sample correctly	2016-11-08 11:20:26 +00:00
Erik Johnston	24772ba56e	Respect use_frozen_dicts option in workers	2016-11-08 11:07:18 +00:00
Erik Johnston	eeda4e618c	Limit the number of prev_events of new events	2016-11-08 11:02:29 +00:00
Erik Johnston	d24197bead	Merge pull request #1198 from euank/more-ip-blacklist ... default config: blacklist more internal ips	2016-11-07 09:41:34 +00:00
Euan Kemp	c6bbad109b	default config: blacklist more internal ips	2016-11-06 17:02:25 -08:00
Erik Johnston	63772443e6	Comment	2016-11-04 10:53:42 +00:00
Erik Johnston	a3f6576084	Remove unused but buggy function	2016-11-04 10:48:20 +00:00
Paul "LeoNerd" Evans	2938a00825	Rename the python-specific metrics now the docs claim that we have done	2016-11-03 17:03:52 +00:00
Paul "LeoNerd" Evans	5219f7e060	Since we don't export per-filetype fd counts any more, delete all the code related to that too	2016-11-03 16:41:32 +00:00
Paul "LeoNerd" Evans	93ebeb2aa8	Remove now-unused 'resource' import	2016-11-03 16:37:09 +00:00
Paul "LeoNerd" Evans	c1b077cd19	Now we have new-style metrics don't bother exporting legacy-named process ones	2016-11-03 16:34:16 +00:00
Erik Johnston	64c6566980	Remove spurious comment	2016-11-03 15:04:32 +00:00
Erik Johnston	8fd4d9129f	Replace postgres GIN with GIST ... This is because GIN can be slow to write too, especially when the table gets large.	2016-11-03 15:00:03 +00:00
David Baker	9084720993	Don't error on non-ascii passwords	2016-11-03 10:42:14 +00:00
Mark Haines	b1c27975d0	Set CORs headers on responses from the media repo	2016-11-02 11:29:25 +00:00
Erik Johnston	2746e805fe	Merge pull request #1188 from matrix-org/erikj/sent_transactions ... Remove sent_transactions table.	2016-11-01 14:52:55 +00:00
Erik Johnston	3c91c5b216	Bump version and changelog	2016-11-01 13:06:36 +00:00
Erik Johnston	760469c812	Continue to clean up received_transactions	2016-11-01 11:42:08 +00:00
Erik Johnston	f09d2b692f	Removed unused stuff	2016-10-31 17:10:56 +00:00
Erik Johnston	4c3eb14d68	Increase batching of sent transaction inserts ... This should further reduce the number of individual inserts, transactions and updates that are required for keeping sent_transactions up to date.	2016-10-31 16:07:45 +00:00
Mark Haines	82e71a259c	Bump changelog and version	2016-10-28 11:16:05 +01:00
Mark Haines	490b97d3e7	Merge branch 'develop' into release-v0.18.2	2016-10-28 11:12:31 +01:00
Paul "LeoNerd" Evans	1cc22da600	Set up the process collector during metrics __init__; that way all split-process workers have it	2016-10-27 18:09:34 +01:00
Paul "LeoNerd" Evans	aac13b1f9a	Pass the Metrics group into the process collector instead of having it find its own one; this avoids it needing to import from synapse.metrics	2016-10-27 18:08:15 +01:00
Paul "LeoNerd" Evans	ccc1a3d54d	Allow creation of a 'subspace' within a Metrics object, returning another one	2016-10-27 18:07:34 +01:00
Erik Johnston	665e53524e	Bump changelog and version	2016-10-27 14:52:47 +01:00
Erik Johnston	a9111786f9	Use most recently added binding, not most recently seen user.	2016-10-27 14:32:45 +01:00
Erik Johnston	1fc1bc2a51	Fix user_threepids schema delta ... The delta `37/user_threepids.sql` aimed to update all the email addresses to be lower case, however duplicate emails may exist in the table already. This commit adds a step where the delta moves the duplicate emails to a new `medium` `email_old`. Only the most recently used account keeps the binding intact. We move rather than delete so that we retain some record of which emails were associated with which account.	2016-10-27 14:14:44 +01:00
Erik Johnston	ab731d8f8e	Bump changelog and version	2016-10-27 11:07:02 +01:00
Mark Haines	177f104432	Merge pull request #1098 from matrix-org/markjh/bearer_token ... Allow clients to supply access_tokens as headers	2016-10-25 17:33:15 +01:00
Erik Johnston	22fbf86e4f	Merge branch 'release-v0.18.2' of github.com:matrix-org/synapse into develop	2016-10-25 15:09:46 +01:00
Erik Johnston	855645c719	Bump version and changelog	2016-10-25 11:16:16 +01:00
Erik Johnston	2ef617bc06	Fix infinite typing bug ... There's a bug somewhere that causes typing notifications to not be timed out properly. By adding a paranoia timer and using correct inequalities notifications should stop being stuck, even if it the root cause hasn't been fixed.	2016-10-24 15:51:22 +01:00
Erik Johnston	b6800a8ecd	Actually use the new function	2016-10-24 13:39:49 +01:00
Erik Johnston	d04e2ff3a4	Fix incredubly slow back pagination query ... If a client didn't specify a from token when paginating backwards synapse would attempt to query the (global) maximum topological token. This a) doesn't make much sense since they're room specific and b) there are no indices that lets postgres do this efficiently.	2016-10-24 13:35:51 +01:00
Paul Evans	a842fed418	Merge pull request #1177 from matrix-org/paul/standard-metric-names ... Standardise prometheus metrics	2016-10-21 13:06:19 +01:00
Luke Barnard	e01a1bc92d	Merge pull request #1175 from matrix-org/luke/feature-configurable-as-rate-limiting ... Allow Configurable Rate Limiting Per AS	2016-10-20 16:21:10 +01:00
Luke Barnard	6fdd31915b	Style	2016-10-20 13:53:15 +01:00
Luke Barnard	07caa749bf	Closing brace on following line	2016-10-20 12:07:16 +01:00
Luke Barnard	f09db236b1	as_user->app_service, less redundant comments, better positioned comments	2016-10-20 12:04:54 +01:00
Luke Barnard	8bfd01f619	flake8	2016-10-20 11:52:46 +01:00
Luke Barnard	1b17d1a106	Use real AS object by passing it through the requester ... This means synapse does not have to check if the AS is interested, but instead it effectively re-uses what it already knew about the requesting user	2016-10-20 11:43:05 +01:00
Paul "LeoNerd" Evans	b01aaadd48	Split callback metric lambda functions down onto their own lines to keep line lengths under 90	2016-10-19 18:26:13 +01:00
Paul "LeoNerd" Evans	1071c7d963	Adjust code for <100 char line limit	2016-10-19 18:23:25 +01:00
Paul "LeoNerd" Evans	6453d03edd	Cut the raw /proc/self/stat line up into named fields at collection time	2016-10-19 18:21:40 +01:00
Paul "LeoNerd" Evans	3ae48a1f99	Move the process metrics collector code into its own file	2016-10-19 18:10:24 +01:00
Paul "LeoNerd" Evans	4cedd53224	A slightly neater way to manage metric collector functions	2016-10-19 17:54:09 +01:00
Paul "LeoNerd" Evans	5663137e03	appease pep8	2016-10-19 16:09:42 +01:00
Paul "LeoNerd" Evans	b202531be6	Also guard /proc/self/fds-related code with a suitable psuedoconstant	2016-10-19 15:37:41 +01:00
Paul "LeoNerd" Evans	1b179455fc	Guard registration of process-wide metrics by existence of the requisite /proc entries	2016-10-19 15:34:38 +01:00
Paul "LeoNerd" Evans	981f852d54	Add standard process_start_time_seconds metric	2016-10-19 15:05:22 +01:00
Paul "LeoNerd" Evans	def63649df	Add standard process_max_fds metric	2016-10-19 15:05:21 +01:00
Paul "LeoNerd" Evans	06f1ad1625	Add standard process_open_fds metric	2016-10-19 15:05:21 +01:00
Paul "LeoNerd" Evans	95fc70216d	Add standard process_*_memory_bytes metrics	2016-10-19 15:05:21 +01:00
Paul "LeoNerd" Evans	9b0316c75a	Use /proc/self/stat to generate the new process_cpu_*_seconds_total metrics	2016-10-19 15:05:21 +01:00
Paul "LeoNerd" Evans	03c2720940	Export CPU usage metrics also under prometheus-standard metric name	2016-10-19 15:05:21 +01:00
Paul "LeoNerd" Evans	b21b9dbc37	Callback metric values might not just be integers - allow floats	2016-10-19 15:05:15 +01:00
Erik Johnston	78c083f159	Merge pull request #1164 from pik/error-codes ... Clarify Error codes for GET /filter/	2016-10-19 14:26:17 +01:00
Erik Johnston	3aa8925091	Merge pull request #1176 from matrix-org/erikj/eager_ratelimit_check ... Check whether to ratelimit sooner to avoid work	2016-10-19 14:25:52 +01:00
Erik Johnston	f2f74ffce6	Comment	2016-10-19 14:21:28 +01:00
David Baker	0108ed8ae6	Latest delta is now 37	2016-10-19 11:40:35 +01:00
David Baker	a7f48320b1	Merge remote-tracking branch 'origin/develop' into dbkr/password_reset_case_insensitive	2016-10-19 11:28:56 +01:00
David Baker	df2a616c7b	Convert emails to lowercase when storing ... And db migration sql to convert existing addresses.	2016-10-19 11:13:55 +01:00
Erik Johnston	550308c7a1	Check whether to ratelimit sooner to avoid work	2016-10-19 10:45:24 +01:00
pik	e8b1d2a452	Refactor test_filter to use real DataStore ... * add tests for filter api errors	2016-10-18 12:17:38 -05:00
Luke Barnard	5b54d51d1e	Allow Configurable Rate Limiting Per AS ... This adds a flag loaded from the registration file of an AS that will determine whether or not its users are rate limited (by ratelimit in _base.py). Needed for IRC bridge reasons - see https://github.com/matrix-org/matrix-appservice-irc/issues/240.	2016-10-18 17:04:09 +01:00
Erik Johnston	8ca05b5755	Fix push notifications for a single unread message	2016-10-18 10:57:33 +01:00
Erik Johnston	f0ca088280	Reduce redundant database work in email pusher ... Update the last stream ordering if the `get_unread_push_actions_for_user_in_range_for_email` returns no new push actions. This reduces the range that it needs to check next iteration.	2016-10-18 10:52:47 +01:00
Erik Johnston	b95dbdcba4	Bump version and changelog	2016-10-17 15:59:12 +01:00
Erik Johnston	6942d68247	Bump schema version	2016-10-17 11:17:45 +01:00
Erik Johnston	b59994b454	Remove TODO	2016-10-17 11:17:02 +01:00
Erik Johnston	816988baaa	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/remove_auth	2016-10-17 11:10:37 +01:00
Erik Johnston	2869a29fd7	Drop some unused indices	2016-10-17 11:08:19 +01:00
Erik Johnston	a68ade6ed3	Merge pull request #1162 from larroy/master ... Use sys.executable instead of hardcoded python. fixes #1161	2016-10-14 21:42:55 +01:00
David Baker	29c5922021	Revert part of 6207399 ... older sqlite doesn't support indexes on expressions, lets just store things lowercase in the db	2016-10-14 16:20:24 +01:00
Alexander Maznev	d9350b0db8	Error codes for filters ... * add tests Signed-off-by: Alexander Maznev <alexander.maznev@gmail.com>	2016-10-14 10:18:28 -05:00
David Baker	bcb1245a2d	Merge remote-tracking branch 'origin/develop' into dbkr/password_reset_case_insensitive	2016-10-14 15:10:38 +01:00
David Baker	62073992c5	Make password reset email field case insensitive	2016-10-14 13:56:53 +01:00
Erik Johnston	6f7540ada4	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/fix_email_notifs	2016-10-14 10:22:43 +01:00
Erik Johnston	1d107d8484	Fix email push notifs being dropped ... A lot of email push notifications were failing to be sent due to an exception being thrown along one of the (many) paths. This was due to a change where we moved from pulling out the full state for each room, but rather pulled out the event ids for the state and separately loaded the full events when needed.	2016-10-13 13:40:38 +01:00
Richard van der Hoff	f7aed3d7a2	Merge pull request #1168 from matrix-org/rav/ui_auth_on_device_delete ... User-interactive auth on delete device	2016-10-13 09:38:41 +01:00
Richard van der Hoff	9009143fb9	Handle delete device requests with no body ... We should probably return a 401 rather than a 400 for existing clients that don't know they have to do the UIA dance to delete a device.	2016-10-12 18:47:28 +01:00
Richard van der Hoff	fbd3866bc6	User-interactive auth on delete device	2016-10-12 16:16:31 +01:00