6537 Commits

Author	SHA1	Message	Date
Erik Johnston	cdbf38728d	Merge pull request #2175 from matrix-org/erikj/prefill_state ... Prefill state caches	2017-05-03 10:54:11 +01:00
Erik Johnston	0c27383dd7	Merge pull request #2170 from matrix-org/erikj/fed_hole_state ... Don't fetch state for missing events that we fetched	2017-05-03 10:49:37 +01:00
Erik Johnston	ef862186dd	Merge together redundant calculations/logging	2017-05-03 10:06:43 +01:00
Erik Johnston	2c2dcf81d0	Update comment	2017-05-03 10:00:29 +01:00
Erik Johnston	1827057acc	Comments	2017-05-03 09:56:05 +01:00
Erik Johnston	8346e6e696	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/prefill_state	2017-05-03 09:46:40 +01:00
Erik Johnston	e4c15fcb5c	Merge pull request #2178 from matrix-org/erikj/message_metrics ... Add more granular event send metrics	2017-05-02 17:57:34 +01:00
Erik Johnston	3e5a62ecd8	Add more granular event send metrics	2017-05-02 14:23:26 +01:00
Richard van der Hoff	82475a18d9	Merge pull request #2180 from matrix-org/rav/fix_timeout_on_timeout ... Instantiate DeferredTimedOutError correctly	2017-05-02 13:32:58 +01:00
Richard van der Hoff	2e996271fe	Instantiate DeferredTimedOutError correctly ... Call `super` correctly, so that we correctly initialise the `errcode` field. Fixes https://github.com/matrix-org/synapse/issues/2179.	2017-05-02 13:26:17 +01:00
Erik Johnston	a2c89a225c	Prefill state caches	2017-05-02 10:40:31 +01:00
Erik Johnston	7166854f41	Add cache for get_current_hosts_in_room	2017-05-02 10:36:35 +01:00
Erik Johnston	3033261891	Merge pull request #2080 from matrix-org/erikj/filter_speed ... Speed up filtering of a single event in push	2017-04-28 14:17:13 +01:00
Erik Johnston	2347efc065	Fixup	2017-04-28 12:46:53 +01:00
Erik Johnston	9b147cd730	Remove unncessary call in _get_missing_events_for_pdu	2017-04-28 11:55:25 +01:00
Erik Johnston	3a9f5bf6dd	Don't fetch state for missing events that we fetched	2017-04-28 11:26:46 +01:00
Erik Johnston	ab37bef83b	Remove unused import	2017-04-28 09:57:23 +01:00
Erik Johnston	ad8b316939	We don't care about forgotten rooms	2017-04-28 09:52:36 +01:00
Erik Johnston	421fdf7460	Speed up filtering of a single event in push	2017-04-28 09:52:36 +01:00
Erik Johnston	25a96e0c63	Merge pull request #2163 from matrix-org/erikj/fix_invite_state ... Fix invite state to always include all events	2017-04-27 17:36:30 +01:00
Erik Johnston	46826bb078	Comment and remove spurious logging	2017-04-27 17:25:44 +01:00
Erik Johnston	f87b287291	Merge pull request #2127 from APwhitehat/alreadystarted ... print something legible if synapse already running	2017-04-27 15:46:53 +01:00
Erik Johnston	bb9246e525	Merge pull request #2131 from matthewjwolff/develop ... web_client_location documentation fix	2017-04-27 15:46:40 +01:00
Richard van der Hoff	c84770b877	Fix bgupdate error if index already exists (#2167) ... When creating a new table index in the background, guard against it existing already. Fixes https://github.com/matrix-org/synapse/issues/2135. Also, make sure we restore the autocommit flag when we're done, otherwise we get more failures from other operations later on. Fixes https://github.com/matrix-org/synapse/issues/1890 (hopefully).	2017-04-27 15:27:48 +01:00
Erik Johnston	380fb87ecc	Merge pull request #2168 from matrix-org/erikj/federation_logging ... Add some extra logging for edge cases of federation	2017-04-27 15:19:12 +01:00
Erik Johnston	87ae59f5e9	Typo	2017-04-27 15:16:21 +01:00
Erik Johnston	e42b4ebf0f	Add some extra logging for edge cases of federation	2017-04-27 14:38:21 +01:00
Erik Johnston	d3c150411c	Merge pull request #2130 from APwhitehat/roomexists ... Check that requested room_id exists	2017-04-27 09:20:26 +01:00
Erik Johnston	1e166470ab	Fix tests	2017-04-26 16:23:30 +01:00
Erik Johnston	34e682d385	Fix invite state to always include all events	2017-04-26 16:18:08 +01:00
David Baker	5fd12dce01	Remove debugging	2017-04-26 12:36:26 +01:00
David Baker	82ae0238f9	Revert accidental commit	2017-04-26 11:43:16 +01:00
David Baker	81804909d3	Merge remote-tracking branch 'origin/develop' into dbkr/http_request_propagate_error	2017-04-26 11:31:55 +01:00
David Baker	c366276056	Fix get_json	2017-04-26 10:07:01 +01:00
David Baker	1a9255c12e	Use CodeMessageException subclass instead ... Parse json errors from get_json client methods and throw special errors.	2017-04-25 19:30:55 +01:00
Erik Johnston	f053a1409e	Make state caches cache in ascii	2017-04-25 17:22:55 +01:00
Erik Johnston	acb58bfb6a	fix up	2017-04-25 15:39:19 +01:00
Erik Johnston	f7181615f2	Don't specify default as dict	2017-04-25 15:22:59 +01:00
Erik Johnston	f144365281	Comment	2017-04-25 15:18:26 +01:00
Erik Johnston	d9aa645f86	Reduce size of joined_user cache ... The _get_joined_users_from_context cache stores a mapping from user_id to avatar_url and display_name. Instead of storing those in a dict, store them in a namedtuple as that uses much less memory. We also try converting the string to ascii to further reduce the size.	2017-04-25 14:38:51 +01:00
Erik Johnston	22f3d3ae76	Reduce _get_state_group_for_event cache size	2017-04-25 11:43:03 +01:00
Erik Johnston	b4da08cad8	Merge pull request #2158 from matrix-org/erikj/reduce_cache_size ... Reduce cache size by not storing deferreds	2017-04-25 11:18:07 +01:00
Erik Johnston	efab1dadde	Remove DEBUG_CACHES	2017-04-25 10:54:09 +01:00
Erik Johnston	119cb9bbcf	Reduce cache size by not storing deferreds ... Currently the cache descriptors store deferreds rather than raw values, this is a simple way of triggering only one database hit and sharing the result if two callers attempt to get the same value. However, there are a few caches that simply store a mapping from string to string (or int). These caches can have a large number of entries, under the assumption that each entry is small. However, the size of a deferred (specifically the size of ObservableDeferred) is signigicantly larger than that of the raw value, 2kb vs 32b. This PR therefore changes the cache descriptors to store the raw values rather than the deferreds. As a side effect cached storage function now either return a deferred or the actual value, as the cached list decriptor already does. This is fine as we always end up just yield'ing on the returned value eventually, which handles that case correctly.	2017-04-25 10:23:11 +01:00
Mark Haines	e6e2627636	Fix code for reporting old verify keys in synapse	2017-04-24 18:51:25 +01:00
Richard van der Hoff	30f7bfa121	Merge pull request #2145 from matrix-org/rav/reject_invite_to_unreachable_server ... Fix rejection of invites to unreachable servers	2017-04-24 15:20:52 +01:00
Erik Johnston	7af825bae4	Merge pull request #2155 from matrix-org/erikj/string_intern ... Only intern ascii strings	2017-04-24 14:28:41 +01:00
Erik Johnston	d134d0935e	Only intern ascii strings	2017-04-24 14:07:48 +01:00
Erik Johnston	e4f3431116	Remove unused cache	2017-04-24 13:27:38 +01:00
David Baker	a46982cee9	Need the HTTP status code	2017-04-21 16:20:12 +01:00
David Baker	70caf49914	Do the same for get_json	2017-04-21 16:09:03 +01:00
David Baker	a1595cec78	Don't error for 3xx responses	2017-04-21 11:51:17 +01:00
David Baker	2e165295b7	Merge remote-tracking branch 'origin/develop' into dbkr/http_request_propagate_error	2017-04-21 11:35:52 +01:00
David Baker	a90a0f5c8a	Propagate errors sensibly from proxied IS requests ... When we're proxying Matrix endpoints, parse out Matrix error responses and turn them into SynapseErrors so they can be propagated sensibly upstream.	2017-04-21 11:32:48 +01:00
Richard van der Hoff	91b3981800	Try harder when sending leave events ... When we're rejecting invites, ignore the backoff data, so that we have a better chance of not getting the room out of sync.	2017-04-21 01:50:36 +01:00
Richard van der Hoff	0cdb32fc43	Remove redundant try/except clauses ... The `except SynapseError` clauses were pointless because the wrapped functions would never throw a `SynapseError` (they either throw a `CodeMessageException` or a `RuntimeError`). The `except CodeMessageException` is now also pointless because the caller treats all exceptions equally, so we may as well just throw the `CodeMessageException`.	2017-04-21 01:32:01 +01:00
Richard van der Hoff	838810b76a	Broaden the conditions for locally_rejecting invites ... The logic for marking invites as locally rejected was all well and good, but didn't happen when the remote server returned a 500, or wasn't reachable, or had no DNS, or whatever. Just expand the except clause to catch everything. Fixes https://github.com/matrix-org/synapse/issues/761.	2017-04-21 01:31:37 +01:00
Richard van der Hoff	736b9a4784	Remove redundant function ... inline `reject_remote_invite`, which only existed to make tracing the callflow more difficult.	2017-04-21 01:31:09 +01:00
Richard van der Hoff	4903ccf159	Fix some lies, and other clarifications, in docstrings ... The documentation on get_json has been wrong ever since the very first commit to synapse...	2017-04-21 01:31:09 +01:00
Matthew Wolff	d4040e9e28	Queried CONDITIONAL_REQUIREMENTS	2017-04-18 16:19:48 -05:00
Luke Barnard	3fb8784c92	m.read_marker -> m.fully_read (#2128) ... Also: - change the REST endpoint to have a "S" on the end (so it's now /read_markers) - change the content of the m.read_up_to event to have the key "event_id" instead of "marker".	2017-04-18 17:46:15 +01:00
Matthew Wolff	54f9a4cb59	Fixed travis build failure ... Signed-off-by: Matthew Wolff <matthewjwolff@gmail.com>	2017-04-17 01:38:27 -05:00
Matthew Wolff	8e780b113d	web_server_root documentation fix ... Signed-off-by: Matthew Wolff <matthewjwolff@gmail.com>	2017-04-17 00:49:11 -05:00
Anant Prakash	574d573ac2	Check that requested room_id exists	2017-04-14 23:50:59 +05:30
Luke Barnard	78f0ddbfad	Merge pull request #2120 from matrix-org/luke/read-markers ... Implement Read Marker API	2017-04-13 14:21:31 +01:00
Luke Barnard	6a70647d45	Correct logic in is_event_after	2017-04-13 13:46:17 +01:00
Anant Prakash	c1f52a321d	synctl.py: Check if synapse is already running	2017-04-13 18:00:02 +05:30
Luke Barnard	b9557064bf	Simplify is_event_after logic	2017-04-12 14:36:20 +01:00
Luke Barnard	cf6121e3da	More null-guard changes	2017-04-12 14:02:03 +01:00
Erik Johnston	247c736b9b	Merge pull request #2115 from matrix-org/erikj/dedupe_federation_repl ... Reduce federation replication traffic	2017-04-12 11:07:13 +01:00
Paul Evans	8fbc0d29ee	Merge pull request #2121 from matrix-org/paul/sent-transactions-metric ... Add a counter metric for successfully-sent transactions	2017-04-12 11:04:31 +01:00
Erik Johnston	c06c00190f	Merge pull request #2116 from matrix-org/erikj/dedupe_federation_repl2 ... Dedupe KeyedEdu and Devices federation repl traffic	2017-04-12 10:57:24 +01:00
Luke Barnard	c0aba0a23e	Remove Unused ref to hs	2017-04-12 10:52:11 +01:00
Luke Barnard	b9676a75f6	Move a space	2017-04-12 10:51:17 +01:00
Luke Barnard	69a18514e9	Only notify user, not entire room	2017-04-12 10:50:37 +01:00
Luke Barnard	122cd52ce4	Remove comment, simplify null-guard	2017-04-12 10:48:32 +01:00
Erik Johnston	26ae5178a4	Add some comments	2017-04-12 10:36:29 +01:00
Erik Johnston	bf9060156a	Merge pull request #2117 from matrix-org/erikj/remove_http_replication ... Remove HTTP replication APIs	2017-04-12 10:21:42 +01:00
Erik Johnston	1745069543	Comment	2017-04-12 10:17:10 +01:00
Erik Johnston	c7ddb5ef7a	Reuse get_interested_parties	2017-04-12 10:16:26 +01:00
Erik Johnston	7b41013102	Merge pull request #2118 from matrix-org/erikj/no_devices ... Fix getting latest device IP for user with no devices	2017-04-12 10:14:32 +01:00
Luke Barnard	77fb2b72ae	Handle no previous RM	2017-04-12 09:47:29 +01:00
Luke Barnard	7f94709066	travis flake8..	2017-04-11 18:35:45 +01:00
Luke Barnard	867822fa1e	flake8	2017-04-11 17:36:04 +01:00
Luke Barnard	73880268ef	Refactor event ordering check to events store	2017-04-11 17:34:09 +01:00
Luke Barnard	131485ef66	Copyright	2017-04-11 17:33:51 +01:00
Paul "LeoNerd" Evans	11dbceb761	Add a counter metric for successfully-sent transactions	2017-04-11 17:16:12 +01:00
Luke Barnard	0127423027	flake8	2017-04-11 17:07:07 +01:00
Erik Johnston	85657eedf8	Bail on where clause instead	2017-04-11 16:24:31 +01:00
Erik Johnston	b48045a8f5	Don't bother with outer check for now	2017-04-11 16:23:24 +01:00
Erik Johnston	9c712a366f	Move get_presence_list_* to SlaveStore	2017-04-11 16:07:33 +01:00
Erik Johnston	a8c8e4efd4	Comment	2017-04-11 15:35:49 +01:00
Erik Johnston	414522aed5	Move get_interested_parties	2017-04-11 15:33:26 +01:00
Erik Johnston	2be8a281d2	Comments	2017-04-11 15:28:24 +01:00
Erik Johnston	6308ac45b0	Move get_interested_remotes back to presence handler	2017-04-11 15:19:26 +01:00
Erik Johnston	b9b72bc6e2	Comments	2017-04-11 15:15:34 +01:00
Luke Barnard	d892079844	Finish implementing RM endpoint ... - This change causes a 405 to be sent if "m.read_marker" is set via /account_data - This also fixes-up the RM endpoint so that it actually Works.	2017-04-11 15:01:39 +01:00
lukebarnard	e263c26690	Initial commit of RM server-side impl ... (See https://docs.google.com/document/d/1UWqdS-e1sdwkLDUY0wA4gZyIkRp-ekjsLZ8k6g_Zvso/edit#heading=h.lndohpg8at5u)	2017-04-11 11:55:30 +01:00
Erik Johnston	f3cf3ff8b6	Merge branch 'master' of github.com:matrix-org/synapse into develop	2017-04-11 11:13:32 +01:00
Erik Johnston	34840cdcef	Fix getting latest device IP for user with no devices	2017-04-11 09:56:54 +01:00
Erik Johnston	28a4649785	Remove HTTP replication APIs	2017-04-11 09:52:11 +01:00
Matthew Hodgson	7c551ec445	trust a hypothetical future riot.im IS	2017-04-10 17:58:36 +01:00
Erik Johnston	84fbb80c8f	Use generators	2017-04-10 16:55:56 +01:00
Erik Johnston	40453b3f84	Dedupe KeyedEdu and Devices federation repl traffic	2017-04-10 16:49:51 +01:00
Erik Johnston	29574fd5b3	Reduce federation presence replication traffic ... This is mainly done by moving the calculation of where to send presence updates from the presence handler to the transaction queue, so we only need to send the presence event (and not the destinations) across the replication connection. Before we were duplicating by sending the full state across once per destination.	2017-04-10 16:48:30 +01:00
Erik Johnston	2e6f5a4910	Typo	2017-04-10 16:17:40 +01:00
David Baker	405ba4178a	Merge pull request #2102 from DanielDent/add-auth-email ... Support authenticated SMTP	2017-04-10 15:42:16 +01:00
Erik Johnston	efcb6db688	Merge pull request #2109 from matrix-org/erikj/send_queue_fix ... Fix up federation SendQueue and document types	2017-04-10 13:09:25 +01:00
Erik Johnston	0018491af2	Rename variable	2017-04-10 12:44:43 +01:00
Erik Johnston	0364d23210	Up replication ping timeout	2017-04-10 11:32:05 +01:00
Erik Johnston	8c5f03cec7	Revert to sending the same data type as before	2017-04-10 10:07:18 +01:00
Erik Johnston	f8434db549	Change name	2017-04-10 10:03:07 +01:00
Erik Johnston	ab904caf33	Comments	2017-04-10 10:02:17 +01:00
Richard van der Hoff	64765e5199	When we do an invite rejection, save the signed leave event to the db ... During a rejection of an invite received over federation, we ask a remote server to make us a `leave` event, then sign it, then send that with `send_leave`. We were saving the *unsigned* version of the event (which has a different event id to the signed version) to our db (and sending it to the clients), whereas other servers in the room will have seen the *signed* version. We're not aware of any actual problems that caused, except that it makes the database confusing to look at and generally leaves the room in a weird state.	2017-04-07 14:39:32 +01:00
Erik Johnston	2a3e822f44	Comment	2017-04-07 13:47:04 +01:00
Erik Johnston	a828a64b75	Comment	2017-04-07 11:54:03 +01:00
Erik Johnston	d4d176e5d0	Add logging	2017-04-07 11:51:28 +01:00
Erik Johnston	449d1297ca	Fix up federation SendQueue and document types	2017-04-07 11:48:33 +01:00
Erik Johnston	d72667fcce	Speed up get_current_state_ids ... Using _simple_select_list is fairly expensive for functions that return a lot of rows and/or get called a lot. (This is because it carefully constructs a list of dicts). get_current_state_ids gets called a lot on startup and e.g. when the IRC bridge decided to send tonnes of joins/leaves (as it invalidates the cache). We therefore replace it with a custon txn function that builds up the final result dict without building up and intermediate representation.	2017-04-07 10:10:49 +01:00
Erik Johnston	a41fe500d6	Bump version and changelog	2017-04-07 10:03:48 +01:00
Erik Johnston	54f59bd7d4	Merge pull request #2107 from HarHarLinks/patch-1 ... fix typo in synctl help	2017-04-07 09:54:37 +01:00
Erik Johnston	98ce212093	Merge pull request #2103 from matrix-org/erikj/no-double-encode ... Don't double encode replication data	2017-04-07 09:39:52 +01:00
Kim Brose	8a1137ceab	fix typo in synctl help	2017-04-06 17:10:20 +02:00
Erik Johnston	877c029c16	Use iteritems	2017-04-06 15:51:22 +01:00
Erik Johnston	944692ef69	Merge pull request #2106 from matrix-org/erikj/reduce_user_sync ... Reduce rate of USER_SYNC repl commands	2017-04-06 13:35:31 +01:00
Erik Johnston	391712a4f9	Comment	2017-04-06 13:35:00 +01:00
Erik Johnston	ad544c803a	Document types of the replication streams	2017-04-06 13:28:52 +01:00
Erik Johnston	dbf87282d3	Docs	2017-04-06 13:11:21 +01:00
Erik Johnston	69b3fd485d	Fix incorrect type when using InvalidateCacheCommand	2017-04-06 09:36:38 +01:00
Daniel Dent	5058292537	Support authenticated SMTP ... Closes (SYN-714) #1385 Signed-off-by: Daniel Dent <matrixcontrib@contactdaniel.net>	2017-04-05 21:01:08 -07:00
Erik Johnston	fcc803b2bf	Add log lines	2017-04-05 17:13:44 +01:00
Erik Johnston	3f213d908d	Rearrange metrics	2017-04-05 14:15:09 +01:00
Erik Johnston	1ca0e78ca1	Fix typo	2017-04-05 13:43:39 +01:00
Erik Johnston	b43d3267e2	Fixup some metrics for tcp repl	2017-04-05 13:34:54 +01:00
Erik Johnston	b5cb6347a4	Don't immediately notify the master about users whose syncs have gone away	2017-04-05 13:25:40 +01:00
Erik Johnston	96b9b6c127	Don't double json encode typing replication data	2017-04-05 11:34:20 +01:00
Erik Johnston	f10ce8944b	Don't double json encode federation replication data	2017-04-05 11:10:28 +01:00
Erik Johnston	a5c401bd12	Merge pull request #2097 from matrix-org/erikj/repl_tcp_client ... Move to using TCP replication	2017-04-05 09:36:21 +01:00
Erik Johnston	d1d5362267	Add comment	2017-04-04 16:41:03 +01:00
Erik Johnston	9f26d3b75b	Deduplicate new deviceinbox rows for replication	2017-04-04 16:21:21 +01:00
Erik Johnston	a76886726b	Merge pull request #2098 from matrix-org/erikj/repl_tcp_fix ... Advance replication streams even if nothing is listening	2017-04-04 15:40:51 +01:00
Erik Johnston	ac66e11f2b	Add the appropriate amount of preserve_fn	2017-04-04 15:22:54 +01:00
Erik Johnston	4264ceb31c	Fiddle tcp replication logging	2017-04-04 14:14:03 +01:00
Erik Johnston	023ee197be	Advance replication streams even if nothing is listening ... Otherwise the streams don't advance and steadily fall behind, so when a worker does connect either a) they'll be streamed lots of old updates or b) the connection will fail as the streams are too far behind.	2017-04-04 13:19:26 +01:00
Erik Johnston	d1605794ad	Remove unused worker config option	2017-04-04 11:17:00 +01:00
Erik Johnston	3376f16012	Shuffle and comment synchrotron presence	2017-04-04 11:14:16 +01:00
Erik Johnston	6ce6bbedcb	Move where we ack federation	2017-04-04 11:02:44 +01:00
Erik Johnston	62b89daac6	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/repl_tcp_server	2017-04-04 09:46:16 +01:00
Richard van der Hoff	773e64cc1a	Merge pull request #2095 from matrix-org/rav/cull_log_preserves ... Cull spurious PreserveLoggingContexts	2017-04-03 17:02:25 +01:00
Richard van der Hoff	2d05eb3cf5	Merge remote-tracking branch 'origin/release-v0.20.0' into develop	2017-04-03 16:23:23 +01:00
Richard van der Hoff	30bcbf775a	Accept join events from all servers ... Make sure that we accept join events from any server, rather than just the origin server, to make the federation join dance work correctly. (Fixes #1893).	2017-04-03 15:58:07 +01:00
Richard van der Hoff	7eb9f34cc3	Remove spurious yield ... In `MessageHandler`, remove `yield` on call to `Notifier.on_new_room_event`: it doesn't return anything anyway.	2017-04-03 15:44:19 +01:00
Richard van der Hoff	0b08c48fc5	Remove more spurious PreserveLoggingContexts ... Remove `PreserveLoggingContext` around calls to `Notifier.on_new_room_event`; there is no problem if the logcontext is set when calling it.	2017-04-03 15:43:37 +01:00
Richard van der Hoff	65e1683680	Remove spurious PreserveLoggingContext ... In `on_new_room_event`, remove `PreserveLoggingContext` - we can call its subroutines with the logcontext set.	2017-04-03 15:42:38 +01:00
Richard van der Hoff	feb496056e	preserve_fn some deferred-returning things ... In `Notifier._on_new_room_event`, `preserve_fn` around its subroutines which return deferreds, so that it is safe to call it with an active logcontext.	2017-04-03 15:41:17 +01:00
Richard van der Hoff	e2eebf1696	Fix fixme in preserve_fn ... `preserve_fn` is no longer used as a decorator anywhere, so we can safely fix a fixme therein.	2017-04-03 15:38:02 +01:00
Erik Johnston	36c28bc467	Update all the workers and master to use TCP replication	2017-04-03 15:35:52 +01:00
Erik Johnston	3a1f3f8388	Change slave storage to use new replication interface ... As the TCP replication uses a slightly different API and streams than the HTTP replication. This breaks HTTP replication.	2017-04-03 15:34:19 +01:00
Erik Johnston	52bfa604e1	Add basic replication client handler and factory	2017-04-03 15:34:13 +01:00
Erik Johnston	0a6a966e2b	Always advance stream tokens	2017-04-03 15:22:56 +01:00
Richard van der Hoff	773e1c6d68	Remove spurious @preserve_fn decorators ... Remove `@preserve_fn` decorators on `on_new_room_event`, `_notify_pending_new_room_events`, `_on_new_room_event`, `on_new_event`, and `on_new_replication_data` - none of these functions return a deferred, and the decorator does nothing unless the wrapped function returns a deferred, so the decorator was a no-op.	2017-04-03 15:14:11 +01:00
Erik Johnston	0d1c85e643	Merge branch 'release-v0.20.0' of github.com:matrix-org/synapse into develop	2017-04-03 14:58:14 +01:00
Erik Johnston	1df7c28661	Use callbacks to notify tcp replication rather than deferreds	2017-03-31 15:42:51 +01:00
Erik Johnston	36d2b66f90	Add a timestamp to USER_SYNC command ... This timestamp is used to indicate when the user last sync'd	2017-03-31 15:42:22 +01:00
Erik Johnston	8a240e4f9c	Merge pull request #2078 from APwhitehat/assertuserfriendly ... add user friendly report of assertion error in synctl.py	2017-03-31 14:41:49 +01:00
Erik Johnston	142b6b4abf	Merge pull request #2011 from matrix-org/matthew/turn_allow_guests ... add setting (on by default) to support TURN for guests	2017-03-31 14:37:09 +01:00
Erik Johnston	2a06b44be2	Merge pull request #1986 from matrix-org/matthew/enable_guest_3p ... enable guest access for the 3pl/3pid APIs	2017-03-31 14:36:03 +01:00
Matthew Hodgson	9ee397b440	switch to allow_guest=True for authing 3Ps as per PR feedback	2017-03-31 13:54:26 +01:00
Erik Johnston	9d0170ac6c	Fix up presence	2017-03-31 11:36:32 +01:00
Erik Johnston	bfcf016714	Fix up docs	2017-03-31 11:19:24 +01:00
Erik Johnston	9cee0ce7db	Merge pull request #2075 from matrix-org/erikj/cache_speed ... Speed up cached function access	2017-03-31 10:10:56 +01:00
Erik Johnston	350333a09a	Merge pull request #2076 from matrix-org/erikj/as_perf ... Make AS's faster	2017-03-31 09:43:10 +01:00
Erik Johnston	4d17add8de	Remove unused instance variable	2017-03-31 09:38:27 +01:00
Erik Johnston	27b1b4a2c9	Speed up copy_and_replace	2017-03-30 17:50:31 +01:00
Erik Johnston	5b5b171f3e	Docs	2017-03-30 17:05:53 +01:00
Erik Johnston	b282fe7170	Revert log context change	2017-03-30 17:03:59 +01:00
Erik Johnston	9ff4e0e91b	Bump version and changelog	2017-03-30 16:37:40 +01:00
Erik Johnston	0834d1a70c	Merge pull request #2079 from matrix-org/erikj/push_regex_cache ... Cache glob to regex at a higher level for push	2017-03-30 16:28:18 +01:00
Erik Johnston	63fcc42990	Remove user from process_presence when stops syncing	2017-03-30 14:26:08 +01:00
Erik Johnston	6194a64ae9	Doc new instance variables	2017-03-30 14:19:10 +01:00
Erik Johnston	014fee93b3	Manually calculate cache key as getcallargs is expensive ... This is because getcallargs recomputes the getargspec, amongst other things, which we don't need to do as its already been done	2017-03-30 14:14:46 +01:00
Erik Johnston	86780a8bc3	Don't convert to deferreds when not necessary	2017-03-30 14:14:36 +01:00
Erik Johnston	3ba2859e0c	Add tcp replication listener type and hook it up	2017-03-30 13:31:10 +01:00
Erik Johnston	e9dd8370b0	Add functions to presence to support remote syncs ... The TCP replication protocol streams deltas of who has started or stopped syncing. This is different from the HTTP API which periodically sends the full list of users who are syncing. This commit adds support for the new TCP style of sending deltas.	2017-03-30 13:25:14 +01:00
Erik Johnston	4d7fc7f977	Add server side resource for tcp replication	2017-03-30 13:24:45 +01:00
Richard van der Hoff	f9b4bb05e0	Fix the logcontext handling in the cache wrappers (#2077) ... The cache wrappers had a habit of leaking the logcontext into the reactor while the lookup function was running, and then not restoring it correctly when the lookup function had completed. It's all the fault of `preserve_context_over_{fn,deferred}` which are basically a bit broken.	2017-03-30 13:22:24 +01:00
Erik Johnston	7450693435	Initial TCP protocol implementation ... This defines the low level TCP replication protocol	2017-03-30 12:54:46 +01:00
Erik Johnston	8da6f0be48	Define the various streams we will replicate	2017-03-30 12:54:46 +01:00
Erik Johnston	11880103b1	Make federation send queue take the current position	2017-03-30 12:54:36 +01:00
Erik Johnston	7984708a55	Add a simple hook to wait for replication traffic	2017-03-30 11:57:52 +01:00
Erik Johnston	24d35ab47b	Add new storage functions for new replication ... The new replication protocol will keep all the streams separate, rather than muxing multiple streams into one.	2017-03-30 11:48:35 +01:00
Erik Johnston	30348c924c	Use txn.fetchall() so we can reuse txn	2017-03-30 10:30:05 +01:00
Anant Prakash	6cdca71079	synctl.py: wait for synapse to stop before restarting (#2020)	2017-03-29 19:20:13 +01:00
Anant Prakash	305d16d612	add user friendly report of assertion error in synctl.py ... Signed-off-by: Anant Prakash <anantprakashjsr@gmail.com>	2017-03-29 20:41:39 +05:30
Erik Johnston	a3810136fe	Cache glob to regex at a higher level for push	2017-03-29 15:53:14 +01:00
Erik Johnston	3ce8d59176	Increase cache size for _get_state_group_for_event	2017-03-29 14:31:46 +01:00
Erik Johnston	b9c2ae6788	Merge pull request #1849 from matrix-org/erikj/state_typo ... Fix bug where current_state_events renamed to current_state_ids	2017-03-29 11:53:42 +01:00
Erik Johnston	85be3dde81	Bail early if remote wouldn't be retried (#2064) ... * Bail early if remote wouldn't be retried * Don't always return true * Just use get_retry_limiter * Spelling	2017-03-29 11:48:27 +01:00
Erik Johnston	2f8b580b64	Merge pull request #2053 from matrix-org/erikj/e2e_one_time_upsert ... Don't user upsert to persist new one time keys	2017-03-29 11:44:23 +01:00
Erik Johnston	c5b0bdd542	Merge pull request #2067 from matrix-org/erikj/notify_on_fed ... Notify on new federation traffic	2017-03-29 11:41:37 +01:00
Erik Johnston	e4df0e189d	Decrank last commit	2017-03-29 11:02:35 +01:00
Erik Johnston	4ad613f6be	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/e2e_one_time_upsert	2017-03-29 10:57:19 +01:00
Erik Johnston	ac6bc55512	Correctly look up key	2017-03-29 10:56:26 +01:00
Erik Johnston	69efd77749	Add comment	2017-03-29 09:50:05 +01:00
Erik Johnston	51b156d48a	Cache whether an AS is interested based on members	2017-03-28 13:27:21 +01:00
Erik Johnston	30f5ffdca2	Remove param and cast at call site	2017-03-28 13:27:21 +01:00
Erik Johnston	650f0e69f2	Compile the regex's used in ASes	2017-03-28 13:27:21 +01:00
Erik Johnston	d28db583da	Merge pull request #2063 from matrix-org/erikj/device_list_batch ... Batch sending of device list pokes	2017-03-28 11:35:41 +01:00
Erik Johnston	58a35366be	The algorithm is part of the key id	2017-03-28 11:34:37 +01:00
Erik Johnston	bac9bf1b12	Typo	2017-03-27 18:02:17 +01:00
Erik Johnston	d82c42837f	Short circuit if all new events have same state group	2017-03-27 18:00:47 +01:00
Erik Johnston	35b4aa04be	Notify on new federation traffic	2017-03-27 14:07:47 +01:00
Erik Johnston	2a28b79e04	Batch sending of device list pokes	2017-03-24 14:44:49 +00:00
Erik Johnston	281553afe6	Merge pull request #2062 from matrix-org/erikj/presence_replication ... Use presence replication stream to invalidate cache	2017-03-24 13:57:45 +00:00
Erik Johnston	987f4945b4	Actually call invalidate	2017-03-24 13:28:20 +00:00
Erik Johnston	31d56c3fb5	Merge pull request #2061 from matrix-org/erikj/add_transaction_store ... Add slave transaction store to workers who send federation requests	2017-03-24 13:24:27 +00:00
Erik Johnston	09f79aaad0	Use presence replication stream to invalidate cache ... Instead of using the cache invalidation replication stream to invalidate the _get_presence_cache, we can instead rely on the presence replication stream. This reduces the amount of replication traffic considerably.	2017-03-24 13:21:08 +00:00
Erik Johnston	23e0ff840a	Merge pull request #2060 from matrix-org/erikj/cache_hosts_in_room ... Cache hosts in room	2017-03-24 13:07:22 +00:00
Erik Johnston	48e7697911	Add slave transaction store	2017-03-24 13:05:30 +00:00
Richard van der Hoff	f136c89d5e	Merge pull request #2058 from matrix-org/rav/logcontext_leaks_2 ... try not to drop context after federation requests	2017-03-24 12:47:26 +00:00
Richard van der Hoff	01fc847f7f	Merge pull request #2057 from matrix-org/rav/missing_yield_2 ... Add another missing yield on check_device_registered	2017-03-24 12:46:43 +00:00
Erik Johnston	7fc1f1e2b6	Cache hosts in room	2017-03-24 11:46:24 +00:00
Erik Johnston	57cfa513f5	Merge pull request #2054 from matrix-org/erikj/user_iter_cursor ... Reduce some CPU work on DB threads	2017-03-24 11:40:08 +00:00
Erik Johnston	d58b1ffe94	Replace some calls to cursor_to_dict ... cursor_to_dict can be surprisinglh expensive for large result sets, so lets only call it when we need to.	2017-03-24 11:07:02 +00:00
Erik Johnston	e71940aa64	Use iter(items|values)	2017-03-24 10:57:02 +00:00
David Baker	e36950dec5	Merge pull request #2055 from matrix-org/dbkr/fix_add_msisdn_requestToken ... Fix token request for addition of phone numbers	2017-03-24 10:32:57 +00:00
David Baker	f902e89d4b	Merge pull request #2056 from matrix-org/dbkr/fix_invite_reject ... Fix rejection of invites not reaching sync	2017-03-24 10:32:51 +00:00
Richard van der Hoff	a380f041c2	try not to drop context after federation requests ... preserve_context_over_fn uses a ContextPreservingDeferred, which only restores context for the duration of its callbacks, which isn't really correct, and means that subsequent operations in the same request can end up without their logcontexts.	2017-03-23 22:36:21 +00:00
Richard van der Hoff	9397edb28b	Merge pull request #2050 from matrix-org/rav/federation_backoff ... push federation retry limiter down to matrixfederationclient	2017-03-23 22:27:01 +00:00
Richard van der Hoff	06ce7335e9	Merge pull request #2052 from matrix-org/rav/time_bound_deferred ... Fix time_bound_deferred to throw the right exception	2017-03-23 22:22:54 +00:00
Richard van der Hoff	13c8749ac9	Add another missing yield on check_device_registered	2017-03-23 22:18:53 +00:00
David Baker	e1f1784f99	Fix rejection of invites not reaching sync ... Always allow the user to see their own leave events, otherwise they won't see the event if they reject an invite for a room whose history visibility is set such that they cannot see events before joining.	2017-03-23 18:50:31 +00:00
David Baker	86e865d7d2	Oops, remove unintentional change	2017-03-23 18:48:30 +00:00
David Baker	a2dfab12c5	Fix token request for addition of phone numbers	2017-03-23 18:46:17 +00:00
Erik Johnston	00957d1aa4	User Cursor.__iter__ instead of fetchall ... This prevents unnecessary construction of lists	2017-03-23 17:53:49 +00:00
pik	250ce11ab9	Add jsonschema to python_dependencies.py ... Signed-off-by: pik <alexander.maznev@gmail.com>	2017-03-23 11:42:47 -03:00
pik	566641a0b5	use jsonschema.FormatChecker for RoomID and UserID strings ... * use a valid filter in rest/client/v2_alpha test Signed-off-by: pik <alexander.maznev@gmail.com>	2017-03-23 11:42:41 -03:00
pik	acafcf1c5b	Add valid filter tests, flake8, fix typo ... Signed-off-by: pik <alexander.maznev@gmail.com>	2017-03-23 11:42:10 -03:00
pik	e56c79c114	check_valid_filter using JSONSchema ... * add invalid filter tests Signed-off-by: pik <alexander.maznev@gmail.com>	2017-03-23 11:42:07 -03:00
Erik Johnston	6ebe2d23b1	Raise a more helpful exception	2017-03-23 13:48:30 +00:00
Erik Johnston	e64655c25d	Don't user upsert to persist new one time keys ... Instead we no-op duplicate one time key uploads, an error if the key_id already exists but encodes a different key.	2017-03-23 13:17:00 +00:00
Richard van der Hoff	5a16cb4bf0	Ignore backoff history for invites, aliases, and roomdirs ... Add a param to the federation client which lets us ignore historical backoff data for federation queries, and set it for a handful of operations.	2017-03-23 12:23:22 +00:00
Richard van der Hoff	b88a323ffb	Fix time_bound_deferred to throw the right exception ... Due to a failure to instantiate DeferredTimedOutError, time_bound_deferred would throw a CancelledError when the deferred timed out, which was rather confusing.	2017-03-23 12:07:11 +00:00
Richard van der Hoff	55366814a6	Merge pull request #2048 from matrix-org/rav/missing_yield ... Add a missing yield in device key upload	2017-03-23 09:29:44 +00:00
Richard van der Hoff	4bd597d9fc	push federation retry limiter down to matrixfederationclient ... rather than having to instrument everywhere we make a federation call, make the MatrixFederationHttpClient manage the retry limiter.	2017-03-23 09:28:46 +00:00
Richard van der Hoff	ad8a26e361	MatrixFederationHttpClient: clean up ... rename _create_request to _request, and push ascii-encoding of `destination` and `path` down into it	2017-03-23 00:27:04 +00:00
Richard van der Hoff	19b9366d73	Fix a couple of logcontext leaks ... Use preserve_fn to correctly manage the logcontexts around things we don't want to yield on.	2017-03-23 00:17:46 +00:00
Richard van der Hoff	e08f81d96a	Add a missing yield in device key upload ... (this would only very very rarely actually be a useful thing, so the main problem was the logcontext leak...)	2017-03-23 00:16:43 +00:00
Richard van der Hoff	35e0cfb54d	Merge pull request #2044 from matrix-org/rav/crypto_docs ... fix up some key verif docstrings	2017-03-22 17:12:55 +00:00
Richard van der Hoff	7b67848042	Merge pull request #2042 from matrix-org/rav/fix_key_caching ... Fix caching of remote servers' signature keys	2017-03-22 17:11:55 +00:00
Richard van der Hoff	95f21c7a66	Fix caching of remote servers' signature keys ... The `@cached` decorator on `KeyStore._get_server_verify_key` was missing its `num_args` parameter, which meant that it was returning the wrong key for any server which had more than one recorded key. By way of a fix, change the default for `num_args` to be *all* arguments. To implement that, factor out a common base class for `CacheDescriptor` and `CacheListDescriptor`.	2017-03-22 15:11:30 +00:00
Matthew Hodgson	d101488c5f	Merge branch 'master' into develop	2017-03-21 22:48:21 +01:00
Richard van der Hoff	64778693be	fix up some key verif docstrings	2017-03-21 13:27:50 +00:00
Erik Johnston	37a187bfab	Merge pull request #2033 from matrix-org/erikj/repl_speed ... Don't send the full event json over replication	2017-03-21 13:11:15 +00:00
Erik Johnston	633dcc316c	Bump changelog and version	2017-03-20 16:37:14 +00:00
Richard van der Hoff	c36d15d2de	Add some debug to help diagnose weird federation issue	2017-03-20 15:36:14 +00:00
Erik Johnston	aac6d1fc9b	PEP8	2017-03-20 13:47:56 +00:00
Richard van der Hoff	bd08ee7a46	Merge pull request #2026 from matrix-org/rav/logcontext_docs ... Logcontext docs	2017-03-20 12:05:21 +00:00
Richard van der Hoff	eddce9d74a	Merge pull request #2027 from matrix-org/rav/logcontext_leaks ... A few fixes to logcontext things	2017-03-20 11:53:36 +00:00
Richard van der Hoff	f40c2db05a	Stop preserve_fn leaking context into the reactor ... Fix a bug in ``logcontext.preserve_fn`` which made it leak context into the reactor, and add a test for it. Also, get rid of ``logcontext.reset_context_after_deferred``, which tried to do the same thing but had its own, different, set of bugs.	2017-03-18 00:07:43 +00:00
Richard van der Hoff	067b00d49d	Run the reactor with the sentinel logcontext ... This fixes a class of 'Unexpected logcontext' messages, which were happening because the logcontext was somewhat arbitrarily swapping between the sentinel and the `run` logcontext.	2017-03-18 00:07:43 +00:00
Richard van der Hoff	994d7ae7c5	Remove broken use of clock.call_later ... background_updates was using `call_later` in a way that leaked the logcontext into the reactor. We could have rewritten it to do it properly, but given that we weren't using the fancier facilities provided by `call_later`, we might as well just use `async.sleep`, which does the logcontext stuff properly.	2017-03-18 00:01:37 +00:00
Richard van der Hoff	d2d146a314	Logcontext docs	2017-03-17 23:59:28 +00:00
Erik Johnston	61f471f779	Don't send the full event json over replication	2017-03-17 15:50:01 +00:00
Richard van der Hoff	0c01f829ae	Avoid resetting state on rejected events ... When we get a rejected event, give it the same state_group as its prev_event, rather than no state_group at all. This should fix https://github.com/matrix-org/synapse/issues/1935.	2017-03-17 15:06:08 +00:00
Richard van der Hoff	5068fb16a5	Refactoring and cleanups ... A few non-functional changes: * A bunch of docstrings to document types * Split `EventsStore._persist_events_txn` up a bit. Hopefully it's a bit more readable. * Rephrase `EventFederationStore._update_min_depth_for_room_txn` to avoid mind-bending conditional. * Rephrase rejected/outlier conditional in `_update_outliers_txn` to avoid mind-bending conditional.	2017-03-17 15:06:07 +00:00
Richard van der Hoff	2abe85d50e	Merge pull request #2016 from matrix-org/rav/queue_pdus_during_join ... Queue up federation PDUs while a room join is in progress	2017-03-17 11:32:44 +00:00
Erik Johnston	da146657c9	Comments	2017-03-16 13:04:07 +00:00
Erik Johnston	a158c36a8a	Comment	2017-03-16 11:57:45 +00:00
Erik Johnston	6957bfdca6	Don't recreate so many sets	2017-03-16 11:54:26 +00:00
Erik Johnston	2ccf3b241c	Implement no op for room stream in sync	2017-03-16 11:06:41 +00:00
Richard van der Hoff	9ce53a3861	Queue up federation PDUs while a room join is in progress ... This just takes the existing `room_queues` logic and moves it out to `on_receive_pdu` instead of `_process_received_pdu`, which ensures that we don't start trying to fetch prev_events and whathaveyou until the join has completed.	2017-03-15 18:01:11 +00:00
Erik Johnston	54d2b7e596	Merge pull request #2014 from Half-Shot/hs/fix-appservice-presence ... Add fallback to last_active_ts if it beats the last sync time on a presence timeout.	2017-03-15 17:37:15 +00:00
Erik Johnston	a8f96c63aa	Comment	2017-03-15 16:01:01 +00:00
Erik Johnston	f83ac78201	Cache set of users whose presence the other user should see	2017-03-15 15:29:19 +00:00
Will Hunt	e6032054bf	Add a great comment to handle_timeout for active vs sync times.	2017-03-15 15:24:48 +00:00
Will Hunt	ebf5a6b14c	Add fallback to last_active_ts if it beats the last sync time.	2017-03-15 15:17:16 +00:00
Erik Johnston	e892457a03	Comment	2017-03-15 15:01:39 +00:00
Erik Johnston	a297155a97	Remove unused import	2017-03-15 14:49:25 +00:00
Erik Johnston	6c82de5100	Format presence events on the edges instead of reformatting them multiple times	2017-03-15 14:27:34 +00:00
David Baker	0ad44acb5a	Merge pull request #1997 from matrix-org/dbkr/cas_partialdownload ... Handle PartialDownloadError in CAS login	2017-03-15 13:52:34 +00:00
Matthew Hodgson	0970e0307e	typo	2017-03-15 12:40:42 +00:00
Matthew Hodgson	5aa42d4292	set default for turn_allow_guests correctly	2017-03-15 12:40:13 +00:00
Matthew Hodgson	e0ff66251f	add setting (on by default) to support TURN for guests	2017-03-15 12:22:18 +00:00
Richard van der Hoff	29ed09e80a	Fix assertion to stop transaction queue getting wedged ... ... and update some docstrings to correctly reflect the types being used. get_new_device_msgs_for_remote can return a long under some circumstances, which was being stored in last_device_list_stream_id_by_dest, and was then upsetting things on the next loop.	2017-03-15 12:16:55 +00:00
Erik Johnston	872e75a3d5	Add some metrics on notifier	2017-03-15 10:56:51 +00:00
Erik Johnston	7827251daf	Merge pull request #1994 from matrix-org/dbkr/msisdn_signin_2 ... Phone number registration / login support v2	2017-03-15 09:59:54 +00:00
Richard van der Hoff	b5d1c68beb	Implement reset_context_after_deferred ... to correctly reset the context when we fire off a deferred we aren't going to wait for.	2017-03-15 02:21:07 +00:00
Richard van der Hoff	f2ed64eaaf	Merge pull request #1992 from matrix-org/rav/fix_media_loop ... Fix routing loop when fetching remote media	2017-03-14 23:40:35 +00:00
Erik Johnston	bad72b0b8e	Merge pull request #2002 from matrix-org/erikj/dont_sync_by_default ... Reduce number of spurious sync result generations.	2017-03-14 16:58:45 +00:00
Erik Johnston	fd2eef49c8	Reduce spurious calls to generate sync	2017-03-14 16:33:45 +00:00
Richard van der Hoff	1d09586599	Address review comments ... - don't blindly proxy all HTTPRequestExceptions - log unexpected exceptions at error - avoid `isinstance` - improve docs on `from_http_response_exception`	2017-03-14 14:15:37 +00:00
Richard van der Hoff	7f237800e9	re-refactor exception heirarchy ... Give CodeMessageException back its `msg` attribute, and use that to hold the HTTP status message for HttpResponseException.	2017-03-14 14:15:37 +00:00
David Baker	1ece06273e	Handle PartialDownloadError in CAS login	2017-03-14 13:37:36 +00:00
Erik Johnston	bb256ac96f	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/public_list_speed	2017-03-14 11:35:05 +00:00
Erik Johnston	cc7a294e2e	Fix current_state_events table to not lie ... If we try and persist two state events that have the same ancestor we calculate the wrong current state when persisting those events.	2017-03-14 10:57:43 +00:00
David Baker	7b6ed9871e	Use extend instead of +=	2017-03-14 10:49:55 +00:00
David Baker	d79a687d85	Oops, remove print	2017-03-14 10:40:20 +00:00
Luke Barnard	f29d85d9e4	Merge pull request #1993 from matrix-org/luke/delete-devices ... Implement delete_devices API	2017-03-14 10:02:56 +00:00
Luke Barnard	bbeeb97f75	Implement _simple_delete_many_txn, use it to delete devices ... (But this doesn't implement the same for deleting access tokens or e2e keys. Also respond to code review.	2017-03-13 17:53:23 +00:00
David Baker	0a9945220e	Fix registration for broken clients ... Only offer msisdn flows if the x_show_msisdn option is given.	2017-03-13 17:29:38 +00:00
David Baker	73a5f06652	Support registration / login with phone number ... Changes from https://github.com/matrix-org/synapse/pull/1971	2017-03-13 17:27:51 +00:00
Luke Barnard	c077c3277b	Flake	2017-03-13 16:45:38 +00:00
Richard van der Hoff	31f3ca1b2b	Merge pull request #1990 from matrix-org/rav/log_config_comments ... Add helpful texts to logger config options	2017-03-13 16:42:12 +00:00
Luke Barnard	c81f33f73d	Implement delete_devices API ... This implements the proposal here https://docs.google.com/document/d/1C-25Gqz3TXy2jIAoeOKxpNtmme0jI4g3yFGqv5GlAAk for deleting multiple devices at once in a single request.	2017-03-13 16:33:51 +00:00
Richard van der Hoff	170ccc9de5	Fix routing loop when fetching remote media ... When we proxy a media request to a remote server, add a query-param, which will tell the remote server to 404 if it doesn't recognise the server_name. This should fix a routing loop where the server keeps forwarding back to itself. Also improves the error handling on remote media fetches, so that we don't always return a rather obscure 502.	2017-03-13 16:30:36 +00:00
Erik Johnston	45c7f12d2a	Add new storage function to slave store	2017-03-13 16:26:44 +00:00
Richard van der Hoff	6037a9804c	Add helpful texts to logger config options	2017-03-13 12:33:35 +00:00
Richard van der Hoff	3c69f32402	Merge remote-tracking branch 'origin/develop' into rav/refactor_received_pdu	2017-03-13 12:20:47 +00:00
Richard van der Hoff	6bfe8e32b5	Merge pull request #1983 from matrix-org/rav/no_redirect_stdio ... Add an option to disable stdio redirect	2017-03-13 12:20:07 +00:00
Richard van der Hoff	5fc9261929	Merge pull request #1982 from matrix-org/rav/sighup_for_logconfig ... Reread log config on SIGHUP	2017-03-13 12:19:54 +00:00
Erik Johnston	0162994983	Comments	2017-03-13 11:53:26 +00:00
Erik Johnston	254b7c5b15	Bump changelog and versions	2017-03-13 10:02:58 +00:00
Erik Johnston	672dcf59d3	Merge branch 'develop' of github.com:matrix-org/synapse into release-v0.19.3	2017-03-13 09:59:54 +00:00
Erik Johnston	7eae6eaa2f	Revert "Support registration & login with phone number"	2017-03-13 09:59:33 +00:00
Erik Johnston	79926e016e	Assume rooms likely haven't changed	2017-03-13 09:50:10 +00:00
Matthew Hodgson	a61dd408ed	enable guest access for the 3pl/3pid APIs	2017-03-12 19:30:45 +00:00
Erik Johnston	8ffbe43ba1	Get current state by using current_state_events table	2017-03-10 17:39:35 +00:00
Richard van der Hoff	bcfa5cd00c	Add an option to disable stdio redirect ... This makes it tractable to run synapse under pdb.	2017-03-10 15:38:29 +00:00
Richard van der Hoff	d84bd51e95	Refactor logger config for workers ... - to make it easier to add more config options.	2017-03-10 15:34:01 +00:00
Richard van der Hoff	9072a8c627	Reread log config on SIGHUP ... When we are using a log_config file, reread it on SIGHUP.	2017-03-10 15:29:55 +00:00
Erik Johnston	8f267fa8a8	Fix it for the workers	2017-03-10 11:22:25 +00:00
Erik Johnston	64d62e41b8	Noop repated delete device inbox calls from /sync	2017-03-10 10:36:43 +00:00
Erik Johnston	3545e17f43	Add setdefault key to ExpiringCache	2017-03-10 10:30:49 +00:00
Richard van der Hoff	29235901b8	Move FederationServer._handle_new_pdu to FederationHandler ... Unfortunately this significantly increases the size of the already-rather-big FederationHandler, but the code fits more naturally here, and it paves the way for the tighter integration that I need between handling incoming PDUs and doing the join dance. Other than renaming the existing `FederationHandler.on_receive_pdu` to `_process_received_pdu` to make way for it, this just consists of the move, and replacing `self.handler` with `self` and `self` with `self.replication_layer`.	2017-03-09 16:20:13 +00:00
Richard van der Hoff	e8b1721290	Move sig check out of _handle_new_pdu ... When we receive PDUs via `get_missing_events`, we have already checked their sigs, so there is no need to do it again.	2017-03-09 15:50:44 +00:00
Richard van der Hoff	3406333a58	Factor _get_missing_events_for_pdu out of _handle_new_pdu ... This should be functionally identical: it just seeks to improve readability by reducing indentation.	2017-03-09 15:50:44 +00:00
Richard van der Hoff	45d173a59a	Fix docstring	2017-03-09 15:50:29 +00:00
David Baker	663396e45d	Merge pull request #1971 from matrix-org/dbkr/msisdn_signin ... Support registration & login with phone number	2017-03-09 10:18:51 +00:00
David Baker	ece7e00048	Comment when our 3pids would be incomplete	2017-03-08 19:07:18 +00:00
David Baker	9d0d40fc15	Docs	2017-03-08 19:05:29 +00:00
David Baker	3edc57296d	Incorrectly copied copyright ... This file post-dates OM	2017-03-08 19:00:51 +00:00
David Baker	727124a762	Not any more, it doesn't	2017-03-08 19:00:23 +00:00
Richard van der Hoff	6ad71cc29d	Remove spurious SQL logging (#1972) ... looks like the upsert function was accidentally sending sql logging to the general logger. We already log the sql in `txn.execute`.	2017-03-08 18:00:44 +00:00
David Baker	d4d3629aaf	Better error message	2017-03-08 17:01:26 +00:00
Erik Johnston	3170c56e07	Bump changelog and version	2017-03-08 13:00:47 +00:00
David Baker	1c99934b28	pep8	2017-03-08 11:58:20 +00:00
David Baker	a9e2b9ec16	Add msisdn util file	2017-03-08 11:53:36 +00:00
David Baker	85bb322333	Pull out datastore in initialiser	2017-03-08 11:51:25 +00:00
David Baker	65d43f3ca5	Minor fixes from PR feedback	2017-03-08 11:48:43 +00:00
David Baker	0e0aee25c4	Fix log line	2017-03-08 11:46:22 +00:00
David Baker	82c5e7de25	Typos	2017-03-08 11:42:44 +00:00
David Baker	2e27339add	Refector out assert_params_in_request ... and replace requestEmailToken where we meant requestMsisdnToken	2017-03-08 11:37:34 +00:00
David Baker	88df6c0c9a	Factor out msisdn canonicalisation ... Plus a couple of other minor fixes	2017-03-08 11:03:39 +00:00
David Baker	402a7bf63d	Fix pep8	2017-03-08 09:33:40 +00:00
David Baker	00466e2feb	Support new login format ... https://docs.google.com/document/d/1-6ZSSW5YvCGhVFDyD2QExAUAdpCWjccvJT5xiyTTG2Y/edit#	2017-03-07 16:37:23 +00:00
Erik Johnston	ac5491f563	Select distinct devices from DB ... Otherwise we might pull out tonnes of duplicate user_ids and this can make synapse sad.	2017-03-06 11:10:14 +00:00
David Baker	b0effa2160	Add msisdns as 3pids during registration ... and support binding them with the bind_msisdn param	2017-03-03 18:34:39 +00:00
Erik Johnston	82f7f1543b	Merge pull request #1964 from matrix-org/erikj/device_list_update_fix ... Fix device list update to not constantly resync	2017-03-03 16:18:16 +00:00
Erik Johnston	f2581ee8b8	Don't keep around old stream IDs forever	2017-03-03 16:02:53 +00:00