2365 Commits

Author	SHA1	Message	Date
Erik Johnston	2f34ad31ac	Add some logging to user directory	2017-06-07 11:50:44 +01:00
Erik Johnston	09e4bc0501	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/state_fixup	2017-06-07 11:05:23 +01:00
Erik Johnston	1a01af079e	Handle profile updates in user directory	2017-06-01 15:39:51 +01:00
Erik Johnston	21e255a8f1	Split the table in two	2017-06-01 14:50:46 +01:00
Erik Johnston	7233341eac	Comments	2017-06-01 13:11:38 +01:00
Erik Johnston	8be6fd95a3	Check if host is still in room	2017-06-01 13:05:39 +01:00
Erik Johnston	9c7db2491b	Fix removing users	2017-06-01 11:36:50 +01:00
Erik Johnston	0fe6f3c521	Bug fixes and logging ... - Check if room is public when a user joins before adding to user dir - Fix typo of field name "content.join_rules" -> "content.join_rule"	2017-06-01 11:09:49 +01:00
Erik Johnston	cc7609aa9f	Comment briefly on how we keep user_directory up to date	2017-05-31 17:11:18 +01:00
Erik Johnston	b2d8d07109	Lifts things into separate function	2017-05-31 17:00:24 +01:00
Erik Johnston	f9791498ae	Typos	2017-05-31 16:50:57 +01:00
Erik Johnston	f091061711	Fix tests	2017-05-31 16:34:40 +01:00
Erik Johnston	dc51af3d03	Pull max id from correct table	2017-05-31 15:13:49 +01:00
Erik Johnston	350622a107	Handle the server leaving a public room	2017-05-31 15:11:36 +01:00
Erik Johnston	63fda37e20	Add comments	2017-05-31 15:00:29 +01:00
Erik Johnston	3b5f22ca40	Add search	2017-05-31 14:00:01 +01:00
Erik Johnston	b5db4ed5f6	Update room column when room becomes unpublic	2017-05-31 13:40:28 +01:00
Erik Johnston	168524543f	Add call later	2017-05-31 11:59:36 +01:00
Erik Johnston	3e123b8497	Start later	2017-05-31 11:56:27 +01:00
Erik Johnston	42137efde7	Don't go round in circles	2017-05-31 11:55:13 +01:00
Erik Johnston	eeb2f9e546	Add user_directory to database	2017-05-31 11:51:01 +01:00
Erik Johnston	23da638360	Fix typing tests	2017-05-26 10:02:04 +01:00
Erik Johnston	11f139a647	Merge pull request #2247 from matrix-org/erikj/auth_event ... Only store event_auth for state events	2017-05-24 16:46:34 +01:00
Erik Johnston	c049472b8a	Only store event_auth for state events	2017-05-24 15:23:31 +01:00
Erik Johnston	b4f59c7e27	Add count of one time keys to sync stream	2017-05-19 15:47:55 +01:00
Erik Johnston	cafe659f72	Store ActionGenerator in HomeServer	2017-05-19 10:09:56 +01:00
Erik Johnston	66d8ffabbd	Faster push rule calculation via push specific cache ... We add a push rule specific cache that ensures that we can reuse calculated push rules appropriately when a user join/leaves.	2017-05-17 16:55:40 +01:00
Erik Johnston	6fa8148ccb	Merge pull request #2228 from matrix-org/erikj/speed_up_get_hosts ... Speed up get_joined_hosts	2017-05-16 17:40:55 +01:00
Erik Johnston	13f540ef1b	Speed up get_joined_hosts	2017-05-16 16:05:22 +01:00
Erik Johnston	ec5c4499f4	Make presence use cached users/hosts in room	2017-05-16 16:01:43 +01:00
Luke Barnard	ccad2ed824	Modify condition on empty localpart	2017-05-10 17:34:30 +01:00
Luke Barnard	369195caa5	Modify register/available to be GET with query param ... - GET is now the method for register/available - a query parameter "username" is now used Also, empty usernames are now handled with an error message on registration or via register/available: `User ID cannot be empty`	2017-05-10 17:23:55 +01:00
Erik Johnston	a3648f84b2	Merge pull request #2208 from matrix-org/erikj/ratelimit_overrid ... Add per user ratelimiting overrides	2017-05-10 15:54:48 +01:00
Erik Johnston	b990b2fce5	Add per user ratelimiting overrides	2017-05-10 11:05:43 +01:00
Richard van der Hoff	aedaba018f	Replace some instances of preserve_context_over_deferred	2017-05-09 19:04:56 +01:00
Richard van der Hoff	de042b3b88	Do some logging when one-time-keys get claimed ... might help us figure out if https://github.com/vector-im/riot-web/issues/3868 has happened.	2017-05-09 19:04:56 +01:00
Richard van der Hoff	a7e9d8762d	Allow clients to upload one-time-keys with new sigs ... When a client retries a key upload, don't give an error if the signature has changed (but the key is the same). Fixes https://github.com/vector-im/riot-android/issues/1208, hopefully.	2017-05-09 19:04:56 +01:00
Erik Johnston	27c28eaa27	Merge pull request #2190 from matrix-org/erikj/mark_remote_as_back_more ... Always mark remotes as up if we receive a signed request from them	2017-05-05 14:08:12 +01:00
Erik Johnston	653d90c1a5	Comment	2017-05-05 14:01:17 +01:00
Erik Johnston	7b222fc56e	Remove redundant reset of destination timers	2017-05-05 11:14:09 +01:00
Erik Johnston	b843631d71	Add comment and TODO	2017-05-05 10:59:32 +01:00
Erik Johnston	f346048a6e	Handle exceptions thrown in handling remote device list updates	2017-05-05 10:34:10 +01:00
David Baker	60833c8978	Merge pull request #2147 from matrix-org/dbkr/http_request_propagate_error ... Propagate errors sensibly from proxied IS requests	2017-05-03 11:23:25 +01:00
Erik Johnston	0c27383dd7	Merge pull request #2170 from matrix-org/erikj/fed_hole_state ... Don't fetch state for missing events that we fetched	2017-05-03 10:49:37 +01:00
Erik Johnston	ef862186dd	Merge together redundant calculations/logging	2017-05-03 10:06:43 +01:00
Erik Johnston	3e5a62ecd8	Add more granular event send metrics	2017-05-02 14:23:26 +01:00
Erik Johnston	2347efc065	Fixup	2017-04-28 12:46:53 +01:00
Erik Johnston	9b147cd730	Remove unncessary call in _get_missing_events_for_pdu	2017-04-28 11:55:25 +01:00
Erik Johnston	3a9f5bf6dd	Don't fetch state for missing events that we fetched	2017-04-28 11:26:46 +01:00
Erik Johnston	25a96e0c63	Merge pull request #2163 from matrix-org/erikj/fix_invite_state ... Fix invite state to always include all events	2017-04-27 17:36:30 +01:00
Erik Johnston	87ae59f5e9	Typo	2017-04-27 15:16:21 +01:00
Erik Johnston	e42b4ebf0f	Add some extra logging for edge cases of federation	2017-04-27 14:38:21 +01:00
Erik Johnston	34e682d385	Fix invite state to always include all events	2017-04-26 16:18:08 +01:00
David Baker	82ae0238f9	Revert accidental commit	2017-04-26 11:43:16 +01:00
David Baker	81804909d3	Merge remote-tracking branch 'origin/develop' into dbkr/http_request_propagate_error	2017-04-26 11:31:55 +01:00
David Baker	1a9255c12e	Use CodeMessageException subclass instead ... Parse json errors from get_json client methods and throw special errors.	2017-04-25 19:30:55 +01:00
David Baker	2e165295b7	Merge remote-tracking branch 'origin/develop' into dbkr/http_request_propagate_error	2017-04-21 11:35:52 +01:00
David Baker	a90a0f5c8a	Propagate errors sensibly from proxied IS requests ... When we're proxying Matrix endpoints, parse out Matrix error responses and turn them into SynapseErrors so they can be propagated sensibly upstream.	2017-04-21 11:32:48 +01:00
Richard van der Hoff	0cdb32fc43	Remove redundant try/except clauses ... The `except SynapseError` clauses were pointless because the wrapped functions would never throw a `SynapseError` (they either throw a `CodeMessageException` or a `RuntimeError`). The `except CodeMessageException` is now also pointless because the caller treats all exceptions equally, so we may as well just throw the `CodeMessageException`.	2017-04-21 01:32:01 +01:00
Richard van der Hoff	838810b76a	Broaden the conditions for locally_rejecting invites ... The logic for marking invites as locally rejected was all well and good, but didn't happen when the remote server returned a 500, or wasn't reachable, or had no DNS, or whatever. Just expand the except clause to catch everything. Fixes https://github.com/matrix-org/synapse/issues/761.	2017-04-21 01:31:37 +01:00
Richard van der Hoff	736b9a4784	Remove redundant function ... inline `reject_remote_invite`, which only existed to make tracing the callflow more difficult.	2017-04-21 01:31:09 +01:00
Luke Barnard	3fb8784c92	m.read_marker -> m.fully_read (#2128) ... Also: - change the REST endpoint to have a "S" on the end (so it's now /read_markers) - change the content of the m.read_up_to event to have the key "event_id" instead of "marker".	2017-04-18 17:46:15 +01:00
Luke Barnard	78f0ddbfad	Merge pull request #2120 from matrix-org/luke/read-markers ... Implement Read Marker API	2017-04-13 14:21:31 +01:00
Luke Barnard	b9557064bf	Simplify is_event_after logic	2017-04-12 14:36:20 +01:00
Erik Johnston	247c736b9b	Merge pull request #2115 from matrix-org/erikj/dedupe_federation_repl ... Reduce federation replication traffic	2017-04-12 11:07:13 +01:00
Luke Barnard	69a18514e9	Only notify user, not entire room	2017-04-12 10:50:37 +01:00
Luke Barnard	122cd52ce4	Remove comment, simplify null-guard	2017-04-12 10:48:32 +01:00
Erik Johnston	c7ddb5ef7a	Reuse get_interested_parties	2017-04-12 10:16:26 +01:00
Luke Barnard	77fb2b72ae	Handle no previous RM	2017-04-12 09:47:29 +01:00
Luke Barnard	867822fa1e	flake8	2017-04-11 17:36:04 +01:00
Luke Barnard	73880268ef	Refactor event ordering check to events store	2017-04-11 17:34:09 +01:00
Luke Barnard	131485ef66	Copyright	2017-04-11 17:33:51 +01:00
Luke Barnard	0127423027	flake8	2017-04-11 17:07:07 +01:00
Erik Johnston	414522aed5	Move get_interested_parties	2017-04-11 15:33:26 +01:00
Erik Johnston	2be8a281d2	Comments	2017-04-11 15:28:24 +01:00
Erik Johnston	6308ac45b0	Move get_interested_remotes back to presence handler	2017-04-11 15:19:26 +01:00
Luke Barnard	d892079844	Finish implementing RM endpoint ... - This change causes a 405 to be sent if "m.read_marker" is set via /account_data - This also fixes-up the RM endpoint so that it actually Works.	2017-04-11 15:01:39 +01:00
lukebarnard	e263c26690	Initial commit of RM server-side impl ... (See https://docs.google.com/document/d/1UWqdS-e1sdwkLDUY0wA4gZyIkRp-ekjsLZ8k6g_Zvso/edit#heading=h.lndohpg8at5u)	2017-04-11 11:55:30 +01:00
Erik Johnston	f3cf3ff8b6	Merge branch 'master' of github.com:matrix-org/synapse into develop	2017-04-11 11:13:32 +01:00
Erik Johnston	29574fd5b3	Reduce federation presence replication traffic ... This is mainly done by moving the calculation of where to send presence updates from the presence handler to the transaction queue, so we only need to send the presence event (and not the destinations) across the replication connection. Before we were duplicating by sending the full state across once per destination.	2017-04-10 16:48:30 +01:00
Richard van der Hoff	64765e5199	When we do an invite rejection, save the signed leave event to the db ... During a rejection of an invite received over federation, we ask a remote server to make us a `leave` event, then sign it, then send that with `send_leave`. We were saving the *unsigned* version of the event (which has a different event id to the signed version) to our db (and sending it to the clients), whereas other servers in the room will have seen the *signed* version. We're not aware of any actual problems that caused, except that it makes the database confusing to look at and generally leaves the room in a weird state.	2017-04-07 14:39:32 +01:00
Erik Johnston	98ce212093	Merge pull request #2103 from matrix-org/erikj/no-double-encode ... Don't double encode replication data	2017-04-07 09:39:52 +01:00
Erik Johnston	877c029c16	Use iteritems	2017-04-06 15:51:22 +01:00
Erik Johnston	96b9b6c127	Don't double json encode typing replication data	2017-04-05 11:34:20 +01:00
Erik Johnston	62b89daac6	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/repl_tcp_server	2017-04-04 09:46:16 +01:00
Richard van der Hoff	773e64cc1a	Merge pull request #2095 from matrix-org/rav/cull_log_preserves ... Cull spurious PreserveLoggingContexts	2017-04-03 17:02:25 +01:00
Richard van der Hoff	30bcbf775a	Accept join events from all servers ... Make sure that we accept join events from any server, rather than just the origin server, to make the federation join dance work correctly. (Fixes #1893).	2017-04-03 15:58:07 +01:00
Richard van der Hoff	7eb9f34cc3	Remove spurious yield ... In `MessageHandler`, remove `yield` on call to `Notifier.on_new_room_event`: it doesn't return anything anyway.	2017-04-03 15:44:19 +01:00
Richard van der Hoff	0b08c48fc5	Remove more spurious PreserveLoggingContexts ... Remove `PreserveLoggingContext` around calls to `Notifier.on_new_room_event`; there is no problem if the logcontext is set when calling it.	2017-04-03 15:43:37 +01:00
Erik Johnston	36d2b66f90	Add a timestamp to USER_SYNC command ... This timestamp is used to indicate when the user last sync'd	2017-03-31 15:42:22 +01:00
Erik Johnston	9d0170ac6c	Fix up presence	2017-03-31 11:36:32 +01:00
Erik Johnston	63fcc42990	Remove user from process_presence when stops syncing	2017-03-30 14:26:08 +01:00
Erik Johnston	e9dd8370b0	Add functions to presence to support remote syncs ... The TCP replication protocol streams deltas of who has started or stopped syncing. This is different from the HTTP API which periodically sends the full list of users who are syncing. This commit adds support for the new TCP style of sending deltas.	2017-03-30 13:25:14 +01:00
Erik Johnston	24d35ab47b	Add new storage functions for new replication ... The new replication protocol will keep all the streams separate, rather than muxing multiple streams into one.	2017-03-30 11:48:35 +01:00
Erik Johnston	7fc1f1e2b6	Cache hosts in room	2017-03-24 11:46:24 +00:00
Richard van der Hoff	9397edb28b	Merge pull request #2050 from matrix-org/rav/federation_backoff ... push federation retry limiter down to matrixfederationclient	2017-03-23 22:27:01 +00:00
Richard van der Hoff	5a16cb4bf0	Ignore backoff history for invites, aliases, and roomdirs ... Add a param to the federation client which lets us ignore historical backoff data for federation queries, and set it for a handful of operations.	2017-03-23 12:23:22 +00:00
Richard van der Hoff	4bd597d9fc	push federation retry limiter down to matrixfederationclient ... rather than having to instrument everywhere we make a federation call, make the MatrixFederationHttpClient manage the retry limiter.	2017-03-23 09:28:46 +00:00
Richard van der Hoff	e08f81d96a	Add a missing yield in device key upload ... (this would only very very rarely actually be a useful thing, so the main problem was the logcontext leak...)	2017-03-23 00:16:43 +00:00
Richard van der Hoff	eddce9d74a	Merge pull request #2027 from matrix-org/rav/logcontext_leaks ... A few fixes to logcontext things	2017-03-20 11:53:36 +00:00
Richard van der Hoff	f40c2db05a	Stop preserve_fn leaking context into the reactor ... Fix a bug in ``logcontext.preserve_fn`` which made it leak context into the reactor, and add a test for it. Also, get rid of ``logcontext.reset_context_after_deferred``, which tried to do the same thing but had its own, different, set of bugs.	2017-03-18 00:07:43 +00:00
Richard van der Hoff	5068fb16a5	Refactoring and cleanups ... A few non-functional changes: * A bunch of docstrings to document types * Split `EventsStore._persist_events_txn` up a bit. Hopefully it's a bit more readable. * Rephrase `EventFederationStore._update_min_depth_for_room_txn` to avoid mind-bending conditional. * Rephrase rejected/outlier conditional in `_update_outliers_txn` to avoid mind-bending conditional.	2017-03-17 15:06:07 +00:00
Richard van der Hoff	2abe85d50e	Merge pull request #2016 from matrix-org/rav/queue_pdus_during_join ... Queue up federation PDUs while a room join is in progress	2017-03-17 11:32:44 +00:00
Erik Johnston	da146657c9	Comments	2017-03-16 13:04:07 +00:00
Erik Johnston	a158c36a8a	Comment	2017-03-16 11:57:45 +00:00
Erik Johnston	6957bfdca6	Don't recreate so many sets	2017-03-16 11:54:26 +00:00
Erik Johnston	2ccf3b241c	Implement no op for room stream in sync	2017-03-16 11:06:41 +00:00
Richard van der Hoff	9ce53a3861	Queue up federation PDUs while a room join is in progress ... This just takes the existing `room_queues` logic and moves it out to `on_receive_pdu` instead of `_process_received_pdu`, which ensures that we don't start trying to fetch prev_events and whathaveyou until the join has completed.	2017-03-15 18:01:11 +00:00
Erik Johnston	54d2b7e596	Merge pull request #2014 from Half-Shot/hs/fix-appservice-presence ... Add fallback to last_active_ts if it beats the last sync time on a presence timeout.	2017-03-15 17:37:15 +00:00
Erik Johnston	f83ac78201	Cache set of users whose presence the other user should see	2017-03-15 15:29:19 +00:00
Will Hunt	e6032054bf	Add a great comment to handle_timeout for active vs sync times.	2017-03-15 15:24:48 +00:00
Will Hunt	ebf5a6b14c	Add fallback to last_active_ts if it beats the last sync time.	2017-03-15 15:17:16 +00:00
Erik Johnston	e892457a03	Comment	2017-03-15 15:01:39 +00:00
Erik Johnston	6c82de5100	Format presence events on the edges instead of reformatting them multiple times	2017-03-15 14:27:34 +00:00
Erik Johnston	7827251daf	Merge pull request #1994 from matrix-org/dbkr/msisdn_signin_2 ... Phone number registration / login support v2	2017-03-15 09:59:54 +00:00
Erik Johnston	bb256ac96f	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/public_list_speed	2017-03-14 11:35:05 +00:00
Luke Barnard	bbeeb97f75	Implement _simple_delete_many_txn, use it to delete devices ... (But this doesn't implement the same for deleting access tokens or e2e keys. Also respond to code review.	2017-03-13 17:53:23 +00:00
David Baker	73a5f06652	Support registration / login with phone number ... Changes from https://github.com/matrix-org/synapse/pull/1971	2017-03-13 17:27:51 +00:00
Richard van der Hoff	3c69f32402	Merge remote-tracking branch 'origin/develop' into rav/refactor_received_pdu	2017-03-13 12:20:47 +00:00
Erik Johnston	0162994983	Comments	2017-03-13 11:53:26 +00:00
Erik Johnston	7eae6eaa2f	Revert "Support registration & login with phone number"	2017-03-13 09:59:33 +00:00
Erik Johnston	79926e016e	Assume rooms likely haven't changed	2017-03-13 09:50:10 +00:00
Erik Johnston	8ffbe43ba1	Get current state by using current_state_events table	2017-03-10 17:39:35 +00:00
Erik Johnston	64d62e41b8	Noop repated delete device inbox calls from /sync	2017-03-10 10:36:43 +00:00
Richard van der Hoff	29235901b8	Move FederationServer._handle_new_pdu to FederationHandler ... Unfortunately this significantly increases the size of the already-rather-big FederationHandler, but the code fits more naturally here, and it paves the way for the tighter integration that I need between handling incoming PDUs and doing the join dance. Other than renaming the existing `FederationHandler.on_receive_pdu` to `_process_received_pdu` to make way for it, this just consists of the move, and replacing `self.handler` with `self` and `self` with `self.replication_layer`.	2017-03-09 16:20:13 +00:00
David Baker	663396e45d	Merge pull request #1971 from matrix-org/dbkr/msisdn_signin ... Support registration & login with phone number	2017-03-09 10:18:51 +00:00
David Baker	0e0aee25c4	Fix log line	2017-03-08 11:46:22 +00:00
David Baker	88df6c0c9a	Factor out msisdn canonicalisation ... Plus a couple of other minor fixes	2017-03-08 11:03:39 +00:00
David Baker	402a7bf63d	Fix pep8	2017-03-08 09:33:40 +00:00
Erik Johnston	f2581ee8b8	Don't keep around old stream IDs forever	2017-03-03 16:02:53 +00:00
Erik Johnston	9834367eea	Spelling	2017-03-03 15:31:57 +00:00
David Baker	ad882cd54d	Just return the deferred straight off ... defer.returnValue doth not maketh a generator: it would need a yield to be a generator, and this doesn't need a yield.	2017-03-01 18:08:51 +00:00
David Baker	3557cf34dc	Merge remote-tracking branch 'origin/develop' into dbkr/msisdn_signin	2017-03-01 17:20:37 +00:00
Erik Johnston	36be39b8b3	Fix device list update to not constantly resync	2017-03-01 14:12:11 +00:00
Erik Johnston	848cf95ea0	Pop with default value to stop throwing	2017-02-28 10:02:54 +00:00
Erik Johnston	64a2cef9bb	Pop rather than del from dict	2017-02-27 19:15:36 +00:00
Erik Johnston	17673404fb	Remove unused param	2017-02-20 15:02:01 +00:00
Erik Johnston	7f026792e1	Fix /context/ visibiltiy rules	2017-02-20 14:54:50 +00:00
David Baker	8c87bb550e	Merge pull request #1922 from matrix-org/dbkr/allow_forget_for_ban ... Allow forgetting rooms you're banned from	2017-02-17 10:52:30 +00:00
David Baker	474c9aadbe	Allow forgetting rooms you're banned from	2017-02-15 19:32:20 +00:00
David Baker	ce3e583d94	WIP support for msisdn 3pid proxy methods	2017-02-14 15:05:55 +00:00
Richard van der Hoff	fc2f29c1d0	Fix bugs in the /keys/changes api ... * `get_forward_extremeties_for_room` takes a numeric `stream_ordering`. We were passing a `RoomStreamToken`, which meant that it returned the *current* extremities, rather than those corresponding to the `from_token`. However: * `get_state_ids_for_events` required a second ('types') parameter; this meant that a `TypeError` was thrown and we ended up acting as though there was *no* prev state. * `get_state_ids_for_events` actually returns a map from event_id to state dictionary - just looking up the state keys in it again meant that we acted as though there was no prev state. We now check if each member's state has changed since *any* of the extremities. Also add/fix some comments.	2017-02-14 13:59:50 +00:00
Erik Johnston	9e617cd4c2	Cache get_presence storage	2017-02-13 13:50:03 +00:00
Erik Johnston	af6da6db2d	Merge pull request #1784 from morteza-araby/user-admin ... Administration functionalities	2017-02-06 16:21:10 +01:00
Erik Johnston	a597994fb6	Measure new device list stuff	2017-02-02 18:36:17 +00:00
Erik Johnston	1232ae41cf	Use new get_users_who_share_room_with_user	2017-02-02 15:25:00 +00:00
Erik Johnston	99fa03e8b5	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/device_list_fixes	2017-02-02 15:23:45 +00:00
Erik Johnston	a8331897aa	Merge pull request #1876 from matrix-org/erikj/shared_member_store ... Make presence.get_new_events a bit faster	2017-02-02 15:20:14 +00:00
Erik Johnston	46ecd9fd6d	Use stream_ordering_to_exterm for /keys/changes	2017-02-02 14:27:19 +00:00
Erik Johnston	54a79c1d37	Make presence.get_new_events a bit faster ... We do this by caching the set of users a user shares rooms with.	2017-02-02 13:07:18 +00:00
Morteza Araby	2849d3f29d	admin,storage: added more administrator functionalities ... administrators can now: - Set displayname of users - Update user avatars - Search for users by user_id - Browse all users in a paginated API - Reset user passwords - Deactivate users Helpers for doing paginated queries has also been added to storage Signed-off-by: Morteza Araby <morteza.araby@ericsson.com>	2017-02-02 14:02:26 +01:00
Erik Johnston	51adaac953	Fix email push in pusher worker ... This was broken when device list updates were implemented, as Mailer could no longer instantiate an AuthHandler due to a dependency on federation sending.	2017-02-02 10:53:36 +00:00
Erik Johnston	fbfe44bb4d	Doc args	2017-02-01 17:52:57 +00:00
Erik Johnston	7e919bdbd0	Include newly joined users in /keys/changes API	2017-02-01 17:33:16 +00:00
Erik Johnston	6d6591880e	Wake sync up for device changes	2017-02-01 15:15:16 +00:00
Erik Johnston	97479d0c54	Implement /keys/changes	2017-02-01 10:30:03 +00:00
Erik Johnston	bfb3d255b1	Merge pull request #1862 from matrix-org/erikj/presence_update ... Use DB cache of joined users for presence	2017-01-31 13:23:24 +00:00
Erik Johnston	c7a26b7c32	Fix unit tests	2017-01-30 17:11:24 +00:00
Erik Johnston	fd1c18c088	Use DB cache of joined users for presence	2017-01-30 17:00:24 +00:00
Erik Johnston	c2c9a78db9	Noop device key changes if they're the same	2017-01-30 16:55:04 +00:00
Erik Johnston	828db669ec	Use get_users_in_room and declare it iterable	2017-01-30 16:37:22 +00:00
Erik Johnston	9636b2407d	Merge pull request #1857 from matrix-org/erikj/device_list_stream ... Implement device lists updates over federation	2017-01-30 14:35:21 +00:00
Erik Johnston	252b503fc8	Hook device list updates to replication	2017-01-27 14:31:35 +00:00
Erik Johnston	888c59c955	Better name	2017-01-27 10:29:47 +00:00
Erik Johnston	76d40f4904	Handle users leaving rooms	2017-01-26 16:39:33 +00:00
Erik Johnston	fbfad76c03	Add comments	2017-01-26 16:33:21 +00:00
Erik Johnston	c974116f19	Implement device key caching over federation	2017-01-26 16:07:24 +00:00
Erik Johnston	51e9fe36e4	Fix up sending of m.device_list_update edus	2017-01-25 16:55:21 +00:00
Erik Johnston	2367c5568c	Add basic implementation of local device list changes	2017-01-25 14:27:27 +00:00
Paul "LeoNerd" Evans	10e48d8310	Don't clobber a displayname or avatar_url if provided by an m.room.member event	2017-01-24 18:06:07 +00:00
Erik Johnston	a55fa2047f	Insert delta of current_state_events to be more efficient	2017-01-20 17:10:18 +00:00
Erik Johnston	09eb08f910	Derive current_state_events from state groups	2017-01-20 11:52:51 +00:00
Erik Johnston	f9058ca785	Merge pull request #1822 from matrix-org/erikj/statE_logging ... Change resolve_state_groups call site logging to DEBUG	2017-01-18 11:02:03 +00:00
Erik Johnston	15f012032c	Merge pull request #1818 from matrix-org/erikj/state_auth_splitout_split ... Optimise state resolution	2017-01-18 10:53:00 +00:00
Erik Johnston	5f027d1fc5	Change resolve_state_groups call site logging to DEBUG	2017-01-17 17:07:15 +00:00
Erik Johnston	e6153e1bd1	Fix couple of federation state bugs	2017-01-17 13:22:34 +00:00
Erik Johnston	5d6bad1b3c	Optimise state resolution	2017-01-17 13:22:19 +00:00
Erik Johnston	7b62d0bc70	Add missing None check	2017-01-11 10:57:03 +00:00
Erik Johnston	657488113e	Merge pull request #1789 from matrix-org/erikj/decouple_presence ... Don't block messages sending on bumping presence	2017-01-10 14:06:05 +00:00
Erik Johnston	3b4de17d2b	Comment	2017-01-10 14:05:53 +00:00
Erik Johnston	7d0981b312	Merge pull request #1787 from matrix-org/erikj/linearize_member ... Linearize updates to membership via PUT /state/	2017-01-10 14:04:54 +00:00
Erik Johnston	586f474a44	Don't block messages sending on bumping presence	2017-01-10 12:46:00 +00:00
Erik Johnston	6823fe5241	Linearize updates to membership via PUT /state/	2017-01-09 18:25:13 +00:00
Erik Johnston	f7085ac84f	Name linearizer's for better logs	2017-01-09 17:17:10 +00:00
Mark Haines	06c00bd19b	Merge branch 'release-v0.18.6' into develop	2017-01-06 14:46:27 +00:00
Patrik Oldsberg	1ef6084b75	handlers/room_member: fix guest access check when joining rooms ... Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>	2017-01-06 14:36:56 +00:00
Matthew Hodgson	bd85434cb3	Merge branch 'release-v0.18.6' into develop	2017-01-05 13:58:19 +00:00
Matthew Hodgson	883ff92a7f	Fix case	2017-01-05 13:45:02 +00:00
Matthew Hodgson	d79d165761	add logging for all the places we call resolve_state_groups. my kingdom for a backtrace that actually works.	2017-01-05 13:40:39 +00:00
Matthew Hodgson	b31ed22738	Merge branch 'release-v0.18.6' into develop	2017-01-05 13:03:02 +00:00
Mark Haines	f784980d2b	Only send events that originate on this server. ... Or events that are sent via the federation "send_join" API. This should match the behaviour from before v0.18.5 and #1635 landed.	2017-01-05 11:26:30 +00:00
David Baker	84cf00c645	Fix another comment typo	2016-12-21 09:51:43 +00:00
David Baker	0c88ab1844	Add /account/3pid/delete endpoint ... Also fix a typo in a comment	2016-12-20 18:27:30 +00:00
Matthew Hodgson	f2a5aebf98	fix ability to change password to a non-ascii one ... https://github.com/vector-im/riot-web/issues/2658	2016-12-18 22:25:21 +00:00
Erik Johnston	b2f8642d3d	Cache network room list queries.	2016-12-16 16:11:43 +00:00
Richard van der Hoff	edc6a1e4f9	Add some logging for syncing to_device events ... Attempt to track down the loss of to_device events (https://github.com/vector-im/riot-web/issues/2711 etc).	2016-12-15 18:16:10 +00:00
Erik Johnston	35129ac998	Merge pull request #1698 from matrix-org/erikj/room_list ... Fix caching on public room list	2016-12-15 15:40:28 +00:00
Erik Johnston	b8255eba26	Comment	2016-12-14 13:49:54 +00:00
Erik Johnston	b2999a7055	Fix /backfill returning events it shouldn't	2016-12-14 13:41:45 +00:00
Erik Johnston	1de53a7a1a	Fix caching on public room list	2016-12-13 17:33:24 +00:00
Erik Johnston	1574b839e0	Merge pull request #1676 from matrix-org/erikj/room_list ... Add new API appservice specific public room list	2016-12-12 17:00:10 +00:00
Erik Johnston	d53a80af25	Merge pull request #1620 from matrix-org/erikj/concurrent_room_access ... Limit the number of events that can be created on a given room concurrently	2016-12-12 10:30:23 +00:00
Richard van der Hoff	85cd30b1fd	Merge pull request #1686 from matrix-org/rav/fix_federation_key_fails ... E2E key query: handle federation fails	2016-12-12 09:33:39 +00:00
Richard van der Hoff	deca951241	Remove unused import	2016-12-12 09:24:35 +00:00
Richard van der Hoff	efa4ccfaee	E2E key query: handle federation fails ... Don't fail the whole request if we can't connect to a particular server.	2016-12-09 18:31:01 +00:00
Erik Johnston	be14c24cea	Fix rare notifier bug where listeners dont timeout ... There was a race condition that caused the notifier to 'miss' the timeout notification, since there were no other checks for the timeout this caused listeners to get stuck in a loop until something happened.	2016-12-09 15:43:18 +00:00
Richard van der Hoff	b08f76bd23	Fix ignored read-receipts ... Don't ignore read-receipts which arrive in the same EDU as a read-receipt for an old event.	2016-12-08 12:13:01 +00:00
Erik Johnston	2687af82d4	Comments	2016-12-07 09:58:33 +00:00
Erik Johnston	f32fb65552	Add new API appservice specific public room list	2016-12-06 16:12:27 +00:00
Richard van der Hoff	1529c19675	Prevent user tokens being used as guest tokens (#1675) ... Make sure that a user cannot pretend to be a guest by adding 'guest = True' caveats.	2016-12-06 15:31:37 +00:00
Erik Johnston	9609c91e7d	Merge pull request #653 from matrix-org/erikj/preset_guest_join ... Enable guest access for private rooms by default	2016-12-05 17:47:14 +00:00
Erik Johnston	338df4f409	Merge pull request #1649 from matrix-org/dbkr/log_ui_auth_args ... Log the args that we have on UI auth completion	2016-12-05 16:40:58 +00:00
Richard van der Hoff	aa09d6b8f0	Rip out more refresh_token code ... We might as well treat all refresh_tokens as invalid. Just return a 403 from /tokenrefresh, so that we don't have a load of dead, untestable code hanging around. Still TODO: removing the table from the schema.	2016-11-30 17:40:18 +00:00
Richard van der Hoff	dc4b23e1a1	Merge branch 'develop' into rav/no_more_refresh_tokens	2016-11-30 17:10:04 +00:00
Richard van der Hoff	1c4f05db41	Stop putting a time caveat on access tokens ... The 'time' caveat on the access tokens was something of a lie, since we weren't enforcing it; more pertinently its presence stops us ever adding useful time caveats. Let's move in the right direction by not lying in our caveats.	2016-11-29 16:49:41 +00:00
Richard van der Hoff	5c4edc83b5	Stop generating refresh tokens ... Since we're not doing refresh tokens any more, we should start killing off the dead code paths. /tokenrefresh itself is a bit of a thornier subject, since there might be apps out there using it, but we can at least not generate refresh tokens on new logins.	2016-11-28 10:13:01 +00:00
David Baker	c9d4e7b716	Clarify that creds doesn not contain passwords.	2016-11-24 10:54:59 +00:00
David Baker	f681aab895	Log the args that we have on UI auth completion ... This will be super helpful for debugging if we have more registration woes.	2016-11-24 10:11:45 +00:00
Erik Johnston	feec718265	Shuffle receipt handler around so that worker apps don't need to load it	2016-11-23 15:14:24 +00:00
Erik Johnston	b69f76c106	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/split_out_fed_txn	2016-11-23 11:31:53 +00:00
Kegan Dougal	83bcdcee61	Return early on /sync code paths if a '*' filter is used ... This is currently very conservative in that it only does this if there is no `since` token. This limits the risk to clients likely to be doing one-off syncs (like bridges), but does mean that normal human clients won't benefit from the time savings here. If the savings are large enough, I would consider generalising this to just check the filter.	2016-11-22 16:38:35 +00:00
Erik Johnston	9687e039e7	Remove explicit calls to send_pdu	2016-11-21 14:48:51 +00:00
Erik Johnston	f8ee66250a	Handle sending events and device messages over federation	2016-11-17 15:48:04 +00:00
Erik Johnston	59ef517e6b	Use new federation_sender DI	2016-11-16 14:47:52 +00:00
Erik Johnston	d56c39cf24	Use external ldap auth pacakge	2016-11-15 13:03:19 +00:00
Erik Johnston	64038b806c	Comments	2016-11-11 10:42:08 +00:00
Erik Johnston	2bd4513a4d	Limit the number of events that can be created on a given room concurretnly	2016-11-10 16:44:35 +00:00
Erik Johnston	a4632783fb	Sample correctly	2016-11-08 11:20:26 +00:00
Erik Johnston	eeda4e618c	Limit the number of prev_events of new events	2016-11-08 11:02:29 +00:00
David Baker	9084720993	Don't error on non-ascii passwords	2016-11-03 10:42:14 +00:00
Erik Johnston	2ef617bc06	Fix infinite typing bug ... There's a bug somewhere that causes typing notifications to not be timed out properly. By adding a paranoia timer and using correct inequalities notifications should stop being stuck, even if it the root cause hasn't been fixed.	2016-10-24 15:51:22 +01:00
Erik Johnston	d04e2ff3a4	Fix incredubly slow back pagination query ... If a client didn't specify a from token when paginating backwards synapse would attempt to query the (global) maximum topological token. This a) doesn't make much sense since they're room specific and b) there are no indices that lets postgres do this efficiently.	2016-10-24 13:35:51 +01:00
Luke Barnard	e01a1bc92d	Merge pull request #1175 from matrix-org/luke/feature-configurable-as-rate-limiting ... Allow Configurable Rate Limiting Per AS	2016-10-20 16:21:10 +01:00
Luke Barnard	f09db236b1	as_user->app_service, less redundant comments, better positioned comments	2016-10-20 12:04:54 +01:00
Luke Barnard	1b17d1a106	Use real AS object by passing it through the requester ... This means synapse does not have to check if the AS is interested, but instead it effectively re-uses what it already knew about the requesting user	2016-10-20 11:43:05 +01:00
Erik Johnston	3aa8925091	Merge pull request #1176 from matrix-org/erikj/eager_ratelimit_check ... Check whether to ratelimit sooner to avoid work	2016-10-19 14:25:52 +01:00
Erik Johnston	f2f74ffce6	Comment	2016-10-19 14:21:28 +01:00
David Baker	df2a616c7b	Convert emails to lowercase when storing ... And db migration sql to convert existing addresses.	2016-10-19 11:13:55 +01:00
Erik Johnston	550308c7a1	Check whether to ratelimit sooner to avoid work	2016-10-19 10:45:24 +01:00
Luke Barnard	5b54d51d1e	Allow Configurable Rate Limiting Per AS ... This adds a flag loaded from the registration file of an AS that will determine whether or not its users are rate limited (by ratelimit in _base.py). Needed for IRC bridge reasons - see https://github.com/matrix-org/matrix-appservice-irc/issues/240.	2016-10-18 17:04:09 +01:00
Erik Johnston	35e2cc8b52	Merge pull request #1155 from matrix-org/erikj/pluggable_pwd_auth ... Implement pluggable password auth	2016-10-12 11:41:20 +01:00
Richard van der Hoff	0061e8744f	Merge pull request #1166 from matrix-org/rav/grandfather_broken_riot_signup ... Work around email-spamming Riot bug	2016-10-11 11:58:58 +01:00
Richard van der Hoff	fa74fcf512	Work around email-spamming Riot bug ... 5d9546f9 introduced a change to synapse behaviour, in that failures in the interactive-auth process would return the flows and params data as well as an error code (as specced in https://github.com/matrix-org/matrix-doc/pull/397). That change exposed a bug in Riot which would make it request a new validation token (and send a new email) each time it got a 401 with a `flows` parameter (see https://github.com/vector-im/vector-web/issues/2447 and the fix at https://github.com/matrix-org/matrix-react-sdk/pull/510). To preserve compatibility with broken versions of Riot, grandfather in the old behaviour for the email validation stage.	2016-10-11 11:34:40 +01:00
Erik Johnston	a2f2516199	Merge pull request #1157 from Rugvip/nolimit ... Remove rate limiting from app service senders and fix get_or_create_user requester	2016-10-11 11:20:54 +01:00
Richard van der Hoff	8681aff4f1	Merge pull request #1160 from matrix-org/rav/401_on_password_fail ... Interactive Auth: Return 401 from for incorrect password	2016-10-07 10:57:43 +01:00
Richard van der Hoff	5d9546f9f4	Interactive Auth: Return 401 from for incorrect password ... This requires a bit of fettling, because I want to return a helpful error message too but we don't want to distinguish between unknown user and invalid password. To avoid hardcoding the error message into 15 places in the code, I've had to refactor a few methods to return None instead of throwing. Fixes https://matrix.org/jira/browse/SYN-744	2016-10-07 00:00:00 +01:00
Patrik Oldsberg	7b5546d077	rest/client/v1/register: use the correct requester in createUser ... Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>	2016-10-06 22:12:32 +02:00
Patrik Oldsberg	3de7c8a4d0	handlers/profile: added admin override for set_displayname and set_avatar_url ... Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>	2016-10-06 15:24:59 +02:00
Patrik Oldsberg	2ff2d36b80	handers: do not ratelimit app service senders ... Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>	2016-10-06 15:24:59 +02:00
Patrik Oldsberg	9bfc617791	storage/appservice: make appservice methods only relying on the cache synchronous	2016-10-06 15:24:59 +02:00
Erik Johnston	850b103b36	Implement pluggable password auth ... Allows delegating the password auth to an external module. This also moves the LDAP auth to using this system, allowing it to be removed from the synapse tree entirely in the future.	2016-10-03 10:36:40 +01:00
Martin Weinelt	3027ea22b0	Restructure ldap authentication ... - properly parse return values of ldap bind() calls - externalize authentication methods - change control flow to be more error-resilient - unbind ldap connections in many places - improve log messages and loglevels	2016-09-29 15:30:08 +01:00
Erik Johnston	655891d179	Move FEDERATION_PING_INTERVAL timer. Update log line	2016-09-23 15:43:34 +01:00
Erik Johnston	22578545a0	Time out typing over federation	2016-09-23 14:00:52 +01:00
Erik Johnston	667fcd54e8	Merge pull request #1136 from matrix-org/erikj/fix_signed_3pid ... Allow invites via 3pid to bypass sender sig check	2016-09-22 13:41:49 +01:00
Erik Johnston	81964aeb90	Merge pull request #1132 from matrix-org/erikj/initial_sync_split ... Support /initialSync in synchrotron worker	2016-09-22 12:45:02 +01:00
Erik Johnston	2e9ee30969	Add comments	2016-09-22 11:59:46 +01:00
Erik Johnston	1168cbd54d	Allow invites via 3pid to bypass sender sig check ... When a server sends a third party invite another server may be the one that the inviting user registers with. In this case it is that remote server that will issue an actual invitation, and wants to do it "in the name of" the original invitee. However, the new proper invite will not be signed by the original server, and thus other servers would reject the invite if it was seen as coming from the original user. To fix this, a special case has been added to the auth rules whereby another server can send an invite "in the name of" another server's user, so long as that user had previously issued a third party invite that is now being accepted.	2016-09-22 10:56:53 +01:00
Erik Johnston	90c070c850	Add total_room_count_estimate to /publicRooms	2016-09-21 13:30:05 +01:00
Erik Johnston	87528f0756	Support /initialSync in synchrotron worker	2016-09-21 11:46:28 +01:00
Erik Johnston	39af634dd2	Merge pull request #1130 from matrix-org/erikj/fix_pubroom_pag ... Handle fact that _generate_room_entry may not return a room entry	2016-09-19 10:13:59 +01:00
Erik Johnston	4d49e0bdfd	PEP8	2016-09-17 18:09:22 +01:00
Erik Johnston	81570abfb2	Handle fact that _generate_room_entry may not return a room entry	2016-09-17 18:01:54 +01:00
Erik Johnston	eb24aecf8c	Merge pull request #1129 from matrix-org/erikj/fix_pubroom_pag ... Fix and clean up publicRooms pagination	2016-09-17 15:30:34 +01:00
Erik Johnston	a298331de4	Spelling	2016-09-17 14:59:40 +01:00
Erik Johnston	71edaae981	Fix and clean up publicRooms pagination	2016-09-17 14:46:19 +01:00
David Baker	5336acd46f	Make public room search case insensitive	2016-09-16 19:02:42 +01:00
Erik Johnston	b7b62bf9ea	Comment	2016-09-16 11:00:29 +01:00
Erik Johnston	d84319ae10	Add remote reoom cache	2016-09-16 10:31:59 +01:00
Erik Johnston	23b6701a28	Support filtering remote room lists	2016-09-16 10:24:15 +01:00
Erik Johnston	e58a9d781c	Filter remote rooms lists locally	2016-09-16 10:19:32 +01:00
Erik Johnston	74d4cdee25	Don't cache searched in /publicRooms	2016-09-16 09:05:11 +01:00
Erik Johnston	098db4aa52	Add very basic filter API to /publicRooms	2016-09-15 17:50:16 +01:00
Erik Johnston	c33b25fd8d	Change the way we calculate new_limit in /publicRooms and add POST API	2016-09-15 17:35:20 +01:00
Erik Johnston	e457034e99	Merge pull request #1121 from matrix-org/erikj/public_room_paginate ... Add pagination support to publicRooms	2016-09-15 13:27:09 +01:00
Erik Johnston	4fb65a1091	Base public room list off of public_rooms stream	2016-09-15 11:27:04 +01:00
Erik Johnston	5810cffd33	Pass since/from parameters over federation	2016-09-15 10:36:19 +01:00
Erik Johnston	f3eead0660	Allow paginating both forwards and backwards	2016-09-15 10:15:37 +01:00
Erik Johnston	4131381123	Remove support for aggregate room lists	2016-09-15 09:28:15 +01:00
Erik Johnston	4f181f361d	Accept optional token to public room list	2016-09-15 09:08:57 +01:00
Erik Johnston	c566f0ee17	Calculate the public room list from a stream_ordering	2016-09-14 17:42:47 +01:00
Erik Johnston	772c6067a3	Refactor public rooms to not pull out the full state for each room	2016-09-14 17:29:25 +01:00
Erik Johnston	264a48aedf	Merge pull request #1117 from matrix-org/erikj/fix_state ... Ensure we don't mutate state cache entries	2016-09-14 16:50:37 +01:00
Erik Johnston	a70a43bc51	Move RoomListHandler into a separate file	2016-09-14 14:07:37 +01:00
Erik Johnston	d5ae1f1291	Ensure we don't mutate state cache entries	2016-09-14 10:03:48 +01:00
Matthew Hodgson	1b01488d27	Merge pull request #1111 from matrix-org/matthew/device-ids ... make device IDs more useful for human disambiguation	2016-09-14 01:18:02 +01:00
Paul Evans	0f73f0e70e	Merge pull request #1116 from matrix-org/paul/tiny-fixes ... Fix typo "persiting"	2016-09-13 13:27:42 +01:00
Paul "LeoNerd" Evans	ca35e54d6b	Fix typo "persiting"	2016-09-13 13:26:33 +01:00
Mark Haines	ad816b0add	Limit how often we ask for keys from dead servers	2016-09-13 11:53:50 +01:00
Mark Haines	18ab019a4a	Move the E2E key handling into the e2e handler	2016-09-13 11:35:35 +01:00
Mark Haines	76b09c29b0	Merge pull request #1110 from matrix-org/markjh/e2e_timeout ... Add a timeout parameter for end2end key queries.	2016-09-13 10:50:45 +01:00
Matthew Hodgson	edbcb4152b	make device IDs more useful for human disambiguation	2016-09-13 00:02:39 +01:00
Mark Haines	949c2c5435	Add a timeout parameter for end2end key queries. ... Add a timeout parameter for controlling how long synapse will wait for responses from remote servers. For servers that fail include how they failed to make it easier to debug. Fetch keys from different servers in parallel rather than in series. Set the default timeout to 10s.	2016-09-12 18:17:09 +01:00
Erik Johnston	48a5a7552d	Add is_direct param to /createRoom	2016-09-12 16:34:20 +01:00
Erik Johnston	3265def8c7	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/batch_edus	2016-09-09 18:06:01 +01:00
Erik Johnston	52b2318777	Clobber EDUs in send queue	2016-09-09 15:59:08 +01:00
Paul Evans	56f38d1776	Merge pull request #1091 from matrix-org/paul/third-party-lookup ... Improvements to 3PE lookup API	2016-09-09 15:43:11 +01:00
Paul "LeoNerd" Evans	ed44c475d8	Reject malformed 3PE query metadata results earlier in AS API handling code	2016-09-09 15:07:04 +01:00
Paul "LeoNerd" Evans	f25d74f69c	Minor fixes from PR comments	2016-09-09 14:54:16 +01:00
Erik Johnston	6c4d582144	Deduplicate presence in _update_states	2016-09-09 14:28:22 +01:00
Paul "LeoNerd" Evans	25eb769b26	Efficiency fix for lookups of a single protocol	2016-09-09 13:25:02 +01:00
Paul "LeoNerd" Evans	033d43e419	Don't corrupt shared cache on subsequent protocol requests	2016-09-09 13:10:36 +01:00
Paul "LeoNerd" Evans	2f267ee160	Collect up all the "instances" lists of individual AS protocol results into one combined answer to the client	2016-09-08 17:43:53 +01:00
Erik Johnston	257025ac89	Merge pull request #1082 from matrix-org/erikj/remote_public_rooms ... Add server param to /publicRooms	2016-09-08 16:04:22 +01:00
Erik Johnston	8b93af662d	Check the user_id for presence/typing matches origin	2016-09-08 15:07:38 +01:00
Mark Haines	fa9d36e050	Merge branch 'develop' into markjh/direct_to_device_federation	2016-09-08 13:43:43 +01:00
Erik Johnston	791658b576	Add server param to /publicRooms	2016-09-08 11:53:05 +01:00
Mark Haines	43954d000e	Add a new method to enqueue the device messages rather than sending a dummy EDU	2016-09-07 16:10:51 +01:00
Mark Haines	d4a35ada28	Send device messages over federation	2016-09-06 18:16:20 +01:00
Erik Johnston	3c4208a057	Record counts of state changes	2016-09-06 11:31:01 +01:00
Erik Johnston	438ef47637	Short circuit if presence is the same	2016-09-06 10:28:35 +01:00
Erik Johnston	74a3b4a650	Fiddle should_notify to better report stats	2016-09-06 10:23:38 +01:00
Erik Johnston	8c93e0bae7	Merge pull request #1070 from matrix-org/erikj/presence_stats ... Record why we have chosen to notify	2016-09-05 15:01:30 +01:00
Erik Johnston	69054e3d4c	Record why we have chosen to notify	2016-09-05 14:12:11 +01:00
Erik Johnston	69a2d4e38c	Use get_joined_users_from_context instead of manually looking up hosts	2016-09-05 13:44:40 +01:00
Mark Haines	274c2f50a5	Merge pull request #1067 from matrix-org/markjh/idempotent ... Fix membership changes to be idempotent	2016-09-05 10:21:25 +01:00
Mark Haines	f2690c6423	Fix membership changes to be idempotent	2016-09-02 19:23:22 +01:00
Mark Haines	81b94c5750	Merge pull request #1066 from matrix-org/markjh/direct_to_device_lowerbound ... Only return new device messages in /sync	2016-09-02 16:18:34 +01:00
Mark Haines	65fa37ac5e	Only return new device messages in /sync	2016-09-02 15:50:37 +01:00
Erik Johnston	273b6bcf22	Merge pull request #1064 from matrix-org/erikj/on_receive_check ... Only check if host is in room if we have state and auth_chain	2016-09-02 14:43:35 +01:00
Erik Johnston	f7f1027d3d	Comment on when auth chain and state are None	2016-09-02 14:42:38 +01:00
Erik Johnston	34e5e17f91	Comment	2016-09-02 14:26:07 +01:00
Erik Johnston	b96c6c3185	Docstrings	2016-09-02 14:19:22 +01:00
Erik Johnston	bd9b8d87ae	Only check if host is in room if we have state and auth_chain	2016-09-02 13:40:28 +01:00
Erik Johnston	2854ee2a52	Only pull out IDs from DB for /state_ids/ request	2016-09-02 10:53:36 +01:00
Erik Johnston	265d847ffd	Fix typo in log line	2016-09-01 14:50:06 +01:00
Erik Johnston	c10cb581c6	Correctly handle the difference between prev and current state	2016-08-31 14:26:22 +01:00
Erik Johnston	c7f665d700	Merge pull request #1055 from matrix-org/erikj/occaisonally_persist ... Occaisonally persist unpersisted presence updates	2016-08-30 15:59:00 +01:00
Erik Johnston	097330bae8	Check correct variable	2016-08-30 15:50:20 +01:00
Erik Johnston	21b977ccfe	Occaisonally persist unpersisted presence updates	2016-08-30 15:39:50 +01:00
Erik Johnston	bc1a8b1f7a	Don't notify for online -> online transitions. ... Specifically, if currently_active remains true then we should not notify if only the last active time changes.	2016-08-30 15:05:32 +01:00
Erik Johnston	c8cd87b21b	Comment about message deletion	2016-08-30 11:23:26 +01:00
Erik Johnston	1ee6285905	Fix check	2016-08-30 11:17:46 +01:00
Erik Johnston	68e1a872fd	Noop get_new_messages_for_device if token hasn't changed	2016-08-30 10:58:46 +01:00
Erik Johnston	55fc17cf4b	Merge pull request #1049 from matrix-org/erikj/presence_users_in_room ... Use state handler instead of get_users_in_room/get_joined_hosts	2016-08-30 10:50:37 +01:00
Erik Johnston	bed10f9880	Use state handler instead of get_users_in_room/get_joined_hosts	2016-08-26 14:54:30 +01:00
Mark Haines	4bbef62124	Merge remote-tracking branch 'origin/develop' into markjh/direct_to_device	2016-08-26 14:35:31 +01:00
Erik Johnston	11faa4296d	Measure _filter_events_for_server	2016-08-26 11:15:40 +01:00
Erik Johnston	f6338d6a3e	Don't pull out full state for _filter_events_for_server	2016-08-26 11:13:16 +01:00
Erik Johnston	30961182f2	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/state_ids	2016-08-26 09:48:13 +01:00
Erik Johnston	c1a133a6b6	Merge pull request #1043 from matrix-org/erikj/backfill_fix ... Fix None check in backfill	2016-08-26 09:07:05 +01:00
Erik Johnston	778fa85f47	Make sync not pull out full state	2016-08-25 18:59:44 +01:00
Mark Haines	ab34fdecb7	Merge branch 'develop' into markjh/direct_to_device	2016-08-25 18:34:46 +01:00
Mark Haines	e993925279	Add store-and-forward direct-to-device messaging	2016-08-25 17:35:37 +01:00
Erik Johnston	a3dc1e9cbe	Replace context.current_state with context.current_state_ids	2016-08-25 17:32:22 +01:00
Paul Evans	d9dcb2ba3a	Merge pull request #1041 from matrix-org/paul/third-party-lookup ... Extend 3PE lookup APIs for metadata query	2016-08-25 17:06:53 +01:00
Paul "LeoNerd" Evans	c435bfee9c	Don't need toplevel cache on 3PE lookup metadata any more	2016-08-25 15:57:07 +01:00
Paul "LeoNerd" Evans	5474824975	Actually query over AS API for 3PE lookup metadata	2016-08-25 15:29:36 +01:00
Erik Johnston	17f4f14df7	Pull out event ids rather than full events for state	2016-08-25 13:42:44 +01:00
Erik Johnston	cd5b264b03	Fix None check in backfill	2016-08-25 10:39:19 +01:00
Erik Johnston	fda97dd58a	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/preserve_log_contexts	2016-08-24 13:22:02 +01:00
Paul "LeoNerd" Evans	8e1ed09dff	Move static knowledge of protocol metadata into AS handler; cache the result	2016-08-24 13:01:53 +01:00
Paul "LeoNerd" Evans	9899824b85	Initial hack at the 3PN protocols metadata lookup API	2016-08-24 12:33:01 +01:00
Erik Johnston	9219139351	Preserve some logcontexts	2016-08-24 11:58:40 +01:00
Erik Johnston	86bcf4d6a7	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/pagination	2016-08-24 10:38:21 +01:00
Erik Johnston	ba07d4a70e	Add None checks to backfill	2016-08-24 10:31:05 +01:00
Kegan Dougal	4b31426a02	Pass through user-supplied content in /join/$room_id ... It was always intended to allow custom keys on the join event, but this has at some point been lost. Restore it. If the user specifies keys like "avatar_url" then they will be clobbered.	2016-08-23 16:32:04 +01:00
Erik Johnston	afbf6b33fc	defer.returnValue must not be called within Measure	2016-08-19 18:23:44 +01:00
Paul Evans	5674ea3e6c	Merge pull request #1026 from matrix-org/paul/thirdpartylookup ... 3rd party entity lookup	2016-08-18 20:52:50 +01:00
Paul "LeoNerd" Evans	65201631a4	Move validation logic for AS 3PE query response into ApplicationServiceApi class, to keep the handler logic neater	2016-08-18 17:33:56 +01:00
Paul "LeoNerd" Evans	697872cf08	More warnings about invalid results from AS 3PE query	2016-08-18 17:24:39 +01:00
Paul "LeoNerd" Evans	b515f844ee	Avoid so much copypasta between 3PU and 3PL query by unifying around a ThirdPartyEntityKind enumeration	2016-08-18 17:19:55 +01:00
Paul "LeoNerd" Evans	2a91799fcc	Minor syntax neatenings	2016-08-18 16:58:25 +01:00
Paul "LeoNerd" Evans	fcf1dec809	Appease pep8	2016-08-18 16:26:19 +01:00
Paul "LeoNerd" Evans	06964c4a0a	Copypasta the 3PU support code to also do 3PL	2016-08-18 16:09:50 +01:00
Paul "LeoNerd" Evans	d7b42afc74	Log a warning if an AS yields an invalid 3PU lookup result	2016-08-18 15:49:55 +01:00
Paul "LeoNerd" Evans	80f4740c8f	Scattergather the call out to ASes; validate received results	2016-08-18 15:40:41 +01:00
Erik Johnston	07229bbdae	Add appservice worker	2016-08-18 14:59:55 +01:00
Paul "LeoNerd" Evans	434bbf2cb5	Filter 3PU lookups by only ASes that declare knowledge of that protocol	2016-08-18 14:56:02 +01:00
Paul "LeoNerd" Evans	d5bf7a4a99	Merge remote-tracking branch 'origin/develop' into paul/thirdpartylookup	2016-08-18 14:21:01 +01:00
Paul "LeoNerd" Evans	f0c73a1e7a	Extend individual list results into the main return list, don't append	2016-08-18 13:53:54 +01:00
Erik Johnston	9da84a9a1e	Make AppserviceHandler stream events from database ... This is for two reasons: 1. Suppresses duplicates correctly, as the notifier doesn't do any duplicate suppression. 2. Makes it easier to connect the AppserviceHandler to the replication stream.	2016-08-18 11:54:41 +01:00
Erik Johnston	320dfe523c	Make notify_interested_services faster	2016-08-17 17:20:50 +01:00
Paul "LeoNerd" Evans	fa87c981e1	Thread 3PU lookup through as far as the AS API object; which currently noöps it	2016-08-17 16:17:28 +01:00
Erik Johnston	62c5245c87	Measure notify_interested_services	2016-08-17 11:12:29 +01:00
Erik Johnston	7c6f4f9427	Merge pull request #1012 from matrix-org/erikj/limit_backfill_uri ... Limit number of extremeties in backfill request	2016-08-16 12:55:42 +01:00
Erik Johnston	2ee1bd124c	Limit number of extremeties in backfill request ... This works around a bug where if we make a backfill request with too many extremeties it causes the request URI to be too long.	2016-08-16 11:34:36 +01:00
Erik Johnston	dc3a00f24f	Refactor user_delete_access_tokens. Invalidate get_user_by_access_token to slaves.	2016-08-15 17:04:39 +01:00
Erik Johnston	75299af4fc	Merge pull request #1009 from matrix-org/erikj/event_split ... Split out /events to synchrotron	2016-08-15 15:39:05 +01:00
Daniel Ehlers	dfaf0fee31	Log the value which is observed in the first place. ... The name 'result' is of bool type and has no len property, resulting in a TypeError. Futhermore in the flow control conn.response is observed and hence should be reported. Signed-off-by: Daniel Ehlers <sargon@toppoint.de>	2016-08-14 16:49:05 +02:00
Daniel Ehlers	e380538b59	Fix AttributeError when bind_dn is not defined. ... In case one does not define bind_dn in ldap configuration, filter attribute is not declared. Since auth code only uses ldap_filter attribute when according LDAP mode is selected, it is safe to only declare the attribute in that case. Signed-off-by: Daniel Ehlers <sargon@toppoint.de>	2016-08-14 16:48:33 +02:00
Erik Johnston	4e1cebd56f	Make synchrotron accept /events	2016-08-12 15:31:44 +01:00
Erik Johnston	448ac6cf0d	Only process one local membership event per room at a time	2016-08-12 09:32:19 +01:00
Erik Johnston	832799dbff	Merge pull request #997 from Half-Shot/develop ... Don't change status_msg on /sync	2016-08-11 14:10:55 +01:00
Erik Johnston	739ea29d1e	Also check if server is in the room	2016-08-10 13:32:23 +01:00
Erik Johnston	7f41bcbeec	Correctly auth /event/ requests	2016-08-10 13:22:20 +01:00
Erik Johnston	11fdfaf03b	Only resign our own events	2016-08-10 13:16:58 +01:00
Will Hunt	2510db3e76	Don't change status_msg on /sync	2016-08-10 12:59:59 +01:00
Richard van der Hoff	79ebfbe7c6	/login: Respond with a 403 when we get an invalid m.login.token	2016-08-09 16:29:28 +01:00
Richard van der Hoff	6fe6a6f029	Fix login with m.login.token ... login with token (as used by CAS auth) was broken by 067596d, such that it always returned a 401.	2016-08-08 16:40:39 +01:00
Erik Johnston	93acf49e9b	Fix backfill auth events	2016-08-05 12:59:04 +01:00
Richard van der Hoff	f131cd9e53	keys/query: Omit device displayname if null ... ... which makes it more consistent with user displaynames.	2016-08-04 10:59:51 +01:00
Richard van der Hoff	68264d7404	Include device name in /keys/query response ... Add an 'unsigned' section which includes the device display name.	2016-08-03 15:42:08 +01:00
Richard van der Hoff	91fa69e029	keys/query: return all users which were asked for ... In the situation where all of a user's devices get deleted, we want to indicate this to a client, so we want to return an empty dictionary, rather than nothing at all.	2016-08-03 15:41:44 +01:00
Richard van der Hoff	530bc862dc	Merge branch 'rav/null_default_device_displayname' into develop	2016-08-03 14:30:32 +01:00
Richard van der Hoff	a6f5cc65d9	PEP8	2016-08-03 14:30:06 +01:00
Richard van der Hoff	a843868fe9	E2eKeysHandler: minor tweaks ... PR feedback	2016-08-03 14:24:33 +01:00
Richard van der Hoff	4fec5e57be	Default device_display_name to null ... It turns out that it's more useful to return a null device display name (and let clients decide how to handle it: eg, falling back to device_id) than using a constant string like "unknown device".	2016-08-03 11:53:00 +01:00
Mark Haines	921f17f938	Merge branch 'develop' into rav/refactor_device_query	2016-08-03 11:12:47 +01:00
Richard van der Hoff	1efee2f52b	E2E keys: Make federation query share code with client query ... Refactor the e2e query handler to separate out the local query, and then make the federation handler use it.	2016-08-02 18:12:00 +01:00
Erik Johnston	7b0f6293f2	Merge pull request #940 from matrix-org/erikj/fed_state_cache ... Cache federation state responses	2016-08-02 15:21:37 +01:00
Richard van der Hoff	986615b0b2	Move e2e query logic into a handler	2016-08-01 18:02:07 +01:00
Paul Evans	18b7eb830b	Merge pull request #958 from matrix-org/paul/SYN-738 ... Forbid non-ASes from registering users whose names begin with '_'	2016-07-29 14:10:45 +01:00
Erik Johnston	c51a52f300	Mention that func will fetch auth events	2016-07-29 11:17:04 +01:00
Erik Johnston	3d13c3a295	Update docstring	2016-07-29 10:45:05 +01:00
Erik Johnston	1e2740caab	Handle the case of missing auth events when joining a room	2016-07-28 16:08:33 +01:00
Paul "LeoNerd" Evans	05f6447301	Forbid non-ASes from registering users whose names begin with '_' (SYN-738)	2016-07-27 17:54:26 +01:00
Richard van der Hoff	d47115ff8b	Delete e2e keys on device delete	2016-07-27 12:24:52 +01:00
Richard van der Hoff	eb359eced4	Add create_requester function ... Wrap the `Requester` constructor with a function which provides sensible defaults, and use it throughout	2016-07-26 16:46:53 +01:00
Richard van der Hoff	8e02494166	Delete refresh tokens when deleting devices	2016-07-26 11:10:37 +01:00
Richard van der Hoff	012b4c1913	Implement updating devices ... You can update the displayname of devices now.	2016-07-26 07:35:48 +01:00
Richard van der Hoff	436bffd15f	Implement deleting devices	2016-07-26 07:35:48 +01:00
David Baker	7ed58bb347	Use get to avoid KeyErrors	2016-07-22 17:18:50 +01:00
David Baker	dad2da7e54	Log the hostname the reCAPTCHA was completed on ... This could be useful information to have in the logs. Also comment about how & why we don't verify the hostname.	2016-07-22 17:00:56 +01:00
Richard van der Hoff	55abbe1850	make /devices return a list ... Turns out I specced this to return a list of devices rather than a dict of them	2016-07-21 15:57:28 +01:00
Richard van der Hoff	1c3c202b96	Fix PEP8 errors	2016-07-21 13:15:15 +01:00
Richard van der Hoff	406f7aa0f6	Implement GET /device/{deviceId}	2016-07-21 12:00:29 +01:00
Erik Johnston	248e6770ca	Cache federation state responses	2016-07-21 10:30:12 +01:00
Richard van der Hoff	bc8f265f0a	GET /devices endpoint ... implement a GET /devices endpoint which lists all of the user's devices. It also returns the last IP where we saw that device, so there is some dancing to fish that out of the user_ips table.	2016-07-20 16:42:32 +01:00
Erik Johnston	aba0b2a39b	Merge pull request #935 from matrix-org/erikj/backfill_notifs ... Don't notify pusher pool for backfilled events	2016-07-20 13:39:16 +01:00
Erik Johnston	57dca35692	Don't notify pusher pool for backfilled events	2016-07-20 13:25:06 +01:00
Richard van der Hoff	c68518dfbb	Merge pull request #933 from matrix-org/rav/type_annotations ... Type annotations	2016-07-20 12:26:32 +01:00
David Baker	e967bc86e7	Merge pull request #932 from matrix-org/rav/register_refactor ... Further registration refactoring	2016-07-20 11:03:33 +01:00
Erik Johnston	1e2a7f18a1	Merge pull request #922 from matrix-org/erikj/file_api2 ... Feature: Add filter to /messages. Add 'contains_url' to filter.	2016-07-20 10:40:48 +01:00
Richard van der Hoff	3413f1e284	Type annotations ... Add some type annotations to help PyCharm (in particular) to figure out the types of a bunch of things.	2016-07-19 18:56:16 +01:00
Richard van der Hoff	40cbffb2d2	Further registration refactoring ... * `RegistrationHandler.appservice_register` no longer issues an access token: instead it is left for the caller to do it. (There are two of these, one in `synapse/rest/client/v1/register.py`, which now simply calls `AuthHandler.issue_access_token`, and the other in `synapse/rest/client/v2_alpha/register.py`, which is covered below). * In `synapse/rest/client/v2_alpha/register.py`, move the generation of access_tokens into `_create_registration_details`. This means that the normal flow no longer needs to call `AuthHandler.issue_access_token`; the shared-secret flow can tell `RegistrationHandler.register` not to generate a token; and the appservice flow continues to work despite the above change.	2016-07-19 18:46:19 +01:00
Richard van der Hoff	9a7a77a22a	Merge pull request #929 from matrix-org/rav/support_deviceid_in_login ... Add device_id support to /login	2016-07-19 15:53:04 +01:00
Mark Haines	c41d52a042	Summary line	2016-07-19 10:28:27 +01:00
Mark Haines	7e554aac86	Update docstring on Handlers. ... To indicate it is deprecated.	2016-07-19 10:20:58 +01:00
Richard van der Hoff	f863a52cea	Add device_id support to /login ... Add a 'devices' table to the storage, as well as a 'device_id' column to refresh_tokens. Allow the client to pass a device_id, and initial_device_display_name, to /login. If login is successful, then register the device in the devices table if it wasn't known already. If no device_id was supplied, make one up. Associate the device_id with the access token and refresh token, so that we can get at it again later. Ensure that the device_id is copied from the refresh token to the access_token when the token is refreshed.	2016-07-18 16:39:44 +01:00
Richard van der Hoff	dcfd71aa4c	Refactor login flow ... Make sure that we have the canonical user_id *before* calling get_login_tuple_for_user_id. Replace login_with_password with a method which just validates the password, and have the caller call get_login_tuple_for_user_id. This brings the password flow into line with the other flows, and will give us a place to register the device_id if necessary.	2016-07-18 15:23:54 +01:00
Richard van der Hoff	6344db659f	Fix a doc-comment ... The `store` in a handler is a generic DataStore, not just an events.StateStore.	2016-07-18 09:48:10 +01:00
Erik Johnston	bd7c51921d	Merge pull request #919 from matrix-org/erikj/auth_fix ... Various auth.py fixes.	2016-07-15 11:38:33 +01:00
Erik Johnston	9e1b43bcbf	Comment	2016-07-15 09:29:54 +01:00
Erik Johnston	ebdafd8114	Check sender signed event	2016-07-14 17:03:24 +01:00
Erik Johnston	a98d215204	Add filter param to /messages API	2016-07-14 16:30:56 +01:00
Negar Fazeli	0136a522b1	Bug fix: expire invalid access tokens	2016-07-13 15:00:37 +02:00
Erik Johnston	a17e7caeb7	Merge branch 'erikj/shared_secret' into erikj/test2	2016-07-06 14:46:31 +01:00
Erik Johnston	651faee698	Add an admin option to shared secret registration	2016-07-05 17:30:22 +01:00
Erik Johnston	8f8798bc0d	Add ReadWriteLock for pagination and history prune	2016-07-05 15:30:25 +01:00
David Baker	ef535178ff	Merge pull request #904 from matrix-org/dbkr/register_email_no_untrusted_id_server ... requestToken update	2016-07-05 15:13:34 +01:00
Kent Shikama	14362bf359	Fix password config	2016-07-05 19:12:53 +09:00
Kent Shikama	1ee2584307	Fix pep8	2016-07-05 19:01:00 +09:00
Erik Johnston	2d21d43c34	Add purge_history API	2016-07-05 10:28:51 +01:00
Kent Shikama	8bdaf5f7af	Add pepper to password hashing ... Signed-off-by: Kent Shikama <kent@kentshikama.com>	2016-07-05 02:13:52 +09:00
Kent Shikama	bb069079bb	Fix style violations ... Signed-off-by: Kent Shikama <kent@kentshikama.com>	2016-07-04 22:07:11 +09:00
Kent Shikama	fc8007dbec	Optionally include password hash in createUser endpoint ... Signed-off-by: Kent Shikama <kent@kentshikama.com>	2016-07-03 15:08:15 +09:00
David Baker	be8be535f7	requestToken update ... Don't send requestToken request to untrusted ID servers Also correct the THREEPID_IN_USE error to add the M_ prefix. This is a backwards incomaptible change, but the only thing using this is the angular client which is now unmaintained, so it's probably better to just do this now.	2016-06-30 17:51:28 +01:00
Martin Weinelt	0a32208e5d	Rework ldap integration with ldap3 ... Use the pure-python ldap3 library, which eliminates the need for a system dependency. Offer both a `search` and `simple_bind` mode, for more sophisticated ldap scenarios. - `search` tries to find a matching DN within the `user_base` while employing the `user_filter`, then tries the bind when a single matching DN was found. - `simple_bind` tries the bind against a specific DN by combining the localpart and `user_base` Offer support for STARTTLS on a plain connection. The configuration was changed to reflect these new possibilities. Signed-off-by: Martin Weinelt <hexa@darmstadt.ccc.de>	2016-06-22 17:51:59 +02:00
Mark Haines	0c13d45522	Add a comment on why we don't create a profile for upgrading users	2016-06-17 19:18:53 +01:00
Mark Haines	9f1800fba8	Remove registered_users from the distributor. ... The only place that was observed was to set the profile. I've made it so that the profile is set within store.register in the same transaction that creates the user. This required some slight changes to the registration code for upgrading guest users, since it previously relied on the distributor swallowing errors if the profile already existed.	2016-06-17 19:14:16 +01:00
Erik Johnston	2884712ca7	Only re-sign our own events	2016-06-17 14:47:33 +01:00
Salvatore LaMendola	ed5f43a55a	Fix TypeError in call to bcrypt.hashpw ... - At the very least, this TypeError caused logins to fail on my own running instance of Synapse, and the simple (explicit) UTF-8 conversion resolved login errors for me. Signed-off-by: Salvatore LaMendola <salvatore.lamendola@gmail.com>	2016-06-16 00:43:42 -04:00
Erik Johnston	b31c49d676	Correctly mark backfilled events as backfilled	2016-06-15 10:59:08 +01:00
Negar Fazeli	6e7dc7c7dd	Fix a bug caused by a change in auth_handler function ... Fix the relevant unit test cases	2016-06-08 23:22:39 +02:00
Erik Johnston	81c07a32fd	Pull full state for each room all at once	2016-06-08 15:51:49 +01:00
Erik Johnston	1a815fb04f	Don't hit DB for noop replications queries	2016-06-08 11:33:30 +01:00
Mark Haines	0b2158719c	Remove dead code. ... Loading push rules now happens in the datastore, so we can remove the methods that loaded them outside the datastore. The ``waiting_for_join_list`` in federation handler is populated by anything, so can be removed. The ``_get_members_events_txn`` method isn't called from anywhere so can be removed.	2016-06-07 15:07:11 +01:00
Erik Johnston	216a05b3e3	.values() returns list of sets	2016-06-06 16:00:09 +01:00
Erik Johnston	96dc600579	Fix typos	2016-06-06 15:44:41 +01:00
Erik Johnston	377eb480ca	Fire after 30s not 8h	2016-06-06 15:14:21 +01:00
Erik Johnston	51773bcbaf	Merge pull request #832 from matrix-org/erikj/presence_coount ... Change def of small delta in presence stream. Add metrics.	2016-06-03 14:57:00 +01:00
Erik Johnston	ab116bdb0c	Fix typo	2016-06-03 14:03:42 +01:00
Erik Johnston	4ce84a1acd	Change metric style	2016-06-03 13:49:16 +01:00
Erik Johnston	a7ff5a1770	Presence metrics. Change def of small delta	2016-06-03 13:40:55 +01:00
Erik Johnston	4982b28868	Merge pull request #829 from matrix-org/erikj/poke_notifier ... Poke notifier on next reactor tick	2016-06-03 11:52:10 +01:00
Erik Johnston	4c04222fa5	Poke notifier on next reactor tick	2016-06-03 11:24:16 +01:00
Erik Johnston	6a0afa582a	Load push rules in storage layer, so that they get cached	2016-06-03 11:10:00 +01:00
David Baker	c71177f285	Merge remote-tracking branch 'origin/dbkr/email_notif_string_fmt_error' into dbkr/email_unsubscribe	2016-06-02 17:20:56 +01:00
Mark Haines	56d15a0530	Store the typing users as user_id strings. (#819) ... Rather than storing them as UserID objects.	2016-06-02 16:28:54 +01:00
David Baker	812b5de0fe	Merge remote-tracking branch 'origin/develop' into dbkr/email_unsubscribe	2016-06-02 15:33:28 +01:00
Mark Haines	661a540dd1	Deduplicate presence entries in sync (#818)	2016-06-02 15:20:28 +01:00
Mark Haines	70599ce925	Allow external processes to mark a user as syncing. (#812) ... * Add infrastructure to the presence handler to track sync requests in external processes * Expire stale entries for dead external processes * Add an http endpoint for making users as syncing Add some docstrings and comments. * Fixes	2016-06-02 15:20:15 +01:00
David Baker	fb2193cc63	Merge pull request #817 from matrix-org/dbkr/split_out_auth_handler ... Split out the auth handler	2016-06-02 14:31:35 +01:00
Erik Johnston	02ac463dbf	Merge pull request #800 from matrix-org/erikj/sync_refactor ... Refactor SyncHandler	2016-06-02 14:02:13 +01:00
David Baker	3a3fb2f6f9	Merge branch 'dbkr/split_out_auth_handler' into dbkr/email_unsubscribe	2016-06-02 13:35:25 +01:00
David Baker	4a10510cd5	Split out the auth handler	2016-06-02 13:31:45 +01:00
David Baker	a15ad60849	Email unsubscribing that may in theory, work ... Were it not for that fact that you can't use the base handler in the pusher because it pulls in the world. Comitting while I fix that on a different branch.	2016-06-02 11:44:15 +01:00
David Baker	e0deeff23e	Fix room list spidering	2016-06-01 17:58:58 +01:00
David Baker	8e539f13c0	Merge remote-tracking branch 'origin/develop' into dbkr/room_list_spider	2016-06-01 09:54:36 +01:00
David Baker	2a449fec4d	Add cache to remote room lists ... Poll for updates from remote servers, waiting for the poll if there's no cache entry.	2016-05-31 18:27:23 +01:00
David Baker	d240796ded	Basic, un-cached support for secondary_directory_servers	2016-05-31 17:20:07 +01:00
Mark Haines	c626fc576a	Move the AS handler out of the Handlers object. ... Access it directly from the homeserver itself. It already wasn't inheriting from BaseHandler storing it on the Handlers object was already somewhat dubious.	2016-05-31 13:53:48 +01:00
David Baker	887c6e6f05	Split out the room list handler ... So I can use it from federation bits without pulling in all the handlers.	2016-05-31 11:05:16 +01:00
Erik Johnston	cc84f7cb8e	Send down correct error response if user not found	2016-05-27 10:35:15 +01:00
Erik Johnston	faad233ea6	Change short circuit path	2016-05-24 14:27:19 +01:00
Erik Johnston	6900303997	Don't send down all ephemeral events	2016-05-24 11:44:55 +01:00
Erik Johnston	1c5ed2a19b	Only work out newly_joined_users for incremental sync	2016-05-24 11:21:34 +01:00
Erik Johnston	b08ad0389e	Only include non-offline presence in initial sync	2016-05-24 11:15:05 +01:00
Erik Johnston	be2c677386	Spell builder correctly	2016-05-24 10:53:03 +01:00
Erik Johnston	79bea8ab9a	Inline function. Make load_filtered_recents private	2016-05-24 10:22:24 +01:00
Erik Johnston	84f94e4cbb	Add comments	2016-05-24 10:14:53 +01:00
Erik Johnston	137e6a4557	Shuffle things room	2016-05-24 09:50:55 +01:00
Negi Fazeli	6fe04ffef2	Fix set profile error with Requester. ... Replace flush_user with delete access token due to function removal Add a new test case for if the user is already registered	2016-05-23 19:50:28 +02:00
Erik Johnston	c0c79ef444	Add back concurrently_execute	2016-05-23 18:21:27 +01:00
Erik Johnston	b5605dfecc	Refactor SyncHandler	2016-05-23 18:08:18 +01:00
Mark Haines	0cb441fedd	Move typing handler out of the Handlers object	2016-05-17 15:58:46 +01:00
Mark Haines	523d5bcd0b	Merge remote-tracking branch 'origin/develop' into markjh/liberate_sync_handler	2016-05-17 10:43:58 +01:00
Mark Haines	526bf8126f	Remove unused get_joined_rooms_for_user	2016-05-17 10:20:51 +01:00
Mark Haines	425e6b4983	Merge branch 'develop' into markjh/member_cleanup	2016-05-17 10:13:16 +01:00
Mark Haines	b153f5b150	Merge pull request #787 from matrix-org/markjh/liberate_presence_handler ... Move the presence handler out of the Handlers object	2016-05-17 10:09:43 +01:00
Mark Haines	f68eea808a	Move SyncHandler out of the Handlers object	2016-05-16 20:19:26 +01:00
Mark Haines	53e171f345	Merge branch 'markjh/liberate_presence_handler' into markjh/liberate_sync_handler	2016-05-16 20:08:32 +01:00
Mark Haines	80cb9becd8	Remove get_joined_rooms_for_user from RoomMemberHandler	2016-05-16 20:06:55 +01:00
Mark Haines	821306120a	Replaces calls to fetch_room_distributions_into with get_joined_hosts_for_room	2016-05-16 19:48:07 +01:00
Mark Haines	1a3a2002ff	Spell "domain" correctly ... s/domian/domain/g	2016-05-16 19:17:23 +01:00
Mark Haines	e168abbcff	Don't inherit PresenceHandler from BaseHandler, remove references to self.hs from presence handler	2016-05-16 19:08:40 +01:00
Mark Haines	3b86ecfa79	Move the presence handler out of the Handlers object	2016-05-16 18:56:37 +01:00
Erik Johnston	99b5a2e560	Merge pull request #741 from negzi/create_user_with_expiry ... Create user with expiry	2016-05-13 14:46:53 +01:00
Negi Fazeli	40aa6e8349	Create user with expiry ... - Add unittests for client, api and handler Signed-off-by: Negar Fazeli <negar.fazeli@ericsson.com>	2016-05-13 15:34:15 +02:00
Mark Haines	a458a40337	missed a spot	2016-05-12 18:19:58 +01:00
Mark Haines	7e23476814	move filter_events_for_client out of base handler	2016-05-11 13:42:37 +01:00
Mark Haines	1620578b13	Shuffle when we get the signing_key attribute. ... Wait until we sign a message to get the signing key from the homeserver config. This means that the message handler can be created without having a signing key in the config which means that separate processes like the pusher that don't send messages and don't need to sign them can still access the handlers.	2016-05-11 12:20:57 +01:00
Erik Johnston	108434e53d	Merge pull request #775 from matrix-org/erikj/password_hash ... Correctly handle NULL password hashes from the database	2016-05-11 12:18:13 +01:00
Erik Johnston	1400bb1663	Correctly handle NULL password hashes from the database	2016-05-11 12:06:02 +01:00
Mark Haines	458a435114	Fix typo	2016-05-11 10:35:33 +01:00
Mark Haines	30057b1e15	Move _create_new_client_event and handle_new_client_event out of base handler	2016-05-11 09:09:20 +01:00
Matthew Hodgson	fe97b81c09	Merge pull request #759 from matrix-org/dbkr/email_notifs ... Send email notifications for missed messages	2016-05-10 16:30:05 +02:00
David Baker	997db04648	Merge remote-tracking branch 'origin/develop' into dbkr/email_notifs	2016-05-10 14:40:19 +02:00
Erik Johnston	c9eb6dfc1b	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/ignore_user	2016-05-09 13:21:06 +01:00
Erik Johnston	def64d6ef3	Merge branch 'develop' of github.com:matrix-org/synapse into erikj/ignore_user	2016-05-09 13:05:09 +01:00
Erik Johnston	08dfa8eee2	Add and use get_domian_from_id	2016-05-09 10:36:03 +01:00
Erik Johnston	56b5e83e36	Reduce database inserts when sending transactions	2016-05-06 11:20:18 +01:00
Erik Johnston	1f0f5ffa1e	Add bulk fetch storage API	2016-05-05 10:03:15 +01:00
Erik Johnston	a438a6d2bc	Implement basic ignore user	2016-05-04 10:16:46 +01:00
David Baker	acded821c4	Merge remote-tracking branch 'origin/develop' into dbkr/email_notifs	2016-04-29 10:05:20 +01:00
Matthew Hodgson	aebd0c9717	fix typo	2016-04-28 15:09:25 +01:00
David Baker	424a7f48f8	Run filter_events_for_client ... so we don't accidentally mail out events people shouldn't see	2016-04-27 17:50:49 +01:00
Erik Johnston	6fd2f685fe	Simplify _check_password	2016-04-15 11:17:18 +01:00
Mark Haines	3c79bdd7a0	Fix check_password rather than inverting the meaning of _check_local_password (#730)	2016-04-14 19:00:21 +01:00
David Baker	4c1b32d7e2	Fix login to error for nonexistent users ... Fixes SYN-680	2016-04-14 18:28:42 +01:00
Erik Johnston	5880bc5417	Merge pull request #718 from matrix-org/erikj/public_room_list ... Don't return empty public rooms	2016-04-13 14:07:26 +01:00
Erik Johnston	c53f9d561e	Don't auto log failed auth checks	2016-04-13 11:11:46 +01:00
Erik Johnston	72f454b752	Don't return empty public rooms	2016-04-12 16:06:18 +01:00
Erik Johnston	c48465dbaa	More comments	2016-04-12 12:48:30 +01:00
Erik Johnston	8be1a37909	More comments	2016-04-12 12:04:19 +01:00
Erik Johnston	d3d0be4167	Don't append to unused list	2016-04-12 11:59:00 +01:00
Erik Johnston	762ada1e07	Add back backfilled parameter that was removed	2016-04-12 11:58:04 +01:00
Erik Johnston	0d3da210f0	Add comment	2016-04-12 11:54:41 +01:00
Erik Johnston	cccf86dd05	Check if we've already backfilled events	2016-04-12 11:19:32 +01:00
David Baker	2547dffccc	Merge pull request #705 from matrix-org/dbkr/pushers_use_event_actions ... Change pushers to use the event_actions table	2016-04-11 12:58:55 +01:00
Erik Johnston	96bd8ff57c	Merge pull request #707 from matrix-org/markjh/remove_changed_presencelike_data ... changed_presencelike_data isn't observed anywhere so can be removed	2016-04-08 14:04:54 +01:00
Mark Haines	7e2f971c08	Remove some unused functions (#711) ... * Remove some unused functions * get_room_events_stream is only used in tests * is_exclusive_room might actually be something we want	2016-04-08 14:01:56 +01:00
Mark Haines	b9ee5650b0	Move all the wrapper functions for distributor.fire ... Move the functions inside the distributor and import them where needed. This reduces duplication and makes it possible for flake8 to detect when the functions aren't used in a given file.	2016-04-08 11:01:38 +01:00
Mark Haines	caef337587	changed_presencelike_data isn't observed anywhere in synapse so can be removed	2016-04-08 10:37:19 +01:00
Mark Haines	b4a5002a6e	Merge pull request #708 from matrix-org/markjh/remove_collect_presencelike_data ... Call profile handler get_displayname directly	2016-04-08 09:51:36 +01:00
Mark Haines	86be915cce	Call profile handler get_displayname directly rather than using collect_presencelike_data	2016-04-07 18:11:49 +01:00
Erik Johnston	f942980c0b	Merge pull request #701 from DoubleMalt/ldap-auth ... Add LDAP authentication	2016-04-07 17:35:28 +01:00
David Baker	e1e042f2a1	Add comments on min_stream_id ... saying that the min stream id won't be completely accurate all the time	2016-04-07 17:09:36 +01:00
Erik Johnston	a294b04bf0	Merge pull request #700 from matrix-org/erikj/deduplicate_joins ... Deduplicate membership changes	2016-04-07 16:35:40 +01:00
David Baker	9c99ab4572	Merge remote-tracking branch 'origin/develop' into dbkr/pushers_use_event_actions	2016-04-07 16:35:22 +01:00
David Baker	92e3071623	Send badge count pushes. ... Also fix bugs with retrying.	2016-04-07 15:39:53 +01:00
Erik Johnston	af03ecf352	Deduplicate joins	2016-04-07 14:19:02 +01:00
Erik Johnston	1ef0365670	Set profile information when joining rooms remotely	2016-04-07 09:42:52 +01:00
Christoph Witzany	ed4d18f516	fix check for failed authentication	2016-04-06 18:30:11 +02:00
Christoph Witzany	9c62fcdb68	remove line	2016-04-06 18:23:46 +02:00
Christoph Witzany	27a0c21c38	make tests for ldap more specific to not be fooled by Mocks	2016-04-06 18:23:46 +02:00
Christoph Witzany	3555a659ec	output ldap version for info and to pacify pep8	2016-04-06 18:23:46 +02:00
Christoph Witzany	4c5e8adf8b	conditionally import ldap	2016-04-06 18:23:46 +02:00
Christoph Witzany	875ed05bdc	fix pep8	2016-04-06 18:23:46 +02:00
Christoph Witzany	67f3a50e9a	fix exception handling	2016-04-06 18:23:46 +02:00
Christoph Witzany	afff321e9a	code style	2016-04-06 18:23:46 +02:00
Christoph Witzany	823b8be4b7	add tls property and twist my head around twisted	2016-04-06 18:23:45 +02:00
Christoph Witzany	7b9319b1c8	move LDAP authentication to AuthenticationHandler	2016-04-06 18:23:45 +02:00
David Baker	7e2c89a37f	Make pushers use the event_push_actions table instead of listening on an event stream & running the rules again. Sytest passes, but remaining to do: ... * Make badges work again * Remove old, unused code	2016-04-06 15:42:15 +01:00
Richard van der Hoff	1e05637e37	Let users see their own leave events ... ... otherwise clients get confused. Fixes https://matrix.org/jira/browse/SYN-662, https://github.com/vector-im/vector-web/issues/368	2016-04-06 15:36:19 +01:00
Erik Johnston	c2b429ab24	Merge pull request #693 from matrix-org/erikj/backfill_self ... Don't backfill from self	2016-04-05 13:04:36 +01:00
Erik Johnston	6222ae51ce	Don't backfill from self	2016-04-05 12:56:29 +01:00
Erik Johnston	b29f98377d	Merge pull request #691 from matrix-org/erikj/member ... Fix stuck invites	2016-04-05 12:44:39 +01:00
Erik Johnston	df727f2126	Fix stuck invites ... If rejecting a remote invite fails with an error response don't fail the entire request; instead mark the invite as locally rejected. This fixes the bug where users can get stuck invites which they can neither accept nor reject.	2016-04-05 11:13:24 +01:00
Erik Johnston	7a77f8b6d5	Merge pull request #690 from matrix-org/erikj/member ... Store invites in a separate table.	2016-04-05 09:12:27 +01:00
Erik Johnston	0c53d750e7	Docs and indents	2016-04-04 18:02:48 +01:00
Erik Johnston	3d76b7cb2b	Store invites in a separate table.	2016-04-04 16:30:15 +01:00
Erik Johnston	bf14883a04	Merge pull request #689 from matrix-org/erikj/member ... Do checks for memberships before creating events	2016-04-04 11:56:40 +01:00
Erik Johnston	d76d89323c	Use computed prev event ids	2016-04-01 17:39:32 +01:00
Erik Johnston	aa82cb38e9	Remove state hack from _create_new_client_event	2016-04-01 16:36:54 +01:00
Mark Haines	89e6839a48	Merge pull request #686 from matrix-org/markjh/doc_strings ... Use google style doc strings.	2016-04-01 16:20:09 +01:00
Erik Johnston	c906f30661	Do checks for memberships before creating events	2016-04-01 16:17:32 +01:00
Mark Haines	2a37467fa1	Use google style doc strings. ... pycharm supports them so there is no need to use the other format. Might as well convert the existing strings to reduce the risk of people accidentally cargo culting the wrong doc string format.	2016-04-01 16:12:07 +01:00
Erik Johnston	8d73cd502b	Add concurrently_execute function	2016-04-01 14:06:00 +01:00
Erik Johnston	35bb465b86	Filter rooms list before chunking	2016-04-01 13:14:53 +01:00
Mark Haines	dc4c1579d4	Remove outlier parameter from compute_event_context ... Use event.internal_metadata.is_outlier instead.	2016-03-31 15:32:24 +01:00
Mark Haines	03e406eefc	Merge pull request #680 from matrix-org/markjh/remove_is_new_state ... Remove the is_new_state argument to persist event.	2016-03-31 15:14:48 +01:00
Mark Haines	76503f95ed	Remove the is_new_state argument to persist event. ... Move the checks for whether an event is new state inside persist event itself. This was harder than expected because there wasn't enough information passed to persist event to correctly handle invites from remote servers for new rooms.	2016-03-31 15:00:42 +01:00
Erik Johnston	d35780eda0	Split out RoomMemberHandler	2016-03-31 13:08:45 +01:00
Erik Johnston	5260db7663	Line length	2016-03-31 10:49:27 +01:00
Erik Johnston	73b6bf4629	Only forget room if you were in the room	2016-03-30 15:09:18 +01:00
Erik Johnston	fddb6fddc1	Require user to have left room to forget room ... This dramatically simplifies the forget API code - in particular it no longer generates a leave event.	2016-03-30 11:03:00 +01:00
Mark Haines	54a546091a	Add a response cache for getting the public room list	2016-03-24 18:02:10 +00:00
Mark Haines	191c7bef6b	Deduplicate identical /sync requests	2016-03-24 17:47:31 +00:00
Erik Johnston	84afeb41f3	Ensure all old public rooms have aliases	2016-03-23 13:59:34 +00:00
Erik Johnston	b2802a1351	Ensure published rooms have public join rules	2016-03-23 13:59:31 +00:00
Erik Johnston	0677fc1c4e	Comment	2016-03-23 13:25:22 +00:00
Erik Johnston	34473a9c7f	Don't require alias in public room list. ... Rooms now no longer require an alias to be published. Also, changes the way we pull out state of each room to not require fetching all state events.	2016-03-23 10:42:19 +00:00
Erik Johnston	d6ac752538	Merge pull request #657 from matrix-org/erikj/roomlist ... Add published room list edit API	2016-03-22 11:57:39 +00:00
Erik Johnston	97785bfc0f	Doc string	2016-03-22 10:41:44 +00:00
Matthew Hodgson	d3654694d0	an invalide is something else...	2016-03-22 00:52:31 +00:00
Mark Haines	5244c0b48e	Remove unused backfilled parameter from persist_event	2016-03-21 18:06:08 +00:00
Erik Johnston	3e7fac0d56	Add published room list edit API	2016-03-21 15:06:07 +00:00
Mark Haines	58f8226c7f	remove unused current_state variable from on_receive_pdu	2016-03-21 14:20:34 +00:00