Matrix-Mirrors/synapse-product
1
0
Fork 0
mirror of https://git.anonymousland.org/anonymousland/synapse-product.git synced 2024-12-24 02:59:23 -05:00
Code Issues Packages Projects Releases Wiki Activity
10,427 Commits 4 Branches 609 Tags 60 MiB
9ef0b179e0
Commit Graph

2089 Commits

Author SHA1 Message Date
Luke Barnard
d892079844 Finish implementing RM endpoint 
- This change causes a 405 to be sent if "m.read_marker" is set via /account_data
 - This also fixes-up the RM endpoint so that it actually Works.
 2017-04-11 15:01:39 +01:00
lukebarnard
e263c26690 Initial commit of RM server-side impl 
(See https://docs.google.com/document/d/1UWqdS-e1sdwkLDUY0wA4gZyIkRp-ekjsLZ8k6g_Zvso/edit#heading=h.lndohpg8at5u)
 2017-04-11 11:55:30 +01:00
Erik Johnston
f3cf3ff8b6 Merge branch 'master' of github.com:matrix-org/synapse into develop 2017-04-11 11:13:32 +01:00
Erik Johnston
29574fd5b3 Reduce federation presence replication traffic 
This is mainly done by moving the calculation of where to send presence
updates from the presence handler to the transaction queue, so we only
need to send the presence event (and not the destinations) across the
replication connection. Before we were duplicating by sending the full
state across once per destination.
 2017-04-10 16:48:30 +01:00
Richard van der Hoff
64765e5199 When we do an invite rejection, save the signed leave event to the db 
During a rejection of an invite received over federation, we ask a remote
server to make us a `leave` event, then sign it, then send that with
`send_leave`.

We were saving the *unsigned* version of the event (which has a different event
id to the signed version) to our db (and sending it to the clients), whereas
other servers in the room will have seen the *signed* version. We're not aware
of any actual problems that caused, except that it makes the database confusing
to look at and generally leaves the room in a weird state.
 2017-04-07 14:39:32 +01:00
Erik Johnston
98ce212093 Merge pull request #2103 from matrix-org/erikj/no-double-encode 
Don't double encode replication data
 2017-04-07 09:39:52 +01:00
Erik Johnston
877c029c16 Use iteritems 2017-04-06 15:51:22 +01:00
Erik Johnston
96b9b6c127 Don't double json encode typing replication data 2017-04-05 11:34:20 +01:00
Erik Johnston
62b89daac6 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/repl_tcp_server 2017-04-04 09:46:16 +01:00
Richard van der Hoff
773e64cc1a Merge pull request #2095 from matrix-org/rav/cull_log_preserves 
Cull spurious PreserveLoggingContexts
 2017-04-03 17:02:25 +01:00
Richard van der Hoff
30bcbf775a Accept join events from all servers 
Make sure that we accept join events from any server, rather than just the
origin server, to make the federation join dance work correctly.

(Fixes #1893).
 2017-04-03 15:58:07 +01:00
Richard van der Hoff
7eb9f34cc3 Remove spurious yield 
In `MessageHandler`, remove `yield` on call to `Notifier.on_new_room_event`:
it doesn't return anything anyway.
 2017-04-03 15:44:19 +01:00
Richard van der Hoff
0b08c48fc5 Remove more spurious PreserveLoggingContexts 
Remove `PreserveLoggingContext` around calls to `Notifier.on_new_room_event`;
there is no problem if the logcontext is set when calling it.
 2017-04-03 15:43:37 +01:00
Erik Johnston
36d2b66f90 Add a timestamp to USER_SYNC command 
This timestamp is used to indicate when the user last sync'd
 2017-03-31 15:42:22 +01:00
Erik Johnston
9d0170ac6c Fix up presence 2017-03-31 11:36:32 +01:00
Erik Johnston
63fcc42990 Remove user from process_presence when stops syncing 2017-03-30 14:26:08 +01:00
Erik Johnston
e9dd8370b0 Add functions to presence to support remote syncs 
The TCP replication protocol streams deltas of who has started or
stopped syncing. This is different from the HTTP API which periodically
sends the full list of users who are syncing. This commit adds support
for the new TCP style of sending deltas.
 2017-03-30 13:25:14 +01:00
Erik Johnston
24d35ab47b Add new storage functions for new replication 
The new replication protocol will keep all the streams separate, rather
than muxing multiple streams into one.
 2017-03-30 11:48:35 +01:00
Erik Johnston
7fc1f1e2b6 Cache hosts in room 2017-03-24 11:46:24 +00:00
Richard van der Hoff
9397edb28b Merge pull request #2050 from matrix-org/rav/federation_backoff 
push federation retry limiter down to matrixfederationclient
 2017-03-23 22:27:01 +00:00
Richard van der Hoff
5a16cb4bf0 Ignore backoff history for invites, aliases, and roomdirs 
Add a param to the federation client which lets us ignore historical backoff
data for federation queries, and set it for a handful of operations.
 2017-03-23 12:23:22 +00:00
Richard van der Hoff
4bd597d9fc push federation retry limiter down to matrixfederationclient 
rather than having to instrument everywhere we make a federation call,
make the MatrixFederationHttpClient manage the retry limiter.
 2017-03-23 09:28:46 +00:00
Richard van der Hoff
e08f81d96a Add a missing yield in device key upload 
(this would only very very rarely actually be a useful thing, so the main
problem was the logcontext leak...)
 2017-03-23 00:16:43 +00:00
Richard van der Hoff
eddce9d74a Merge pull request #2027 from matrix-org/rav/logcontext_leaks 
A few fixes to logcontext things
 2017-03-20 11:53:36 +00:00
Richard van der Hoff
f40c2db05a Stop preserve_fn leaking context into the reactor 
Fix a bug in ``logcontext.preserve_fn`` which made it leak context into the
reactor, and add a test for it.

Also, get rid of ``logcontext.reset_context_after_deferred``, which tried to do
the same thing but had its own, different, set of bugs.
 2017-03-18 00:07:43 +00:00
Richard van der Hoff
5068fb16a5 Refactoring and cleanups 
A few non-functional changes:

* A bunch of docstrings to document types
* Split `EventsStore._persist_events_txn` up a bit. Hopefully it's a bit more
  readable.
* Rephrase `EventFederationStore._update_min_depth_for_room_txn` to avoid
  mind-bending conditional.
* Rephrase rejected/outlier conditional in `_update_outliers_txn` to avoid
  mind-bending conditional.
 2017-03-17 15:06:07 +00:00
Richard van der Hoff
2abe85d50e Merge pull request #2016 from matrix-org/rav/queue_pdus_during_join 
Queue up federation PDUs while a room join is in progress
 2017-03-17 11:32:44 +00:00
Erik Johnston
da146657c9 Comments 2017-03-16 13:04:07 +00:00
Erik Johnston
a158c36a8a Comment 2017-03-16 11:57:45 +00:00
Erik Johnston
6957bfdca6 Don't recreate so many sets 2017-03-16 11:54:26 +00:00
Erik Johnston
2ccf3b241c Implement no op for room stream in sync 2017-03-16 11:06:41 +00:00
Richard van der Hoff
9ce53a3861 Queue up federation PDUs while a room join is in progress 
This just takes the existing `room_queues` logic and moves it out to
`on_receive_pdu` instead of `_process_received_pdu`, which ensures that we
don't start trying to fetch prev_events and whathaveyou until the join has
completed.
 2017-03-15 18:01:11 +00:00
Erik Johnston
54d2b7e596 Merge pull request #2014 from Half-Shot/hs/fix-appservice-presence 
Add fallback to last_active_ts if it beats the last sync time on a presence timeout.
 2017-03-15 17:37:15 +00:00
Erik Johnston
f83ac78201 Cache set of users whose presence the other user should see 2017-03-15 15:29:19 +00:00
Will Hunt
e6032054bf Add a great comment to handle_timeout for active vs sync times. 2017-03-15 15:24:48 +00:00
Will Hunt
ebf5a6b14c Add fallback to last_active_ts if it beats the last sync time. 2017-03-15 15:17:16 +00:00
Erik Johnston
e892457a03 Comment 2017-03-15 15:01:39 +00:00
Erik Johnston
6c82de5100 Format presence events on the edges instead of reformatting them multiple times 2017-03-15 14:27:34 +00:00
Erik Johnston
7827251daf Merge pull request #1994 from matrix-org/dbkr/msisdn_signin_2 
Phone number registration / login support v2
 2017-03-15 09:59:54 +00:00
Erik Johnston
bb256ac96f Merge branch 'develop' of github.com:matrix-org/synapse into erikj/public_list_speed 2017-03-14 11:35:05 +00:00
Luke Barnard
bbeeb97f75 Implement _simple_delete_many_txn, use it to delete devices 
(But this doesn't implement the same for deleting access tokens or e2e keys.

Also respond to code review.
 2017-03-13 17:53:23 +00:00
David Baker
73a5f06652 Support registration / login with phone number 
Changes from https://github.com/matrix-org/synapse/pull/1971
 2017-03-13 17:27:51 +00:00
Richard van der Hoff
3c69f32402 Merge remote-tracking branch 'origin/develop' into rav/refactor_received_pdu 2017-03-13 12:20:47 +00:00
Erik Johnston
0162994983 Comments 2017-03-13 11:53:26 +00:00
Erik Johnston
7eae6eaa2f Revert "Support registration & login with phone number" 2017-03-13 09:59:33 +00:00
Erik Johnston
79926e016e Assume rooms likely haven't changed 2017-03-13 09:50:10 +00:00
Erik Johnston
8ffbe43ba1 Get current state by using current_state_events table 2017-03-10 17:39:35 +00:00
Erik Johnston
64d62e41b8 Noop repated delete device inbox calls from /sync 2017-03-10 10:36:43 +00:00
Richard van der Hoff
29235901b8 Move FederationServer._handle_new_pdu to FederationHandler 
Unfortunately this significantly increases the size of the already-rather-big
FederationHandler, but the code fits more naturally here, and it paves the way
for the tighter integration that I need between handling incoming PDUs and
doing the join dance.

Other than renaming the existing `FederationHandler.on_receive_pdu` to
`_process_received_pdu` to make way for it, this just consists of the move, and
replacing `self.handler` with `self` and `self` with `self.replication_layer`.
 2017-03-09 16:20:13 +00:00
David Baker
663396e45d Merge pull request #1971 from matrix-org/dbkr/msisdn_signin 
Support registration & login with phone number
 2017-03-09 10:18:51 +00:00
David Baker
0e0aee25c4 Fix log line 2017-03-08 11:46:22 +00:00
David Baker
88df6c0c9a Factor out msisdn canonicalisation 
Plus a couple of other minor fixes
 2017-03-08 11:03:39 +00:00
David Baker
402a7bf63d Fix pep8 2017-03-08 09:33:40 +00:00
Erik Johnston
f2581ee8b8 Don't keep around old stream IDs forever 2017-03-03 16:02:53 +00:00
Erik Johnston
9834367eea Spelling 2017-03-03 15:31:57 +00:00
David Baker
ad882cd54d Just return the deferred straight off 
defer.returnValue doth not maketh a generator: it would need a
yield to be a generator, and this doesn't need a yield.
 2017-03-01 18:08:51 +00:00
David Baker
3557cf34dc Merge remote-tracking branch 'origin/develop' into dbkr/msisdn_signin 2017-03-01 17:20:37 +00:00
Erik Johnston
36be39b8b3 Fix device list update to not constantly resync 2017-03-01 14:12:11 +00:00
Erik Johnston
848cf95ea0 Pop with default value to stop throwing 2017-02-28 10:02:54 +00:00
Erik Johnston
64a2cef9bb Pop rather than del from dict 2017-02-27 19:15:36 +00:00
Erik Johnston
17673404fb Remove unused param 2017-02-20 15:02:01 +00:00
Erik Johnston
7f026792e1 Fix /context/ visibiltiy rules 2017-02-20 14:54:50 +00:00
David Baker
8c87bb550e Merge pull request #1922 from matrix-org/dbkr/allow_forget_for_ban 
Allow forgetting rooms you're banned from
 2017-02-17 10:52:30 +00:00
David Baker
474c9aadbe Allow forgetting rooms you're banned from 2017-02-15 19:32:20 +00:00
David Baker
ce3e583d94 WIP support for msisdn 3pid proxy methods 2017-02-14 15:05:55 +00:00
Richard van der Hoff
fc2f29c1d0 Fix bugs in the /keys/changes api 
* `get_forward_extremeties_for_room` takes a numeric `stream_ordering`. We were
  passing a `RoomStreamToken`, which meant that it returned the *current*
  extremities, rather than those corresponding to the `from_token`. However:
* `get_state_ids_for_events` required a second ('types') parameter; this meant
  that a `TypeError` was thrown and we ended up acting as though there was *no*
  prev state.
* `get_state_ids_for_events` actually returns a map from event_id to state
  dictionary - just looking up the state keys in it again meant that we acted
  as though there was no prev state. We now check if each member's state has
  changed since *any* of the extremities.

Also add/fix some comments.
 2017-02-14 13:59:50 +00:00
Erik Johnston
9e617cd4c2 Cache get_presence storage 2017-02-13 13:50:03 +00:00
Erik Johnston
af6da6db2d Merge pull request #1784 from morteza-araby/user-admin 
Administration functionalities
 2017-02-06 16:21:10 +01:00
Erik Johnston
a597994fb6 Measure new device list stuff 2017-02-02 18:36:17 +00:00
Erik Johnston
1232ae41cf Use new get_users_who_share_room_with_user 2017-02-02 15:25:00 +00:00
Erik Johnston
99fa03e8b5 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/device_list_fixes 2017-02-02 15:23:45 +00:00
Erik Johnston
a8331897aa Merge pull request #1876 from matrix-org/erikj/shared_member_store 
Make presence.get_new_events a bit faster
 2017-02-02 15:20:14 +00:00
Erik Johnston
46ecd9fd6d Use stream_ordering_to_exterm for /keys/changes 2017-02-02 14:27:19 +00:00
Erik Johnston
54a79c1d37 Make presence.get_new_events a bit faster 
We do this by caching the set of users a user shares rooms with.
 2017-02-02 13:07:18 +00:00
Morteza Araby
2849d3f29d admin,storage: added more administrator functionalities 
administrators can now:
 - Set displayname of users
 - Update user avatars
 - Search for users by user_id
 - Browse all users in a paginated API
 - Reset user passwords
 - Deactivate users

Helpers for doing paginated queries has also been added to storage

Signed-off-by: Morteza Araby <morteza.araby@ericsson.com>
 2017-02-02 14:02:26 +01:00
Erik Johnston
51adaac953 Fix email push in pusher worker 
This was broken when device list updates were implemented, as Mailer
could no longer instantiate an AuthHandler due to a dependency on
federation sending.
 2017-02-02 10:53:36 +00:00
Erik Johnston
fbfe44bb4d Doc args 2017-02-01 17:52:57 +00:00
Erik Johnston
7e919bdbd0 Include newly joined users in /keys/changes API 2017-02-01 17:33:16 +00:00
Erik Johnston
6d6591880e Wake sync up for device changes 2017-02-01 15:15:16 +00:00
Erik Johnston
97479d0c54 Implement /keys/changes 2017-02-01 10:30:03 +00:00
Erik Johnston
bfb3d255b1 Merge pull request #1862 from matrix-org/erikj/presence_update 
Use DB cache of joined users for presence
 2017-01-31 13:23:24 +00:00
Erik Johnston
c7a26b7c32 Fix unit tests 2017-01-30 17:11:24 +00:00
Erik Johnston
fd1c18c088 Use DB cache of joined users for presence 2017-01-30 17:00:24 +00:00
Erik Johnston
c2c9a78db9 Noop device key changes if they're the same 2017-01-30 16:55:04 +00:00
Erik Johnston
828db669ec Use get_users_in_room and declare it iterable 2017-01-30 16:37:22 +00:00
Erik Johnston
9636b2407d Merge pull request #1857 from matrix-org/erikj/device_list_stream 
Implement device lists updates over federation
 2017-01-30 14:35:21 +00:00
Erik Johnston
252b503fc8 Hook device list updates to replication 2017-01-27 14:31:35 +00:00
Erik Johnston
888c59c955 Better name 2017-01-27 10:29:47 +00:00
Erik Johnston
76d40f4904 Handle users leaving rooms 2017-01-26 16:39:33 +00:00
Erik Johnston
fbfad76c03 Add comments 2017-01-26 16:33:21 +00:00
Erik Johnston
c974116f19 Implement device key caching over federation 2017-01-26 16:07:24 +00:00
Erik Johnston
51e9fe36e4 Fix up sending of m.device_list_update edus 2017-01-25 16:55:21 +00:00
Erik Johnston
2367c5568c Add basic implementation of local device list changes 2017-01-25 14:27:27 +00:00
Paul "LeoNerd" Evans
10e48d8310 Don't clobber a displayname or avatar_url if provided by an m.room.member event 2017-01-24 18:06:07 +00:00
Erik Johnston
a55fa2047f Insert delta of current_state_events to be more efficient 2017-01-20 17:10:18 +00:00
Erik Johnston
09eb08f910 Derive current_state_events from state groups 2017-01-20 11:52:51 +00:00
Erik Johnston
f9058ca785 Merge pull request #1822 from matrix-org/erikj/statE_logging 
Change resolve_state_groups call site logging to DEBUG
 2017-01-18 11:02:03 +00:00
Erik Johnston
15f012032c Merge pull request #1818 from matrix-org/erikj/state_auth_splitout_split 
Optimise state resolution
 2017-01-18 10:53:00 +00:00
Erik Johnston
5f027d1fc5 Change resolve_state_groups call site logging to DEBUG 2017-01-17 17:07:15 +00:00
Erik Johnston
e6153e1bd1 Fix couple of federation state bugs 2017-01-17 13:22:34 +00:00
Erik Johnston
5d6bad1b3c Optimise state resolution 2017-01-17 13:22:19 +00:00
Erik Johnston
7b62d0bc70 Add missing None check 2017-01-11 10:57:03 +00:00
Erik Johnston
657488113e Merge pull request #1789 from matrix-org/erikj/decouple_presence 
Don't block messages sending on bumping presence
 2017-01-10 14:06:05 +00:00
Erik Johnston
3b4de17d2b Comment 2017-01-10 14:05:53 +00:00
Erik Johnston
7d0981b312 Merge pull request #1787 from matrix-org/erikj/linearize_member 
Linearize updates to membership via PUT /state/
 2017-01-10 14:04:54 +00:00
Erik Johnston
586f474a44 Don't block messages sending on bumping presence 2017-01-10 12:46:00 +00:00
Erik Johnston
6823fe5241 Linearize updates to membership via PUT /state/ 2017-01-09 18:25:13 +00:00
Erik Johnston
f7085ac84f Name linearizer's for better logs 2017-01-09 17:17:10 +00:00
Mark Haines
06c00bd19b Merge branch 'release-v0.18.6' into develop 2017-01-06 14:46:27 +00:00
Patrik Oldsberg
1ef6084b75 handlers/room_member: fix guest access check when joining rooms 
Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>
 2017-01-06 14:36:56 +00:00
Matthew Hodgson
bd85434cb3 Merge branch 'release-v0.18.6' into develop 2017-01-05 13:58:19 +00:00
Matthew Hodgson
883ff92a7f Fix case 2017-01-05 13:45:02 +00:00
Matthew Hodgson
d79d165761 add logging for all the places we call resolve_state_groups. my kingdom for a backtrace that actually works. 2017-01-05 13:40:39 +00:00
Matthew Hodgson
b31ed22738 Merge branch 'release-v0.18.6' into develop 2017-01-05 13:03:02 +00:00
Mark Haines
f784980d2b Only send events that originate on this server. 
Or events that are sent via the federation "send_join" API.

This should match the behaviour from before v0.18.5 and #1635 landed.
 2017-01-05 11:26:30 +00:00
David Baker
84cf00c645 Fix another comment typo 2016-12-21 09:51:43 +00:00
David Baker
0c88ab1844 Add /account/3pid/delete endpoint 
Also fix a typo in a comment
 2016-12-20 18:27:30 +00:00
Matthew Hodgson
f2a5aebf98 fix ability to change password to a non-ascii one 
https://github.com/vector-im/riot-web/issues/2658
 2016-12-18 22:25:21 +00:00
Erik Johnston
b2f8642d3d Cache network room list queries. 2016-12-16 16:11:43 +00:00
Richard van der Hoff
edc6a1e4f9 Add some logging for syncing to_device events 
Attempt to track down the loss of to_device events
(https://github.com/vector-im/riot-web/issues/2711 etc).
 2016-12-15 18:16:10 +00:00
Erik Johnston
35129ac998 Merge pull request #1698 from matrix-org/erikj/room_list 
Fix caching on public room list
 2016-12-15 15:40:28 +00:00
Erik Johnston
b8255eba26 Comment 2016-12-14 13:49:54 +00:00
Erik Johnston
b2999a7055 Fix /backfill returning events it shouldn't 2016-12-14 13:41:45 +00:00
Erik Johnston
1de53a7a1a Fix caching on public room list 2016-12-13 17:33:24 +00:00
Erik Johnston
1574b839e0 Merge pull request #1676 from matrix-org/erikj/room_list 
Add new API appservice specific public room list
 2016-12-12 17:00:10 +00:00
Erik Johnston
d53a80af25 Merge pull request #1620 from matrix-org/erikj/concurrent_room_access 
Limit the number of events that can be created on a given room concurrently
 2016-12-12 10:30:23 +00:00
Richard van der Hoff
85cd30b1fd Merge pull request #1686 from matrix-org/rav/fix_federation_key_fails 
E2E key query: handle federation fails
 2016-12-12 09:33:39 +00:00
Richard van der Hoff
deca951241 Remove unused import 2016-12-12 09:24:35 +00:00
Richard van der Hoff
efa4ccfaee E2E key query: handle federation fails 
Don't fail the whole request if we can't connect to a particular server.
 2016-12-09 18:31:01 +00:00
Erik Johnston
be14c24cea Fix rare notifier bug where listeners dont timeout 
There was a race condition that caused the notifier to 'miss' the
timeout notification, since there were no other checks for the timeout
this caused listeners to get stuck in a loop until something happened.
 2016-12-09 15:43:18 +00:00
Richard van der Hoff
b08f76bd23 Fix ignored read-receipts 
Don't ignore read-receipts which arrive in the same EDU as a read-receipt for
an old event.
 2016-12-08 12:13:01 +00:00
Erik Johnston
2687af82d4 Comments 2016-12-07 09:58:33 +00:00
Erik Johnston
f32fb65552 Add new API appservice specific public room list 2016-12-06 16:12:27 +00:00
Richard van der Hoff
1529c19675 Prevent user tokens being used as guest tokens (#1675) 
Make sure that a user cannot pretend to be a guest by adding 'guest = True'
caveats.
 2016-12-06 15:31:37 +00:00
Erik Johnston
9609c91e7d Merge pull request #653 from matrix-org/erikj/preset_guest_join 
Enable guest access for private rooms by default
 2016-12-05 17:47:14 +00:00
Erik Johnston
338df4f409 Merge pull request #1649 from matrix-org/dbkr/log_ui_auth_args 
Log the args that we have on UI auth completion
 2016-12-05 16:40:58 +00:00
Richard van der Hoff
aa09d6b8f0 Rip out more refresh_token code 
We might as well treat all refresh_tokens as invalid. Just return a 403 from
/tokenrefresh, so that we don't have a load of dead, untestable code hanging
around.

Still TODO: removing the table from the schema.
 2016-11-30 17:40:18 +00:00
Richard van der Hoff
dc4b23e1a1 Merge branch 'develop' into rav/no_more_refresh_tokens 2016-11-30 17:10:04 +00:00
Richard van der Hoff
1c4f05db41 Stop putting a time caveat on access tokens 
The 'time' caveat on the access tokens was something of a lie, since we weren't
enforcing it; more pertinently its presence stops us ever adding useful time
caveats.

Let's move in the right direction by not lying in our caveats.
 2016-11-29 16:49:41 +00:00
Richard van der Hoff
5c4edc83b5 Stop generating refresh tokens 
Since we're not doing refresh tokens any more, we should start killing off the
dead code paths. /tokenrefresh itself is a bit of a thornier subject, since
there might be apps out there using it, but we can at least not generate
refresh tokens on new logins.
 2016-11-28 10:13:01 +00:00
David Baker
c9d4e7b716 Clarify that creds doesn not contain passwords. 2016-11-24 10:54:59 +00:00
David Baker
f681aab895 Log the args that we have on UI auth completion 
This will be super helpful for debugging if we have more
registration woes.
 2016-11-24 10:11:45 +00:00
Erik Johnston
feec718265 Shuffle receipt handler around so that worker apps don't need to load it 2016-11-23 15:14:24 +00:00
Erik Johnston
b69f76c106 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/split_out_fed_txn 2016-11-23 11:31:53 +00:00
Kegan Dougal
83bcdcee61 Return early on /sync code paths if a '*' filter is used 
This is currently very conservative in that it only does this if there is no
`since` token. This limits the risk to clients likely to be doing one-off
syncs (like bridges), but does mean that normal human clients won't benefit
from the time savings here. If the savings are large enough, I would consider
generalising this to just check the filter.
 2016-11-22 16:38:35 +00:00
Erik Johnston
9687e039e7 Remove explicit calls to send_pdu 2016-11-21 14:48:51 +00:00
Erik Johnston
f8ee66250a Handle sending events and device messages over federation 2016-11-17 15:48:04 +00:00
Erik Johnston
59ef517e6b Use new federation_sender DI 2016-11-16 14:47:52 +00:00
Erik Johnston
d56c39cf24 Use external ldap auth pacakge 2016-11-15 13:03:19 +00:00
Erik Johnston
64038b806c Comments 2016-11-11 10:42:08 +00:00
Erik Johnston
2bd4513a4d Limit the number of events that can be created on a given room concurretnly 2016-11-10 16:44:35 +00:00
Erik Johnston
a4632783fb Sample correctly 2016-11-08 11:20:26 +00:00
Erik Johnston
eeda4e618c Limit the number of prev_events of new events 2016-11-08 11:02:29 +00:00
David Baker
9084720993 Don't error on non-ascii passwords 2016-11-03 10:42:14 +00:00
Erik Johnston
2ef617bc06 Fix infinite typing bug 
There's a bug somewhere that causes typing notifications to not be timed
out properly. By adding a paranoia timer and using correct inequalities
notifications should stop being stuck, even if it the root cause hasn't
been fixed.
 2016-10-24 15:51:22 +01:00
Erik Johnston
d04e2ff3a4 Fix incredubly slow back pagination query 
If a client didn't specify a from token when paginating backwards
synapse would attempt to query the (global) maximum topological token.
This a) doesn't make much sense since they're room specific and b) there
are no indices that lets postgres do this efficiently.
 2016-10-24 13:35:51 +01:00
Luke Barnard
e01a1bc92d Merge pull request #1175 from matrix-org/luke/feature-configurable-as-rate-limiting 
Allow Configurable Rate Limiting Per AS
 2016-10-20 16:21:10 +01:00
Luke Barnard
f09db236b1 as_user->app_service, less redundant comments, better positioned comments 2016-10-20 12:04:54 +01:00
Luke Barnard
1b17d1a106 Use real AS object by passing it through the requester 
This means synapse does not have to check if the AS is interested, but instead it effectively re-uses what it already knew about the requesting user
 2016-10-20 11:43:05 +01:00
Erik Johnston
3aa8925091 Merge pull request #1176 from matrix-org/erikj/eager_ratelimit_check 
Check whether to ratelimit sooner to avoid work
 2016-10-19 14:25:52 +01:00
Erik Johnston
f2f74ffce6 Comment 2016-10-19 14:21:28 +01:00
David Baker
df2a616c7b Convert emails to lowercase when storing 
And db migration sql to convert existing addresses.
 2016-10-19 11:13:55 +01:00
Erik Johnston
550308c7a1 Check whether to ratelimit sooner to avoid work 2016-10-19 10:45:24 +01:00
Luke Barnard
5b54d51d1e Allow Configurable Rate Limiting Per AS 
This adds a flag loaded from the registration file of an AS that will determine whether or not its users are rate limited (by ratelimit in _base.py). Needed for IRC bridge reasons - see https://github.com/matrix-org/matrix-appservice-irc/issues/240.
 2016-10-18 17:04:09 +01:00
Erik Johnston
35e2cc8b52 Merge pull request #1155 from matrix-org/erikj/pluggable_pwd_auth 
Implement pluggable password auth
 2016-10-12 11:41:20 +01:00
Richard van der Hoff
0061e8744f Merge pull request #1166 from matrix-org/rav/grandfather_broken_riot_signup 
Work around email-spamming Riot bug
 2016-10-11 11:58:58 +01:00
Richard van der Hoff
fa74fcf512 Work around email-spamming Riot bug 
5d9546f9 introduced a change to synapse behaviour, in that failures in the
interactive-auth process would return the flows and params data as well as an
error code (as specced in https://github.com/matrix-org/matrix-doc/pull/397).

That change exposed a bug in Riot which would make it request a new validation
token (and send a new email) each time it got a 401 with a `flows` parameter
(see https://github.com/vector-im/vector-web/issues/2447 and the fix at
https://github.com/matrix-org/matrix-react-sdk/pull/510).

To preserve compatibility with broken versions of Riot, grandfather in the old
behaviour for the email validation stage.
 2016-10-11 11:34:40 +01:00
Erik Johnston
a2f2516199 Merge pull request #1157 from Rugvip/nolimit 
Remove rate limiting from app service senders and fix get_or_create_user requester
 2016-10-11 11:20:54 +01:00
Richard van der Hoff
8681aff4f1 Merge pull request #1160 from matrix-org/rav/401_on_password_fail 
Interactive Auth: Return 401 from for incorrect password
 2016-10-07 10:57:43 +01:00
Richard van der Hoff
5d9546f9f4 Interactive Auth: Return 401 from for incorrect password 
This requires a bit of fettling, because I want to return a helpful error
message too but we don't want to distinguish between unknown user and invalid
password. To avoid hardcoding the error message into 15 places in the code,
I've had to refactor a few methods to return None instead of throwing.

Fixes https://matrix.org/jira/browse/SYN-744
 2016-10-07 00:00:00 +01:00
Patrik Oldsberg
7b5546d077 rest/client/v1/register: use the correct requester in createUser 
Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>
 2016-10-06 22:12:32 +02:00
Patrik Oldsberg
3de7c8a4d0 handlers/profile: added admin override for set_displayname and set_avatar_url 
Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>
 2016-10-06 15:24:59 +02:00
Patrik Oldsberg
2ff2d36b80 handers: do not ratelimit app service senders 
Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>
 2016-10-06 15:24:59 +02:00
Patrik Oldsberg
9bfc617791 storage/appservice: make appservice methods only relying on the cache synchronous 2016-10-06 15:24:59 +02:00
Erik Johnston
850b103b36 Implement pluggable password auth 
Allows delegating the password auth to an external module. This also
moves the LDAP auth to using this system, allowing it to be removed from
the synapse tree entirely in the future.
 2016-10-03 10:36:40 +01:00
Martin Weinelt
3027ea22b0 Restructure ldap authentication 
- properly parse return values of ldap bind() calls
- externalize authentication methods
- change control flow to be more error-resilient
- unbind ldap connections in many places
- improve log messages and loglevels
 2016-09-29 15:30:08 +01:00
Erik Johnston
655891d179 Move FEDERATION_PING_INTERVAL timer. Update log line 2016-09-23 15:43:34 +01:00
Erik Johnston
22578545a0 Time out typing over federation 2016-09-23 14:00:52 +01:00
Erik Johnston
667fcd54e8 Merge pull request #1136 from matrix-org/erikj/fix_signed_3pid 
Allow invites via 3pid to bypass sender sig check
 2016-09-22 13:41:49 +01:00
Erik Johnston
81964aeb90 Merge pull request #1132 from matrix-org/erikj/initial_sync_split 
Support /initialSync in synchrotron worker
 2016-09-22 12:45:02 +01:00
Erik Johnston
2e9ee30969 Add comments 2016-09-22 11:59:46 +01:00
Erik Johnston
1168cbd54d Allow invites via 3pid to bypass sender sig check 
When a server sends a third party invite another server may be the one
that the inviting user registers with. In this case it is that remote
server that will issue an actual invitation, and wants to do it "in the
name of" the original invitee. However, the new proper invite will not
be signed by the original server, and thus other servers would reject
the invite if it was seen as coming from the original user.

To fix this, a special case has been added to the auth rules whereby
another server can send an invite "in the name of" another server's
user, so long as that user had previously issued a third party invite
that is now being accepted.
 2016-09-22 10:56:53 +01:00
Erik Johnston
90c070c850 Add total_room_count_estimate to /publicRooms 2016-09-21 13:30:05 +01:00
Erik Johnston
87528f0756 Support /initialSync in synchrotron worker 2016-09-21 11:46:28 +01:00
Erik Johnston
39af634dd2 Merge pull request #1130 from matrix-org/erikj/fix_pubroom_pag 
Handle fact that _generate_room_entry may not return a room entry
 2016-09-19 10:13:59 +01:00
Erik Johnston
4d49e0bdfd PEP8 2016-09-17 18:09:22 +01:00
Erik Johnston
81570abfb2 Handle fact that _generate_room_entry may not return a room entry 2016-09-17 18:01:54 +01:00
Erik Johnston
eb24aecf8c Merge pull request #1129 from matrix-org/erikj/fix_pubroom_pag 
Fix and clean up publicRooms pagination
 2016-09-17 15:30:34 +01:00
Erik Johnston
a298331de4 Spelling 2016-09-17 14:59:40 +01:00
Erik Johnston
71edaae981 Fix and clean up publicRooms pagination 2016-09-17 14:46:19 +01:00
David Baker
5336acd46f Make public room search case insensitive 2016-09-16 19:02:42 +01:00
Erik Johnston
b7b62bf9ea Comment 2016-09-16 11:00:29 +01:00
Erik Johnston
d84319ae10 Add remote reoom cache 2016-09-16 10:31:59 +01:00
Erik Johnston
23b6701a28 Support filtering remote room lists 2016-09-16 10:24:15 +01:00
Erik Johnston
e58a9d781c Filter remote rooms lists locally 2016-09-16 10:19:32 +01:00
Erik Johnston
74d4cdee25 Don't cache searched in /publicRooms 2016-09-16 09:05:11 +01:00
Erik Johnston
098db4aa52 Add very basic filter API to /publicRooms 2016-09-15 17:50:16 +01:00
Erik Johnston
c33b25fd8d Change the way we calculate new_limit in /publicRooms and add POST API 2016-09-15 17:35:20 +01:00
Erik Johnston
e457034e99 Merge pull request #1121 from matrix-org/erikj/public_room_paginate 
Add pagination support to publicRooms
 2016-09-15 13:27:09 +01:00
Erik Johnston
4fb65a1091 Base public room list off of public_rooms stream 2016-09-15 11:27:04 +01:00
Erik Johnston
5810cffd33 Pass since/from parameters over federation 2016-09-15 10:36:19 +01:00
Erik Johnston
f3eead0660 Allow paginating both forwards and backwards 2016-09-15 10:15:37 +01:00
Erik Johnston
4131381123 Remove support for aggregate room lists 2016-09-15 09:28:15 +01:00
Erik Johnston
4f181f361d Accept optional token to public room list 2016-09-15 09:08:57 +01:00
Erik Johnston
c566f0ee17 Calculate the public room list from a stream_ordering 2016-09-14 17:42:47 +01:00
Erik Johnston
772c6067a3 Refactor public rooms to not pull out the full state for each room 2016-09-14 17:29:25 +01:00
Erik Johnston
264a48aedf Merge pull request #1117 from matrix-org/erikj/fix_state 
Ensure we don't mutate state cache entries
 2016-09-14 16:50:37 +01:00
Erik Johnston
a70a43bc51 Move RoomListHandler into a separate file 2016-09-14 14:07:37 +01:00
Erik Johnston
d5ae1f1291 Ensure we don't mutate state cache entries 2016-09-14 10:03:48 +01:00
Matthew Hodgson
1b01488d27 Merge pull request #1111 from matrix-org/matthew/device-ids 
make device IDs more useful for human disambiguation
 2016-09-14 01:18:02 +01:00
Paul Evans
0f73f0e70e Merge pull request #1116 from matrix-org/paul/tiny-fixes 
Fix typo "persiting"
 2016-09-13 13:27:42 +01:00
Paul "LeoNerd" Evans
ca35e54d6b Fix typo "persiting" 2016-09-13 13:26:33 +01:00
Mark Haines
ad816b0add Limit how often we ask for keys from dead servers 2016-09-13 11:53:50 +01:00
Mark Haines
18ab019a4a Move the E2E key handling into the e2e handler 2016-09-13 11:35:35 +01:00
Mark Haines
76b09c29b0 Merge pull request #1110 from matrix-org/markjh/e2e_timeout 
Add a timeout parameter for end2end key queries.
 2016-09-13 10:50:45 +01:00
Matthew Hodgson
edbcb4152b make device IDs more useful for human disambiguation 2016-09-13 00:02:39 +01:00
Mark Haines
949c2c5435 Add a timeout parameter for end2end key queries. 
Add a timeout parameter for controlling how long synapse will wait
for responses from remote servers. For servers that fail include how
they failed to make it easier to debug.

Fetch keys from different servers in parallel rather than in series.

Set the default timeout to 10s.
 2016-09-12 18:17:09 +01:00
Erik Johnston
48a5a7552d Add is_direct param to /createRoom 2016-09-12 16:34:20 +01:00
Erik Johnston
3265def8c7 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/batch_edus 2016-09-09 18:06:01 +01:00
Erik Johnston
52b2318777 Clobber EDUs in send queue 2016-09-09 15:59:08 +01:00
Paul Evans
56f38d1776 Merge pull request #1091 from matrix-org/paul/third-party-lookup 
Improvements to 3PE lookup API
 2016-09-09 15:43:11 +01:00
Paul "LeoNerd" Evans
ed44c475d8 Reject malformed 3PE query metadata results earlier in AS API handling code 2016-09-09 15:07:04 +01:00
Paul "LeoNerd" Evans
f25d74f69c Minor fixes from PR comments 2016-09-09 14:54:16 +01:00
Erik Johnston
6c4d582144 Deduplicate presence in _update_states 2016-09-09 14:28:22 +01:00
Paul "LeoNerd" Evans
25eb769b26 Efficiency fix for lookups of a single protocol 2016-09-09 13:25:02 +01:00
Paul "LeoNerd" Evans
033d43e419 Don't corrupt shared cache on subsequent protocol requests 2016-09-09 13:10:36 +01:00
Paul "LeoNerd" Evans
2f267ee160 Collect up all the "instances" lists of individual AS protocol results into one combined answer to the client 2016-09-08 17:43:53 +01:00
Erik Johnston
257025ac89 Merge pull request #1082 from matrix-org/erikj/remote_public_rooms 
Add server param to /publicRooms
 2016-09-08 16:04:22 +01:00
Erik Johnston
8b93af662d Check the user_id for presence/typing matches origin 2016-09-08 15:07:38 +01:00
Mark Haines
fa9d36e050 Merge branch 'develop' into markjh/direct_to_device_federation 2016-09-08 13:43:43 +01:00
Erik Johnston
791658b576 Add server param to /publicRooms 2016-09-08 11:53:05 +01:00
Mark Haines
43954d000e Add a new method to enqueue the device messages rather than sending a dummy EDU 2016-09-07 16:10:51 +01:00
Mark Haines
d4a35ada28 Send device messages over federation 2016-09-06 18:16:20 +01:00
Erik Johnston
3c4208a057 Record counts of state changes 2016-09-06 11:31:01 +01:00
Erik Johnston
438ef47637 Short circuit if presence is the same 2016-09-06 10:28:35 +01:00
Erik Johnston
74a3b4a650 Fiddle should_notify to better report stats 2016-09-06 10:23:38 +01:00
Erik Johnston
8c93e0bae7 Merge pull request #1070 from matrix-org/erikj/presence_stats 
Record why we have chosen to notify
 2016-09-05 15:01:30 +01:00
Erik Johnston
69054e3d4c Record why we have chosen to notify 2016-09-05 14:12:11 +01:00
Erik Johnston
69a2d4e38c Use get_joined_users_from_context instead of manually looking up hosts 2016-09-05 13:44:40 +01:00
Mark Haines
274c2f50a5 Merge pull request #1067 from matrix-org/markjh/idempotent 
Fix membership changes to be idempotent
 2016-09-05 10:21:25 +01:00
Mark Haines
f2690c6423 Fix membership changes to be idempotent 2016-09-02 19:23:22 +01:00
Mark Haines
81b94c5750 Merge pull request #1066 from matrix-org/markjh/direct_to_device_lowerbound 
Only return new device messages in /sync
 2016-09-02 16:18:34 +01:00
Mark Haines
65fa37ac5e Only return new device messages in /sync 2016-09-02 15:50:37 +01:00
Erik Johnston
273b6bcf22 Merge pull request #1064 from matrix-org/erikj/on_receive_check 
Only check if host is in room if we have state and auth_chain
 2016-09-02 14:43:35 +01:00
Erik Johnston
f7f1027d3d Comment on when auth chain and state are None 2016-09-02 14:42:38 +01:00
Erik Johnston
34e5e17f91 Comment 2016-09-02 14:26:07 +01:00
Erik Johnston
b96c6c3185 Docstrings 2016-09-02 14:19:22 +01:00
Erik Johnston
bd9b8d87ae Only check if host is in room if we have state and auth_chain 2016-09-02 13:40:28 +01:00
Erik Johnston
2854ee2a52 Only pull out IDs from DB for /state_ids/ request 2016-09-02 10:53:36 +01:00
Erik Johnston
265d847ffd Fix typo in log line 2016-09-01 14:50:06 +01:00
Erik Johnston
c10cb581c6 Correctly handle the difference between prev and current state 2016-08-31 14:26:22 +01:00
Erik Johnston
c7f665d700 Merge pull request #1055 from matrix-org/erikj/occaisonally_persist 
Occaisonally persist unpersisted presence updates
 2016-08-30 15:59:00 +01:00
Erik Johnston
097330bae8 Check correct variable 2016-08-30 15:50:20 +01:00
Erik Johnston
21b977ccfe Occaisonally persist unpersisted presence updates 2016-08-30 15:39:50 +01:00
Erik Johnston
bc1a8b1f7a Don't notify for online -> online transitions. 
Specifically, if currently_active remains true then we should not notify
if only the last active time changes.
 2016-08-30 15:05:32 +01:00
Erik Johnston
c8cd87b21b Comment about message deletion 2016-08-30 11:23:26 +01:00
Erik Johnston
1ee6285905 Fix check 2016-08-30 11:17:46 +01:00
Erik Johnston
68e1a872fd Noop get_new_messages_for_device if token hasn't changed 2016-08-30 10:58:46 +01:00
Erik Johnston
55fc17cf4b Merge pull request #1049 from matrix-org/erikj/presence_users_in_room 
Use state handler instead of get_users_in_room/get_joined_hosts
 2016-08-30 10:50:37 +01:00
Erik Johnston
bed10f9880 Use state handler instead of get_users_in_room/get_joined_hosts 2016-08-26 14:54:30 +01:00
Mark Haines
4bbef62124 Merge remote-tracking branch 'origin/develop' into markjh/direct_to_device 2016-08-26 14:35:31 +01:00
Erik Johnston
11faa4296d Measure _filter_events_for_server 2016-08-26 11:15:40 +01:00
Erik Johnston
f6338d6a3e Don't pull out full state for _filter_events_for_server 2016-08-26 11:13:16 +01:00
Erik Johnston
30961182f2 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/state_ids 2016-08-26 09:48:13 +01:00
Erik Johnston
c1a133a6b6 Merge pull request #1043 from matrix-org/erikj/backfill_fix 
Fix None check in backfill
 2016-08-26 09:07:05 +01:00
Erik Johnston
778fa85f47 Make sync not pull out full state 2016-08-25 18:59:44 +01:00
Mark Haines
ab34fdecb7 Merge branch 'develop' into markjh/direct_to_device 2016-08-25 18:34:46 +01:00
Mark Haines
e993925279 Add store-and-forward direct-to-device messaging 2016-08-25 17:35:37 +01:00
Erik Johnston
a3dc1e9cbe Replace context.current_state with context.current_state_ids 2016-08-25 17:32:22 +01:00
Paul Evans
d9dcb2ba3a Merge pull request #1041 from matrix-org/paul/third-party-lookup 
Extend 3PE lookup APIs for metadata query
 2016-08-25 17:06:53 +01:00
Paul "LeoNerd" Evans
c435bfee9c Don't need toplevel cache on 3PE lookup metadata any more 2016-08-25 15:57:07 +01:00
Paul "LeoNerd" Evans
5474824975 Actually query over AS API for 3PE lookup metadata 2016-08-25 15:29:36 +01:00
Erik Johnston
17f4f14df7 Pull out event ids rather than full events for state 2016-08-25 13:42:44 +01:00
Erik Johnston
cd5b264b03 Fix None check in backfill 2016-08-25 10:39:19 +01:00
Erik Johnston
fda97dd58a Merge branch 'develop' of github.com:matrix-org/synapse into erikj/preserve_log_contexts 2016-08-24 13:22:02 +01:00
Paul "LeoNerd" Evans
8e1ed09dff Move static knowledge of protocol metadata into AS handler; cache the result 2016-08-24 13:01:53 +01:00
Paul "LeoNerd" Evans
9899824b85 Initial hack at the 3PN protocols metadata lookup API 2016-08-24 12:33:01 +01:00
Erik Johnston
9219139351 Preserve some logcontexts 2016-08-24 11:58:40 +01:00
Erik Johnston
86bcf4d6a7 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/pagination 2016-08-24 10:38:21 +01:00
Erik Johnston
ba07d4a70e Add None checks to backfill 2016-08-24 10:31:05 +01:00
Kegan Dougal
4b31426a02 Pass through user-supplied content in /join/$room_id 
It was always intended to allow custom keys on the join event, but this has
at some point been lost. Restore it.

If the user specifies keys like "avatar_url" then they will be clobbered.
 2016-08-23 16:32:04 +01:00
Erik Johnston
afbf6b33fc defer.returnValue must not be called within Measure 2016-08-19 18:23:44 +01:00
Paul Evans
5674ea3e6c Merge pull request #1026 from matrix-org/paul/thirdpartylookup 
3rd party entity lookup
 2016-08-18 20:52:50 +01:00
Paul "LeoNerd" Evans
65201631a4 Move validation logic for AS 3PE query response into ApplicationServiceApi class, to keep the handler logic neater 2016-08-18 17:33:56 +01:00
Paul "LeoNerd" Evans
697872cf08 More warnings about invalid results from AS 3PE query 2016-08-18 17:24:39 +01:00
Paul "LeoNerd" Evans
b515f844ee Avoid so much copypasta between 3PU and 3PL query by unifying around a ThirdPartyEntityKind enumeration 2016-08-18 17:19:55 +01:00
Paul "LeoNerd" Evans
2a91799fcc Minor syntax neatenings 2016-08-18 16:58:25 +01:00
Paul "LeoNerd" Evans
fcf1dec809 Appease pep8 2016-08-18 16:26:19 +01:00
Paul "LeoNerd" Evans
06964c4a0a Copypasta the 3PU support code to also do 3PL 2016-08-18 16:09:50 +01:00
Paul "LeoNerd" Evans
d7b42afc74 Log a warning if an AS yields an invalid 3PU lookup result 2016-08-18 15:49:55 +01:00
Paul "LeoNerd" Evans
80f4740c8f Scattergather the call out to ASes; validate received results 2016-08-18 15:40:41 +01:00
Erik Johnston
07229bbdae Add appservice worker 2016-08-18 14:59:55 +01:00
Paul "LeoNerd" Evans
434bbf2cb5 Filter 3PU lookups by only ASes that declare knowledge of that protocol 2016-08-18 14:56:02 +01:00
Paul "LeoNerd" Evans
d5bf7a4a99 Merge remote-tracking branch 'origin/develop' into paul/thirdpartylookup 2016-08-18 14:21:01 +01:00
Paul "LeoNerd" Evans
f0c73a1e7a Extend individual list results into the main return list, don't append 2016-08-18 13:53:54 +01:00
Erik Johnston
9da84a9a1e Make AppserviceHandler stream events from database 
This is for two reasons:

1. Suppresses duplicates correctly, as the notifier doesn't do any
   duplicate suppression.
2. Makes it easier to connect the AppserviceHandler to the replication
   stream.
 2016-08-18 11:54:41 +01:00
Erik Johnston
320dfe523c Make notify_interested_services faster 2016-08-17 17:20:50 +01:00
Paul "LeoNerd" Evans
fa87c981e1 Thread 3PU lookup through as far as the AS API object; which currently noöps it 2016-08-17 16:17:28 +01:00
Erik Johnston
62c5245c87 Measure notify_interested_services 2016-08-17 11:12:29 +01:00
Erik Johnston
7c6f4f9427 Merge pull request #1012 from matrix-org/erikj/limit_backfill_uri 
Limit number of extremeties in backfill request
 2016-08-16 12:55:42 +01:00
Erik Johnston
2ee1bd124c Limit number of extremeties in backfill request 
This works around a bug where if we make a backfill request with too
many extremeties it causes the request URI to be too long.
 2016-08-16 11:34:36 +01:00
Erik Johnston
dc3a00f24f Refactor user_delete_access_tokens. Invalidate get_user_by_access_token to slaves. 2016-08-15 17:04:39 +01:00
Erik Johnston
75299af4fc Merge pull request #1009 from matrix-org/erikj/event_split 
Split out /events to synchrotron
 2016-08-15 15:39:05 +01:00
Daniel Ehlers
dfaf0fee31
Log the value which is observed in the first place. 
The name 'result' is of bool type and has no len property,
resulting in a TypeError. Futhermore in the flow control
conn.response is observed and hence should be reported.

Signed-off-by: Daniel Ehlers <sargon@toppoint.de>
 2016-08-14 16:49:05 +02:00
Daniel Ehlers
e380538b59
Fix AttributeError when bind_dn is not defined. 
In case one does not define bind_dn in ldap configuration, filter
attribute is not declared. Since auth code only uses ldap_filter attribute
when according LDAP mode is selected, it is safe to only declare the
attribute in that case.

Signed-off-by: Daniel Ehlers <sargon@toppoint.de>
 2016-08-14 16:48:33 +02:00
Erik Johnston
4e1cebd56f Make synchrotron accept /events 2016-08-12 15:31:44 +01:00
Erik Johnston
448ac6cf0d Only process one local membership event per room at a time 2016-08-12 09:32:19 +01:00
Erik Johnston
832799dbff Merge pull request #997 from Half-Shot/develop 
Don't change status_msg on /sync
 2016-08-11 14:10:55 +01:00
Erik Johnston
739ea29d1e Also check if server is in the room 2016-08-10 13:32:23 +01:00
Erik Johnston
7f41bcbeec Correctly auth /event/ requests 2016-08-10 13:22:20 +01:00
Erik Johnston
11fdfaf03b Only resign our own events 2016-08-10 13:16:58 +01:00
Will Hunt
2510db3e76 Don't change status_msg on /sync 2016-08-10 12:59:59 +01:00
Richard van der Hoff
79ebfbe7c6 /login: Respond with a 403 when we get an invalid m.login.token 2016-08-09 16:29:28 +01:00
Richard van der Hoff
6fe6a6f029 Fix login with m.login.token 
login with token (as used by CAS auth) was broken by 067596d, such that it
always returned a 401.
 2016-08-08 16:40:39 +01:00
Erik Johnston
93acf49e9b Fix backfill auth events 2016-08-05 12:59:04 +01:00
Richard van der Hoff
f131cd9e53 keys/query: Omit device displayname if null 
... which makes it more consistent with user displaynames.
 2016-08-04 10:59:51 +01:00
Richard van der Hoff
68264d7404 Include device name in /keys/query response 
Add an 'unsigned' section which includes the device display name.
 2016-08-03 15:42:08 +01:00
Richard van der Hoff
91fa69e029 keys/query: return all users which were asked for 
In the situation where all of a user's devices get deleted, we want to
indicate this to a client, so we want to return an empty dictionary, rather
than nothing at all.
 2016-08-03 15:41:44 +01:00
Richard van der Hoff
530bc862dc Merge branch 'rav/null_default_device_displayname' into develop 2016-08-03 14:30:32 +01:00
Richard van der Hoff
a6f5cc65d9 PEP8 2016-08-03 14:30:06 +01:00
Richard van der Hoff
a843868fe9 E2eKeysHandler: minor tweaks 
PR feedback
 2016-08-03 14:24:33 +01:00
Richard van der Hoff
4fec5e57be Default device_display_name to null 
It turns out that it's more useful to return a null device display name (and
let clients decide how to handle it: eg, falling back to device_id) than using
a constant string like "unknown device".
 2016-08-03 11:53:00 +01:00
Mark Haines
921f17f938 Merge branch 'develop' into rav/refactor_device_query 2016-08-03 11:12:47 +01:00
Richard van der Hoff
1efee2f52b E2E keys: Make federation query share code with client query 
Refactor the e2e query handler to separate out the local query, and then make
the federation handler use it.
 2016-08-02 18:12:00 +01:00
Erik Johnston
7b0f6293f2 Merge pull request #940 from matrix-org/erikj/fed_state_cache 
Cache federation state responses
 2016-08-02 15:21:37 +01:00
Richard van der Hoff
986615b0b2 Move e2e query logic into a handler 2016-08-01 18:02:07 +01:00
Paul Evans
18b7eb830b Merge pull request #958 from matrix-org/paul/SYN-738 
Forbid non-ASes from registering users whose names begin with '_'
 2016-07-29 14:10:45 +01:00
Erik Johnston
c51a52f300 Mention that func will fetch auth events 2016-07-29 11:17:04 +01:00
Erik Johnston
3d13c3a295 Update docstring 2016-07-29 10:45:05 +01:00
Erik Johnston
1e2740caab Handle the case of missing auth events when joining a room 2016-07-28 16:08:33 +01:00
Paul "LeoNerd" Evans
05f6447301 Forbid non-ASes from registering users whose names begin with '_' (SYN-738) 2016-07-27 17:54:26 +01:00
Richard van der Hoff
d47115ff8b Delete e2e keys on device delete 2016-07-27 12:24:52 +01:00
Richard van der Hoff
eb359eced4 Add create_requester function 
Wrap the `Requester` constructor with a function which provides sensible
defaults, and use it throughout
 2016-07-26 16:46:53 +01:00
Richard van der Hoff
8e02494166 Delete refresh tokens when deleting devices 2016-07-26 11:10:37 +01:00
Richard van der Hoff
012b4c1913 Implement updating devices 
You can update the displayname of devices now.
 2016-07-26 07:35:48 +01:00
Richard van der Hoff
436bffd15f Implement deleting devices 2016-07-26 07:35:48 +01:00
David Baker
7ed58bb347 Use get to avoid KeyErrors 2016-07-22 17:18:50 +01:00
David Baker
dad2da7e54 Log the hostname the reCAPTCHA was completed on 
This could be useful information to have in the logs. Also comment about how & why we don't verify the hostname.
 2016-07-22 17:00:56 +01:00
Richard van der Hoff
55abbe1850 make /devices return a list 
Turns out I specced this to return a list of devices rather than a dict of them
 2016-07-21 15:57:28 +01:00
Richard van der Hoff
1c3c202b96 Fix PEP8 errors 2016-07-21 13:15:15 +01:00
Richard van der Hoff
406f7aa0f6 Implement GET /device/{deviceId} 2016-07-21 12:00:29 +01:00
Erik Johnston
248e6770ca Cache federation state responses 2016-07-21 10:30:12 +01:00
Richard van der Hoff
bc8f265f0a GET /devices endpoint 
implement a GET /devices endpoint which lists all of the user's devices.

It also returns the last IP where we saw that device, so there is some dancing
to fish that out of the user_ips table.
 2016-07-20 16:42:32 +01:00
Erik Johnston
aba0b2a39b Merge pull request #935 from matrix-org/erikj/backfill_notifs 
Don't notify pusher pool for backfilled events
 2016-07-20 13:39:16 +01:00
Erik Johnston
57dca35692 Don't notify pusher pool for backfilled events 2016-07-20 13:25:06 +01:00
Richard van der Hoff
c68518dfbb Merge pull request #933 from matrix-org/rav/type_annotations 
Type annotations
 2016-07-20 12:26:32 +01:00
David Baker
e967bc86e7 Merge pull request #932 from matrix-org/rav/register_refactor 
Further registration refactoring
 2016-07-20 11:03:33 +01:00
Erik Johnston
1e2a7f18a1 Merge pull request #922 from matrix-org/erikj/file_api2 
Feature: Add filter to /messages. Add 'contains_url' to filter.
 2016-07-20 10:40:48 +01:00
Richard van der Hoff
3413f1e284 Type annotations 
Add some type annotations to help PyCharm (in particular) to figure out the
types of a bunch of things.
 2016-07-19 18:56:16 +01:00
Richard van der Hoff
40cbffb2d2 Further registration refactoring 
* `RegistrationHandler.appservice_register` no longer issues an access token:
  instead it is left for the caller to do it. (There are two of these, one in
  `synapse/rest/client/v1/register.py`, which now simply calls
  `AuthHandler.issue_access_token`, and the other in
  `synapse/rest/client/v2_alpha/register.py`, which is covered below).

* In `synapse/rest/client/v2_alpha/register.py`, move the generation of
  access_tokens into `_create_registration_details`. This means that the normal
  flow no longer needs to call `AuthHandler.issue_access_token`; the
  shared-secret flow can tell `RegistrationHandler.register` not to generate a
  token; and the appservice flow continues to work despite the above change.
 2016-07-19 18:46:19 +01:00
Richard van der Hoff
9a7a77a22a Merge pull request #929 from matrix-org/rav/support_deviceid_in_login 
Add device_id support to /login
 2016-07-19 15:53:04 +01:00
Mark Haines
c41d52a042 Summary line 2016-07-19 10:28:27 +01:00
Mark Haines
7e554aac86 Update docstring on Handlers. 
To indicate it is deprecated.
 2016-07-19 10:20:58 +01:00
Richard van der Hoff
f863a52cea Add device_id support to /login 
Add a 'devices' table to the storage, as well as a 'device_id' column to
refresh_tokens.

Allow the client to pass a device_id, and initial_device_display_name, to
/login. If login is successful, then register the device in the devices table
if it wasn't known already. If no device_id was supplied, make one up.

Associate the device_id with the access token and refresh token, so that we can
get at it again later. Ensure that the device_id is copied from the refresh
token to the access_token when the token is refreshed.
 2016-07-18 16:39:44 +01:00
Richard van der Hoff
dcfd71aa4c Refactor login flow 
Make sure that we have the canonical user_id *before* calling
get_login_tuple_for_user_id.

Replace login_with_password with a method which just validates the password,
and have the caller call get_login_tuple_for_user_id. This brings the password
flow into line with the other flows, and will give us a place to register the
device_id if necessary.
 2016-07-18 15:23:54 +01:00
Richard van der Hoff
6344db659f Fix a doc-comment 
The `store` in a handler is a generic DataStore, not just an events.StateStore.
 2016-07-18 09:48:10 +01:00
Erik Johnston
bd7c51921d Merge pull request #919 from matrix-org/erikj/auth_fix 
Various auth.py fixes.
 2016-07-15 11:38:33 +01:00
Erik Johnston
9e1b43bcbf Comment 2016-07-15 09:29:54 +01:00
Erik Johnston
ebdafd8114 Check sender signed event 2016-07-14 17:03:24 +01:00
Erik Johnston
a98d215204 Add filter param to /messages API 2016-07-14 16:30:56 +01:00
Negar Fazeli
0136a522b1 Bug fix: expire invalid access tokens 2016-07-13 15:00:37 +02:00
Erik Johnston
a17e7caeb7 Merge branch 'erikj/shared_secret' into erikj/test2 2016-07-06 14:46:31 +01:00
Erik Johnston
651faee698 Add an admin option to shared secret registration 2016-07-05 17:30:22 +01:00
Erik Johnston
8f8798bc0d Add ReadWriteLock for pagination and history prune 2016-07-05 15:30:25 +01:00
David Baker
ef535178ff Merge pull request #904 from matrix-org/dbkr/register_email_no_untrusted_id_server 
requestToken update
 2016-07-05 15:13:34 +01:00
Kent Shikama
14362bf359
Fix password config 2016-07-05 19:12:53 +09:00
Kent Shikama
1ee2584307
Fix pep8 2016-07-05 19:01:00 +09:00
Erik Johnston
2d21d43c34 Add purge_history API 2016-07-05 10:28:51 +01:00
Kent Shikama
8bdaf5f7af
Add pepper to password hashing 
Signed-off-by: Kent Shikama <kent@kentshikama.com>
 2016-07-05 02:13:52 +09:00
Kent Shikama
bb069079bb
Fix style violations 
Signed-off-by: Kent Shikama <kent@kentshikama.com>
 2016-07-04 22:07:11 +09:00
Kent Shikama
fc8007dbec
Optionally include password hash in createUser endpoint 
Signed-off-by: Kent Shikama <kent@kentshikama.com>
 2016-07-03 15:08:15 +09:00
David Baker
be8be535f7 requestToken update 
Don't send requestToken request to untrusted ID servers

Also correct the THREEPID_IN_USE error to add the M_ prefix. This is a backwards incomaptible change, but the only thing using this is the angular client which is now unmaintained, so it's probably better to just do this now.
 2016-06-30 17:51:28 +01:00
Martin Weinelt
0a32208e5d Rework ldap integration with ldap3 
Use the pure-python ldap3 library, which eliminates the need for a
system dependency.

Offer both a `search` and `simple_bind` mode, for more sophisticated
ldap scenarios.
- `search` tries to find a matching DN within the `user_base` while
  employing the `user_filter`, then tries the bind when a single
  matching DN was found.
- `simple_bind` tries the bind against a specific DN by combining the
  localpart and `user_base`

Offer support for STARTTLS on a plain connection.

The configuration was changed to reflect these new possibilities.

Signed-off-by: Martin Weinelt <hexa@darmstadt.ccc.de>
 2016-06-22 17:51:59 +02:00
Mark Haines
0c13d45522 Add a comment on why we don't create a profile for upgrading users 2016-06-17 19:18:53 +01:00
Mark Haines
9f1800fba8 Remove registered_users from the distributor. 
The only place that was observed was to set the profile. I've made it
so that the profile is set within store.register in the same transaction
that creates the user.

This required some slight changes to the registration code for upgrading
guest users, since it previously relied on the distributor swallowing errors
if the profile already existed.
 2016-06-17 19:14:16 +01:00
Erik Johnston
2884712ca7 Only re-sign our own events 2016-06-17 14:47:33 +01:00
Salvatore LaMendola
ed5f43a55a
Fix TypeError in call to bcrypt.hashpw 
- At the very least, this TypeError caused logins to fail on my own
  running instance of Synapse, and the simple (explicit) UTF-8
  conversion resolved login errors for me.

Signed-off-by: Salvatore LaMendola <salvatore.lamendola@gmail.com>
 2016-06-16 00:43:42 -04:00
Erik Johnston
b31c49d676 Correctly mark backfilled events as backfilled 2016-06-15 10:59:08 +01:00
Negar Fazeli
6e7dc7c7dd Fix a bug caused by a change in auth_handler function 
Fix the relevant unit test cases
 2016-06-08 23:22:39 +02:00
Erik Johnston
81c07a32fd Pull full state for each room all at once 2016-06-08 15:51:49 +01:00
Erik Johnston
1a815fb04f Don't hit DB for noop replications queries 2016-06-08 11:33:30 +01:00
Mark Haines
0b2158719c Remove dead code. 
Loading push rules now happens in the datastore, so we can remove
the methods that loaded them outside the datastore.

The ``waiting_for_join_list`` in federation handler is populated by
anything, so can be removed.

The ``_get_members_events_txn`` method isn't called from anywhere
so can be removed.
 2016-06-07 15:07:11 +01:00
Erik Johnston
216a05b3e3 .values() returns list of sets 2016-06-06 16:00:09 +01:00
Erik Johnston
96dc600579 Fix typos 2016-06-06 15:44:41 +01:00
Erik Johnston
377eb480ca Fire after 30s not 8h 2016-06-06 15:14:21 +01:00
Erik Johnston
51773bcbaf Merge pull request #832 from matrix-org/erikj/presence_coount 
Change def of small delta in presence stream. Add metrics.
 2016-06-03 14:57:00 +01:00
Erik Johnston
ab116bdb0c Fix typo 2016-06-03 14:03:42 +01:00
Erik Johnston
4ce84a1acd Change metric style 2016-06-03 13:49:16 +01:00
Erik Johnston
a7ff5a1770 Presence metrics. Change def of small delta 2016-06-03 13:40:55 +01:00
Erik Johnston
4982b28868 Merge pull request #829 from matrix-org/erikj/poke_notifier 
Poke notifier on next reactor tick
 2016-06-03 11:52:10 +01:00
Erik Johnston
4c04222fa5 Poke notifier on next reactor tick 2016-06-03 11:24:16 +01:00
Erik Johnston
6a0afa582a Load push rules in storage layer, so that they get cached 2016-06-03 11:10:00 +01:00
David Baker
c71177f285 Merge remote-tracking branch 'origin/dbkr/email_notif_string_fmt_error' into dbkr/email_unsubscribe 2016-06-02 17:20:56 +01:00
Mark Haines
56d15a0530 Store the typing users as user_id strings. (#819) 
Rather than storing them as UserID objects.
 2016-06-02 16:28:54 +01:00
David Baker
812b5de0fe Merge remote-tracking branch 'origin/develop' into dbkr/email_unsubscribe 2016-06-02 15:33:28 +01:00
Mark Haines
661a540dd1 Deduplicate presence entries in sync (#818) 2016-06-02 15:20:28 +01:00
Mark Haines
70599ce925 Allow external processes to mark a user as syncing. (#812) 
* Add infrastructure to the presence handler to track sync requests in external processes

* Expire stale entries for dead external processes

* Add an http endpoint for making users as syncing

Add some docstrings and comments.

* Fixes
 2016-06-02 15:20:15 +01:00
David Baker
fb2193cc63 Merge pull request #817 from matrix-org/dbkr/split_out_auth_handler 
Split out the auth handler
 2016-06-02 14:31:35 +01:00
Erik Johnston
02ac463dbf Merge pull request #800 from matrix-org/erikj/sync_refactor 
Refactor SyncHandler
 2016-06-02 14:02:13 +01:00
David Baker
3a3fb2f6f9 Merge branch 'dbkr/split_out_auth_handler' into dbkr/email_unsubscribe 2016-06-02 13:35:25 +01:00