Matthew Hodgson
3241c7aac3
untested WIP but might actually work
2017-11-29 18:27:05 +00:00
Matthew Hodgson
47d99a20d5
Add user_directory_include_pattern config param to expand search results to additional users
...
Initial commit; this doesn't work yet - the LIKE filtering seems too aggressive.
It also needs _do_initial_spam to be aware of prepopulating the whole user_directory_search table with all users...
...and it needs a handle_user_signup() or something to be added so that new signups get incrementally added to the table too.
Committing it here as a WIP
2017-11-29 16:46:45 +00:00
Richard van der Hoff
ad7e570d07
Delete devices in various logout situations
...
Make sure that we delete devices whenever a user is logged out due to any of
the following situations:
* /logout
* /logout_all
* change password
* deactivate account (by the user or by an admin)
* invalidate access token from a dynamic module
Fixes #2672 .
2017-11-29 16:44:35 +00:00
Richard van der Hoff
ae31f8ce45
Move set_password into its own handler
...
Non-functional refactoring to move set_password. This means that we'll be able
to properly deactivate devices and access tokens without introducing a
dependency loop.
2017-11-29 16:44:35 +00:00
Richard van der Hoff
7ca5c68233
Move deactivate_account into its own handler
...
Non-functional refactoring to move deactivate_account. This means that we'll be
able to properly deactivate devices and access tokens without introducing a
dependency loop.
2017-11-29 16:44:35 +00:00
Richard van der Hoff
2c6d63922a
Remove pushers when deleting access tokens
...
Whenever an access token is invalidated, we should remove the associated
pushers.
2017-11-29 16:44:35 +00:00
Erik Johnston
dfbc45302e
PEP8
2017-11-28 15:23:26 +00:00
Erik Johnston
c4c1d170af
Fix wrong avatars when inviting multiple users when creating room
...
We reused the `content` dictionary between invite requests, which meant they could end up reusing the profile info for a previous user
2017-11-28 15:19:15 +00:00
Luke Barnard
cdf6fc15b0
Merge pull request #2686 from matrix-org/luke/as-flair
...
Add automagical AS Publicised Group(s)
2017-11-17 10:13:46 +00:00
Jurek
eded7084d2
Fix auth handler #2678
2017-11-17 10:07:27 +00:00
Matthew Hodgson
34c3d0a386
typo
2017-11-17 01:54:02 +00:00
Luke Barnard
5b48eec4a1
Make sure we check AS groups for lookup on bulk
2017-11-16 17:55:15 +00:00
Jurek
624a8bbd67
Fix auth handler #2678
2017-11-16 17:19:02 +00:00
Luke Barnard
97bd18af4e
Add automagical AS Publicised Group(s)
...
via registration file "users" namespace:
```YAML
...
namespaces:
users:
- exclusive: true
regex: '.*luke.*'
group_id: '+all_the_lukes:hsdomain'
...
```
This is part of giving App Services their own groups for matching users. With this, ghost users will be given the appeareance that they are in a group and that they have publicised the fact, but _only_ from the perspective of the `get_publicised_groups_for_user` API.
2017-11-16 16:44:55 +00:00
Richard van der Hoff
1b83c09c03
Merge pull request #2675 from matrix-org/rav/remove_broken_logcontext_funcs
...
Remove preserve_context_over_{fn, deferred}
2017-11-15 11:13:53 +00:00
Richard van der Hoff
7e6fa29cb5
Remove preserve_context_over_{fn, deferred}
...
Both of these functions ae known to leak logcontexts. Replace the remaining
calls to them and kill them off.
2017-11-14 11:22:42 +00:00
Richard van der Hoff
44a1bfd6a6
Reshuffle room list request code
...
I'm not entirely sure if this will actually help anything, but it simplifies
the code and might give further clues about why room list search requests are
blowing out the get_current_state_ids caches.
2017-11-14 10:29:58 +00:00
Richard van der Hoff
1fc66c7460
Add a load of logging to the room_list handler
...
So we can see what it gets up to.
2017-11-14 10:23:47 +00:00
Richard van der Hoff
9b803ccc98
Revert "Allow upper-case characters in mxids"
...
This reverts commit b70b646903
2017-11-09 21:57:24 +00:00
Richard van der Hoff
b70b646903
Allow upper-case characters in mxids
...
Because we're never going to be able to fix this :'(
2017-11-09 19:36:13 +00:00
Erik Johnston
e8814410ef
Have an explicit API to update room config
2017-11-08 16:13:27 +00:00
Erik Johnston
94ff2cda73
Revert "Modify group room association API to allow modification of is_public"
2017-11-08 15:43:34 +00:00
Erik Johnston
d305987b40
Merge pull request #2631 from xyzz/fix_appservice_event_backlog
...
Fix appservices being backlogged and not receiving new events due to a bug in notify_interested_services
2017-11-08 11:54:10 +00:00
Richard van der Hoff
02a9a93bde
Merge pull request #2649 from matrix-org/rav/fix_delta_on_state_res
...
Fix bug in state group storage
2017-11-08 09:22:13 +00:00
Richard van der Hoff
e148438e97
s/items/iteritems/
2017-11-08 09:21:41 +00:00
Ilya Zhuravlev
d46386d57e
Remove useless assignment in notify_interested_services
2017-11-07 22:23:22 +03:00
Matthew Hodgson
228ccf1fe3
Merge pull request #2643 from matrix-org/matthew/user_dir_typos
...
Fix various embarrassing typos around user_directory and add some doc.
2017-11-07 17:31:11 +00:00
Richard van der Hoff
780dbb378f
Update deltas when doing auth resolution
...
Fixes a bug where the persisted state groups were different to those actually
being used after auth resolution.
2017-11-07 16:43:00 +00:00
Richard van der Hoff
1ca4288135
factor out _update_context_for_auth_events
...
This is duplicated, so let's factor it out before fixing it
2017-11-07 16:43:00 +00:00
Erik Johnston
5ef5e14ecc
Merge pull request #2636 from farialima/me-master
...
Fix for #2635 : correctly update rooms avatar/display name when modified by admin
2017-11-07 13:49:27 +00:00
Matthew Hodgson
d1622e080f
s/intial/initial/
2017-11-04 19:35:14 +00:00
Francois Granade
f103b91ffa
removed unused import flagged by flake8a
2017-11-03 18:45:49 +01:00
Francois Granade
fa4f337b49
Fix for issue 2635: correctly update rooms avatar/display name when modified by admin
2017-11-03 18:25:04 +01:00
Ilya Zhuravlev
45fbe4ff67
Fix appservices being backlogged and not receiving new events due to a bug in notify_interested_services
2017-11-02 22:49:43 +03:00
Richard van der Hoff
1189be43a2
Factor _AccountHandler proxy out to ModuleApi
...
We're going to need to use this from places that aren't password auth, so let's
move it to a proper class.
2017-11-02 14:36:11 +00:00
David Baker
b19d9e2174
Merge pull request #2624 from matrix-org/rav/password_provider_notify_logout
...
Notify auth providers on logout
2017-11-02 10:55:17 +00:00
David Baker
1f080a6c97
Merge pull request #2623 from matrix-org/rav/callbacks_for_auth_providers
...
Allow password_auth_providers to return a callback
2017-11-02 10:49:03 +00:00
David Baker
04897c9dc1
Merge pull request #2622 from matrix-org/rav/db_access_for_auth_providers
...
Let auth providers get to the database
2017-11-02 10:41:25 +00:00
Richard van der Hoff
979eed4362
Fix user-interactive password auth
...
this got broken in the previous commit
2017-11-01 17:03:20 +00:00
Richard van der Hoff
bc8a5c0330
Notify auth providers on logout
...
Provide a hook by which auth providers can be notified of logouts.
2017-11-01 16:51:51 +00:00
Richard van der Hoff
4c8f94ac94
Allow password_auth_providers to return a callback
...
... so that they have a way to record access tokens.
2017-11-01 16:51:03 +00:00
Richard van der Hoff
846a94fbc9
Merge pull request #2620 from matrix-org/rav/auth_non_password
...
Let password auth providers handle arbitrary login types
2017-11-01 16:45:33 +00:00
Richard van der Hoff
3cd6b22c7b
Let password auth providers handle arbitrary login types
...
Provide a hook where password auth providers can say they know about other
login types, and get passed the relevant parameters
2017-11-01 16:43:57 +00:00
David Baker
4f0488b307
Merge remote-tracking branch 'origin/develop' into rav/refactor_accesstoken_delete
2017-11-01 16:20:19 +00:00
David Baker
e5e930aec3
Merge pull request #2615 from matrix-org/rav/break_auth_device_dep
...
Break dependency of auth_handler on device_handler
2017-11-01 16:06:31 +00:00
David Baker
fbbacb284e
Merge pull request #2613 from matrix-org/rav/kill_refresh_tokens
...
Remove the last vestiges of refresh_tokens
2017-11-01 15:57:35 +00:00
Richard van der Hoff
dd13310fb8
Move access token deletion into auth handler
...
Also move duplicated deactivation code into the auth handler.
I want to add some hooks when we deactivate an access token, so let's bring it
all in here so that there's somewhere to put it.
2017-11-01 15:46:22 +00:00
Richard van der Hoff
74c56f794c
Break dependency of auth_handler on device_handler
...
I'm going to need to make the device_handler depend on the auth_handler, so I
need to break this dependency to avoid a cycle.
It turns out that the auth_handler was only using the device_handler in one
place which was an edge case which we can more elegantly handle by throwing an
error rather than fixing it up.
2017-11-01 10:27:06 +00:00
Richard van der Hoff
356bcafc44
Remove the last vestiges of refresh_tokens
2017-10-31 20:35:58 +00:00
Richard van der Hoff
3e0aaad190
Let auth providers get to the database
...
Somewhat open to abuse, but also somewhat unavoidable :/
2017-10-31 17:22:29 +00:00
Luke Barnard
20fe347906
Modify group room association API to allow modification of is_public
...
also includes renamings to make things more consistent.
2017-10-31 17:04:28 +00:00
Richard van der Hoff
1b65ae00ac
Refactor some logic from LoginRestServlet into AuthHandler
...
I'm going to need some more flexibility in handling login types in password
auth providers, so as a first step, move some stuff from LoginRestServlet into
AuthHandler.
In particular, we pass everything other than SAML, JWT and token logins down to
the AuthHandler, which now has responsibility for checking the login type and
fishing the password out of the login dictionary, as well as qualifying the
user_id if need be. Ideally SAML, JWT and token would go that way too, but
there's no real need for it right now and I'm trying to minimise impact.
This commit *should* be non-functional.
2017-10-31 10:48:41 +00:00
Richard van der Hoff
110b373e9c
Merge pull request #2589 from matrix-org/rav/as_deactivate_account
...
Allow ASes to deactivate their own users
2017-10-27 12:29:32 +01:00
Richard van der Hoff
785bd7fd75
Allow ASes to deactivate their own users
2017-10-27 00:01:00 +01:00
Erik Johnston
566e21eac8
Update room_list.py
2017-10-26 11:39:54 +01:00
Erik Johnston
2a7e9faeec
Do logcontexts outside ResponseCache
2017-10-25 15:21:08 +01:00
Erik Johnston
33a9026cdf
Add logging and fix log contexts for publicRooms
2017-10-25 10:26:06 +01:00
Richard van der Hoff
eaaabc6c4f
replace 'except:' with 'except Exception:'
...
what could possibly go wrong
2017-10-23 15:52:32 +01:00
Richard van der Hoff
3267b81b81
Merge pull request #2561 from matrix-org/rav/id_checking
...
Updates to ID checking
2017-10-23 14:39:20 +01:00
Richard van der Hoff
29812c628b
Allow = in mxids and groupids
...
... because the spec says we should.
2017-10-20 23:42:53 +01:00
Richard van der Hoff
58fbbe0f1d
Disallow capital letters in userids
...
Factor out a common function for checking user ids and group ids, which forbids
capitals.
2017-10-20 23:37:22 +01:00
Richard van der Hoff
631d7b87b5
Remove pointless create() method
...
It just calls the constructor, so we may as well kill it rather than having
random codepaths.
2017-10-20 22:14:55 +01:00
Luke Barnard
a3ac4f6b0a
_create_rererouter for get_invited_users_in_group
2017-10-16 15:41:03 +01:00
Luke Barnard
2c5972f87f
Implement GET /groups/$groupId/invited_users
2017-10-16 15:31:11 +01:00
Luke Barnard
6079d0027a
Log a warning when no profile for invited member
...
And return empty profile
2017-10-16 14:20:45 +01:00
David Baker
b8d8ed1ba9
Merge pull request #2531 from matrix-org/dbkr/spamcheck_error_messages
...
Allow error strings from spam checker
2017-10-12 10:31:03 +01:00
Erik Johnston
b752507b48
Fix fetching remote summaries
2017-10-11 16:59:18 +01:00
Richard van der Hoff
68fd82e840
Merge pull request #2530 from matrix-org/rav/fix_receipt_logcontext
...
fix a logcontext leak in read receipt handling
2017-10-11 15:08:53 +01:00
David Baker
b78bae2d51
fix isinstance
2017-10-11 14:49:09 +01:00
David Baker
c3b7a45e84
Allow error strings from spam checker
2017-10-11 14:39:22 +01:00
Richard van der Hoff
c3e190ce67
fix a logcontext leak in read receipt handling
2017-10-11 14:37:20 +01:00
Erik Johnston
4ce4379235
Fix attestations to check correct server name
2017-10-11 14:11:43 +01:00
Erik Johnston
535cc49f27
Merge pull request #2466 from matrix-org/erikj/groups_merged
...
Initial Group Implementation
2017-10-11 13:20:07 +01:00
Erik Johnston
bc7f3eb32f
Merge pull request #2483 from jeremycline/unfreeze-ujson-dump
...
Unfreeze event before serializing with ujson
2017-10-11 13:18:52 +01:00
Erik Johnston
84e27a592d
Merge pull request #2490 from matrix-org/erikj/drop_left_room_events
...
Ignore incoming events for rooms that we have left
2017-10-10 11:58:32 +01:00
Richard van der Hoff
c8f568ddf9
Fix up deferred handling in federation.py
...
* Avoid preserve_context_over_deferred, which is broken
* set consumeErrors=True on defer.gatherResults, to avoid spurious "unhandled
failure" erros
2017-10-06 22:14:24 +01:00
David Baker
f878e6f8af
Spam checking: add the invitee to user_may_invite
2017-10-05 14:02:28 +01:00
David Baker
d8ce68b09b
spam check room publishing
2017-10-04 14:29:33 +01:00
David Baker
78d4ced829
un-double indent
2017-10-04 12:44:27 +01:00
David Baker
197c14dbcf
Add room creation checks to spam checker
...
Lets the spam checker deny attempts to create rooms and add aliases
to them.
2017-10-04 10:47:54 +01:00
David Baker
1e375468de
pass room id too
2017-10-03 17:13:14 +01:00
David Baker
c2c188b699
Federation was passing strings anyway
...
so pass string everywhere
2017-10-03 15:46:19 +01:00
David Baker
c46a0d7eb4
this shouldn't be debug
2017-10-03 15:20:14 +01:00
David Baker
bd769a81e1
better logging
2017-10-03 15:16:40 +01:00
David Baker
41fd9989a2
Skip spam check for admin users
2017-10-03 14:17:44 +01:00
Erik Johnston
e4ab96021e
Update comments
2017-10-03 14:10:41 +01:00
David Baker
2a7ed700d5
Fix param name & lint
2017-10-03 14:04:10 +01:00
David Baker
84716d267c
Allow spam checker to reject invites too
2017-10-03 13:56:43 +01:00
Erik Johnston
f2da6df568
Remove spurious line feed
2017-10-03 11:31:06 +01:00
Erik Johnston
30848c0fcd
Ignore incoming events for rooms that we have left
...
When synapse receives an event for a room its not in over federation, it
double checks with the remote server to see if it is in fact in the
room. This is done so that if the server has forgotten about the room
(usually as a result of the database being dropped) it can recover from
it.
However, in the presence of state resets in large rooms, this can cause
a lot of work for servers that have legitimately left. As a hacky
solution that supports both cases we drop incoming events for rooms that
we have explicitly left.
This means that we no longer support the case of servers having
forgotten that they've rejoined a room, but that is sufficiently rare
that we're not going to support it for now.
2017-10-03 11:18:21 +01:00
David Baker
27955056e0
Merge branch 'develop' into erikj/groups_merged
2017-10-02 16:20:41 +01:00
Jeremy Cline
cafb8de132
Unfreeze event before serializing with ujson
...
In newer versions of https://github.com/esnme/ultrajson , ujson does not
serialize frozendicts (introduced in esnme/ultrajson@53f85b1 ). Although
the PyPI version is still 1.35, Fedora ships with a build from commit
esnme/ultrajson@2f1d487 . This causes the serialization to fail if the
distribution-provided package is used.
This runs the event through the unfreeze utility before serializing it.
Thanks to @ignatenkobrain for tracking down the root cause.
fixes #2351
Signed-off-by: Jeremy Cline <jeremy@jcline.org>
2017-09-30 11:22:37 -04:00
Erik Johnston
8090fd4664
Fix /joined_members to work with AS users
2017-09-28 10:09:32 +01:00
Erik Johnston
adec03395d
Fix bug where /joined_members didn't check user was in room
2017-09-27 15:14:39 +01:00
David Baker
6cd5fcd536
Make the spam checker a module
2017-09-26 19:20:23 +01:00
Erik Johnston
17b8e2bd02
Add remove room API
2017-09-26 15:52:41 +01:00
Erik Johnston
95298783bb
Add is_publicised to group summary
2017-09-26 11:04:37 +01:00
Erik Johnston
e3edca3b5d
Refactor to speed up incremental syncs
2017-09-25 17:35:39 +01:00
Erik Johnston
3166ed55b2
Fix device list when rejoining room ( #2461 )
2017-09-22 14:44:17 +01:00
Richard van der Hoff
aa620d09a0
Add a config option to block all room invites ( #2457 )
...
- allows sysadmins the ability to lock down their servers so that people can't
send their users room invites.
2017-09-19 16:08:14 +01:00
Richard van der Hoff
2eabdf3f98
add some comments to on_exchange_third_party_invite_request
2017-09-19 12:20:36 +01:00
Richard van der Hoff
5ed109d59f
PoC for filtering spammy events ( #2456 )
...
Demonstration of how you might add some hooks to filter out spammy events.
2017-09-19 12:20:11 +01:00
Erik Johnston
93e504d04e
Ensure that creator of group sees group down /sync
2017-09-19 11:08:16 +01:00
Erik Johnston
b5feaa5a49
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/groups_merged
2017-09-19 11:07:45 +01:00
Richard van der Hoff
3f405b34e9
Fix overzealous kicking of guest users ( #2453 )
...
We should only kick guest users if the guest access event is authorised.
2017-09-19 08:52:52 +01:00
Erik Johnston
2d1b7955ae
Don't filter out current state events from timeline
2017-09-18 17:13:03 +01:00
Erik Johnston
d6dadd95ac
Correctly handle leaving room in /key/changes
2017-09-18 15:38:22 +01:00
Erik Johnston
4a94eb3ea4
Fix typo
2017-09-15 09:56:54 +01:00
Erik Johnston
3a0cee28d6
Actually hook leave notifs up
2017-09-14 11:49:37 +01:00
Erik Johnston
4f845a0713
Handle joining/leaving rooms in /keys/changes
2017-09-13 16:28:08 +01:00
Erik Johnston
473700f016
Get left rooms
2017-09-13 15:13:41 +01:00
Erik Johnston
9ce866ed4f
In sync handle device lists for newly joined/left rooms
2017-09-12 16:44:26 +01:00
Erik Johnston
69ef4987a6
Add left section to /keys/changes
2017-09-08 14:44:36 +01:00
Erik Johnston
53cc8ad35a
Send down device list change notif when member leaves/rejoins room
2017-09-07 15:08:39 +01:00
Erik Johnston
4a9b1cf253
Add user profiles to summary from group server
2017-08-25 16:23:58 +01:00
Erik Johnston
258409ef61
Fix typos and reinherit
2017-08-25 14:45:20 +01:00
Erik Johnston
bf81f3cf2c
Split out profile handler to fix tests
2017-08-25 14:34:56 +01:00
Erik Johnston
27ebc5c8f2
Add remote profile cache
2017-08-25 11:25:47 +01:00
Erik Johnston
ba3ff7918b
Fixup
2017-08-11 13:42:42 +01:00
Erik Johnston
ef8e578677
Add bulk group publicised lookup API
2017-08-09 13:36:22 +01:00
Erik Johnston
05e21285aa
Store whether the user wants to publicise their membership of a group
2017-08-08 13:01:46 +01:00
Erik Johnston
a1e67bcb97
Remove stale TODO comments
2017-08-04 10:07:10 +01:00
Erik Johnston
d5e32c843f
Correctly add joins to correct segment
2017-07-24 13:31:26 +01:00
Erik Johnston
96917d5552
Merge pull request #2378 from matrix-org/erikj/group_sync_support
...
Add groups to sync stream
2017-07-21 11:05:39 +01:00
Erik Johnston
960dae3340
Add notifier
2017-07-20 17:14:44 +01:00
Erik Johnston
2cc998fed8
Fix replication. And notify
2017-07-20 17:13:18 +01:00
Erik Johnston
139fe30f47
Remember to cast to bool
2017-07-20 16:47:35 +01:00
Erik Johnston
c544188ee3
Add groups to sync stream
2017-07-20 16:36:42 +01:00
Erik Johnston
6f443a74cf
Add update group profile API
2017-07-20 09:46:33 +01:00
Erik Johnston
14a34f12d7
Comments
2017-07-18 17:28:42 +01:00
Erik Johnston
94ecd871a0
Fix typos
2017-07-18 16:38:54 +01:00
Erik Johnston
332839f6ea
Update federation client pokes
2017-07-18 14:45:37 +01:00
Erik Johnston
e5ea6dd021
Add client apis
2017-07-18 14:37:06 +01:00
Erik Johnston
cccfcfa7b9
Comments
2017-07-18 10:35:18 +01:00
Erik Johnston
68f34e85ce
Use transport client directly
2017-07-18 10:29:57 +01:00
Erik Johnston
2f9eafdd36
Add local group server support
2017-07-17 12:03:49 +01:00
Erik Johnston
e9aec001f4
Use less DB for device list handling in sync
2017-07-12 10:30:10 +01:00
Erik Johnston
28e8c46f29
Merge pull request #2352 from matrix-org/erikj/group_server_split
...
Initial Group Server
2017-07-12 10:26:29 +01:00
Erik Johnston
9a6fd3ef29
Don't compute push actions for backfilled events
2017-07-11 10:02:21 +01:00
Erik Johnston
b8ca494ee9
Initial group server implementation
2017-07-10 15:44:15 +01:00
Erik Johnston
5a7f561a9b
Fix bug where pusherpool didn't start and broke some rooms
...
Since we didn't instansiate the PusherPool at start time it could fail
at run time, which it did for some users.
This may or may not fix things for those users, but it should happen at
start time and stop the server from starting.
2017-07-06 17:55:51 +01:00
Mark Haines
3e279411fe
Improve the error handling for bad invites received over federation
2017-06-30 16:20:30 +01:00
Erik Johnston
c72058bcc6
Use an ExpiringCache for storing registration sessions
...
This is because pruning them was a significant performance drain on
matrix.org
2017-06-29 14:08:37 +01:00
Erik Johnston
816605a137
Merge pull request #2307 from matrix-org/erikj/user_ip_batch
...
Batch upsert user ips
2017-06-27 15:08:32 +01:00
Erik Johnston
a0a561ae85
Fix up client ips to read from pending data
2017-06-27 14:46:12 +01:00
Erik Johnston
1bce3e6b35
Remove unused variables
2017-06-26 14:03:27 +01:00
Erik Johnston
e3cbec10c1
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/ensure_round_trip
2017-06-26 14:02:44 +01:00
Erik Johnston
44c722931b
Make some more params configurable
2017-06-22 14:59:52 +01:00
Erik Johnston
2d520a9826
Typo. ARGH.
2017-06-22 14:42:39 +01:00
Erik Johnston
24d894e2e2
Fix thinko in unhandled user spam
2017-06-22 14:39:05 +01:00
Erik Johnston
1217c7da91
Don't work out users who share room with appservice users
2017-06-21 12:00:41 +01:00
Erik Johnston
e5ae386ea4
Handle all cases of sending membership events
2017-06-19 16:07:54 +01:00
Erik Johnston
b490299a3b
Change to create new room and join other users
2017-06-19 14:10:13 +01:00
Erik Johnston
d7fe6b356c
Add shutdown room API
2017-06-19 12:37:27 +01:00
Erik Johnston
fcf01dd88e
Reject local events that don't round trip the DB
2017-06-19 11:33:40 +01:00
Erik Johnston
6aa5bc8635
Initial worker impl
2017-06-16 11:47:11 +01:00
Erik Johnston
a9d6fa8b2b
Include users who share room with requester in user directory
2017-06-15 10:17:21 +01:00
Erik Johnston
4564b05483
Implement updating users who share rooms on the fly
2017-06-15 10:17:17 +01:00
Erik Johnston
72613bc379
Implement initial population of users who share rooms table
2017-06-15 09:59:04 +01:00
Erik Johnston
d9fd937e39
Fix user directory insertion due to missing room_id
2017-06-13 11:50:24 +01:00
Erik Johnston
6ae8373d40
Don't assume existance of events when updating user directory
2017-06-13 10:19:26 +01:00
Erik Johnston
a837765e8c
Merge pull request #2266 from matrix-org/erikj/host_in_room
...
Change is_host_joined to use current_state table
2017-06-12 09:49:51 +01:00
Erik Johnston
1664948e41
Comment
2017-06-09 13:05:05 +01:00
Erik Johnston
0185b75381
Change is_host_joined to use current_state table
...
This bypasses a bug where using the state groups to figure out if a host
is in a room sometimes errors if the servers isn't in the room. (For
example when the server rejected an invite to a remote room)
2017-06-09 10:52:26 +01:00
Erik Johnston
7132e5cdff
Mark remote invite rejections as outliers
2017-06-09 10:08:18 +01:00
Erik Johnston
f45f07ab86
Merge pull request #2258 from matrix-org/erikj/user_dir
...
Don't start user_directory handling on workers
2017-06-07 14:04:50 +01:00
Erik Johnston
a053ff3979
Merge pull request #2248 from matrix-org/erikj/state_fixup
...
Faster cache for get_joined_hosts
2017-06-07 14:01:06 +01:00
Erik Johnston
ecdd2a3658
Don't start user_directory handling on workers
2017-06-07 12:02:53 +01:00
Erik Johnston
2f34ad31ac
Add some logging to user directory
2017-06-07 11:50:44 +01:00
Erik Johnston
09e4bc0501
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/state_fixup
2017-06-07 11:05:23 +01:00
Erik Johnston
1a01af079e
Handle profile updates in user directory
2017-06-01 15:39:51 +01:00
Erik Johnston
21e255a8f1
Split the table in two
2017-06-01 14:50:46 +01:00
Erik Johnston
7233341eac
Comments
2017-06-01 13:11:38 +01:00
Erik Johnston
8be6fd95a3
Check if host is still in room
2017-06-01 13:05:39 +01:00
Erik Johnston
9c7db2491b
Fix removing users
2017-06-01 11:36:50 +01:00
Erik Johnston
0fe6f3c521
Bug fixes and logging
...
- Check if room is public when a user joins before adding to user dir
- Fix typo of field name "content.join_rules" -> "content.join_rule"
2017-06-01 11:09:49 +01:00
Erik Johnston
cc7609aa9f
Comment briefly on how we keep user_directory up to date
2017-05-31 17:11:18 +01:00
Erik Johnston
b2d8d07109
Lifts things into separate function
2017-05-31 17:00:24 +01:00
Erik Johnston
f9791498ae
Typos
2017-05-31 16:50:57 +01:00
Erik Johnston
f091061711
Fix tests
2017-05-31 16:34:40 +01:00
Erik Johnston
dc51af3d03
Pull max id from correct table
2017-05-31 15:13:49 +01:00
Erik Johnston
350622a107
Handle the server leaving a public room
2017-05-31 15:11:36 +01:00
Erik Johnston
63fda37e20
Add comments
2017-05-31 15:00:29 +01:00
Erik Johnston
3b5f22ca40
Add search
2017-05-31 14:00:01 +01:00
Erik Johnston
b5db4ed5f6
Update room column when room becomes unpublic
2017-05-31 13:40:28 +01:00
Erik Johnston
168524543f
Add call later
2017-05-31 11:59:36 +01:00
Erik Johnston
3e123b8497
Start later
2017-05-31 11:56:27 +01:00
Erik Johnston
42137efde7
Don't go round in circles
2017-05-31 11:55:13 +01:00
Erik Johnston
eeb2f9e546
Add user_directory to database
2017-05-31 11:51:01 +01:00
Erik Johnston
23da638360
Fix typing tests
2017-05-26 10:02:04 +01:00
Erik Johnston
11f139a647
Merge pull request #2247 from matrix-org/erikj/auth_event
...
Only store event_auth for state events
2017-05-24 16:46:34 +01:00
Erik Johnston
c049472b8a
Only store event_auth for state events
2017-05-24 15:23:31 +01:00
Erik Johnston
b4f59c7e27
Add count of one time keys to sync stream
2017-05-19 15:47:55 +01:00
Erik Johnston
cafe659f72
Store ActionGenerator in HomeServer
2017-05-19 10:09:56 +01:00
Erik Johnston
66d8ffabbd
Faster push rule calculation via push specific cache
...
We add a push rule specific cache that ensures that we can reuse
calculated push rules appropriately when a user join/leaves.
2017-05-17 16:55:40 +01:00
Erik Johnston
6fa8148ccb
Merge pull request #2228 from matrix-org/erikj/speed_up_get_hosts
...
Speed up get_joined_hosts
2017-05-16 17:40:55 +01:00
Erik Johnston
13f540ef1b
Speed up get_joined_hosts
2017-05-16 16:05:22 +01:00
Erik Johnston
ec5c4499f4
Make presence use cached users/hosts in room
2017-05-16 16:01:43 +01:00
Luke Barnard
ccad2ed824
Modify condition on empty localpart
2017-05-10 17:34:30 +01:00
Luke Barnard
369195caa5
Modify register/available to be GET with query param
...
- GET is now the method for register/available
- a query parameter "username" is now used
Also, empty usernames are now handled with an error message on registration or via register/available: `User ID cannot be empty`
2017-05-10 17:23:55 +01:00
Erik Johnston
a3648f84b2
Merge pull request #2208 from matrix-org/erikj/ratelimit_overrid
...
Add per user ratelimiting overrides
2017-05-10 15:54:48 +01:00
Erik Johnston
b990b2fce5
Add per user ratelimiting overrides
2017-05-10 11:05:43 +01:00
Richard van der Hoff
aedaba018f
Replace some instances of preserve_context_over_deferred
2017-05-09 19:04:56 +01:00
Richard van der Hoff
de042b3b88
Do some logging when one-time-keys get claimed
...
might help us figure out if https://github.com/vector-im/riot-web/issues/3868
has happened.
2017-05-09 19:04:56 +01:00
Richard van der Hoff
a7e9d8762d
Allow clients to upload one-time-keys with new sigs
...
When a client retries a key upload, don't give an error if the signature has
changed (but the key is the same).
Fixes https://github.com/vector-im/riot-android/issues/1208 , hopefully.
2017-05-09 19:04:56 +01:00
Erik Johnston
27c28eaa27
Merge pull request #2190 from matrix-org/erikj/mark_remote_as_back_more
...
Always mark remotes as up if we receive a signed request from them
2017-05-05 14:08:12 +01:00
Erik Johnston
653d90c1a5
Comment
2017-05-05 14:01:17 +01:00
Erik Johnston
7b222fc56e
Remove redundant reset of destination timers
2017-05-05 11:14:09 +01:00
Erik Johnston
b843631d71
Add comment and TODO
2017-05-05 10:59:32 +01:00
Erik Johnston
f346048a6e
Handle exceptions thrown in handling remote device list updates
2017-05-05 10:34:10 +01:00
David Baker
60833c8978
Merge pull request #2147 from matrix-org/dbkr/http_request_propagate_error
...
Propagate errors sensibly from proxied IS requests
2017-05-03 11:23:25 +01:00
Erik Johnston
0c27383dd7
Merge pull request #2170 from matrix-org/erikj/fed_hole_state
...
Don't fetch state for missing events that we fetched
2017-05-03 10:49:37 +01:00
Erik Johnston
ef862186dd
Merge together redundant calculations/logging
2017-05-03 10:06:43 +01:00
Erik Johnston
3e5a62ecd8
Add more granular event send metrics
2017-05-02 14:23:26 +01:00
Erik Johnston
2347efc065
Fixup
2017-04-28 12:46:53 +01:00
Erik Johnston
9b147cd730
Remove unncessary call in _get_missing_events_for_pdu
2017-04-28 11:55:25 +01:00
Erik Johnston
3a9f5bf6dd
Don't fetch state for missing events that we fetched
2017-04-28 11:26:46 +01:00
Erik Johnston
25a96e0c63
Merge pull request #2163 from matrix-org/erikj/fix_invite_state
...
Fix invite state to always include all events
2017-04-27 17:36:30 +01:00
Erik Johnston
87ae59f5e9
Typo
2017-04-27 15:16:21 +01:00
Erik Johnston
e42b4ebf0f
Add some extra logging for edge cases of federation
2017-04-27 14:38:21 +01:00
Erik Johnston
34e682d385
Fix invite state to always include all events
2017-04-26 16:18:08 +01:00
David Baker
82ae0238f9
Revert accidental commit
2017-04-26 11:43:16 +01:00
David Baker
81804909d3
Merge remote-tracking branch 'origin/develop' into dbkr/http_request_propagate_error
2017-04-26 11:31:55 +01:00
David Baker
1a9255c12e
Use CodeMessageException subclass instead
...
Parse json errors from get_json client methods and throw special
errors.
2017-04-25 19:30:55 +01:00
David Baker
2e165295b7
Merge remote-tracking branch 'origin/develop' into dbkr/http_request_propagate_error
2017-04-21 11:35:52 +01:00
David Baker
a90a0f5c8a
Propagate errors sensibly from proxied IS requests
...
When we're proxying Matrix endpoints, parse out Matrix error
responses and turn them into SynapseErrors so they can be
propagated sensibly upstream.
2017-04-21 11:32:48 +01:00
Richard van der Hoff
0cdb32fc43
Remove redundant try/except clauses
...
The `except SynapseError` clauses were pointless because the wrapped functions
would never throw a `SynapseError` (they either throw a `CodeMessageException`
or a `RuntimeError`).
The `except CodeMessageException` is now also pointless because the caller
treats all exceptions equally, so we may as well just throw the
`CodeMessageException`.
2017-04-21 01:32:01 +01:00
Richard van der Hoff
838810b76a
Broaden the conditions for locally_rejecting invites
...
The logic for marking invites as locally rejected was all well and good, but
didn't happen when the remote server returned a 500, or wasn't reachable, or
had no DNS, or whatever.
Just expand the except clause to catch everything.
Fixes https://github.com/matrix-org/synapse/issues/761 .
2017-04-21 01:31:37 +01:00
Richard van der Hoff
736b9a4784
Remove redundant function
...
inline `reject_remote_invite`, which only existed to make tracing the callflow
more difficult.
2017-04-21 01:31:09 +01:00
Luke Barnard
3fb8784c92
m.read_marker -> m.fully_read ( #2128 )
...
Also:
- change the REST endpoint to have a "S" on the end (so it's now /read_markers)
- change the content of the m.read_up_to event to have the key "event_id" instead of "marker".
2017-04-18 17:46:15 +01:00
Luke Barnard
78f0ddbfad
Merge pull request #2120 from matrix-org/luke/read-markers
...
Implement Read Marker API
2017-04-13 14:21:31 +01:00
Luke Barnard
b9557064bf
Simplify is_event_after logic
2017-04-12 14:36:20 +01:00
Erik Johnston
247c736b9b
Merge pull request #2115 from matrix-org/erikj/dedupe_federation_repl
...
Reduce federation replication traffic
2017-04-12 11:07:13 +01:00
Luke Barnard
69a18514e9
Only notify user, not entire room
2017-04-12 10:50:37 +01:00
Luke Barnard
122cd52ce4
Remove comment, simplify null-guard
2017-04-12 10:48:32 +01:00
Erik Johnston
c7ddb5ef7a
Reuse get_interested_parties
2017-04-12 10:16:26 +01:00
Luke Barnard
77fb2b72ae
Handle no previous RM
2017-04-12 09:47:29 +01:00
Luke Barnard
867822fa1e
flake8
2017-04-11 17:36:04 +01:00
Luke Barnard
73880268ef
Refactor event ordering check to events store
2017-04-11 17:34:09 +01:00
Luke Barnard
131485ef66
Copyright
2017-04-11 17:33:51 +01:00
Luke Barnard
0127423027
flake8
2017-04-11 17:07:07 +01:00
Erik Johnston
414522aed5
Move get_interested_parties
2017-04-11 15:33:26 +01:00
Erik Johnston
2be8a281d2
Comments
2017-04-11 15:28:24 +01:00
Erik Johnston
6308ac45b0
Move get_interested_remotes back to presence handler
2017-04-11 15:19:26 +01:00
Luke Barnard
d892079844
Finish implementing RM endpoint
...
- This change causes a 405 to be sent if "m.read_marker" is set via /account_data
- This also fixes-up the RM endpoint so that it actually Works.
2017-04-11 15:01:39 +01:00
lukebarnard
e263c26690
Initial commit of RM server-side impl
...
(See https://docs.google.com/document/d/1UWqdS-e1sdwkLDUY0wA4gZyIkRp-ekjsLZ8k6g_Zvso/edit#heading=h.lndohpg8at5u )
2017-04-11 11:55:30 +01:00
Erik Johnston
f3cf3ff8b6
Merge branch 'master' of github.com:matrix-org/synapse into develop
2017-04-11 11:13:32 +01:00
Erik Johnston
29574fd5b3
Reduce federation presence replication traffic
...
This is mainly done by moving the calculation of where to send presence
updates from the presence handler to the transaction queue, so we only
need to send the presence event (and not the destinations) across the
replication connection. Before we were duplicating by sending the full
state across once per destination.
2017-04-10 16:48:30 +01:00
Richard van der Hoff
64765e5199
When we do an invite rejection, save the signed leave event to the db
...
During a rejection of an invite received over federation, we ask a remote
server to make us a `leave` event, then sign it, then send that with
`send_leave`.
We were saving the *unsigned* version of the event (which has a different event
id to the signed version) to our db (and sending it to the clients), whereas
other servers in the room will have seen the *signed* version. We're not aware
of any actual problems that caused, except that it makes the database confusing
to look at and generally leaves the room in a weird state.
2017-04-07 14:39:32 +01:00
Erik Johnston
98ce212093
Merge pull request #2103 from matrix-org/erikj/no-double-encode
...
Don't double encode replication data
2017-04-07 09:39:52 +01:00
Erik Johnston
877c029c16
Use iteritems
2017-04-06 15:51:22 +01:00
Erik Johnston
96b9b6c127
Don't double json encode typing replication data
2017-04-05 11:34:20 +01:00
Erik Johnston
62b89daac6
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/repl_tcp_server
2017-04-04 09:46:16 +01:00
Richard van der Hoff
773e64cc1a
Merge pull request #2095 from matrix-org/rav/cull_log_preserves
...
Cull spurious PreserveLoggingContexts
2017-04-03 17:02:25 +01:00
Richard van der Hoff
30bcbf775a
Accept join events from all servers
...
Make sure that we accept join events from any server, rather than just the
origin server, to make the federation join dance work correctly.
(Fixes #1893 ).
2017-04-03 15:58:07 +01:00
Richard van der Hoff
7eb9f34cc3
Remove spurious yield
...
In `MessageHandler`, remove `yield` on call to `Notifier.on_new_room_event`:
it doesn't return anything anyway.
2017-04-03 15:44:19 +01:00
Richard van der Hoff
0b08c48fc5
Remove more spurious PreserveLoggingContexts
...
Remove `PreserveLoggingContext` around calls to `Notifier.on_new_room_event`;
there is no problem if the logcontext is set when calling it.
2017-04-03 15:43:37 +01:00
Erik Johnston
36d2b66f90
Add a timestamp to USER_SYNC command
...
This timestamp is used to indicate when the user last sync'd
2017-03-31 15:42:22 +01:00
Erik Johnston
9d0170ac6c
Fix up presence
2017-03-31 11:36:32 +01:00
Erik Johnston
63fcc42990
Remove user from process_presence when stops syncing
2017-03-30 14:26:08 +01:00
Erik Johnston
e9dd8370b0
Add functions to presence to support remote syncs
...
The TCP replication protocol streams deltas of who has started or
stopped syncing. This is different from the HTTP API which periodically
sends the full list of users who are syncing. This commit adds support
for the new TCP style of sending deltas.
2017-03-30 13:25:14 +01:00
Erik Johnston
24d35ab47b
Add new storage functions for new replication
...
The new replication protocol will keep all the streams separate, rather
than muxing multiple streams into one.
2017-03-30 11:48:35 +01:00
Erik Johnston
7fc1f1e2b6
Cache hosts in room
2017-03-24 11:46:24 +00:00
Richard van der Hoff
9397edb28b
Merge pull request #2050 from matrix-org/rav/federation_backoff
...
push federation retry limiter down to matrixfederationclient
2017-03-23 22:27:01 +00:00
Richard van der Hoff
5a16cb4bf0
Ignore backoff history for invites, aliases, and roomdirs
...
Add a param to the federation client which lets us ignore historical backoff
data for federation queries, and set it for a handful of operations.
2017-03-23 12:23:22 +00:00
Richard van der Hoff
4bd597d9fc
push federation retry limiter down to matrixfederationclient
...
rather than having to instrument everywhere we make a federation call,
make the MatrixFederationHttpClient manage the retry limiter.
2017-03-23 09:28:46 +00:00
Richard van der Hoff
e08f81d96a
Add a missing yield in device key upload
...
(this would only very very rarely actually be a useful thing, so the main
problem was the logcontext leak...)
2017-03-23 00:16:43 +00:00
Richard van der Hoff
eddce9d74a
Merge pull request #2027 from matrix-org/rav/logcontext_leaks
...
A few fixes to logcontext things
2017-03-20 11:53:36 +00:00
Richard van der Hoff
f40c2db05a
Stop preserve_fn leaking context into the reactor
...
Fix a bug in ``logcontext.preserve_fn`` which made it leak context into the
reactor, and add a test for it.
Also, get rid of ``logcontext.reset_context_after_deferred``, which tried to do
the same thing but had its own, different, set of bugs.
2017-03-18 00:07:43 +00:00
Richard van der Hoff
5068fb16a5
Refactoring and cleanups
...
A few non-functional changes:
* A bunch of docstrings to document types
* Split `EventsStore._persist_events_txn` up a bit. Hopefully it's a bit more
readable.
* Rephrase `EventFederationStore._update_min_depth_for_room_txn` to avoid
mind-bending conditional.
* Rephrase rejected/outlier conditional in `_update_outliers_txn` to avoid
mind-bending conditional.
2017-03-17 15:06:07 +00:00
Richard van der Hoff
2abe85d50e
Merge pull request #2016 from matrix-org/rav/queue_pdus_during_join
...
Queue up federation PDUs while a room join is in progress
2017-03-17 11:32:44 +00:00
Erik Johnston
da146657c9
Comments
2017-03-16 13:04:07 +00:00
Erik Johnston
a158c36a8a
Comment
2017-03-16 11:57:45 +00:00
Erik Johnston
6957bfdca6
Don't recreate so many sets
2017-03-16 11:54:26 +00:00
Erik Johnston
2ccf3b241c
Implement no op for room stream in sync
2017-03-16 11:06:41 +00:00
Richard van der Hoff
9ce53a3861
Queue up federation PDUs while a room join is in progress
...
This just takes the existing `room_queues` logic and moves it out to
`on_receive_pdu` instead of `_process_received_pdu`, which ensures that we
don't start trying to fetch prev_events and whathaveyou until the join has
completed.
2017-03-15 18:01:11 +00:00
Erik Johnston
54d2b7e596
Merge pull request #2014 from Half-Shot/hs/fix-appservice-presence
...
Add fallback to last_active_ts if it beats the last sync time on a presence timeout.
2017-03-15 17:37:15 +00:00
Erik Johnston
f83ac78201
Cache set of users whose presence the other user should see
2017-03-15 15:29:19 +00:00
Will Hunt
e6032054bf
Add a great comment to handle_timeout for active vs sync times.
2017-03-15 15:24:48 +00:00
Will Hunt
ebf5a6b14c
Add fallback to last_active_ts if it beats the last sync time.
2017-03-15 15:17:16 +00:00
Erik Johnston
e892457a03
Comment
2017-03-15 15:01:39 +00:00
Erik Johnston
6c82de5100
Format presence events on the edges instead of reformatting them multiple times
2017-03-15 14:27:34 +00:00
Erik Johnston
7827251daf
Merge pull request #1994 from matrix-org/dbkr/msisdn_signin_2
...
Phone number registration / login support v2
2017-03-15 09:59:54 +00:00
Erik Johnston
bb256ac96f
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/public_list_speed
2017-03-14 11:35:05 +00:00
Luke Barnard
bbeeb97f75
Implement _simple_delete_many_txn, use it to delete devices
...
(But this doesn't implement the same for deleting access tokens or e2e keys.
Also respond to code review.
2017-03-13 17:53:23 +00:00
David Baker
73a5f06652
Support registration / login with phone number
...
Changes from https://github.com/matrix-org/synapse/pull/1971
2017-03-13 17:27:51 +00:00
Richard van der Hoff
3c69f32402
Merge remote-tracking branch 'origin/develop' into rav/refactor_received_pdu
2017-03-13 12:20:47 +00:00
Erik Johnston
0162994983
Comments
2017-03-13 11:53:26 +00:00
Erik Johnston
7eae6eaa2f
Revert "Support registration & login with phone number"
2017-03-13 09:59:33 +00:00
Erik Johnston
79926e016e
Assume rooms likely haven't changed
2017-03-13 09:50:10 +00:00
Erik Johnston
8ffbe43ba1
Get current state by using current_state_events table
2017-03-10 17:39:35 +00:00
Erik Johnston
64d62e41b8
Noop repated delete device inbox calls from /sync
2017-03-10 10:36:43 +00:00
Richard van der Hoff
29235901b8
Move FederationServer._handle_new_pdu to FederationHandler
...
Unfortunately this significantly increases the size of the already-rather-big
FederationHandler, but the code fits more naturally here, and it paves the way
for the tighter integration that I need between handling incoming PDUs and
doing the join dance.
Other than renaming the existing `FederationHandler.on_receive_pdu` to
`_process_received_pdu` to make way for it, this just consists of the move, and
replacing `self.handler` with `self` and `self` with `self.replication_layer`.
2017-03-09 16:20:13 +00:00
David Baker
663396e45d
Merge pull request #1971 from matrix-org/dbkr/msisdn_signin
...
Support registration & login with phone number
2017-03-09 10:18:51 +00:00
David Baker
0e0aee25c4
Fix log line
2017-03-08 11:46:22 +00:00
David Baker
88df6c0c9a
Factor out msisdn canonicalisation
...
Plus a couple of other minor fixes
2017-03-08 11:03:39 +00:00
David Baker
402a7bf63d
Fix pep8
2017-03-08 09:33:40 +00:00
Erik Johnston
f2581ee8b8
Don't keep around old stream IDs forever
2017-03-03 16:02:53 +00:00
Erik Johnston
9834367eea
Spelling
2017-03-03 15:31:57 +00:00
David Baker
ad882cd54d
Just return the deferred straight off
...
defer.returnValue doth not maketh a generator: it would need a
yield to be a generator, and this doesn't need a yield.
2017-03-01 18:08:51 +00:00
