Erik Johnston
bf7598f582
Log when we 3pid/unbind request fails
2018-08-09 10:09:56 +01:00
Erik Johnston
2bdafaf3c1
Merge pull request #3632 from matrix-org/erikj/refactor_repl_servlet
...
Add helper base class for generating new replication endpoints
2018-08-09 10:06:23 +01:00
Richard van der Hoff
bb89c84614
Merge pull request #3664 from matrix-org/rav/federation_metrics
...
more metrics for the federation and appservice senders
2018-08-08 23:16:58 +01:00
Neil Johnson
e92fb00f32
sync auth blocking
2018-08-08 17:54:49 +01:00
Richard van der Hoff
8521ae13e3
Merge pull request #3654 from matrix-org/rav/room_versions
...
Support for room versioning
2018-08-08 17:10:53 +01:00
Neil Johnson
990fe9fc23
Merge pull request #3633 from matrix-org/neilj/mau_tracker
...
API for monthly_active_users table
2018-08-08 12:44:15 +00:00
Erik Johnston
360ba89c50
Don't fail requests to unbind 3pids for non supporting ID servers
...
Older identity servers may not support the unbind 3pid request, so we
shouldn't fail the requests if we received one of 400/404/501. The
request still fails if we receive e.g. 500 responses, allowing clients
to retry requests on transient identity server errors that otherwise do
support the API.
Fixes #3661
2018-08-08 12:06:18 +01:00
Richard van der Hoff
53bca4690b
more metrics for the federation and appservice senders
2018-08-07 19:09:48 +01:00
Richard van der Hoff
3523f5432a
Don't expose default_room_version as config opt
2018-08-07 12:51:57 +01:00
Erik Johnston
495cb100d1
Allow profile changes to happen on workers
2018-08-07 10:50:26 +01:00
Erik Johnston
a3f5bf79a0
Add EDU/query handling over replication
2018-08-06 15:23:31 +01:00
Erik Johnston
e26dbd82ef
Add replication APIs for persisting federation events
2018-08-06 15:02:28 +01:00
Richard van der Hoff
f900d50824
include known room versions in outgoing make_joins
2018-08-06 13:45:37 +01:00
Neil Johnson
e40a510fbf
py3 fix
2018-08-03 23:19:13 +01:00
Neil Johnson
886be75ad1
bug fixes
2018-08-03 22:29:03 +01:00
Will Hunt
16d9701892
Return M_NOT_FOUND when a profile could not be found. ( #3596 )
2018-08-03 19:08:05 +01:00
Richard van der Hoff
0ca459ea33
Basic support for room versioning
...
This is the first tranche of support for room versioning. It includes:
* setting the default room version in the config file
* new room_version param on the createRoom API
* storing the version of newly-created rooms in the m.room.create event
* fishing the version of existing rooms out of the m.room.create event
2018-08-03 16:08:32 +01:00
Neil Johnson
897c51d274
Merge branch 'develop' of github.com:matrix-org/synapse into neilj/mau_tracker
2018-08-03 13:40:47 +01:00
Erik Johnston
cb298ff623
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/refactor_repl_servlet
2018-08-03 09:25:15 +01:00
Richard van der Hoff
1fa98495d0
Merge pull request #3639 from matrix-org/rav/refactor_error_handling
...
Clean up handling of errors from outbound requests
2018-08-02 17:38:24 +01:00
Neil Johnson
74b1d46ad9
do mau checks based on monthly_active_users table
2018-08-02 16:57:35 +01:00
Richard van der Hoff
704c3e6239
Merge branch 'master' into develop
2018-08-02 15:43:30 +01:00
Richard van der Hoff
0bf5ec0db7
Check room visibility for /event/ requests
...
Make sure that the user has permission to view the requeseted event for
/event/{eventId} and /room/{roomId}/event/{eventId} requests.
Also check that the event is in the given room for
/room/{roomId}/event/{eventId}, for sanity.
2018-08-02 15:03:27 +01:00
Richard van der Hoff
14fa9d4d92
Avoid extra db lookups
...
Since we're about to look up the events themselves anyway, we can skip the
extra db queries here.
2018-08-02 13:55:51 +01:00
Richard van der Hoff
0a65450d04
Validation for events/rooms in fed requests
...
When we get a federation request which refers to an event id, make sure that
said event is in the room the caller claims it is in.
(patch supplied by @turt2live)
2018-08-02 13:48:40 +01:00
Erik Johnston
40c1c59cf4
Merge pull request #3621 from matrix-org/erikj/split_fed_store
...
Split out DB writes in federation handler
2018-08-02 10:41:42 +01:00
Neil Johnson
085435e13a
Merge pull request #3630 from matrix-org/neilj/mau_sign_in_log_in_limits
...
Initial impl of capping MAU
2018-08-01 15:58:45 +00:00
Richard van der Hoff
01e93f48ed
Kill off MatrixCodeMessageException
...
This code brings the SimpleHttpClient into line with the
MatrixFederationHttpClient by having it raise HttpResponseExceptions when a
request fails (rather than trying to parse for matrix errors and maybe raising
MatrixCodeMessageException).
Then, whenever we were checking for MatrixCodeMessageException and turning them
into SynapseErrors, we now need to check for HttpResponseExceptions and call
to_synapse_error.
2018-08-01 16:02:46 +01:00
Amber Brown
da7785147d
Python 3: Convert some unicode/bytes uses ( #3569 )
2018-08-02 00:54:06 +10:00
Neil Johnson
303f1c851f
Merge branch 'develop' of github.com:matrix-org/synapse into neilj/mau_sign_in_log_in_limits
2018-08-01 13:42:50 +01:00
Erik Johnston
a6d7b74915
update docs
2018-08-01 13:39:14 +01:00
Erik Johnston
4b256b9271
_persist_auth_tree no longer returns anything
2018-08-01 13:39:07 +01:00
Neil Johnson
0aba3d361a
count_monthly_users() async
2018-08-01 11:47:58 +01:00
Neil Johnson
7931393495
make count_monthly_users async synapse/handlers/auth.py
2018-08-01 10:21:56 +01:00
Richard van der Hoff
bdbdceeafa
rename replication_layer to federation_client
...
I have HAD ENOUGH of trying to remember wtf a replication layer is in terms of
classes.
2018-07-31 15:44:05 +01:00
Erik Johnston
443da003bc
Use new helper base class for membership requests
2018-07-31 14:32:23 +01:00
Erik Johnston
729b672823
Use new helper base class for ReplicationSendEventRestServlet
2018-07-31 14:32:23 +01:00
Neil Johnson
df2235e7fa
coding style
2018-07-31 13:16:20 +01:00
Neil Johnson
251e6c1210
limit register and sign in on number of monthly users
2018-07-30 15:55:57 +01:00
Erik Johnston
143f1a2532
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/split_fed_store
2018-07-30 09:56:18 +01:00
Matthew Hodgson
e9b2d047f6
make /context lazyload & filter aware ( #3567 )
...
make /context lazyload & filter aware.
2018-07-27 15:12:50 +01:00
Richard van der Hoff
b0b5566f36
Merge pull request #3391 from t3chguy/t3chguy/default_inviter_display_name_3pid
...
if inviter_display_name == ""||None then default to inviter MXID
2018-07-27 10:08:39 +01:00
Matthew Hodgson
a75231b507
Deduplicate redundant lazy-loaded members ( #3331 )
...
* attempt at deduplicating lazy-loaded members
as per the proposal; we can deduplicate redundant lazy-loaded members
which are sent in the same sync sequence. we do this heuristically
rather than requiring the client to somehow tell us which members it
has chosen to cache, by instead caching the last N members sent to
a client, and not sending them again. For now we hardcode N to 100.
Each cache for a given (user,device) tuple is in turn cached for up to
X minutes (to avoid the caches building up). For now we hardcode X to 30.
* add include_redundant_members filter option & make it work
* remove stale todo
* add tests for _get_some_state_from_cache
* incorporate review
2018-07-26 22:51:30 +01:00
Richard van der Hoff
03751a6420
Fix some looping_call calls which were broken in #3604
...
It turns out that looping_call does check the deferred returned by its
callback, and (at least in the case of client_ips), we were relying on this,
and I broke it in #3604 .
Update run_as_background_process to return the deferred, and make sure we
return it to clock.looping_call.
2018-07-26 11:48:08 +01:00
Matthew Hodgson
1bcd0490c2
Merge pull request #2970 from matrix-org/matthew/filter_members
...
Implement the lazy_load_members room state filter parameter
2018-07-26 00:03:01 +01:00
Travis Ralston
d8e65ed7e1
Fix a minor documentation typo in on_make_leave
2018-07-25 15:44:41 -06:00
Matthew Hodgson
2565804030
Merge branch 'develop' into matthew/filter_members
2018-07-25 17:27:49 +01:00
Erik Johnston
78a691d005
Split out DB writes in federation handler
...
This will allow us to easily add an internal replication API to proxy
these reqeusts to master, so that we can move federation APIs to
workers.
2018-07-25 16:22:56 +01:00
Richard van der Hoff
55acd6856c
Fix updating of cached remote profiles
...
_update_remote_profile_cache was missing its `defer.inlineCallbacks`, so when
it was called, would just return a generator object, without actually running
any of the method body.
2018-07-25 10:34:48 +01:00
Michael Telatynski
87951d3891
Merge branch 'develop' of github.com:matrix-org/synapse into t3chguy/default_inviter_display_name_3pid
2018-07-24 17:17:46 +01:00
Erik Johnston
8b8c4f34a3
Replace usage of get_current_toke with StreamToken.START
...
This allows us to handle /context/ requests on the client_reader worker
without having to pull in all the various stream handlers (e.g.
precence, typing, pushers etc). The only thing the token gets used for
is pagination, and that ignores everything but the room portion of the
token.
2018-07-24 16:49:17 +01:00
Richard van der Hoff
a321f78991
Merge pull request #3586 from matrix-org/rav/optimise_resolve_state_groups
...
Fixes and optimisations for resolve_state_groups
2018-07-24 14:11:45 +01:00
Matthew Hodgson
1a01a5b964
clarify comment on p_ids
2018-07-24 14:03:15 +01:00
Matthew Hodgson
eb1d911ab7
rather than adding ll_ids, remove them from p_ids
2018-07-24 13:40:49 +01:00
Matthew Hodgson
d19fba3655
Merge branch 'develop' into matthew/filter_members
2018-07-24 12:39:54 +01:00
Matthew Hodgson
cd241d6bda
incorporate more review
2018-07-24 12:39:40 +01:00
Richard van der Hoff
a678145010
Merge branch 'develop' into rav/logcontext_fixes
2018-07-24 10:43:30 +01:00
Erik Johnston
536bc63a4e
Merge branch 'develop' into erikj/client_apis_move
2018-07-24 09:57:05 +01:00
Richard van der Hoff
cf2d15c6a9
another couple of logcontext leaks
2018-07-24 00:57:48 +01:00
Richard van der Hoff
8dff6e0322
Logcontext fixes
...
Fix some random logcontext leaks.
2018-07-24 00:37:17 +01:00
Richard van der Hoff
c1f80effbe
Handle delta_ids being None in _update_context_for_auth_events
...
it's easier to create the new state group as a delta from the existing one.
(There's an outside chance this will help with
https://github.com/matrix-org/synapse/issues/3364 )
2018-07-23 22:06:50 +01:00
Matthew Hodgson
adfe29ec0b
Merge branch 'develop' into matthew/filter_members
2018-07-23 19:21:37 +01:00
Matthew Hodgson
254fb430d1
incorporate review
2018-07-23 19:21:20 +01:00
Erik Johnston
0faa3223cd
Fix missing attributes on workers.
...
This was missed during the transition from attribute to getter for
getting state from context.
2018-07-23 16:28:00 +01:00
Erik Johnston
0b0b24cb82
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/client_apis_move
2018-07-23 13:21:15 +01:00
Erik Johnston
027bc01a1b
Add support for updating state
2018-07-23 13:17:25 +01:00
Erik Johnston
e42510ba63
Use new getters
2018-07-23 13:17:22 +01:00
Amber Brown
3132b89f12
Make the rest of the .iterwhatever go away ( #3562 )
2018-07-21 15:47:18 +10:00
Erik Johnston
5c88bb722f
Move PaginationHandler to its own file
2018-07-20 15:32:23 +01:00
Erik Johnston
0ecf68aedc
Move check_in_room_or_world_readable to Auth
2018-07-20 15:30:59 +01:00
Richard van der Hoff
7c712f95bb
Combine Limiter and Linearizer
...
Linearizer was effectively a Limiter with max_count=1, so rather than
maintaining two sets of code, let's combine them.
2018-07-20 13:11:43 +01:00
Richard van der Hoff
8462c26485
Improvements to the Limiter
...
* give them names, to improve logging
* use a deque rather than a list for efficiency
2018-07-20 12:50:27 +01:00
Matthew Hodgson
bcaec2915a
incorporate review
2018-07-19 19:03:50 +01:00
Matthew Hodgson
924eb34d94
add a filtered_types param to limit filtering to specific types
2018-07-19 18:32:02 +01:00
Matthew Hodgson
be3adfc331
merge develop pydoc for _get_state_for_groups
2018-07-19 11:26:04 +01:00
Erik Johnston
bacdf0cbf9
Move RoomContextHandler out of Handlers
...
This is in preparation for moving GET /context/ to a worker
2018-07-18 15:33:03 +01:00
Erik Johnston
8cb8df55e9
Split MessageHandler into read only and writers
...
This will let us call the read only parts from workers, and so be able
to move some APIs off of master, e.g. the `/state` API.
2018-07-18 15:33:03 +01:00
Richard van der Hoff
dab00faa83
Merge pull request #3367 from matrix-org/rav/drop_re_signing_hacks
...
Remove event re-signing hacks
2018-07-18 12:46:27 +01:00
Richard van der Hoff
9c04b4abf9
Merge pull request #3541 from matrix-org/rav/optimize_filter_events_for_server
...
Refactor and optimze filter_events_for_server
2018-07-17 14:01:39 +01:00
Richard van der Hoff
ea69d35651
Move filter_events_for_server out of FederationHandler
...
for easier unit testing.
2018-07-16 13:06:24 +01:00
Matthew Hodgson
ba22b6a456
typo
2018-07-13 12:03:39 +01:00
Matthew Hodgson
ea752bdd99
s/becuase/because/g
2018-07-10 17:58:18 +01:00
Amber Brown
49af402019
run isort
2018-07-09 16:09:20 +10:00
Richard van der Hoff
a4ab491371
Merge branch 'develop' into rav/drop_re_signing_hacks
2018-07-04 07:13:38 +01:00
Erik Johnston
e3b4043800
Merge pull request #3456 from matrix-org/hawkowl/federation-prevevent-checking
...
Check the state of prev_events a bit more thoroughly when coming over federation
2018-06-29 13:55:02 +01:00
Amber Brown
6350bf925e
Attempt to be more performant on PyPy ( #3462 )
2018-06-28 14:49:57 +01:00
Amber Brown
99800de63d
try and clean up
2018-06-27 11:40:27 +01:00
Amber Brown
f03a5d1a17
pep8
2018-06-27 11:38:14 +01:00
Amber Brown
a7ecf34b70
cleanups
2018-06-27 11:36:03 +01:00
Amber Brown
77078d6c8e
handle federation not telling us about prev_events
2018-06-27 11:27:32 +01:00
Matthew Hodgson
c7f6b420ae
Merge pull request #3448 from matrix-org/matthew/gdpr-deactivate-admin-api
...
add GDPR erase param to deactivate API
2018-06-26 10:43:14 +01:00
Matthew Hodgson
1e788db430
add GDPR erase param to deactivate API
2018-06-26 10:26:54 +01:00
Erik Johnston
d72fb9a448
Merge pull request #3442 from matrix-org/matthew/allow-unconsented-parts
...
allow non-consented users to still part rooms (to let us autopart them)
2018-06-25 20:14:18 +01:00
Erik Johnston
df48f7ef37
Actually fix it
2018-06-25 20:03:41 +01:00
Erik Johnston
a0e8a53c6d
Comment
2018-06-25 19:57:38 +01:00
Erik Johnston
7bdc5c8fa3
Fix bug with assuming wrong type
2018-06-25 19:56:02 +01:00
Erik Johnston
ea7a9c0483
Add fast path to _filter_events_for_server
...
Most rooms have a trivial history visibility like "shared" or
"world_readable", especially large rooms, so lets not bother getting the
full membership of those rooms in that case.
2018-06-25 19:49:13 +01:00
Matthew Hodgson
0269367f18
allow non-consented users to still part rooms (to let us autopart them)
2018-06-25 17:56:10 +01:00
Matthew Hodgson
784189b1f4
typos
2018-06-25 17:37:16 +01:00
Matthew Hodgson
6eb861b67f
typo
2018-06-25 17:37:16 +01:00
