Commit Graph

178 Commits

Author SHA1 Message Date
Erik Johnston
3e831f24ff
Don't hammer the database for destination retry timings every ~5mins (#10036) 2021-05-21 17:57:08 +01:00
Patrick Cloke
551d2c3f4b
Allow a user who could join a restricted room to see it in spaces summary. (#9922)
This finishes up the experimental implementation of MSC3083 by showing
the restricted rooms in the spaces summary (from MSC2946).
2021-05-20 11:10:36 -04:00
Patrick Cloke
f4833e0c06
Support fetching the spaces summary via GET over federation. (#9947)
Per changes in MSC2946, the C-S and S-S APIs for spaces summary
should use GET requests.

Until this is stable, the POST endpoints still exist.

This does not switch federation requests to use the GET version yet
since it is newly added and already deployed servers might not support
it. When switching to the stable endpoint we should switch to GET
requests.
2021-05-11 12:21:43 -04:00
Jonathan de Jong
4b965c862d
Remove redundant "coding: utf-8" lines (#9786)
Part of #9744

Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now.

`Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`
2021-04-14 15:34:27 +01:00
Richard van der Hoff
f946450184
Fix duplicate logging of exceptions in transaction processing (#9780)
There's no point logging this twice.
2021-04-09 18:12:15 +01:00
Patrick Cloke
d959d28730
Add type hints to the federation handler and server. (#9743) 2021-04-06 07:21:57 -04:00
Richard van der Hoff
4ecba9bd5c
Federation API for Space summary (#9652)
Builds on the work done in #9643 to add a federation API for space summaries.

There's a bit of refactoring of the existing client-server code first, to avoid too much duplication.
2021-03-23 11:51:12 +00:00
AndrewFerr
9bc74743d5
Add configs to make profile data more private (#9203)
Add off-by-default configuration settings to:
- disable putting an invitee's profile info in invite events
- disable profile lookup via federation

Signed-off-by: Andrew Ferrazzutti <fair@miscworks.net>
2021-02-19 09:50:41 +00:00
Patrick Cloke
d2f0ec12d5
Add type hints to groups code. (#9393) 2021-02-17 08:41:47 -05:00
Eric Eastwood
0a00b7ff14
Update black, and run auto formatting over the codebase (#9381)
- Update black version to the latest
 - Run black auto formatting over the codebase
    - Run autoformatting according to [`docs/code_style.md
`](80d6dc9783/docs/code_style.md)
 - Update `code_style.md` docs around installing black to use the correct version
2021-02-16 22:32:34 +00:00
Richard van der Hoff
0cd2938bc8
Support icons for Identity Providers (#9154) 2021-01-20 08:15:14 -05:00
Patrick Cloke
1c9a850562
Add type hints to the crypto module. (#8999) 2021-01-04 10:04:50 -05:00
Richard van der Hoff
7ea85302f3 fix up various test cases
A few test cases were relying on being able to mount non-client servlets on the
test resource. it's better to give them their own Resources.
2020-12-02 16:30:01 +00:00
Richard van der Hoff
950bb0305f
Consistently use room_id from federation request body (#8776)
* Consistently use room_id from federation request body

Some federation APIs have a redundant `room_id` path param (see
https://github.com/matrix-org/matrix-doc/issues/2330). We should make sure we
consistently use either the path param or the body param, and the body param is
easier.

* Kill off some references to "context"

Once upon a time, "rooms" were known as "contexts". I think this kills of the
last references to "contexts".
2020-11-19 10:05:33 +00:00
Erik Johnston
f21e24ffc2
Add ability for access tokens to belong to one user but grant access to another user. (#8616)
We do it this way round so that only the "owner" can delete the access token (i.e. `/logout/all` by the "owner" also deletes that token, but `/logout/all` by the "target user" doesn't).

A future PR will add an API for creating such a token.

When the target user and authenticated entity are different the `Processed request` log line will be logged with a: `{@admin:server as @bob:server} ...`. I'm not convinced by that format (especially since it adds spaces in there, making it harder to use `cut -d ' '` to chop off the start of log lines). Suggestions welcome.
2020-10-29 15:58:44 +00:00
Patrick Cloke
8a4a4186de
Simplify super() calls to Python 3 syntax. (#8344)
This converts calls like super(Foo, self) -> super().

Generated with:

    sed -i "" -Ee 's/super\([^\(]+\)/super()/g' **/*.py
2020-09-18 09:56:44 -04:00
Erik Johnston
14b5b48a22
Fix ratelimiting for federation /send requests. (#8342)
c.f. #8295 for rationale
2020-09-18 10:49:29 +01:00
Patrick Cloke
c619253db8
Stop sub-classing object (#8249) 2020-09-04 06:54:56 -04:00
Richard van der Hoff
4876af06dd
Abort federation requests if the client disconnects early (#7930)
For inbound federation requests, if a given remote server makes too many
requests at once, we start stacking them up rather than processing them
immediatedly.

However, that means that there is a fair chance that the requesting server will
disconnect before we start processing the request. In that case, if it was a
read-only request (ie, a GET request), there is absolutely no point in
building a response (and some requests are quite expensive to handle).

Even in the case of a POST request, one of two things will happen:

 * Most likely, the requesting server will retry the request and we'll get the
   information anyway.

 * Even if it doesn't, the requesting server has to assume that we didn't get
   the memo, and act accordingly.

In short, we're better off aborting the request at this point rather than
ploughing on with what might be a quite expensive request.
2020-07-23 16:52:33 +01:00
Patrick Cloke
de119063f2
Convert room list handler to async/await. (#7912) 2020-07-21 07:51:48 -04:00
Patrick Cloke
38e1fac886
Fix some spelling mistakes / typos. (#7811) 2020-07-09 09:52:58 -04:00
Erik Johnston
5cdca53aa0
Merge different Resource implementation classes (#7732) 2020-07-03 19:02:19 +01:00
Patrick Cloke
7581d30e9f
Remove unused federation endpoint (query_auth) (#7026) 2020-03-17 08:04:49 -04:00
Erik Johnston
ed630ea17c
Reduce amount of logging at INFO level. (#6862)
A lot of the things we log at INFO are now a bit superfluous, so lets
make them DEBUG logs to reduce the amount we log by default.

Co-Authored-By: Brendan Abolivier <babolivier@matrix.org>
Co-authored-by: Brendan Abolivier <github@brendanabolivier.com>
2020-02-06 13:31:05 +00:00
Richard van der Hoff
b0d112e78b
Fix room_version in on_invite_request flow (#6827)
I messed this up a bit in #6805, but fortunately we weren't actually doing
anything with the room_version so it didn't matter that it was a str not a RoomVersion.
2020-02-03 13:15:23 +00:00
Erik Johnston
c3d4ad8afd
Fix sending server up commands from workers (#6811)
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2020-01-30 16:42:11 +00:00
Erik Johnston
a8a50f5b57
Wake up transaction queue when remote server comes back online (#6706)
This will be used to retry outbound transactions to a remote server if
we think it might have come back up.
2020-01-17 10:27:19 +00:00
Brendan Abolivier
e126d83f74 Merge branch 'develop' into babolivier/msc1802 2019-12-05 21:00:43 +00:00
Amber Brown
0f87b912ab
Implementation of MSC2314 (#6176) 2019-11-28 08:54:07 +11:00
Andrew Morgan
bc29a19731 Replace instance variations of homeserver with correct case/spacing 2019-11-12 13:08:12 +00:00
Brendan Abolivier
5e18dc7955
Fix prefix for v2/send_leave 2019-11-11 16:46:09 +00:00
Brendan Abolivier
74897de01f
Add server-side support to the v2 API 2019-11-11 16:40:45 +00:00
Andrew Morgan
54fef094b3
Remove usage of deprecated logger.warn method from codebase (#6271)
Replace every instance of `logger.warn` with `logger.warning` as the former is deprecated.
2019-10-31 10:23:24 +00:00
Erik Johnston
a5166e4d5f
Land improved room list based on room stats (#6019)
Use room_stats and room_state for room directory search
2019-10-02 14:08:35 +01:00
Richard van der Hoff
9d94313209 Fix exception when resetting retry timings
Fixes:
> TypeError: set_destination_retry_timings() missing 1 required positional
argument: 'retry_interval'

Introduced in #6016.
2019-09-20 12:09:39 +01:00
Andrew Morgan
cd17a2085e
Remove origin parameter from add_display_name_to_third_party_invite and add params to docstring (#6010)
Another small fixup noticed during work on a larger PR. The `origin` field of `add_display_name_to_third_party_invite` is not used and likely was just carried over from the `on_PUT` method of `FederationThirdPartyInviteExchangeServlet` which, like all other servlets, provides an `origin` argument.

Since it's not used anywhere in the handler function though, we should remove it from the function arguments.
2019-09-11 10:37:17 +01:00
Jorik Schellekens
909827b422
Add opentracing to all client servlets (#5983) 2019-09-05 14:46:04 +01:00
reivilibre
7ccc251415
Merge pull request #5859 from matrix-org/rei/msc2197
MSC2197 Search Filters over Federation
2019-08-28 09:00:21 +01:00
Jorik Schellekens
812ed6b0d5
Opentracing across workers (#5771)
Propagate opentracing contexts across workers


Also includes some Convenience modifications to opentracing for servlets, notably:
- Add boolean to skip the whitelisting check on inject
  extract methods. - useful when injecting into carriers
  locally. Otherwise we'd always have to include our
  own servername and whitelist our servername
- start_active_span_from_request instead of header
- Add boolean to decide whether to extract context
  from a request to a servlet
2019-08-22 18:08:07 +01:00
Olivier Wilkinson (reivilibre)
bb29bc2937 Use MSC2197 on stable prefix as it has almost finished FCP
Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
2019-08-20 08:49:31 +01:00
Jorik Schellekens
87fa26006b
Opentracing misc (#5856)
Add authenticated_entity and servlet_names tags.

Functionally:
- Add a tag for authenticated_entity
- Add a tag for servlet_names

Stylistically:
Moved to importing methods directly from opentracing.
2019-08-16 16:13:25 +01:00
Olivier Wilkinson (reivilibre)
2253b083d9 Add support for inbound MSC2197 requests on unstable Federation API
Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
2019-08-15 11:06:21 +01:00
Olivier Wilkinson (reivilibre)
1b63ccd848 Wrap get_local_public_room_list call in maybeDeferred because it
is cached and so does not always return a `Deferred`.
`await` does not silently pass-through non-Deferreds like `yield` used to.

Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
2019-08-14 14:53:49 +01:00
Jorik Schellekens
cf2972c818
Fix servlet metric names (#5734)
* Fix servlet metric names

Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>

* Remove redundant check

* Cover all return paths
2019-07-24 13:07:35 +01:00
Richard van der Hoff
fa8271c5ac
Convert synapse.federation.transport.server to async (#5689)
* Convert BaseFederationServlet._wrap to async

Empirically, this fixes some lost stacktraces. It should be safe because the
wrapped function is called from JsonResource._async_render, which is already
async.

* Convert the rest of synapse.federation.transport.server to async

We may as well do the whole file while we're here.

* changelog

* flake8
2019-07-18 11:46:47 +01:00
Jorik Schellekens
38a6d3eea7
Add basic opentracing support (#5544)
* Configure and initialise tracer

Includes config options for the tracer and sets up JaegerClient.

* Scope manager using LogContexts

We piggy-back our tracer scopes by using log context.
The current log context gives us the current scope. If new scope is
created we create a stack of scopes in the context.

* jaeger is a dependency now

* Carrier inject and extraction for Twisted Headers

* Trace federation requests on the way in and out.

The span is created in _started_processing and closed in
_finished_processing because we need a meaningful log context.

* Create logcontext for new scope.

Instead of having a stack of scopes in a logcontext we create a new
context for a new scope if the current logcontext already has a scope.

* Remove scope from logcontext if logcontext is top level

* Disable tracer if not configured

* typo

* Remove dependence on jaeger internals

* bools

* Set service name

* :Explicitely state that the tracer is disabled

* Black is the new black

* Newsfile

* Code style

* Use the new config setup.

* Generate config.

* Copyright

* Rename config to opentracing

* Remove user whitelisting

* Empty whitelist by default

* User ConfigError instead of RuntimeError

* Use isinstance

* Use tag constants for opentracing.

* Remove debug comment and no need to explicitely record error

* Two errors a "s(c)entry"

* Docstrings!

* Remove debugging brainslip

* Homeserver Whitlisting

* Better opentracing config comment

* linting

* Inclue worker name in service_name

* Make opentracing an optional dependency

* Neater config retreival

* Clean up dummy tags

* Instantiate tracing as object instead of global class

* Inlcude opentracing as a homeserver member.

* Thread opentracing to the request level

* Reference opetnracing through hs

* Instantiate dummy opentracin g for tests.

* About to revert, just keeping the unfinished changes just in case

* Revert back to global state, commit number:

9ce4a3d9067bf9889b86c360c05ac88618b85c4f

* Use class level methods in tracerutils

* Start and stop requests spans in a place where we
have access to the authenticated entity

* Seen it, isort it

* Make sure to close the active span.

* I'm getting black and blue from this.

* Logger formatting

Co-Authored-By: Erik Johnston <erik@matrix.org>

* Outdated comment

* Import opentracing at the top

* Return a contextmanager

* Start tracing client requests from the servlet

* Return noop context manager if not tracing

* Explicitely say that these are federation requests

* Include servlet name in client requests

* Use context manager

* Move opentracing to logging/

* Seen it, isort it again!

* Ignore twisted return exceptions on context exit

* Escape the scope

* Scopes should be entered to make them useful.

* Nicer decorator names

* Just one init, init?

* Don't need to close something that isn't open

* Docs make you smarter
2019-07-11 10:36:03 +01:00
Amber Brown
463b072b12
Move logging utilities out of the side drawer of util/ and into logging/ (#5606) 2019-07-04 00:07:04 +10:00
Brendan Abolivier
bfe84e051e Split public rooms directory auth config in two 2019-06-24 15:42:31 +01:00
Amber Brown
32e7c9e7f2
Run Black. (#5482) 2019-06-20 19:32:02 +10:00
Richard van der Hoff
cea9750d11 Associate a request_name with each verify request, for logging
Also:
* rename VerifyKeyRequest->VerifyJsonRequest
* calculate key_ids on VerifyJsonRequest construction
* refactor things to pass around VerifyJsonRequests instead of 4-tuples
2019-06-05 10:46:26 +01:00