Commit Graph

75 Commits

Author SHA1 Message Date
Patrick Cloke
d8cc86eff4
Remove redundant types from comments. ()
Remove type hints from comments which have been added
as Python type hints. This helps avoid drift between comments
and reality, as well as removing redundant information.

Also adds some missing type hints which were simple to fill in.
2022-11-16 15:25:24 +00:00
Hugh Nimmo-Smith
4eaf3eb840
Implementation of HTTP 307 response for MSC3886 POST endpoint ()
Co-authored-by: reivilibre <olivier@librepush.net>
Co-authored-by: Andrew Morgan <andrewm@element.io>
2022-10-18 15:52:25 +00:00
Eric Eastwood
5093cbf88d
Be able to correlate timeouts in reverse-proxy layer in front of Synapse (pull request ID from header) ()
Fix https://github.com/matrix-org/synapse/issues/13685

New config:

```diff
  listeners:
    - port: 8008
      tls: false
      type: http
      x_forwarded: true
+     request_id_header: "cf-ray"
      bind_addresses: ['::1', '127.0.0.1', '0.0.0.0']
```
2022-09-15 15:32:25 -05:00
Quentin Gliech
3dd175b628
synapse.api.auth.Auth cleanup: make permission-related methods use Requester instead of the UserID ()
Part of 

This changes all the permission-related methods to rely on the Requester instead of the UserID. This is a first step towards enabling scoped access tokens at some point, since I expect the Requester to have scope-related informations in it.

It also changes methods which figure out the user/device/appservice out of the access token to return a Requester instead of something else. This avoids having store-related objects in the methods signatures.
2022-08-22 14:17:59 +01:00
Sean Quah
c997bfb926
Capture the Deferred for request cancellation in _AsyncResource ()
All async request processing goes through `_AsyncResource`, so this is
the only place where a `Deferred` needs to be captured for cancellation.

Unfortunately, the same isn't true for determining whether a request
can be cancelled. Each of `RestServlet`, `BaseFederationServlet`,
`DirectServe{Html,Json}Resource` and `ReplicationEndpoint` have
different wrappers around the method doing the request handling and they
all need to be handled separately.

Signed-off-by: Sean Quah <seanq@element.io>
2022-05-10 20:39:05 +01:00
Sean Quah
5cfb004595
Add ability to cancel disconnected requests to SynapseRequest ()
Signed-off-by: Sean Quah <seanq@element.io>
2022-05-10 14:06:08 +01:00
Patrick Cloke
7fbf42499d
Use getClientAddress instead of getClientIP. ()
getClientIP was deprecated in Twisted 18.4.0, which also added
getClientAddress. The Synapse minimum version for Twisted is
currently 18.9.0, so all supported versions have the new API.
2022-05-04 14:11:21 -04:00
David Robertson
d8df8e6c14
Don't print HTTPStatus.* in "Processed..." logs ()
* Don't print HTTPStatus.* in "Processed..." logs

Fixes . See also  and
https://github.com/matrix-org/synapse/pull/7188#r401719326 in
particular.

Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
2022-01-26 12:47:34 +00:00
Patrick Cloke
10a88ba91c
Use auto_attribs/native type hints for attrs classes. () 2022-01-13 13:49:28 +00:00
Richard van der Hoff
60fa4935b5
Improve opentracing for incoming HTTP requests ()
* remove `start_active_span_from_request`

Instead, pull out a separate function, `span_context_from_request`, to extract
the parent span, which we can then pass into `start_active_span` as
normal. This seems to be clearer all round.

* Remove redundant tags from `incoming-federation-request`

These are all wrapped up inside a parent span generated in AsyncResource, so
there's no point duplicating all the tags that are set there.

* Leave request spans open until the request completes

It may take some time for the response to be encoded into JSON, and that JSON
to be streamed back to the client, and really we want that inside the top-level
span, so let's hand responsibility for closure to the SynapseRequest.

* opentracing logs for HTTP request events

* changelog
2021-12-20 17:45:03 +00:00
Patrick Cloke
33abbc3278
Add missing type hints to synapse.http. () 2021-12-14 07:00:47 -05:00
Erik Johnston
50022cff96
Add reactor to SynapseRequest and fix up types. () 2021-09-24 11:01:25 +01:00
Erik Johnston
b25a494779
Add types to http.site () 2021-09-21 16:41:27 +00:00
Erik Johnston
66d72b7e17
Change logging of puppeted requests to better differentiate users ()
This used to be a comma and got accidentally changed to a period in , but a pipe character is more easier to parse visually.
2021-09-08 12:59:15 +00:00
Jonathan de Jong
95e47b2e78
[pyupgrade] synapse/ ()
This PR is tantamount to running 
```
pyupgrade --py36-plus --keep-percent-format `find synapse/ -type f -name "*.py"`
```

Part of 
2021-07-19 15:28:05 +01:00
Jonathan de Jong
bf72d10dbf
Use inline type hints in various other places (in synapse/) () 2021-07-15 11:02:43 +01:00
Erik Johnston
3f96dbbda7
Log method and path when dropping request due to size limit () 2021-05-28 15:57:53 +01:00
Richard van der Hoff
3ff2251754
Improved validation for received requests ()
* Simplify `start_listening` callpath

* Correctly check the size of uploaded files
2021-04-23 19:20:44 +01:00
Richard van der Hoff
59d24c5bef
pass a reactor into SynapseSite () 2021-04-23 17:06:47 +01:00
Dan Callahan
1d5f0e3529
Bump black configuration to target py36 ()
Signed-off-by: Dan Callahan <danc@element.io>
2021-04-13 10:41:34 +01:00
Patrick Cloke
48d44ab142
Record more information into structured logs. ()
Records additional request information into the structured logs,
e.g. the requester, IP address, etc.
2021-04-08 08:01:14 -04:00
Patrick Cloke
33a02f0f52
Fix additional type hints from Twisted upgrade. () 2021-03-03 15:47:38 -05:00
Richard van der Hoff
d8e95e5452
Add support for X-Forwarded-Proto ()
rewrite XForwardedForRequest to set `isSecure()` based on
`X-Forwarded-Proto`. Also implement `getClientAddress()` while we're here.
2021-02-24 18:11:33 +00:00
Eric Eastwood
0a00b7ff14
Update black, and run auto formatting over the codebase ()
- Update black version to the latest
 - Run black auto formatting over the codebase
    - Run autoformatting according to [`docs/code_style.md
`](80d6dc9783/docs/code_style.md)
 - Update `code_style.md` docs around installing black to use the correct version
2021-02-16 22:32:34 +00:00
Richard van der Hoff
2ec8ca5e60
Remove SynapseRequest.get_user_agent ()
SynapseRequest is in danger of becoming a bit of a dumping-ground for "useful stuff relating to Requests",
which isn't really its intention (its purpose is to override render, finished and connectionLost to set up the 
LoggingContext and write the right entries to the request log).

Putting utility functions inside SynapseRequest means that lots of our code ends up requiring a
SynapseRequest when there is nothing synapse-specific about the Request at all, and any old
twisted.web.iweb.IRequest will do. This increases code coupling and makes testing more difficult.

In short: move get_user_agent out to a utility function.
2021-01-12 12:34:16 +00:00
Patrick Cloke
1619802228
Various clean-ups to the logging context code () 2020-12-14 14:19:47 -05:00
Erik Johnston
f21e24ffc2
Add ability for access tokens to belong to one user but grant access to another user. ()
We do it this way round so that only the "owner" can delete the access token (i.e. `/logout/all` by the "owner" also deletes that token, but `/logout/all` by the "target user" doesn't).

A future PR will add an API for creating such a token.

When the target user and authenticated entity are different the `Processed request` log line will be logged with a: `{@admin:server as @bob:server} ...`. I'm not convinced by that format (especially since it adds spaces in there, making it harder to use `cut -d ' '` to chop off the start of log lines). Suggestions welcome.
2020-10-29 15:58:44 +00:00
Patrick Cloke
34a5696f93
Fix typos and spelling errors. () 2020-10-23 12:38:40 -04:00
Erik Johnston
c850dd9a8e
Fix handling of User-Agent headers with bad utf-8. () 2020-10-23 17:12:59 +01:00
Erik Johnston
2f9fd5ab00
Don't log OPTIONS request at INFO () 2020-08-07 14:53:05 +01:00
Erik Johnston
7620912d84
Add health check endpoint () 2020-08-07 14:21:24 +01:00
Patrick Cloke
c36228c403
Convert run_as_background_process inner function to async. () 2020-08-06 08:20:42 -04:00
Richard van der Hoff
1ec688bf21
Downgrade warning on client disconnect to INFO ()
Clients disconnecting before we finish processing the request happens from time
to time. We don't need to yell about it
2020-07-24 09:55:47 +01:00
Richard van der Hoff
03619324fc
Create a ListenerConfig object ()
This ended up being a bit more invasive than I'd hoped for (not helped by
generic_worker duplicating some of the code from homeserver), but hopefully
it's an improvement.

The idea is that, rather than storing unstructured `dict`s in the config for
the listener configurations, we instead parse it into a structured
`ListenerConfig` object.
2020-06-16 12:44:07 +01:00
Richard van der Hoff
d84bdfe599
mypy for synapse.http.site () 2020-05-22 10:12:17 +01:00
Andrew Morgan
5611644519
Workaround for failure to wrap reason in Failure () 2020-05-14 17:07:24 +01:00
Richard van der Hoff
60adcbed91
Fix "'NoneType' has no attribute start|stop" logcontext errors ()
Fixes .
2020-03-31 15:18:41 +01:00
Erik Johnston
ed630ea17c
Reduce amount of logging at INFO level. ()
A lot of the things we log at INFO are now a bit superfluous, so lets
make them DEBUG logs to reduce the amount we log by default.

Co-Authored-By: Brendan Abolivier <babolivier@matrix.org>
Co-authored-by: Brendan Abolivier <github@brendanabolivier.com>
2020-02-06 13:31:05 +00:00
Richard van der Hoff
feee819973
Fix exceptions on requests for non-ascii urls ()
Fixes 
2020-01-13 12:41:51 +00:00
Richard van der Hoff
b6b57ecb4e
Kill off redundant SynapseRequestFactory ()
We already get the Site via the Channel, so there's no need for a dedicated
RequestFactory: we can just use the right constructor.
2020-01-03 14:19:48 +00:00
Andrew Morgan
54fef094b3
Remove usage of deprecated logger.warn method from codebase ()
Replace every instance of `logger.warn` with `logger.warning` as the former is deprecated.
2019-10-31 10:23:24 +00:00
Amber Brown
463b072b12
Move logging utilities out of the side drawer of util/ and into logging/ () 2019-07-04 00:07:04 +10:00
Amber Brown
32e7c9e7f2
Run Black. () 2019-06-20 19:32:02 +10:00
Erik Johnston
334e075dd8 Fix error when logging incomplete requests
If a connection is lost before a request is read from Request, Twisted
sets `method` (and `uri`) attributes to dummy values. These dummy values
have incorrect types (i.e. they're not bytes), and so things like
`__repr__` would raise an exception.

To fix this we had a helper method to return the method with a
consistent type.
2018-10-02 12:06:22 +01:00
Amber Brown
1f3f5fcf52
Fix client IPs being broken on Python 3 () 2018-09-20 20:14:34 +10:00
Amber Brown
aeca5a5ed5
Add a regression test for logging on failed connections () 2018-09-20 16:28:18 +10:00
Richard van der Hoff
ac80cb08fe Fix more b'abcd' noise in metrics 2018-09-17 17:16:50 +01:00
Richard van der Hoff
f00a9d2636 Fix some b'abcd' noise in logs and metrics
Python 3 compatibility: make sure that we decode some byte sequences before we
use them to create log lines and metrics labels.
2018-09-17 16:15:42 +01:00
Amber Brown
c971aa7b9d fix 2018-09-14 03:57:02 +10:00
Amber Brown
8f08d848f5 fix 2018-09-14 03:53:56 +10:00