dependabot[bot]
f53a3a56e2
Bump treq from 23.11.0 to 24.9.1 ( #17744 )
...
Bumps [treq](https://github.com/twisted/treq ) from 23.11.0 to 24.9.1.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/twisted/treq/releases ">treq's
releases</a>.</em></p>
<blockquote>
<h2>Treq 24.9.0</h2>
<h2>Features</h2>
<ul>
<li>treq now ships type annotations. (<a
href="https://redirect.github.com/twisted/treq/issues/366 ">#366</a>)</li>
<li>The new <code>treq.cookies</code> module provides helper functions
for working with <code>http.cookiejar.Cookie</code> and
<code>CookieJar</code> objects. (<a
href="https://redirect.github.com/twisted/treq/issues/384 ">#384</a>)</li>
<li>Python 3.13 is now supported. (<a
href="https://redirect.github.com/twisted/treq/issues/391 ">#391</a>)</li>
</ul>
<h2>Bugfixes</h2>
<ul>
<li><code>treq.content.text_content()</code> no longer generates
deprecation warnings due to use of the <code>cgi</code> module. (<a
href="https://redirect.github.com/twisted/treq/issues/355 ">#355</a>)</li>
</ul>
<h2>Deprecations and Removals</h2>
<ul>
<li>Mixing the <em>json</em> argument with <em>files</em> or
<em>data</em> now raises <code>TypeError</code>. (<a
href="https://redirect.github.com/twisted/treq/issues/297 ">#297</a>)</li>
<li>Passing non-string (<code>str</code> or <code>bytes</code>) values
as part of a dict to the <em>headers</em> argument now results in a
<code>TypeError</code>, as does passing any collection other than a
<code>dict</code> or <code>Headers</code> instance. (<a
href="https://redirect.github.com/twisted/treq/issues/302 ">#302</a>)</li>
<li>Support for Python 3.7 and PyPy 3.8, which have reached end of
support, has been dropped. (<a
href="https://redirect.github.com/twisted/treq/issues/378 ">#378</a>)</li>
</ul>
<h2>Misc</h2>
<ul>
<li><a
href="https://redirect.github.com/twisted/treq/issues/336 ">#336</a>, <a
href="https://redirect.github.com/twisted/treq/issues/382 ">#382</a>, <a
href="https://redirect.github.com/twisted/treq/issues/395 ">#395</a></li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/twisted/treq/blob/trunk/CHANGELOG.rst ">treq's
changelog</a>.</em></p>
<blockquote>
<h1>24.9.1 (2024-09-19)</h1>
<h2>Bugfixes</h2>
<ul>
<li>treq has vendored its dependency on the <code>multipart</code>
library to avoid import
conflicts with <code>python-multipart</code>; it should now be
installable alongside
that library. (<code>[#399 ](https://github.com/twisted/treq/issues/399 )
<https://github.com/twisted/treq/issues/399> ;</code>__)</li>
</ul>
<h1>24.9.0 (2024-09-17)</h1>
<h2>Features</h2>
<ul>
<li>treq now ships type annotations.
(<code>[#366 ](https://github.com/twisted/treq/issues/366 )
<https://github.com/twisted/treq/issues/366> ;</code>__)</li>
<li>The new :mod:<code>treq.cookies</code> module provides helper
functions for working with <code>http.cookiejar.Cookie</code> and
<code>CookieJar</code> objects.
(<code>[#384 ](https://github.com/twisted/treq/issues/384 )
<https://github.com/twisted/treq/issues/384> ;</code>__)</li>
<li>Python 3.13 is now supported.
(<code>[#391 ](https://github.com/twisted/treq/issues/391 )
<https://github.com/twisted/treq/issues/391> ;</code>__)</li>
</ul>
<h2>Bugfixes</h2>
<ul>
<li>:mod:<code>treq.content.text_content()</code> no longer generates
deprecation warnings due to use of the <code>cgi</code> module.
(<code>[#355 ](https://github.com/twisted/treq/issues/355 )
<https://github.com/twisted/treq/issues/355> ;</code>__)</li>
</ul>
<h2>Deprecations and Removals</h2>
<ul>
<li>Mixing the <em>json</em> argument with <em>files</em> or
<em>data</em> now raises <code>TypeError</code>.
(<code>[#297 ](https://github.com/twisted/treq/issues/297 )
<https://github.com/twisted/treq/issues/297> ;</code>__)</li>
<li>Passing non-string (<code>str</code> or <code>bytes</code>) values
as part of a dict to the <em>headers</em> argument now results in a
<code>TypeError</code>, as does passing any collection other than a
<code>dict</code> or <code>Headers</code> instance.
(<code>[#302 ](https://github.com/twisted/treq/issues/302 )
<https://github.com/twisted/treq/issues/302> ;</code>__)</li>
<li>Support for Python 3.7 and PyPy 3.8, which have reached end of
support, has been dropped.
(<code>[#378 ](https://github.com/twisted/treq/issues/378 )
<https://github.com/twisted/treq/issues/378> ;</code>__)</li>
</ul>
<h2>Misc</h2>
<ul>
<li><code>[#336 ](https://github.com/twisted/treq/issues/336 )
<https://github.com/twisted/treq/issues/336> ;</code><strong>,
<code>[#382 ](https://github.com/twisted/treq/issues/382 )
<https://github.com/twisted/treq/issues/382> ;</code></strong>,
<code>[#395 ](https://github.com/twisted/treq/issues/395 )
<https://github.com/twisted/treq/issues/395> ;</code>__</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="caaf9fcb629cedb088b4https://redirect.github.com/twisted/treq/issues/400 ">#400</a> from
twisted/vendor-multipart-for-now</li>
<li><a
href="4aa1ee8a3cd7c16de8f54fd3c842c2f0a5148cba7f16b87f0a1526431a37https://github.com/defnull/multipa ">https://github.com/defnull/multipa </a>...</li>
<li><a
href="7c52d4917fca3966f57ahttps://redirect.github.com/twisted/treq/issues/398 ">#398</a> from
twisted/397-release-24.9.0</li>
<li>Additional commits viewable in <a
href="https://github.com/twisted/treq/compare/release-23.11.0...treq-24.9.1 ">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores )
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Quentin Gliech <quenting@element.io>
2024-09-25 11:19:03 +02:00
V02460
2fc43e4219
Remove the deprecated cgi module ( #17741 )
...
Removes all uses of the `cgi` module from Synapse. It was deprecated in
Python version 3.11 and removed in version 3.13 ([“dead
battery”](https://docs.python.org/3.13/whatsnew/3.13.html#pep-594-remove-dead-batteries-from-the-standard-library )).
### Pull Request Checklist
<!-- Please read
https://element-hq.github.io/synapse/latest/development/contributing_guide.html
before submitting your pull request -->
* [x] Pull request is based on the develop branch
* [x] Pull request includes a [changelog
file](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#changelog ).
The entry should:
- Be a short description of your change which makes sense to users.
"Fixed a bug that prevented receiving messages from other servers."
instead of "Moved X method from `EventStore` to `EventWorkerStore`.".
- Use markdown where necessary, mostly for `code blocks`.
- End with either a period (.) or an exclamation mark (!).
- Start with a capital letter.
- Feel free to credit yourself, by adding a sentence "Contributed by
@github_username." or "Contributed by [Your Name]." to the end of the
entry.
* [x] [Code
style](https://element-hq.github.io/synapse/latest/code_style.html ) is
correct
(run the
[linters](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-linters ))
---------
Co-authored-by: Quentin Gliech <quenting@element.io>
2024-09-25 11:15:34 +02:00
Éloi Rivard
ebad618bf0
import pydantic objects from the _pydantic_compat module ( #17667 )
...
This PR changes `from pydantic import BaseModel` to `from
synapse._pydantic_compat import BaseModel` (as well as `constr`,
`conbytes`, `conint`, `confloat`).
It allows `check_pydantic_models.py` to mock those pydantic objects only
in the synapse module, and not interfere with pydantic objects in
external dependencies.
This should solve the CI problems for #17144 , which breaks because
`check_pydantic_models.py` patches pydantic models from
[scim2-models](https://scim2-models.readthedocs.io/ ).
/cc @DMRobertson @gotmax23
fixes #17659
### Pull Request Checklist
<!-- Please read
https://element-hq.github.io/synapse/latest/development/contributing_guide.html
before submitting your pull request -->
* [x] Pull request is based on the develop branch
* [x] Pull request includes a [changelog
file](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#changelog ).
The entry should:
- Be a short description of your change which makes sense to users.
"Fixed a bug that prevented receiving messages from other servers."
instead of "Moved X method from `EventStore` to `EventWorkerStore`.".
- Use markdown where necessary, mostly for `code blocks`.
- End with either a period (.) or an exclamation mark (!).
- Start with a capital letter.
- Feel free to credit yourself, by adding a sentence "Contributed by
@github_username." or "Contributed by [Your Name]." to the end of the
entry.
* [x] [Code
style](https://element-hq.github.io/synapse/latest/code_style.html ) is
correct
(run the
[linters](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-linters ))
2024-09-11 21:01:43 +00:00
Quentin Gliech
7d52ce7d4b
Format files with Ruff ( #17643 )
...
I thought ruff check would also format, but it doesn't.
This runs ruff format in CI and dev scripts. The first commit is just a
run of `ruff format .` in the root directory.
2024-09-02 12:39:04 +01:00
Erik Johnston
b21134de3b
Fix starting non-media repos ( #17626 )
...
Regressed in #17543 .
The `max_download_size` config is not available on workers that don't
load the media repo.
Besides, we should honour the max_size param that was passed into the
function.
2024-08-29 12:26:17 +00:00
Till
573c6d7e69
Use max_upload_size as the limit when following the Location header ( #17543 )
...
Otherwise we use the `expected_size` from the initial federation
request, which might be far too low.
### Pull Request Checklist
<!-- Please read
https://element-hq.github.io/synapse/latest/development/contributing_guide.html
before submitting your pull request -->
* [x] Pull request is based on the develop branch
* [x] Pull request includes a [changelog
file](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#changelog ).
The entry should:
- Be a short description of your change which makes sense to users.
"Fixed a bug that prevented receiving messages from other servers."
instead of "Moved X method from `EventStore` to `EventWorkerStore`.".
- Use markdown where necessary, mostly for `code blocks`.
- End with either a period (.) or an exclamation mark (!).
- Start with a capital letter.
- Feel free to credit yourself, by adding a sentence "Contributed by
@github_username." or "Contributed by [Your Name]." to the end of the
entry.
* [x] [Code
style](https://element-hq.github.io/synapse/latest/code_style.html ) is
correct
(run the
[linters](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-linters ))
---------
Co-authored-by: Erik Johnston <erikj@element.io>
2024-08-29 09:25:10 +02:00
Shay
b05b2e14bb
Handle lower-case http headers in _Mulitpart_Parser_Protocol ( #17545 )
2024-08-14 09:49:01 +01:00
Erik Johnston
aaa3c36420
Remove logging in multipart ( #17563 )
...
This is really spurious and causes a lot of spam. I don't think there is
a use for it even at DEBUG level.
2024-08-13 15:56:18 +01:00
Erik Johnston
9f9ec92526
Speed up responding to media requests ( #17558 )
...
We do this by reading from a threadpool, rather than blocking the main
thread.
This is broadly what we do in the [S3 storage
provider](https://github.com/matrix-org/synapse-s3-storage-provider/blob/main/s3_storage_provider.py#L234 )
2024-08-13 14:06:17 +01:00
Olivier 'reivilibre
53db8a914e
Merge branch 'master' into develop
2024-07-30 17:10:46 +01:00
reivilibre
c56b070e6f
Upgrade locked dependency on Twisted to 24.7.0rc1. ( #17502 )
...
I also update the tests and HTTP Proxy code to fix it for this new
Twisted release.
Pulls in fix for
https://github.com/twisted/twisted/security/advisories/GHSA-c8m8-j448-xjx7
Signed-off-by: Olivier 'reivilibre <oliverw@matrix.org>
2024-07-30 15:23:23 +01:00
Shay
429ecb7564
Handle remote download responses with UNKNOWN_LENGTH more gracefully ( #17439 )
...
Prior to this PR, remote downloads which did not provide a
`content-length` were decremented from the remote download ratelimiter
at the max allowable size, leading to excessive ratelimiting - see
https://github.com/element-hq/synapse/issues/17394 .
This PR adds a linearizer to limit concurrent remote downloads to 6 per
IP address, and decrements remote downloads without a `content-length`
from the ratelimiter *after* the download is complete and the response
length is known.
Also adds logic to ensure that responses with a known length respect the
`max_download_size`.
2024-07-16 11:13:55 +01:00
Shay
8f890447b0
Support MSC3916 by adding _matrix/client/v1/media/download endpoint ( #17365 )
2024-07-02 14:07:04 +01:00
Denis Kasak
700d2cc4a0
Tidy up integer parsing ( #17339 )
...
The parse_integer function was previously made to reject negative values by
default in https://github.com/element-hq/synapse/pull/16920 , but the
documentation stated otherwise. This fixes the documentation and also:
- Removes explicit negative=False parameters from call sites.
- Brings the negative default of parse_integer_from_args in alignment with
parse_integer.
2024-06-24 15:12:14 +02:00
Shay
fcbc79bb87
Ratelimiting of remote media downloads ( #17256 )
2024-06-05 13:43:36 +01:00
Quentin Gliech
2e92b718d5
MSC4108 implementation ( #17056 )
...
Co-authored-by: Hugh Nimmo-Smith <hughns@element.io>
Co-authored-by: Hugh Nimmo-Smith <hughns@users.noreply.github.com>
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2024-04-25 12:50:12 +00:00
Gordan Trevis
1d47532310
Parse json validation ( #16923 )
...
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2024-04-18 13:57:38 +01:00
Quentin Gliech
c8e0bed426
Support for MSC4108 via delegation ( #17086 )
...
This adds support for MSC4108 via delegation, similar to what has been done for MSC3886
---------
Co-authored-by: Hugh Nimmo-Smith <hughns@element.io>
2024-04-17 16:47:35 +02:00
Gordan Trevis
f0d6f14047
Parse Integer negative value validation ( #16920 )
2024-04-16 19:12:36 +00:00
Erik Johnston
3a30846bd0
Fix mypy on latest Twisted release ( #17036 )
...
`ITransport.abortConnection` isn't a thing, but
`HTTPChannel.forceAbortClient` calls it, so lets just use that
Fixes https://github.com/element-hq/synapse/issues/16728
2024-04-11 16:03:45 +01:00
Erik Johnston
c900d18647
Fix OIDC login regression ( #17031 )
...
Requests may require a User-Agent header, and the change in #16972
accidentally removed it, resulting in requests getting rejected causing
login to fail.
2024-03-26 13:26:46 +00:00
dependabot[bot]
1e68b56a62
Bump black from 23.10.1 to 24.2.0 ( #16936 )
2024-03-13 16:46:44 +00:00
Erik Johnston
23740eaa3d
Correctly mention previous copyright ( #16820 )
...
During the migration the automated script to update the copyright
headers accidentally got rid of some of the existing copyright lines.
Reinstate them.
2024-01-23 11:26:48 +00:00
FadhlanR
eabedd9520
Fix email verification redirection ( #16761 )
...
Previously, the response status of `HTMLResource` was hardcoded as
`200`. However, for proper redirection after the user verifies their
email, we require the status to be `302`. This PR addresses that issue
by using `code` as response status.
2024-01-02 16:25:26 +00:00
Erik Johnston
eaad9bb156
Merge remote-tracking branch 'gitlab/clokep/license-license' into new_develop
2023-12-13 15:11:56 +00:00
Patrick Cloke
d6c3b7584f
Request & follow redirects for /media/v3/download ( #16701 )
...
Implement MSC3860 to follow redirects for federated media downloads.
Note that the Client-Server API doesn't support this (yet) since the media
repository in Synapse doesn't have a way of supporting redirects.
2023-11-29 19:03:42 +00:00
Patrick Cloke
8e1e62c9e0
Update license headers
2023-11-21 15:29:58 -05:00
Patrick Cloke
f2f2c7c1f0
Use full GitHub links instead of bare issue numbers. ( #16637 )
2023-11-15 08:02:11 -05:00
Richard Brežák
95076f77c1
Fix http/s proxy authentication with long username/passwords ( #16504 )
2023-10-24 13:45:21 +00:00
Patrick Cloke
26b960b08b
Register media servlets via regex. ( #16419 )
...
This converts the media servlet URLs in the same way as
(most) of the rest of Synapse. This will give more flexibility
in the versions each endpoint exists under.
2023-10-06 07:22:55 -04:00
Maxwell G
12611bfcdd
Add support for pydantic v2 via pydantic.v1 compat module ( #16332 )
...
While maintaining support with pydantic v1.
2023-09-25 15:19:08 +00:00
Patrick Cloke
d38d0dffc9
Use StrCollection in additional places. ( #16301 )
2023-09-13 07:57:19 -04:00
Travis Ralston
b1d71c687a
Add MSC4040 matrix-fed service lookups ( #16137 )
2023-09-05 15:45:39 -04:00
Erik Johnston
f84baecb6f
Don't reset retry timers on "valid" error codes ( #16221 )
2023-09-04 14:04:43 +01:00
David Robertson
62a1a9be52
Describe which rate limiter was hit in logs ( #16135 )
2023-08-30 00:39:39 +01:00
dependabot[bot]
001fc7bd19
Bump ruff from 0.0.277 to 0.0.286 ( #16198 )
...
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-29 09:41:43 -04:00
Erik Johnston
fd44053b84
Don't log exceptions for every non-200 response ( #15969 )
...
Introduced in #15913
2023-07-20 11:07:58 +01:00
Eric Eastwood
1c802de626
Re-introduce the outbound federation proxy ( #15913 )
...
Allow configuring the set of workers to proxy outbound federation traffic through (`outbound_federation_restricted_to`).
This is useful when you have a worker setup with `federation_sender` instances responsible for sending outbound federation requests and want to make sure *all* outbound federation traffic goes through those instances. Before this change, the generic workers would still contact federation themselves for things like profile lookups, backfill, etc. This PR allows you to set more strict access controls/firewall for all workers and only allow the `federation_sender`'s to contact the outside world.
2023-07-18 09:49:21 +01:00
Jason Little
224ef0b669
Unix Sockets for HTTP Replication ( #15708 )
...
Unix socket support for `federation` and `client` Listeners has existed now for a little while(since [1.81.0](https://github.com/matrix-org/synapse/pull/15353 )), but there was one last hold out before it could be complete: HTTP Replication communication. This should finish it up. The Listeners would have always worked, but would have had no way to be talked to/at.
---------
Co-authored-by: Eric Eastwood <madlittlemods@gmail.com>
Co-authored-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>
Co-authored-by: Eric Eastwood <erice@element.io>
2023-07-11 13:08:06 -05:00
Michael Telatynski
b516d91999
Add Server to Access-Control-Expose-Headers header ( #15908 )
2023-07-11 09:18:50 +01:00
Eric Eastwood
c9bf644fa0
Revert "Federation outbound proxy" ( #15910 )
...
Revert "Federation outbound proxy (#15773 )"
This reverts commit b07b14b494
2023-07-10 11:10:20 -05:00
Eric Eastwood
b07b14b494
Federation outbound proxy ( #15773 )
...
Allow configuring the set of workers to proxy outbound federation traffic through (`outbound_federation_restricted_to`).
This is useful when you have a worker setup with `federation_sender` instances responsible for sending outbound federation requests and want to make sure *all* outbound federation traffic goes through those instances. Before this change, the generic workers would still contact federation themselves for things like profile lookups, backfill, etc. This PR allows you to set more strict access controls/firewall for all workers and only allow the `federation_sender`'s to contact the outside world.
The original code is from @erikjohnston's branches which I've gotten in-shape to merge.
2023-07-05 18:53:55 -05:00
Erik Johnston
25c55a9d22
Add login spam checker API ( #15838 )
2023-06-26 14:12:20 +00:00
Mathieu Velten
496f73103d
Allow for the configuration of max request retries and min/max retry delays in the matrix federation client ( #15783 )
2023-06-21 10:41:11 +02:00
Eric Eastwood
887fa4b66b
Switch from matrix:// to matrix-federation:// scheme for internal Synapse routing of outbound federation traffic ( #15806 )
...
`matrix://` is a registered specced scheme nowadays and doesn't make sense for
our internal to Synapse use case anymore. ([discussion]
(https://github.com/matrix-org/synapse/pull/15773#discussion_r1227598679 ))
2023-06-20 10:05:31 +01:00
Mathieu Velten
ef0d3d7bd9
Revert "Allow for the configuration of max request retries and min/max retry delays in the matrix federation client ( #12504 )"
...
This reverts commit d84e66144d
2023-06-14 11:55:57 +02:00
Shay
d84e66144d
Allow for the configuration of max request retries and min/max retry delays in the matrix federation client ( #12504 )
...
Co-authored-by: Mathieu Velten <mathieuv@matrix.org>
Co-authored-by: Erik Johnston <erik@matrix.org>
2023-06-09 09:00:46 +02:00
Erik Johnston
f6321e386c
Merge branch 'master' into develop
2023-06-08 13:16:46 +01:00
Erik Johnston
733342ad3e
Fix using TLS for replication ( #15746 )
...
Fixes #15744 .
2023-06-08 13:03:48 +01:00
Eric Eastwood
33c3550887
Add context for when/why to use the long_retries option when sending Federation requests ( #15721 )
2023-06-06 16:25:03 -05:00
