Matrix-Mirrors/synapse-product
1
0
Fork 0
mirror of https://git.anonymousland.org/anonymousland/synapse-product.git synced 2024-12-21 03:34:22 -05:00
Code Issues Packages Projects Releases Wiki Activity
19,361 Commits 4 Branches 609 Tags 60 MiB
f455b0e420
Commit Graph

13606 Commits

Author SHA1 Message Date
Erik Johnston
92b6ac31b2
Speed up MultiWriterIdGenerator when lots of IDs are in flight. (#10755) 2021-09-03 18:23:46 +01:00
Patrick Cloke
ae3c16318b
Support MSC3375: room version 9. (#10747) 2021-09-03 12:51:15 -04:00
Sean
924276f482
Add a partial index to presence_stream to speed up startups (#10748) 
Signed-off-by: Sean Quah <seanq@element.io>
 2021-09-03 17:16:56 +01:00
Erik Johnston
0eae330a26
Use execute_values more in PostgreSQL (#10754) 
`execute_values` is a faster version of `execute_batch`.
 2021-09-03 15:35:49 +00:00
Patrick Cloke
2cb85bdf75
Raise an error if an unknown preset is used to create a room. (#10738) 
Raises a 400 error instead of a 500 if an unknown preset is passed
from a client to create a room.
 2021-09-03 13:46:18 +00:00
Patrick Cloke
ecbfa4fe4f
Additional type hints for client REST servlets (part 5) (#10736) 
Additionally this enforce type hints on all function signatures inside
of the synapse.rest.client package.
 2021-09-03 09:22:22 -04:00
Erik Johnston
f58d202e3f
Fix bug with reusing 'txn' when persisting event. (#10743) 
This will only happen when a server has multiple out of band membership
events in a single room.
 2021-09-03 10:59:25 +01:00
Patrick Cloke
c586d6803a
Ignore rooms with unknown room versions in the spaces summary. (#10727) 
This avoids breaking the entire endpoint if a room with
an unsupported room version is encountered.
 2021-09-01 17:01:08 +00:00
Patrick Cloke
6258730ebe
Consider the origin_server_ts of the m.space.child event when ordering rooms. (#10730) 
This updates the ordering of the returned events from the spaces
summary API to that defined in MSC2946 (which updates MSC1772).

Previously a step was skipped causing ordering to be inconsistent with
clients.
 2021-09-01 12:59:52 -04:00
Patrick Cloke
d1f1b46c2c
Additional type hints for client REST servlets (part 4) (#10728) 2021-09-01 11:59:32 -04:00
Eric Eastwood
dc75fb7f05
Populate rooms.creator field for easy lookup (#10697) 
Part of https://github.com/matrix-org/synapse/pull/10566

 - Fill in creator whenever we insert into the rooms table
 - Add background update to backfill any missing creator values
 2021-09-01 16:27:58 +01:00
Olivier Wilkinson (reivilibre)
e059094119 Synapse 1.42.0rc1 (2021-09-01) 
==============================
 
 Server administrators are reminded to read [the upgrade notes](docs/upgrade.md#upgrading-to-v1420).
 
 Features
 --------
 
 - Add support for [MSC3231](https://github.com/matrix-org/matrix-doc/pull/3231): Token authenticated registration. Users can be required to submit a token during registration to authenticate themselves. Contributed by Callum Brown. ([\#10142](https://github.com/matrix-org/synapse/issues/10142))
 - Add support for [MSC3283](https://github.com/matrix-org/matrix-doc/pull/3283): Expose `enable_set_displayname` in capabilities. ([\#10452](https://github.com/matrix-org/synapse/issues/10452))
 - Port the `PresenceRouter` module interface to the new generic interface. ([\#10524](https://github.com/matrix-org/synapse/issues/10524))
 - Add pagination to the spaces summary based on updates to [MSC2946](https://github.com/matrix-org/matrix-doc/pull/2946). ([\#10613](https://github.com/matrix-org/synapse/issues/10613), [\#10725](https://github.com/matrix-org/synapse/issues/10725))
 
 Bugfixes
 --------
 
 - Validate new `m.room.power_levels` events. Contributed by @aaronraimist. ([\#10232](https://github.com/matrix-org/synapse/issues/10232))
 - Display an error on User-Interactive Authentication fallback pages when authentication fails. Contributed by Callum Brown. ([\#10561](https://github.com/matrix-org/synapse/issues/10561))
 - Remove pushers when deleting an e-mail address from an account. Pushers for old unlinked emails will also be deleted. ([\#10581](https://github.com/matrix-org/synapse/issues/10581), [\#10734](https://github.com/matrix-org/synapse/issues/10734))
 - Reject Client-Server `/keys/query` requests which provide `device_ids` incorrectly. ([\#10593](https://github.com/matrix-org/synapse/issues/10593))
 - Rooms with unsupported room versions are no longer returned via `/sync`. ([\#10644](https://github.com/matrix-org/synapse/issues/10644))
 - Enforce the maximum length for per-room display names and avatar URLs. ([\#10654](https://github.com/matrix-org/synapse/issues/10654))
 - Fix a bug which caused the `synapse_user_logins_total` Prometheus metric not to be correctly initialised on restart. ([\#10677](https://github.com/matrix-org/synapse/issues/10677))
 - Improve `ServerNoticeServlet` to avoid duplicate requests and add unit tests. ([\#10679](https://github.com/matrix-org/synapse/issues/10679))
 - Fix long-standing issue which caused an error when a thumbnail is requested and there are multiple thumbnails with the same quality rating. ([\#10684](https://github.com/matrix-org/synapse/issues/10684))
 - Fix a regression introduced in v1.41.0 which affected the performance of concurrent fetches of large sets of events, in extreme cases causing the process to hang. ([\#10703](https://github.com/matrix-org/synapse/issues/10703))
 - Fix a regression introduced in Synapse 1.41 which broke email transmission on Systems using older versions of the Twisted library. ([\#10713](https://github.com/matrix-org/synapse/issues/10713))
 
 Improved Documentation
 ----------------------
 
 - Add documentation on how to connect Django with Synapse using OpenID Connect and django-oauth-toolkit. Contributed by @HugoDelval. ([\#10192](https://github.com/matrix-org/synapse/issues/10192))
 - Advertise https://matrix-org.github.io/synapse documentation in the `README` and `CONTRIBUTING` files. ([\#10595](https://github.com/matrix-org/synapse/issues/10595))
 - Fix some of the titles not rendering in the OpenID Connect documentation. ([\#10639](https://github.com/matrix-org/synapse/issues/10639))
 - Minor clarifications to the documentation for reverse proxies. ([\#10708](https://github.com/matrix-org/synapse/issues/10708))
 - Remove table of contents from the top of installation and contributing documentation pages. ([\#10711](https://github.com/matrix-org/synapse/issues/10711))
 
 Deprecations and Removals
 -------------------------
 
 - Remove deprecated Shutdown Room and Purge Room Admin API. ([\#8830](https://github.com/matrix-org/synapse/issues/8830))
 
 Internal Changes
 ----------------
 
 - Improve type hints for the proxy agent and SRV resolver modules. Contributed by @dklimpel. ([\#10608](https://github.com/matrix-org/synapse/issues/10608))
 - Clean up some of the federation event authentication code for clarity. ([\#10614](https://github.com/matrix-org/synapse/issues/10614), [\#10615](https://github.com/matrix-org/synapse/issues/10615), [\#10624](https://github.com/matrix-org/synapse/issues/10624), [\#10640](https://github.com/matrix-org/synapse/issues/10640))
 - Add a comment asking developers to leave a reason when bumping the database schema version. ([\#10621](https://github.com/matrix-org/synapse/issues/10621))
 - Remove not needed database updates in modify user admin API. ([\#10627](https://github.com/matrix-org/synapse/issues/10627))
 - Convert room member storage tuples to `attrs` classes. ([\#10629](https://github.com/matrix-org/synapse/issues/10629), [\#10642](https://github.com/matrix-org/synapse/issues/10642))
 - Use auto-attribs for the attrs classes used in sync. ([\#10630](https://github.com/matrix-org/synapse/issues/10630))
 - Make `backfill` and `get_missing_events` use the same codepath. ([\#10645](https://github.com/matrix-org/synapse/issues/10645))
 - Improve the performance of the `/hierarchy` API (from [MSC2946](https://github.com/matrix-org/matrix-doc/pull/2946)) by caching responses received over federation. ([\#10647](https://github.com/matrix-org/synapse/issues/10647))
 - Run a nightly CI build against Twisted trunk. ([\#10651](https://github.com/matrix-org/synapse/issues/10651), [\#10672](https://github.com/matrix-org/synapse/issues/10672))
 - Do not print out stack traces for network errors when fetching data over federation. ([\#10662](https://github.com/matrix-org/synapse/issues/10662))
 - Simplify tests for device admin rest API. ([\#10664](https://github.com/matrix-org/synapse/issues/10664))
 - Add missing type hints to REST servlets. ([\#10665](https://github.com/matrix-org/synapse/issues/10665), [\#10666](https://github.com/matrix-org/synapse/issues/10666), [\#10674](https://github.com/matrix-org/synapse/issues/10674))
 - Flatten the `tests.synapse.rests` package by moving the contents of `v1` and `v2_alpha` into the parent. ([\#10667](https://github.com/matrix-org/synapse/issues/10667))
 - Update `complement.sh` to rebuild the base Docker image when run with workers. ([\#10686](https://github.com/matrix-org/synapse/issues/10686))
 - Split the event-processing methods in `FederationHandler` into a separate `FederationEventHandler`. ([\#10692](https://github.com/matrix-org/synapse/issues/10692))
 - Remove unused `compare_digest` function. ([\#10706](https://github.com/matrix-org/synapse/issues/10706))
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCgAdFiEE8SRSDO7gYkSP4chELS76LzL74EcFAmEvd3cACgkQLS76LzL7
 4EcF0hAAvOjDeH2hn0eI4YX03/BTCesJ5pCTbEJeC9Hsui3SbWamyYarH0Ae0gM6
 BP9x5/XkCdsZhKT2OJedtZQUQ0cUws5FdJP87NnE1m7KBW9djbTxmkue3JKaHNTT
 ajvEoLe8UjOFbjvyMc+188uWVgtKQ6KLcetwllTpYY3FJ3+yk9CQP1ydYzwfHcHe
 hXW3b20WoRdAmH0B27mj1oQzCeXclbj1kCthcBcxH5RytA8bBc1itBtpAP2FFaeS
 Ml6paoXSs0wZaXXL5T1MhQtcVOOG5Yh7EPsemdyfqkwhIk9QtV3nmGfC7K5iEeq0
 DldTgBNE0wLO+ycdgEZ2oCvitCLT4WDoKkVqKPA91wpuvCMeK5lnDdqbFUSN3hfd
 l8Oz+ydhjsCx7+pBBopnsyGrPIHNaN/vySYGF3lPa3ANKS2J0IVlAaJ59bZqzg2C
 vzdyMBCnn0fhnXMkADPLnIYqtebvC09JlPBRGo5BKmwmlM8Ea1dLQ+Lcy5vXalcw
 +GaYw3U805c03EvsWQxWchjwq6IwMeP06YpK/1ruMjlrCWYJp3V7f/4hXjlTdYm5
 OEWc/kHOnlLfH2cA4aS0sV5WsnEuftotQIzBXuXWw5hSMdFmH7eAviPg0vXgfeA/
 i0FOkOgEYR8O3RMh4Cip8+h44t8SPNjYUFJK9ip3DmNateeUCH0=
 =XxMc
 -----END PGP SIGNATURE-----

Merge tag 'v1.42.0rc1' into develop

Synapse 1.42.0rc1 (2021-09-01)
==============================

Server administrators are reminded to read [the upgrade notes](docs/upgrade.md#upgrading-to-v1420).

Features
--------

- Add support for [MSC3231](https://github.com/matrix-org/matrix-doc/pull/3231): Token authenticated registration. Users can be required to submit a token during registration to authenticate themselves. Contributed by Callum Brown. ([\#10142](https://github.com/matrix-org/synapse/issues/10142))
- Add support for [MSC3283](https://github.com/matrix-org/matrix-doc/pull/3283): Expose `enable_set_displayname` in capabilities. ([\#10452](https://github.com/matrix-org/synapse/issues/10452))
- Port the `PresenceRouter` module interface to the new generic interface. ([\#10524](https://github.com/matrix-org/synapse/issues/10524))
- Add pagination to the spaces summary based on updates to [MSC2946](https://github.com/matrix-org/matrix-doc/pull/2946). ([\#10613](https://github.com/matrix-org/synapse/issues/10613), [\#10725](https://github.com/matrix-org/synapse/issues/10725))

Bugfixes
--------

- Validate new `m.room.power_levels` events. Contributed by @aaronraimist. ([\#10232](https://github.com/matrix-org/synapse/issues/10232))
- Display an error on User-Interactive Authentication fallback pages when authentication fails. Contributed by Callum Brown. ([\#10561](https://github.com/matrix-org/synapse/issues/10561))
- Remove pushers when deleting an e-mail address from an account. Pushers for old unlinked emails will also be deleted. ([\#10581](https://github.com/matrix-org/synapse/issues/10581), [\#10734](https://github.com/matrix-org/synapse/issues/10734))
- Reject Client-Server `/keys/query` requests which provide `device_ids` incorrectly. ([\#10593](https://github.com/matrix-org/synapse/issues/10593))
- Rooms with unsupported room versions are no longer returned via `/sync`. ([\#10644](https://github.com/matrix-org/synapse/issues/10644))
- Enforce the maximum length for per-room display names and avatar URLs. ([\#10654](https://github.com/matrix-org/synapse/issues/10654))
- Fix a bug which caused the `synapse_user_logins_total` Prometheus metric not to be correctly initialised on restart. ([\#10677](https://github.com/matrix-org/synapse/issues/10677))
- Improve `ServerNoticeServlet` to avoid duplicate requests and add unit tests. ([\#10679](https://github.com/matrix-org/synapse/issues/10679))
- Fix long-standing issue which caused an error when a thumbnail is requested and there are multiple thumbnails with the same quality rating. ([\#10684](https://github.com/matrix-org/synapse/issues/10684))
- Fix a regression introduced in v1.41.0 which affected the performance of concurrent fetches of large sets of events, in extreme cases causing the process to hang. ([\#10703](https://github.com/matrix-org/synapse/issues/10703))
- Fix a regression introduced in Synapse 1.41 which broke email transmission on Systems using older versions of the Twisted library. ([\#10713](https://github.com/matrix-org/synapse/issues/10713))

Improved Documentation
----------------------

- Add documentation on how to connect Django with Synapse using OpenID Connect and django-oauth-toolkit. Contributed by @HugoDelval. ([\#10192](https://github.com/matrix-org/synapse/issues/10192))
- Advertise https://matrix-org.github.io/synapse documentation in the `README` and `CONTRIBUTING` files. ([\#10595](https://github.com/matrix-org/synapse/issues/10595))
- Fix some of the titles not rendering in the OpenID Connect documentation. ([\#10639](https://github.com/matrix-org/synapse/issues/10639))
- Minor clarifications to the documentation for reverse proxies. ([\#10708](https://github.com/matrix-org/synapse/issues/10708))
- Remove table of contents from the top of installation and contributing documentation pages. ([\#10711](https://github.com/matrix-org/synapse/issues/10711))

Deprecations and Removals
-------------------------

- Remove deprecated Shutdown Room and Purge Room Admin API. ([\#8830](https://github.com/matrix-org/synapse/issues/8830))

Internal Changes
----------------

- Improve type hints for the proxy agent and SRV resolver modules. Contributed by @dklimpel. ([\#10608](https://github.com/matrix-org/synapse/issues/10608))
- Clean up some of the federation event authentication code for clarity. ([\#10614](https://github.com/matrix-org/synapse/issues/10614), [\#10615](https://github.com/matrix-org/synapse/issues/10615), [\#10624](https://github.com/matrix-org/synapse/issues/10624), [\#10640](https://github.com/matrix-org/synapse/issues/10640))
- Add a comment asking developers to leave a reason when bumping the database schema version. ([\#10621](https://github.com/matrix-org/synapse/issues/10621))
- Remove not needed database updates in modify user admin API. ([\#10627](https://github.com/matrix-org/synapse/issues/10627))
- Convert room member storage tuples to `attrs` classes. ([\#10629](https://github.com/matrix-org/synapse/issues/10629), [\#10642](https://github.com/matrix-org/synapse/issues/10642))
- Use auto-attribs for the attrs classes used in sync. ([\#10630](https://github.com/matrix-org/synapse/issues/10630))
- Make `backfill` and `get_missing_events` use the same codepath. ([\#10645](https://github.com/matrix-org/synapse/issues/10645))
- Improve the performance of the `/hierarchy` API (from [MSC2946](https://github.com/matrix-org/matrix-doc/pull/2946)) by caching responses received over federation. ([\#10647](https://github.com/matrix-org/synapse/issues/10647))
- Run a nightly CI build against Twisted trunk. ([\#10651](https://github.com/matrix-org/synapse/issues/10651), [\#10672](https://github.com/matrix-org/synapse/issues/10672))
- Do not print out stack traces for network errors when fetching data over federation. ([\#10662](https://github.com/matrix-org/synapse/issues/10662))
- Simplify tests for device admin rest API. ([\#10664](https://github.com/matrix-org/synapse/issues/10664))
- Add missing type hints to REST servlets. ([\#10665](https://github.com/matrix-org/synapse/issues/10665), [\#10666](https://github.com/matrix-org/synapse/issues/10666), [\#10674](https://github.com/matrix-org/synapse/issues/10674))
- Flatten the `tests.synapse.rests` package by moving the contents of `v1` and `v2_alpha` into the parent. ([\#10667](https://github.com/matrix-org/synapse/issues/10667))
- Update `complement.sh` to rebuild the base Docker image when run with workers. ([\#10686](https://github.com/matrix-org/synapse/issues/10686))
- Split the event-processing methods in `FederationHandler` into a separate `FederationEventHandler`. ([\#10692](https://github.com/matrix-org/synapse/issues/10692))
- Remove unused `compare_digest` function. ([\#10706](https://github.com/matrix-org/synapse/issues/10706))
 2021-09-01 14:58:14 +01:00
David Robertson
d9069388f3
Correctly include room avatars in email notifications (#10658) 
Judging by the template, this was intended ages ago, but we never
actually passed an avatar URL to the template. So let's provide one.

Closes #1546.

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
 2021-09-01 13:48:41 +01:00
Sean
f8bf83b811
Skip the final GC on shutdown to improve restart times (#10712) 
Use `gc.freeze()` on exit to exclude all existing objects from the final GC.
In testing, this sped up shutdown by up to a few seconds.

`gc.freeze()` runs in constant time, so there is little chance of performance
regression.

Signed-off-by: Sean Quah <seanq@element.io>
 2021-09-01 11:55:31 +01:00
Olivier Wilkinson (reivilibre)
6b2aca473a 1.42.0rc1 2021-09-01 11:47:24 +01:00
Andrew Morgan
3693ea61f5
Fix iteration in _remove_deleted_email_pushers background job. (#10734) 2021-09-01 09:13:01 +00:00
Patrick Cloke
e2481dbe93
Allow configuration of the oEmbed URLs. (#10714) 
This adds configuration options (under an `oembed` section) to
configure which URLs are matched to use oEmbed for URL
previews.
 2021-08-31 18:37:07 -04:00
Patrick Cloke
287918e2d4
Additional type hints for the client REST servlets (part 3). (#10707) 2021-08-31 17:22:29 +00:00
Patrick Cloke
78e590d473
Move the sessions delta to the latest schema version. (#10725) 
This was erroneously put under schema version 62 instead of 63.
 2021-08-31 16:38:43 +00:00
Richard van der Hoff
5d9e7e0c71 Merge branch 'master' into develop 2021-08-31 14:09:48 +01:00
Richard van der Hoff
a4c8a2f08b 1.41.1 2021-08-31 13:43:28 +01:00
reivilibre
cb35df940a
Merge pull request from GHSA-jj53-8fmw-f2w2 2021-08-31 11:24:09 +01:00
reivilibre
52c7a51cfc
Merge pull request from GHSA-3x4c-pq33-4w3q 
* Add some tests to characterise the problem

Some failing. Current states:

  RoomsMemberListTestCase
test_get_member_list ...
[OK]
test_get_member_list_mixed_memberships ...
[OK]
test_get_member_list_no_permission ...
[OK]
test_get_member_list_no_permission_former_member ...
[OK]
test_get_member_list_no_permission_former_member_with_at_token ...
[FAIL]
test_get_member_list_no_room ...
[OK]
test_get_member_list_no_permission_with_at_token ...
[FAIL]

* Correct the tests

* Check user is/was member before divulging room membership

* Pull out only the 1 membership event we want.

* Update tests/rest/client/v1/test_rooms.py

Co-authored-by: Erik Johnston <erik@matrix.org>

* Fixup tests (following apply review suggestion)

Co-authored-by: Erik Johnston <erik@matrix.org>
 2021-08-31 10:09:58 +01:00
Richard van der Hoff
e3abc0a5cc Merge remote-tracking branch 'origin/release-v1.41' into develop 2021-08-27 16:35:02 +01:00
Richard van der Hoff
8f98260552
Fix incompatibility with Twisted < 21. (#10713) 
Turns out that the functionality added in #10546 to skip TLS was incompatible
with older Twisted versions, so we need to be a bit more inventive.

Also, add a test to (hopefully) not break this in future. Sadly, testing TLS is
really hard.
 2021-08-27 16:33:41 +01:00
Patrick Cloke
029b7ad7b9
Remove unused compare_digest function. (#10706) 2021-08-27 07:08:02 -04:00
Dirk Klimpel
e62cdbef1a
Improve ServerNoticeServlet to avoid duplicate requests (#10679) 
Fixes: #9544
 2021-08-27 09:16:40 +00:00
Erik Johnston
c4fa4f37cb
Fix perf of fetching the same events many times. (#10703) 
The code to deduplicate repeated fetches of the same set of events was
N^2 (over the number of events requested), which could lead to a process
being completely wedged.

The main fix is to deduplicate the returned deferreds so we only await
on a deferred once rather than many times. Seperately, when handling the
returned events from the defrered we only add the events we care about
to the event map to be returned (so that we don't pay the price of
inserting extraneous events into the dict).
 2021-08-27 09:15:50 +00:00
Richard van der Hoff
1800aabfc2
Split FederationHandler in half (#10692) 
The idea here is to take anything to do with incoming events and move it out to a separate handler, as a way of making FederationHandler smaller.
 2021-08-26 21:41:44 +01:00
Richard van der Hoff
96715d7633
Make backfill and get_missing_events use the same codepath (#10645) 
Given that backfill and get_missing_events are basically the same thing, it's somewhat crazy that we have entirely separate code paths for them. This makes backfill use the existing get_missing_events code, and then clears up all the unused code.
 2021-08-26 18:34:57 +01:00
Aaron Raimist
40f619eaa5
Validate new m.room.power_levels events (#10232) 
Signed-off-by: Aaron Raimist <aaron@raim.ist>
 2021-08-26 17:07:58 +01:00
Azrenbeth
ad17fbd20e
Remove pushers when deleting 3pid from account (#10581) 
When a user deletes an email from their account it will
now also remove all pushers for that email and that user
(even if these pushers were created by a different client)
 2021-08-26 13:53:57 +01:00
Patrick Cloke
1aa0dad021
Additional type hints for REST servlets (part 2). (#10674) 
Applies the changes from #10665 to additional modules.
 2021-08-26 11:53:52 +00:00
Patrick Cloke
5548fe0978
Cache the result of fetching the room hierarchy over federation. (#10647) 2021-08-26 07:16:53 -04:00
Andrew Morgan
b45cc1530b
Make a note to leave a summary when one is bumping the schema version (#10621) 
I found this easy to miss (and evidently, it looks like it was missed for schema version 62).
 2021-08-25 17:00:44 +01:00
Sean
7367473f96
Fix error when selecting between thumbnails with the same quality (#10684) 
Fixes #10318
 2021-08-25 09:51:08 +00:00
Erik Johnston
cd22fb568a Merge branch 'master' into develop 2021-08-24 17:13:29 +01:00
Erik Johnston
6f77a3d433 1.41.0 2021-08-24 15:31:55 +01:00
Patrick Cloke
d12ba52f17
Persist room hierarchy pagination sessions to the database. (#10613) 2021-08-24 08:14:03 -04:00
Richard van der Hoff
15db8b7c7f
Correctly initialise the synapse_user_logins metric. (#10677) 
Fix a bug where the prometheus metrics for SSO logins wouldn't be initialised
until the first user logged in with a given auth provider.
 2021-08-24 09:17:51 +00:00
Azrenbeth
0c1d6f65d7
Enforce the max length for per-room display names / avatar URLs. (#10654) 
To match the maximum lengths allowed for profile data.
 2021-08-23 11:25:33 -04:00
Patrick Cloke
bd7d398b05
Additional type hints for the sync REST servlet. (#10666) 2021-08-23 08:14:42 -04:00
Patrick Cloke
2af6d31b78
Addtional type hints for the REST servlets. (#10665) 2021-08-23 08:14:17 -04:00
Patrick Cloke
31dac7ffee
Do not include stack traces for known exceptions when trying multiple federation destinations. (#10662) 2021-08-23 08:00:25 -04:00
Callum Brown
947dbbdfd1
Implement MSC3231: Token authenticated registration (#10142) 
Signed-off-by: Callum Brown <callum@calcuode.com>

This is part of my GSoC project implementing [MSC3231](https://github.com/matrix-org/matrix-doc/pull/3231).
 2021-08-21 22:14:43 +01:00
David Robertson
ee3b2ac59a
Validate device_keys for C-S /keys/query requests (#10593) 
* Validate device_keys for C-S /keys/query requests

Closes #10354

A small, not particularly critical fix. I'm interested in seeing if we
can find a more systematic approach though. #8445 is the place for any discussion.
 2021-08-20 15:47:03 +01:00
Patrick Cloke
5cda75fede
Set room version 8 as preferred for restricted rooms. (#10571) 2021-08-20 07:17:50 -04:00
Richard van der Hoff
e81d62009e
Split on_receive_pdu in half (#10640) 
Here we split on_receive_pdu into two functions (on_receive_pdu and process_pulled_event), rather than having both cases in the same method. There's a tiny bit of overlap, but not that much.
 2021-08-19 17:05:12 +00:00
Richard van der Hoff
50af1efe4b
Extract _resolve_state_at_missing_prevs (#10624) 
This is a follow-up to #10615: it takes the code that constructs the state at a backwards extremity, and extracts it to a separate method.
 2021-08-19 17:31:40 +01:00
Patrick Cloke
000aa89be6
Do not include rooms with an unknown room version in a sync response. (#10644) 
A user will still see this room if it is in a local cache, but it will
not reappear if clearing the cache and reloading.
 2021-08-19 11:12:55 -04:00
Dirk Klimpel
b5fef6054a
Support MSC3283: Expose enable_set_displayname in capabilities (#10452) 2021-08-19 09:40:40 +00:00
Dirk Klimpel
220f901229
Remove not needed database updates in modify user admin API (#10627) 2021-08-19 10:25:05 +01:00
Dirk Klimpel
0c3565da4c
Additional type hints for the proxy agent and SRV resolver modules. (#10608) 2021-08-18 13:53:20 -04:00
Erik Johnston
78a70a2e0b Merge branch 'release-v1.41' into develop 2021-08-18 17:02:47 +01:00
Patrick Cloke
d9856d9150
Fix weakref_slot parameter for room member storage attrs. (#10642) 
Follow-up to #10629 which set it to true, not false.
 2021-08-18 15:00:37 +00:00
Erik Johnston
49cb7eae97 1.41.0rc1 2021-08-18 15:52:11 +01:00
Patrick Cloke
bec01c0758
Convert room member storage tuples to attrs. (#10629) 
Instead of using namedtuples. This helps with asserting type hints
and code completion.
 2021-08-18 09:22:07 -04:00
Andrew Morgan
3692f7fd33
Mount /_synapse/admin/v1/users/{userId}/media admin API on media workers only (#10628) 
Co-authored-by: Patrick Cloke <patrickc@matrix.org>
 2021-08-18 13:25:12 +01:00
Callum Brown
6e613a10d0
Display an error page during failure of fallback UIA. (#10561) 2021-08-18 08:13:35 -04:00
Richard van der Hoff
964f29cb6f
Refactor on_receive_pdu code (#10615) 
* drop room pdu linearizer sooner

No point holding onto it while we recheck the db

* move out `missing_prevs` calculation

we're going to need `missing_prevs` whatever we do, so we may as well calculate
it eagerly and just update it if it gets outdated.

* Add another `if missing_prevs` condition

this should be a no-op, since all the code inside the block already checks `if
missing_prevs`

* reorder if conditions

This shouldn't change the logic at all.

* Push down `min_depth` read

No point reading it from the database unless we're going to use it.

* Collect the sent_to_us_directly code together

Move the remaining `sent_to_us_directly` code inside the `if
sent_to_us_directly` block.

* Properly separate the `not sent_to_us_directly` branch

Since the only way this second block is now reachable is if we
*didn't* go into the `sent_to_us_directly` branch, we can replace it with a
simple `else`.

* changelog
 2021-08-18 12:36:22 +01:00
Patrick Cloke
6a5f8fbcda
Use auto-attribs for attrs classes for sync. (#10630) 2021-08-18 07:27:32 -04:00
Erik Johnston
5581dd7bf7
Allow modules to run looping call on all instances (#10638) 
By default the calls only ran on the worker configured to run background
tasks.
 2021-08-18 10:21:11 +00:00
Dirk Klimpel
430241a1e9
Remove deprecated Shutdown Room and Purge Room Admin API (#8830) 2021-08-17 21:19:13 +01:00
Andrew Morgan
703e3a9e85
Allow /createRoom to be run on workers (#10564) 
Fixes https://github.com/matrix-org/synapse/issues/7867
 2021-08-17 14:33:16 +01:00
Azrenbeth
1a9f531c79
Port the PresenceRouter module interface to the new generic interface (#10524) 
Port the PresenceRouter module interface to the new generic interface introduced in v1.37.0
 2021-08-17 13:22:45 +00:00
Andrew Morgan
84469bdac7
Remove the unused public_room_list_stream (#10565) 
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
 2021-08-17 14:02:50 +01:00
Brendan Abolivier
5639759980
Centralise the custom template directory (#10596) 
Several configuration sections are using separate settings for custom template directories, which can be confusing. This PR adds a new top-level configuration for a custom template directory which is then used for every module. The only exception is the consent templates, since the consent template directory require a specific hierarchy, so it's probably better that it stays separate from everything else.
 2021-08-17 14:45:24 +02:00
Patrick Cloke
c4cf0c0473
Attempt to pull from the legacy spaces summary API over federation. (#10583) 
If the new /hierarchy API does not exist on all destinations,
fallback to querying the /spaces API and translating the results.

This is a backwards compatibility hack since not all of the
federated homeservers will update at the same time.
 2021-08-17 08:19:12 -04:00
Richard van der Hoff
272b89d547
Stop setting the outlier flag for things that aren't (#10614) 
Marking things as outliers to inhibit pushes is a sledgehammer to crack a
nut. Move the test further down the stack so that we just inhibit the thing we
want.
 2021-08-17 13:13:42 +01:00
reivilibre
5f7b1e1f27
Make PeriodicallyFlushingMemoryHandler the default logging handler. (#10518) 2021-08-17 13:13:11 +01:00
reivilibre
642a42edde
Flatten the synapse.rest.client package (#10600) 2021-08-17 11:57:58 +00:00
Andrew Morgan
b62eba7705
Always list fallback key types in /sync (#10623) 2021-08-17 12:32:25 +01:00
Dirk Klimpel
3bcd525b46
Allow to edit external_ids by Edit User admin API (#10598) 
Signed-off-by: Dirk Klimpel dirk@klimpel.org
 2021-08-17 11:56:11 +01:00
Richard van der Hoff
58f0d97275
update links to schema doc (#10620) 2021-08-17 10:45:35 +00:00
Brendan Abolivier
ae2714c1f3
Allow using several custom template directories (#10587) 
Allow using several directories in read_templates.
 2021-08-17 10:23:14 +00:00
Will Hunt
a933c2c7d8
Add an admin API to check if a username is available (#10578) 
This adds a new API GET /_synapse/admin/v1/username_available?username=foo to check if a username is available. It is the counterpart to https://matrix.org/docs/spec/client_server/r0.6.0#get-matrix-client-r0-register-available, except that it works even if registration is disabled.
 2021-08-17 10:52:38 +01:00
reivilibre
19e51b14d2
Manhole: wrap coroutines in defer.ensureDeferred automatically (#10602) 2021-08-16 18:11:48 +01:00
Patrick Cloke
5af83efe8d
Validate the max_rooms_per_space parameter to ensure it is non-negative. (#10611) 2021-08-16 12:01:30 -04:00
Michael Telatynski
0ace38b7b3
Experimental support for MSC3266 Room Summary API. (#10394) 2021-08-16 14:49:12 +00:00
Patrick Cloke
87b62f8bb2
Split synapse.federation.transport.server into multiple files. (#10590) 2021-08-16 10:14:31 -04:00
Richard van der Hoff
2d9ca4ca77
Clean up some logging in the federation event handler (#10591) 
* Include outlier status in `str(event)`

In places where we log event objects, knowing whether or not you're dealing
with an outlier is super useful.

* Remove duplicated logging in get_missing_events

When we process events received from get_missing_events, we log them twice
(once in `_get_missing_events_for_pdu`, and once in `on_receive_pdu`). Reduce
the duplication by removing the logging in `on_receive_pdu`, and ensuring the
call sites do sensible logging.

* log in `on_receive_pdu` when we already have the event

* Log which prev_events we are missing

* changelog
 2021-08-16 13:19:02 +01:00
Patrick Cloke
7de445161f
Support federation in the new spaces summary API (MSC2946). (#10569) 2021-08-16 08:06:17 -04:00
Šimon Brandner
a3a7514570
Handle string read receipt data (#10606) 
* Handle string read receipt data

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Test that we handle string read receipt data

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Add changelog for #10606

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Add docs

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Ignore malformed RRs

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Only surround hidden = ...

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Remove unnecessary argument

Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>

* Update changelog.d/10606.bugfix

Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
 2021-08-16 12:22:38 +01:00
Eric Eastwood
c8d54be44c
Move /batch_send to /v2_alpha directory (MSC2716) (#10576) 
* Move /batch_send to /v2_alpha directory

As pointed out by @erikjohnston,
https://github.com/matrix-org/synapse/pull/10552#discussion_r685836624
 2021-08-13 14:37:24 -05:00
Patrick Cloke
c12b5577f2
Fix a harmless exception when the staged events queue is empty. (#10592) 2021-08-13 11:49:06 +00:00
Patrick Cloke
98a3355d9a
Update the pagination parameter name based on MSC2946 review. (#10579) 2021-08-11 15:44:45 -04:00
Dirk Klimpel
915b37e5ef
Admin API to delete media for a specific user (#10558) 2021-08-11 19:29:59 +00:00
Patrick Cloke
3ebb6694f0
Allow requesting the summary of a space which is joinable. (#10580) 
As opposed to only allowing the summary of spaces which the user is
already in or has world-readable visibility.

This makes the logic consistent with whether a space/room is returned
as part of a space and whether a space summary can start at a space.
 2021-08-11 15:04:51 -04:00
Patrick Cloke
5acd8b5a96
Expire old spaces summary pagination sessions. (#10574) 2021-08-11 18:52:09 +00:00
Patrick Cloke
2ae2a04616
Clarify error message when joining a restricted room. (#10572) 2021-08-11 14:31:39 -04:00
Dirk Klimpel
339c3918e1
support federation queries through http connect proxy (#10475) 
Signed-off-by: Marcus Hoffmann <bubu@bubu1.eu>
Signed-off-by: Dirk Klimpel dirk@klimpel.org
 2021-08-11 15:34:59 +01:00
Eric Eastwood
8c654b7309
Only return state events that the AS passed in via state_events_at_start (MSC2716) (#10552) 
* Only return state events that the AS passed in via state_events_at_start

As discovered by @Half-Shot in
https://github.com/matrix-org/matrix-doc/pull/2716#discussion_r684158448

Part of MSC2716

* Add changelog

* Fix changelog extension
 2021-08-10 18:10:40 -05:00
Patrick Cloke
fe1d0c8618
Add local support for the new spaces summary endpoint (MSC2946) (#10549) 
This adds support for the /hierarchy endpoint, which is an
update to MSC2946. Currently this only supports rooms known
locally to the homeserver.
 2021-08-10 13:08:17 -04:00
Patrick Cloke
691593bf71
Fix an edge-case with invited rooms over federation in the spaces summary. (#10560) 
If a room which the requesting user was invited to was queried over
federation it will now properly appear in the spaces summary (instead
of being stripped out by the requesting server).
 2021-08-10 14:56:54 +00:00
Brendan Abolivier
b5de77cf86 Merge branch 'master' into develop 2021-08-10 14:23:57 +01:00
Brendan Abolivier
9f7c038272 1.40.0 2021-08-10 13:50:58 +01:00
Eric Eastwood
7afb615839
When redacting, keep event fields around that maintain the historical event structure intact (MSC2716) (#10538) 
* Keep event fields that maintain the historical event structure intact

Fix https://github.com/matrix-org/synapse/issues/10521

* Add changelog

* Bump room version

* Better changelog text

* Fix up room version after develop merge
 2021-08-09 20:23:31 -05:00
Dirk Klimpel
6b61debf5c
Do not remove status_msg when user going offline (#10550) 
Signed-off-by: Dirk Klimpel dirk@klimpel.org
 2021-08-09 16:21:04 +00:00
Brendan Abolivier
f8e86b7d2e
Merge branch 'release-v1.40' into develop 2021-08-09 14:44:28 +01:00
Brendan Abolivier
ad35b7739e 1.40.0rc3 2021-08-09 13:41:29 +01:00