Commit Graph

1500 Commits

Author SHA1 Message Date
jejo86
668597214f
Improve documentation on becoming server admin (#13230)
* Improved section regarding server admin

Added steps describing how to elevate an existing user to administrator by manipulating a `postgres` database.

Signed-off-by: jejo86 28619134+jejo86@users.noreply.github.com

* Improved section regarding server admin

* Reference database settings

Add instructions to check database settings to find out the database name, instead of listing all available PostgreSQL databases.

* Add suggestions from PR conversation

Replace config filename `homeserver.yaml`. with "config file".
Remove instructions to switch to `postgres` user.
Add instructions how to connect to SQLite database.

* Update changelog.d/13230.doc

Co-authored-by: reivilibre <olivier@librepush.net>
2022-08-03 11:15:23 +01:00
Dirk Klimpel
fb7a2cc4cc
Update doc for setting macaroon_secret_key (#13443)
* Update doc for setting `macaroon_secret_key`

* newsfile
2022-08-03 10:41:19 +01:00
Dirk Klimpel
5eccfdfafd
Remove 'Contents' section from the Configuration Manual (#13438)
Fixes: #13053
2022-08-03 09:19:20 +00:00
Dirk Klimpel
ec6758d472
Fix wrong headline for url_preview_accept_language in docs (#13437)
Fixes: #13433
2022-08-03 09:41:57 +01:00
Olivier Wilkinson (reivilibre)
a2a867b521 Merge branch 'master' into develop 2022-08-02 11:56:02 +01:00
Olivier Wilkinson (reivilibre)
cb209638ea Add upgrade notes 2022-08-02 11:10:26 +01:00
Richard van der Hoff
d548d8f18d Synapse 1.64.0rc2 (2022-07-29)
==============================
 
 This RC reintroduces support for `account_threepid_delegates.email`, which was removed in 1.64.0rc1. It remains deprecated and will be removed altogether in a future release. ([\#13406](https://github.com/matrix-org/synapse/issues/13406))
 -----BEGIN PGP SIGNATURE-----
 
 iQEzBAABCgAdFiEEv27Axt/F4vrTL/8QOSor00I9eP8FAmLj4a8ACgkQOSor00I9
 eP8biQf/c8yY2mbeRZcBKtp6yoQCRYQvboSMEXyi+dLe1hNqdhSZwRQcAoFuAFwE
 WdScDvoTaElUxv0v6eCI1x9CoXnZ6xpDShvK39j5Yhzv+1tNsm5Uq9imyG3jK5i6
 U/3Gt6CrCsS01VkGslQ3B5I6MFtbC6ZZK9O48yg+GD8Oqw2HH/gllr5swyVbKdbc
 GGhRBHvgXn+w6d/KnKt8uRxJqIpDt9JMga+WdB8CwFR5WnWbGdw24KsyxmBuOLC3
 caQRiluJL/X4jApUpfsJMBBd/jrDod5wWDFO/4P+v0+2d3Ts+hKezZbt5h1VIYSw
 szZXbzxn5RNDkNiJDpOOOMYQ5DXGmA==
 =3/nK
 -----END PGP SIGNATURE-----

Merge tag 'v1.64.0rc2' into develop

Synapse 1.64.0rc2 (2022-07-29)
==============================

This RC reintroduces support for `account_threepid_delegates.email`, which was removed in 1.64.0rc1. It remains deprecated and will be removed altogether in a future release. ([\#13406](https://github.com/matrix-org/synapse/issues/13406))
2022-07-29 15:15:21 +01:00
3nprob
98fb610cc0
Revert "Drop support for delegating email validation (#13192)" (#13406)
Reverts commit fa71bb18b5, and tweaks documentation.

Signed-off-by: 3nprob <git@3n.anonaddy.com>
2022-07-29 10:29:23 +00:00
Brendan Abolivier
24ef1460f6
Explicitly mention which resources support compression in the config guide (#13221) 2022-07-29 09:09:57 +00:00
Matt Holt
935e73efed
Update Caddy reverse proxy documentation (#13344)
Improve/simplify Caddy examples. Remove Caddy v1 (has long been EOL'ed)

Signed-off-by: Matthew Holt <mholt@users.noreply.github.com>
2022-07-25 16:07:26 +00:00
Jan Schär
e8519e0ed2
Support Implicit TLS for sending emails (#13317)
Previously, TLS could only be used with STARTTLS.
Add a new option `force_tls`, where TLS is used from the start.
Implicit TLS is recommended over STARTLS,
see https://datatracker.ietf.org/doc/html/rfc8314

Fixes #8046.

Signed-off-by: Jan Schär <jan@jschaer.ch>
2022-07-25 16:27:19 +01:00
Richard van der Hoff
c7c84b81e3
Update config_documentation.md (#13364)
"changed in" goes before the example
2022-07-22 13:50:20 +01:00
Sean Quah
0fa41a7b17
Update locked frozendict version to 2.3.3 (#13352)
frozendict 2.3.3 includes fixes for memory leaks that get triggered during `/sync`.
2022-07-22 10:26:09 +01:00
David Robertson
4f57ef0b18
Merge branch 'master' into develop 2022-07-21 11:27:08 +01:00
David Teller
b909d5327b
Document rc_invites.per_issuer, added in v1.63.
Resolves #13330.
Missed in #13125.

Signed-off-by: David Teller <davidt@element.io>
2022-07-21 11:26:34 +01:00
Brendan Abolivier
47822fd2e8
Merge branch 'master' into develop 2022-07-19 16:14:02 +02:00
Andrew Morgan
6faaf76a32
Remove 'anonymised' from the phone home stats documentation (#13321) 2022-07-19 12:38:29 +00:00
Jörg Behrmann
87a917e8c8
Add notes when config options were changed to config documentation (#13314)
Signed-off-by: Jörg Behrmann <behrmann@physik.fu-berlin.de>
2022-07-19 12:36:29 +00:00
David Robertson
b977867358
Rate limit joins per-room (#13276) 2022-07-19 11:45:17 +00:00
David Robertson
e9ce4d089b
Use and recommend poetry 1.1.14, up from 1.1.12 (#13285) 2022-07-15 16:18:47 +01:00
jejo86
2341032cf2
Document advising against publicly exposing the Admin API and provide a usage example (#13231)
* Admin API request explanation improved

Pointed out, that the Admin API is not accessible by default from any remote computer, but only from the PC `matrix-synapse` is running on.
Added a full, working example, making sure to include the cURL flag `-X`, which needs to be prepended to `GET`, `POST`, `PUT` etc. and listing the full query string including protocol, IP address and port.

* Admin API request explanation improved

* Apply suggestions from code review

Update changelog. Reword prose.

Co-authored-by: David Robertson <david.m.robertson1@gmail.com>
2022-07-13 19:33:33 +01:00
Patrick Cloke
1381563988
Inline URL preview documentation. (#13261)
Inline URL preview documentation near the implementation.
2022-07-12 15:01:58 -04:00
Richard van der Hoff
fa71bb18b5
Drop support for delegating email validation (#13192)
* Drop support for delegating email validation

Delegating email validation to an IS is insecure (since it allows the owner of
the IS to do a password reset on your HS), and has long been deprecated. It
will now cause a config error at startup.

* Update unit test which checks for email verification

Give it an `email` config instead of a threepid delegate

* Remove unused method `requestEmailToken`

* Simplify config handling for email verification

Rather than an enum and a boolean, all we need here is a single bool, which
says whether we are or are not doing email verification.

* update docs

* changelog

* upgrade.md: fix typo

* update version number

this will be in 1.64, not 1.63

* update version number

this one too
2022-07-12 19:18:53 +01:00
Shay
6f30eb5b8e
Add info about configuration in the url preview docs (#13233)
Cross-link doc pages for easier navigation.
2022-07-12 13:48:47 -04:00
andrew do
2d82cdafd2
expose whether a room is a space in the Admin API (#13208) 2022-07-12 15:30:53 +01:00
Andrew Morgan
5ef2f87569
Document the 'databases' homeserver config option (#13212) 2022-07-11 14:05:24 +00:00
reivilibre
fb7d24ab6d
Check that auto_vacuum is disabled when porting a SQLite database to Postgres, as VACUUMs must not be performed between runs of the script. (#13195) 2022-07-07 10:08:04 +00:00
Patrick Cloke
dcc7873700
Add information on how the Synapse team does reviews. (#13132) 2022-07-06 07:30:58 -04:00
reivilibre
65e675504f
Add the ability to set the log level using the SYNAPSE_TEST_LOG_LEVEL environment when using complement.sh. (#13152) 2022-07-05 09:46:20 +00:00
Dirk Klimpel
e514495465
Add missing links to config options (#13166) 2022-07-05 10:10:26 +01:00
Andrew Morgan
c0efc689cb
Add documentation for phone home stats (#13086) 2022-06-30 22:12:28 +01:00
Shay
8330fc9953
Cleanup references to sample config in the docs and redirect users to configuration manual (#13077) 2022-06-30 09:21:39 -07:00
Andrew Morgan
0ceb3af10b
Add a link to the configuration manual from the homeserver sample config documentation page (#13139) 2022-06-30 15:59:11 +01:00
Moritz Stückler
e714b8a057
Fix documentation header for allow_public_rooms_over_federation (#13116)
Signed-off-by: Moritz Stückler <moritz.stueckler@gmail.com>
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2022-06-29 18:41:39 +02:00
jejo86
cdc0259449
Document the --report-stats argument (#13029)
Signed-off-by: jejo86 <28619134+jejo86@users.noreply.github.com>
2022-06-29 10:24:10 +01:00
Aaron Raimist
3ceaf1462d
Remove docs for Delete Group Admin API (#13112)
This API no longer exists.

Signed-off-by: Aaron Raimist <aaron@raim.ist>
2022-06-27 11:15:25 +01:00
Shay
3d94d07db3
Update opentracing docs to reference the configuration manual rather than the configuation file. (#13076) 2022-06-17 10:47:38 -07:00
Richard van der Hoff
5d6f55959e
Update info on downstream debs (#13095) 2022-06-17 12:47:22 +01:00
Jacek Kuśnierz
0ef1307619
Add custom well-known (#13035)
Co-authored-by: David Robertson <david.m.robertson1@gmail.com>
2022-06-16 11:48:18 +01:00
reivilibre
ffe2464836
Add instructions for running Complement with gotestfmt-formatted output locally. (#13073) 2022-06-16 09:31:10 +00:00
Shay
cba1c5cbc2
Add headers to individual options in config documentation to allow for linking. (#13055) 2022-06-15 11:31:46 -07:00
Erik Johnston
c95b04bb0e
Change default sync_response_cache_duration (#13042) 2022-06-15 16:55:20 +00:00
Hannes Lerchl
7d99414edf
Replace pyjwt with authlib in org.matrix.login.jwt (#13011) 2022-06-15 16:45:16 +00:00
Brendan Abolivier
bdb6628dcf
Fix version number in spam checker callbacks doc (#13047) 2022-06-14 17:24:25 +02:00
Shay
493c2fc44a
Remove code generating comments in configuration file (#12941) 2022-06-14 07:53:42 -07:00
Erik Johnston
5d139f578d Merge branch 'release-v1.61' into develop 2022-06-14 12:00:02 +01:00
Erik Johnston
b8bf61230c Fixup upgrades 2022-06-14 11:56:45 +01:00
Sami Olmari
7b54badd31 Mention removed community/group worker endpoints in upgrade.md (#13023) 2022-06-14 11:40:02 +01:00
Sami Olmari
a542a92c57
Mention removed community/group worker endpoints in upgrade.md (#13023) 2022-06-14 11:35:22 +01:00
David Teller
a164a46038
Uniformize spam-checker API, part 4: port other spam-checker callbacks to return Union[Allow, Codes]. (#12857)
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
2022-06-13 18:16:16 +00:00
David Robertson
4579445cc5
Add missing TOC link to poetry cheat sheet (#13022)
Missed by #12475.
2022-06-10 15:47:49 +01:00
reivilibre
67f51c84f8
Merge the Complement testing Docker images into a single, multi-purpose image. (#12881)
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2022-06-08 09:57:05 +00:00
Andrew Morgan
a47636c570
Prevent local quarantined media from being claimed by media retention (#12972) 2022-06-07 10:53:47 +00:00
Jan Christian Grünhage
fcd8703508
Allow updating passwords using the admin api without logging out devices (#12952) 2022-06-06 12:10:13 +01:00
Patrick Cloke
7bc08f3201
Remove remaining bits of groups code. (#12936)
* Update worker docs to remove group endpoints.
* Removes an unused parameter to `ApplicationService`.
* Break dependency between media repo and groups.
* Avoid copying `m.room.related_groups` state events during room upgrades.
2022-06-01 09:41:25 -04:00
Andrew Morgan
2fc787c341
Add config options for media retention (#12732) 2022-05-31 16:35:29 +00:00
Erik Johnston
3594f6c1f3 Merge branch 'master' into develop 2022-05-31 14:48:22 +01:00
Brendan Abolivier
8fd87739bf
Fix import in module_api module and docs on the new check_event_for_spam signature (#12918)
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2022-05-31 12:04:53 +02:00
Brendan Abolivier
28989cb301
Add a background job to automatically delete stale devices (#12855)
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2022-05-27 17:47:32 +02:00
Sean Quah
053ca5f3ca Synapse 1.60.0rc2 (2022-05-27)
==============================
 
 This release of Synapse adds a unique index to the `state_group_edges` table, in
 order to prevent accidentally introducing duplicate information (for example,
 because a database backup was restored multiple times). If your Synapse database
 already has duplicate rows in this table, this could fail with an error and
 require manual remediation.
 
 Additionally, the signature of the `check_event_for_spam` module callback has changed.
 The previous signature has been deprecated and remains working for now. Module authors
 should update their modules to use the new signature where possible.
 
 See [the upgrade notes](https://github.com/matrix-org/synapse/blob/develop/docs/upgrade.md#upgrading-to-v1600)
 for more details.
 
 Features
 --------
 
 - Add an option allowing users to use their password to reauthenticate for privileged actions even though password login is disabled. ([\#12883](https://github.com/matrix-org/synapse/issues/12883))
 
 Bugfixes
 --------
 
 - Explicitly close `ijson` coroutines once we are done with them, instead of leaving the garbage collector to close them. ([\#12875](https://github.com/matrix-org/synapse/issues/12875))
 
 Internal Changes
 ----------------
 
 - Improve URL previews by not including the content of media tags in the generated description. ([\#12887](https://github.com/matrix-org/synapse/issues/12887))
 -----BEGIN PGP SIGNATURE-----
 
 iQGzBAABCgAdFiEEWMTnW8Z8khaaf90R+84KzgcyGG8FAmKQqMcACgkQ+84Kzgcy
 GG9Z2Av+N+b/fvaB3D56UkFqTW/xLmCEyri65njcXU8625bWiLSPM6hssmyJB1FA
 xlc2RBKr8QxlnHRS/v31wDtONC8YZ2O3fyzYPFfY1fF5Ul7Kg3XCzLeUH4/j1/Ar
 5bqriDqaN9FQ/6QJybShXlA4l7lY1Fs0C4P23jDBgqfKjnlToeVLqhVA70dDaFu/
 ir+vVprKCkQI1iqnYXwIxGRmgBzLWGoVqQFGbSI6hugGwXpGIyX7+2I+0v8tI6vA
 SZ99vLFWcvnd6DJTyBhIeV22Ff4qA7eQsyPvSrMETdsaZmrxGlG+t332HNCgplv8
 gv2gUpJL0br++5WTAX+nRc7HpfKo/74vKeTktqPmlvFP8kUOg+PbzmoJFUu21PhA
 rnq5TzgsPHK0dqBhM1RC2vtOiJ5v3ZBqzJJzSRXl6lsFpWxxFmwesEcIDAYS0Nmh
 QoJb7/L8cPCHksHvZM76bzB465tSH9NhuFYZQoLGHcpxa0kYekrdlYasP8U0FU7L
 nF3C0Pgw
 =D3F+
 -----END PGP SIGNATURE-----

Merge tag 'v1.60.0rc2' into develop

Synapse 1.60.0rc2 (2022-05-27)
==============================

This release of Synapse adds a unique index to the `state_group_edges` table, in
order to prevent accidentally introducing duplicate information (for example,
because a database backup was restored multiple times). If your Synapse database
already has duplicate rows in this table, this could fail with an error and
require manual remediation.

Additionally, the signature of the `check_event_for_spam` module callback has changed.
The previous signature has been deprecated and remains working for now. Module authors
should update their modules to use the new signature where possible.

See [the upgrade notes](https://github.com/matrix-org/synapse/blob/develop/docs/upgrade.md#upgrading-to-v1600)
for more details.

Features
--------

- Add an option allowing users to use their password to reauthenticate for privileged actions even though password login is disabled. ([\#12883](https://github.com/matrix-org/synapse/issues/12883))

Bugfixes
--------

- Explicitly close `ijson` coroutines once we are done with them, instead of leaving the garbage collector to close them. ([\#12875](https://github.com/matrix-org/synapse/issues/12875))

Internal Changes
----------------

- Improve URL previews by not including the content of media tags in the generated description. ([\#12887](https://github.com/matrix-org/synapse/issues/12887))
2022-05-27 12:07:18 +01:00
reivilibre
7b88f5a107
Add an option allowing users to use their password to reauthenticate even though password authentication is disabled. (#12883) 2022-05-27 09:44:51 +00:00
Patrick Cloke
1885ee0113
Remove unstable APIs for /hierarchy. (#12851)
Removes the unstable endpoint as well as a duplicated field
which was modified during stabilization.
2022-05-26 07:10:28 -04:00
Patrick Cloke
a8db8c6eba
Remove user-visible groups/communities code (#12553)
Makes it so that groups/communities no longer exist from a user-POV. E.g. we remove:

* All API endpoints (including Client-Server, Server-Server, and admin).
* Documented configuration options (and the experimental flag, which is now unused).
* Special handling during room upgrades.
* The `groups` section of the `/sync` response.
2022-05-25 07:53:40 -04:00
Nick Mills-Barrett
1f9013ce60
Add the batch_send endpoint to generic workers (#12868) 2022-05-25 09:51:07 +00:00
Nick Mills-Barrett
2e5f88b5e6
Add the /account/whoami endpoint to generic workers (#12866) 2022-05-25 10:41:41 +01:00
Nick Mills-Barrett
b4fab0b14f
Fix incorrect worker-allowed path in documentation (#12867) 2022-05-25 09:20:34 +00:00
Dirk Klimpel
298911555c
Fix typos in documentation (#12863) 2022-05-25 10:14:03 +01:00
Richard van der Hoff
0b3423fd51
contributing_guide.md: fix link to DCO 2022-05-24 11:48:11 +01:00
David Teller
28199e9357
Uniformize spam-checker API, part 2: check_event_for_spam (#12808)
Signed-off-by: David Teller <davidt@element.io>
2022-05-23 17:27:39 +00:00
Jess Porter
a608ac847b
add SpamChecker callback for silently dropping inbound federated events (#12744)
Signed-off-by: jesopo <github@lolnerd.net>
2022-05-23 16:36:21 +00:00
Shay
eb4aaa1b4b
Add detail to cache_autotuning config option documentation (#12776) 2022-05-19 07:47:07 -07:00
Shay
ab2a615cfb
Update configuration manual to document size-related suffixes (#12777) 2022-05-19 07:46:33 -07:00
reivilibre
66a5f6c400
Add a unique index to state_group_edges to prevent duplicates being accidentally introduced and the consequential impact to performance. (#12687) 2022-05-19 14:16:49 +01:00
Sami Olmari
47619017f9
Add missing user directory search endpoint to the generic worker documentation (#12773)
Signed-off-by: Sami Olmari <sami@olmari.fi>
2022-05-19 12:03:12 +01:00
Richard van der Hoff
5675cebfaa
openid.md: fix some links
docbook doesn't auto-linkify links
2022-05-19 10:28:18 +01:00
Patrick Cloke
a1cb05b3e8
Fix federation in demo scripts. (#12783) 2022-05-18 14:49:33 -04:00
Andrew Morgan
0fce474a40
Fix YAML parsing error in url_preview_accept_language (#12785) 2022-05-18 17:24:44 +00:00
Sean Quah
3d8839c30c
Add documentation for cancellation of request processing (#12761)
Signed-off-by: Sean Quah <seanq@matrix.org>
2022-05-18 17:56:23 +01:00
Andrew Morgan
a167304c8b
Switch the 'Configuration' link in the docs homepage to the config manual (#12748) 2022-05-18 11:29:32 +00:00
Andrew Morgan
deca250e3f
Add some documentation around the rc_invites option to the config docs (#12759) 2022-05-18 12:21:32 +01:00
David Robertson
0d17357fcd
Suggest using docker when testing against postgres (#12765)
Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
2022-05-17 18:05:53 +00:00
Andrew Morgan
6d8d1218dd
Fix typo in name of 'run_background_tasks_on' option in config manual (#12749) 2022-05-16 17:31:12 +00:00
Dirk Klimpel
8060034612
Fix typo in listener config (#12742) 2022-05-16 13:50:07 +00:00
Shay
cde8af9a49
Add config flags to allow for cache auto-tuning (#12701) 2022-05-13 12:32:39 -07:00
Niklas
c9fc2c0d22
Update issuer URL in example OIDC Keycloak config (#12727)
* Update openid.md

Newer versions of keycloak returning a 404 when using the `/auth` prefix.

Related: https://github.com/matrix-org/synapse/issues/12714
2022-05-13 10:15:51 +00:00
Andy Balaam
de1e599b9d
add default_power_level_content_override config option. (#12618)
Co-authored-by: Matthew Hodgson <matthew@matrix.org>
2022-05-12 10:41:35 +00:00
Andrew Morgan
409573f6d0
Fix reference to the wrong symbol in the media admin api docs (#12715) 2022-05-12 09:29:37 +01:00
David Robertson
d38d242411
Reload cache factors from disk on SIGHUP (#12673) 2022-05-11 13:43:22 +00:00
reivilibre
699192fc1a
Add the update_user_directory_from_worker configuration option (superseding update_user_directory) to allow a generic worker to be designated as the worker to update the user directory. (#12654)
Co-authored-by: Shay <hillerys@element.io>
2022-05-10 11:08:45 +01:00
Shay
d80a7ab151
Update replication.md with info on TCP module structure (#12621) 2022-05-09 14:46:43 -07:00
Dirk Klimpel
615d96ad6e
Update SQL statements in docs for Synapse Admins (#12536) 2022-05-09 14:43:02 -07:00
Eric Eastwood
18d6c18aa1
Fix docs on how to run specific Complement tests after recent complement.sh change (#12664) 2022-05-09 10:38:32 +01:00
reivilibre
c2d50e9f6c
Add the notify_appservices_from_worker configuration option (superseding notify_appservices) to allow a generic worker to be designated as the worker to send traffic to Application Services. (#12452) 2022-05-06 11:43:53 +01:00
Will Hunt
2d74a8c178
Add mau_appservice_trial_days config (#12619)
* Add mau_appservice_trial_days

* Add a test

* Tweaks

* changelog

* Ensure we sync after the delay

* Fix types

* Add config statement

* Fix test

* Reinstate logging that got removed

* Fix feature name
2022-05-04 19:33:26 +01:00
Andrew Morgan
332cce8dcf
Disable device name lookup over federation by default (#12616) 2022-05-04 16:41:40 +01:00
Patrick Cloke
ba3fd54bad
Remove unstable/unspecced login types. (#12597)
* `m.login.jwt`, which was never specced and has been deprecated
  since Synapse 1.16.0. (`org.matrix.login.jwt` can be used instead.)
* `uk.half-shot.msc2778.login.application_service`, which was
  stabilized as part of the Matrix spec v1.2 release.
2022-05-04 13:53:21 +00:00
Sean Quah
b2df0716bc
Improve logging for cancelled requests (#12587)
Don't log stack traces for cancelled requests and use a custom HTTP
status code of 499.

Signed-off-by: Sean Quah <seanq@element.io>
2022-05-04 13:38:55 +01:00
Richard van der Hoff
8d156ec0ba
Remove special-case for twisted logger (#12589)
This was originally added when we first added a `MemoryHandler` to the default
log config back in https://github.com/matrix-org/synapse/pull/8040, to ensure
that we didn't explode with an infinite loop if there was an error formatting
the logs.

Since then, we made additional improvements to logging which make this
workaround redundant. In particular:

 * we no longer attempt to log un-UTF8-decodable byte sequences, which were the
   most likely cause of an error in the first place.

 * https://github.com/matrix-org/synapse/pull/8268 ensures that in the unlikely
   case that there *is* an error, it won't cause an infinite loop.
2022-04-29 22:05:18 +01:00
Will Hunt
e8d1ec0e92
Add option to enable token registration without requiring 3pids (#12526) 2022-04-27 12:57:53 +00:00
Shay
e75c7e3b6d
Add a table of contents to config manual (#12527)
* Update config_documentation.md
2022-04-25 11:43:59 -07:00
Sami Olmari
813d728d09
Correct typo in user_admin_api.md device deletion JSON (#12533)
Signed-off-by: Sami Olmari <sami@olmari.fi>
2022-04-25 12:39:15 +00:00
villepeh
d9b71410c2
Add HAProxy delegation example to docs (#12501)
Signed-off-by: Ville Petteri Huh
2022-04-25 13:18:18 +01:00
Dirk Klimpel
a36a38b1ca
Add some example configurations for worker (#12492)
Signed-off-by: Dirk Klimpel <dirk@klimpel.org>
2022-04-25 13:17:03 +01:00
Richard van der Hoff
f46b223354
turn-howto: fix some links 2022-04-22 14:23:40 +01:00
David Robertson
ecef741add
Recommend poetry in docs (#12475)
* Recommend poetry in docs

- readme
- contributor guide
- upgrade notes
- new dev cheat sheet for poetry

Co-authored-by: Shay <hillerys@element.io>
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2022-04-20 15:18:21 +01:00
Olivier Wilkinson (reivilibre)
0921d93dcd Merge branch 'master' into develop 2022-04-19 13:28:28 +01:00
Richard van der Hoff
9f512ff537
fix typo in upgrade notes 2022-04-19 11:25:12 +01:00
Shay
3c758d9808
Add a manual documenting config file options (#12368) 2022-04-18 11:32:30 -07:00
reivilibre
7efddbebef
Update documentation to reflect that run_background_tasks_on is no longer experimental. (#12451)
* Background workers aren't experimental anymore

* Newsfile

Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>

* Stream writers aren't experimental either
2022-04-14 15:25:22 +01:00
Dirk Klimpel
a743f7d33e
Replace federation_reader with generic_worker in docs (#12457) 2022-04-14 13:09:07 +01:00
David Robertson
535a689cfc
Reintroduce the lint targets in the linter script (#12455) 2022-04-14 11:33:06 +01:00
Andrew Morgan
0922462fc7
docs: Don't render the table of contents on the print page (#12340) 2022-04-13 08:27:51 -07:00
Erik Johnston
a468768104 Merge branch 'release-v1.57' into develop 2022-04-12 15:28:41 +01:00
Patrick Cloke
9535fd0f9c
Disable groups/communities by default. (#12344)
This disables the endpoints (and sync response fields) for
groups/communities by default.
2022-04-12 10:20:46 -04:00
Richard van der Hoff
643c0c50c1
Update UPGRADE notes to include more details about v1.57 upgrade failure mode (#12448) 2022-04-12 14:56:05 +01:00
Erik Johnston
e31d06f6f0 Split changelogs by year 2022-04-12 13:54:36 +01:00
David Robertson
dd5cc37aa4
Stop maintaining a list of lint targets in lint.sh (#12420) 2022-04-08 14:36:09 +01:00
Eric Eastwood
350062661c
Clarify that we mark as outliers because we don't have any state for them (#12345)
As discussed at https://github.com/matrix-org/synapse/pull/12179#discussion_r837263852
2022-04-06 16:34:33 -05:00
Patrick Cloke
6902e9ff2b
Remove docs for converting a legacy structured logging config. (#12392)
And update the upgrade notes to point at old versions of the documentation.
2022-04-06 13:41:14 -04:00
Sean Quah
ab3fdcf960 Merge branch 'master' into develop 2022-04-05 15:14:53 +01:00
reivilibre
708d88b1a2
Allow specifying the Postgres database's port when running unit tests with Postgres. (#12376) 2022-04-05 12:44:48 +01:00
Shay
f608e6c8cf
Update broken link in workers.md (#12369) 2022-04-04 17:40:31 -07:00
Nick Mills-Barrett
993d90f82b
Use a sequence to generate AS transaction IDs, drop last_txn AS state (#12209)
Switching to a sequence means there's no need to track `last_txn` on the
AS state table to generate new TXN IDs. This also means that there is
no longer contention between the AS scheduler and AS handler on updates
to the `application_services_state` table, which will prevent serialization
errors during the complete AS txn transaction.
2022-04-01 13:33:25 +00:00
Brendan Abolivier
e4409301ba
Add a module callback to react to account data changes (#12327)
Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2022-04-01 11:22:48 +02:00
Brendan Abolivier
5e88143dff
Add a callback to react to 3PID associations (#12302) 2022-03-31 18:27:21 +02:00
David Robertson
15cdcf8f30
Remove dockerfile-pgtests (#12336) 2022-03-31 11:30:07 +01:00
reivilibre
2fc15ac718
Flesh out documentation for running SyTest against Synapse, including use of Postgres and worker mode. (#12271)
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2022-03-30 12:55:05 +00:00
Brendan Abolivier
437a8ed9ef
Add a configuration to exclude rooms from sync response (#12310) 2022-03-30 09:43:04 +00:00
Andrew Morgan
013f3f5e44
Move warning of open registration to v1.56 upgrade notes (#12322) 2022-03-29 14:12:41 +01:00
villepeh
a4643a685c
HAProxy guide update (#12279)
HAproxy reverse proxy guide update to stop sending IPv4-mapped address to HS.
2022-03-28 10:48:18 +00:00
Shay
3c41d87b67
Add restrictions by default to open registration in Synapse (#12091) 2022-03-25 10:11:01 -07:00
IronTooch
61aae18d45
Authentik OpenID minor doc update (#12275) 2022-03-25 12:40:10 +00:00
Brendan Abolivier
5859e2fe0c
Mention the new behaviour on unsafe database locale in the upgrade notes (#12288)
Co-authored-by: Shay <hillerys@element.io>
2022-03-25 10:56:18 +01:00
Shay
e78d4f61fc
Refuse to start if DB has an unsafe locale (#12262) 2022-03-23 10:23:05 -07:00
Jonathan de Jong
c5776780f0
Remove mutual_rooms update_user_directory check, and add extra documentation (#12038)
Resolves #10339
2022-03-23 12:47:07 +00:00
Patrick Cloke
2177e356bc
Sync more worker regexes in the documentation. (#12243) 2022-03-18 12:51:27 -04:00
David Robertson
872dbb0181
Correct check_username_for_spam annotations and docs (#12246)
* Formally type the UserProfile in user searches
* export UserProfile in synapse.module_api
* Update docs

Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
2022-03-18 13:51:41 +00:00
Patrick Cloke
54f674f7a9
Deprecate the groups/communities endpoints and add an experimental configuration flag. (#12200) 2022-03-12 13:23:37 -05:00
Shay
ef3619e61d
Add config settings for background update parameters (#11980) 2022-03-11 10:46:45 -08:00
Brendan Abolivier
e6a106fd5e
Implement a Jinja2 filter to extract localparts from email addresses (#12212) 2022-03-11 15:15:11 +00:00
Brendan Abolivier
003cc6910a
Update the SSO username picker template to comply with SIWA guidelines (#12210)
Fixes https://github.com/matrix-org/synapse/issues/12205
2022-03-11 13:20:00 +00:00
Andrew Morgan
3b12f6d61b
Note that contributors can sign off privately (#12204)
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2022-03-11 11:10:20 +00:00
~creme
7577894bec
Document that most streams can only have a single writer. (#12196)
This includes the `typing`, `to_device`, `account_data`, `receipts`, and `presence`
streams (really anything except the `events` stream).
2022-03-10 18:15:19 +00:00
reivilibre
72e7f1c420
Remove workaround introduced in Synapse v1.50.0rc1 for Mjolnir compatibility. Breaks compatibility with Mjolnir v1.3.1 and earlier. (#11700) 2022-03-10 15:53:23 +00:00
Richard van der Hoff
52a947dc46
Updates to the Room DAG concepts development document (#12179)
Some stuff that came up while we were talking about #12173.
2022-03-10 15:18:31 +00:00
Will Hunt
15382b1afa
Add third_party module callbacks to check if a user can delete a room and deactivate a user (#12028)
* Add check_can_deactivate_user

* Add check_can_shutdown_rooms

* Documentation

* callbacks, not functions

* Various suggested tweaks

* Add tests for test_check_can_shutdown_room and test_check_can_deactivate_user

* Update check_can_deactivate_user to not take a Requester

* Fix check_can_shutdown_room docs

* Renegade and use `by_admin` instead of `admin_user_id`

* fix lint

* Update docs/modules/third_party_rules_callbacks.md

Co-authored-by: Brendan Abolivier <babolivier@matrix.org>

* Update docs/modules/third_party_rules_callbacks.md

Co-authored-by: Brendan Abolivier <babolivier@matrix.org>

* Update docs/modules/third_party_rules_callbacks.md

Co-authored-by: Brendan Abolivier <babolivier@matrix.org>

* Update docs/modules/third_party_rules_callbacks.md

Co-authored-by: Brendan Abolivier <babolivier@matrix.org>

Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
2022-03-09 18:23:57 +00:00
Patrick Cloke
9a0172d49f
Clean-up demo scripts & documentation (#12143)
* Rewrites the demo documentation to be clearer, accurate, and moves it to our documentation tree.
* Improvements to the demo scripts:
	* `clean.sh` now runs `stop.sh` first to avoid zombie processes.
	* Uses more modern Synapse configuration (and removes some obsolete configuration).
	* Consistently use the HTTP ports for server name, etc.
	* Remove the `demo/etc` directory and place everything into the `demo/808x` directories.
2022-03-08 15:02:59 -05:00
David Robertson
4aeb00ca20
Move synctl into synapse._scripts and expose as an entrypoint (#12140) 2022-03-04 11:58:49 +00:00
David Robertson
1fbe0316a9
Add suffices to scripts in scripts-dev (#12137)
* Rename scripts-dev to have suffices

* Update references to `scripts-dev`

* Changelog

* These scripts don't pass mypy
2022-03-02 18:00:26 +00:00
Andrew Morgan
b4461e7d8a
Enable complexity checking in complexity checking docs example (#11998) 2022-03-02 16:11:16 +00:00
David Robertson
f3f0ab10fe
Move scripts directory inside synapse, exposing as setuptools entry_points (#12118)
* Two scripts are basically entry_points already
* Move and rename scripts/* to synapse/_scripts/*.py
* Delete sync_room_to_group.pl
* Expose entry points in setup.py
* Update linter script and config
* Fixup scripts & docs mentioning scripts that moved

Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2022-03-02 13:00:16 +00:00
Brendan Abolivier
300ed0b8a6
Add module callbacks called for reacting to deactivation status change and profile update (#12062) 2022-03-01 15:00:03 +00:00
Patrick Cloke
7754af24ab
Remove the unstable /spaces endpoint. (#12073)
...and various code supporting it.

The /spaces endpoint was from an old version of MSC2946 and included
both a Client-Server and Server-Server API. Note that the unstable
/hierarchy endpoint (from the final version of MSC2946) is not yet
removed.
2022-02-28 18:33:00 +00:00
Patrick Cloke
c56bfb08bc
Add documentation for missing worker types. (#11599)
And clean-up the endpoints which should be routed to workers.
2022-02-23 22:49:04 +00:00
Richard van der Hoff
e24ff8ebe3
Remove HomeServer.get_datastore() (#12031)
The presence of this method was confusing, and mostly present for backwards
compatibility. Let's get rid of it.

Part of #11733
2022-02-23 11:04:02 +00:00
Patrick Cloke
7bcc28f82f
Use room version 9 as the default room version (per MSC3589). (#12058) 2022-02-22 14:09:40 +00:00
Erik Johnston
79c18e0709 Merge remote-tracking branch 'origin/master' into develop 2022-02-22 12:05:48 +00:00
Patrick Cloke
444b04058b
Document why auth providers aren't validated in the admin API. (#12004)
Since it is reasonable to give a future or past auth provider,
which might not be in the current configuration.
2022-02-18 17:24:25 +00:00
Brendan Abolivier
707049c6ff
Allow modules to set a display name on registration (#12009)
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2022-02-17 16:54:16 +00:00
Patrick Cloke
e69f8f0a8e
Remove support for the legacy structured logging configuration. (#12008) 2022-02-17 13:32:18 +00:00
David Robertson
73fc488783
Explain the meaning of spam checker callbacks' return values (#12003)
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2022-02-16 12:25:43 +00:00
Andrew Morgan
7812fe9edd
Note when unstable MSC3283 prefixes will be removed (#11989) 2022-02-14 19:07:59 +00:00
H. Shay
9c4563c5cd remove empty file, reword/rename newsfragment, and add line to upgrade doc 2022-02-14 10:21:00 -08:00
Shay
b2b971f28a
Enable cache time-based expiry by default (#11849) 2022-02-11 11:05:20 -08:00
Brendan Abolivier
0171fa5226
Remove deprecated user_may_create_room_with_invites callback (#11950)
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2022-02-11 13:58:11 +00:00
Dirk Klimpel
d36943c4df
Support the stable API endpoint for MSC3283: new settings in /capabilities endpoint (#11933) 2022-02-11 09:32:11 +00:00
Patrick Cloke
e4fdf459e2
Basic documentation for the release cycle. (#11954) 2022-02-10 13:15:10 +00:00
Brad Jones
3914576b2b
Fix example for structured logging. (#11946)
The StreamHandler takes a stream parameter, not location.
2022-02-09 20:56:33 +00:00
Brendan Abolivier
6f440fd859
Recommend upgrading treq alongside twisted (#11943) 2022-02-09 15:06:51 +00:00
Brendan Abolivier
6b91315ddf Merge branch 'master' into develop 2022-02-08 13:26:09 +00:00
Brendan Abolivier
1aa2231e27
Fix wording 2022-02-08 12:03:49 +00:00
Brendan Abolivier
5cdd491310
Add words about the Twisted security fix 2022-02-08 11:47:35 +00:00
Brendan Abolivier
0640f8ebaa
Add a callback to allow modules to deny 3PID (#11854)
Part of the Tchap Synapse mainlining.

This allows modules to implement extra logic to figure out whether a given 3PID can be added to the local homeserver. In the Tchap use case, this will allow a Synapse module to interface with the custom endpoint /internal_info.
2022-02-08 11:20:32 +01:00
Dirk Klimpel
65ef21b1c7
Clarify that users' media are also preview images (#11862) 2022-02-04 14:39:14 +00:00
Jonathan de Jong
02632b3504
Stabilise MSC3231 (Token Based Registration) (#11867) 2022-02-04 12:15:13 +00:00
Richard van der Hoff
b3d155a749
Delete MSC1711_certificates_FAQ.md (#11907)
This document isn't really relevant any more, and its existence is more confusing than helpful.
2022-02-04 11:27:00 +00:00
Patrick Cloke
119edf51eb
Remove support for the webclient listener. (#11895)
Also remove support for non-HTTP(S) web_client_location.
2022-02-03 18:36:49 +00:00
Christian Paul
6b1c265c21
Fix typo: unpind -> unbind (#11859)
Co-authored-by: reivilibre <olivier@librepush.net>
2022-02-03 17:20:44 +00:00
Brendan Abolivier
d80d39b035
Add a ratelimiter for 3pid invite (#11892) 2022-02-03 14:28:15 +01:00
Dirk Klimpel
0da2301b21
Consolidate the access_token information in the admin api (#11861)
Co-authored-by: reivilibre <olivier@librepush.net>
2022-01-31 16:24:29 +00:00
Dirk Klimpel
7eb198ddc8
Remove not needed old table of contents in documentation (#11860) 2022-01-31 14:40:20 +00:00
Brendan Abolivier
bf60da1a60
Configurable limits on avatars (#11846)
Only allow files which file size and content types match configured
limits to be set as avatar.

Most of the inspiration from the non-test code comes from matrix-org/synapse-dinsic#19
2022-01-28 15:41:33 +01:00
Dirk Klimpel
fd65139714
Fix some indentation inconsistencies in the sample config (modules) (#11838) 2022-01-27 10:06:29 +00:00
Shay
ec07062e31
Update installation docs to indicate that we support Python 3.10 (#11820) 2022-01-26 16:05:29 -08:00
Brendan Abolivier
2d3bd9aa67
Add a module callback to set username at registration (#11790)
This is in the context of mainlining the Tchap fork of Synapse. Currently in Tchap usernames are derived from the user's email address (extracted from the UIA results, more specifically the m.login.email.identity step).
This change also exports the check_username method from the registration handler as part of the module API, so that a module can check if the username it's trying to generate is correct and doesn't conflict with an existing one, and fallback gracefully if not.

Co-authored-by: David Robertson <davidr@element.io>
2022-01-26 14:21:13 +00:00
Brendan Abolivier
95b3f952fa
Add a config flag to inhibit M_USER_IN_USE during registration (#11743)
This is mostly motivated by the tchap use case, where usernames are automatically generated from the user's email address (in a way that allows figuring out the email address from the username). Therefore, it's an issue if we respond to requests on /register and /register/available with M_USER_IN_USE, because it can potentially leak email addresses (which include the user's real name and place of work).

This commit adds a flag to inhibit the M_USER_IN_USE errors that are raised both by /register/available, and when providing a username early into the registration process. This error will still be raised if the user completes the registration process but the username conflicts. This is particularly useful when using modules (https://github.com/matrix-org/synapse/pull/11790 adds a module callback to set the username of users at registration) or SSO, since they can ensure the username is unique.

More context is available in the PR that introduced this behaviour to synapse-dinsic: matrix-org/synapse-dinsic#48 - as well as the issue in the matrix-dinsic repo: matrix-org/matrix-dinsic#476
2022-01-26 13:02:54 +01:00
Dirk Klimpel
6a72c910f1
Add admin API to get a list of federated rooms (#11658) 2022-01-25 16:11:40 +00:00
Richard van der Hoff
fc8598bc87
Minor updates, and docs, for schema delta files (#11823)
* Make functions in python deltas optional

It's annoying to always have to write stubs for these.

* Documentation for delta files

* changelog
2022-01-25 14:11:13 +00:00
Forest Johnson
4210143f53
Docs: add missing PR submission process how-tos (#11821)
* Docs: add missing PR submission process how-tos

The documentation says that in order to submit a pull request you have to run the linter and links to [Run the linters](https://matrix-org.github.io/synapse/latest/development/contributing_guide.html#run-the-linters).  IMO "Run the linters" should explain that development dependencies are a pre-requisite.

I also included `pip install wheel`  which I had to run inside my virtual environment on ubuntu before I `pip install -e ".[all,dev]"` would succeed.
2022-01-25 14:09:56 +00:00
Dirk Klimpel
0d6cfea9b8
Add admin API to reset connection timeouts for remote server (#11639)
* Fix get federation status of destination if no error occured
2022-01-25 12:06:29 +00:00
reivilibre
df54c8485a
Remove account data (including client config, push rules and ignored users) upon user deactivation. (#11621)
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2022-01-24 13:37:00 +00:00
Shay
9006ee36d1
Drop support for and remove references to EOL Python 3.6 (#11683)
* remove reference in comments to python3.6

* upgrade tox python env in script

* bump python version in example for completeness

* upgrade python version requirement in setup doc

* upgrade necessary python version in __init__.py

* upgrade python version in setup.py

* newsfragment

* drops refs to bionic and replace with focal

* bump refs to postgres 9.6 to 10

* fix hanging ci

* try installing tzdata first

* revert change made in b979f336

* ignore new random mypy error while debugging other error

* fix lint error for temporary workaround

* revert change to install list

* try passing env var

* export debian frontend var?

* move line and add comment

* bump pillow dependency

* bump lxml depenency

* install libjpeg-dev for pillow

* bump automat version to one compatible with py3.8

* add libwebp for pillow

* bump twisted trunk python version

* change suffix of newsfragment

* remove redundant python 3.7 checks

* lint
2022-01-21 14:23:26 -08:00
Richard van der Hoff
56834ab779
installation.md: drop python 3.6 support (#11781)
#11595 dropped support for python 3.6, but forgot to update this doc.
2022-01-20 14:37:11 +00:00
Patrick Cloke
91221b6961
Add deprecation warnings for webclient listener and non-HTTP(S) web_client_location. (#11774)
This changes the behaviour of the root endpoint to redirect
directly to the configuration of `web_client_location` if it is
given an HTTP(S) URL.
2022-01-20 14:21:06 +00:00
Philippe Daouadi
15ffc4143c
Fix preview of imgur and Tenor URLs. (#11669)
By scraping Open Graph information from the HTML even
when an autodiscovery endpoint is found. The results are
then combined to capture as much information as possible
from the page.
2022-01-18 13:20:24 -05:00
AndrewFerr
cefd4b87a3
Warn against using Let's Encrypt certs for encrypted TURN (#11686)
* Warn against using Let's Encrypt certs for encrypted TURN

This helps to avoid client-side issues:
* https://github.com/vector-im/element-android/issues/1533
* https://github.com/vector-im/element-ios/issues/2712

Signed-off-by: Andrew Ferrazzutti <fair@miscworks.net>
2022-01-17 15:13:09 +00:00
Richard van der Hoff
86615aa965
Fix up docs for track_puppeted_user_ips (again) (#11757)
Fixes #11741
2022-01-17 14:55:30 +00:00
Richard van der Hoff
b0352f9c08
Update documentation for configuring facebook login (#11755)
... and a minor thinko fix in the sample config.
2022-01-17 12:35:00 +00:00
Andrew Morgan
18862f20b5
Remove the 'password_hash' from the Users Admin API endpoint response dictionary (#11576) 2022-01-14 14:53:33 +00:00
Jason Robinson
904bb04409
Fix sample_config.yaml in regards track_puppeted_user_ips (#11749)
* Fix sample_config.yaml in regards track_puppeted_user_ips

Closes #11741

Signed-off-by: Jason Robinson <jasonr@matrix.org>
2022-01-14 14:11:55 +00:00