Commit Graph

1340 Commits

Author SHA1 Message Date
David Robertson
73fc488783
Explain the meaning of spam checker callbacks' return values (#12003)
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2022-02-16 12:25:43 +00:00
Andrew Morgan
7812fe9edd
Note when unstable MSC3283 prefixes will be removed (#11989) 2022-02-14 19:07:59 +00:00
H. Shay
9c4563c5cd remove empty file, reword/rename newsfragment, and add line to upgrade doc 2022-02-14 10:21:00 -08:00
Shay
b2b971f28a
Enable cache time-based expiry by default (#11849) 2022-02-11 11:05:20 -08:00
Brendan Abolivier
0171fa5226
Remove deprecated user_may_create_room_with_invites callback (#11950)
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2022-02-11 13:58:11 +00:00
Dirk Klimpel
d36943c4df
Support the stable API endpoint for MSC3283: new settings in /capabilities endpoint (#11933) 2022-02-11 09:32:11 +00:00
Patrick Cloke
e4fdf459e2
Basic documentation for the release cycle. (#11954) 2022-02-10 13:15:10 +00:00
Brad Jones
3914576b2b
Fix example for structured logging. (#11946)
The StreamHandler takes a stream parameter, not location.
2022-02-09 20:56:33 +00:00
Brendan Abolivier
6f440fd859
Recommend upgrading treq alongside twisted (#11943) 2022-02-09 15:06:51 +00:00
Brendan Abolivier
6b91315ddf Merge branch 'master' into develop 2022-02-08 13:26:09 +00:00
Brendan Abolivier
1aa2231e27
Fix wording 2022-02-08 12:03:49 +00:00
Brendan Abolivier
5cdd491310
Add words about the Twisted security fix 2022-02-08 11:47:35 +00:00
Brendan Abolivier
0640f8ebaa
Add a callback to allow modules to deny 3PID (#11854)
Part of the Tchap Synapse mainlining.

This allows modules to implement extra logic to figure out whether a given 3PID can be added to the local homeserver. In the Tchap use case, this will allow a Synapse module to interface with the custom endpoint /internal_info.
2022-02-08 11:20:32 +01:00
Dirk Klimpel
65ef21b1c7
Clarify that users' media are also preview images (#11862) 2022-02-04 14:39:14 +00:00
Jonathan de Jong
02632b3504
Stabilise MSC3231 (Token Based Registration) (#11867) 2022-02-04 12:15:13 +00:00
Richard van der Hoff
b3d155a749
Delete MSC1711_certificates_FAQ.md (#11907)
This document isn't really relevant any more, and its existence is more confusing than helpful.
2022-02-04 11:27:00 +00:00
Patrick Cloke
119edf51eb
Remove support for the webclient listener. (#11895)
Also remove support for non-HTTP(S) web_client_location.
2022-02-03 18:36:49 +00:00
Christian Paul
6b1c265c21
Fix typo: unpind -> unbind (#11859)
Co-authored-by: reivilibre <olivier@librepush.net>
2022-02-03 17:20:44 +00:00
Brendan Abolivier
d80d39b035
Add a ratelimiter for 3pid invite (#11892) 2022-02-03 14:28:15 +01:00
Dirk Klimpel
0da2301b21
Consolidate the access_token information in the admin api (#11861)
Co-authored-by: reivilibre <olivier@librepush.net>
2022-01-31 16:24:29 +00:00
Dirk Klimpel
7eb198ddc8
Remove not needed old table of contents in documentation (#11860) 2022-01-31 14:40:20 +00:00
Brendan Abolivier
bf60da1a60
Configurable limits on avatars (#11846)
Only allow files which file size and content types match configured
limits to be set as avatar.

Most of the inspiration from the non-test code comes from matrix-org/synapse-dinsic#19
2022-01-28 15:41:33 +01:00
Dirk Klimpel
fd65139714
Fix some indentation inconsistencies in the sample config (modules) (#11838) 2022-01-27 10:06:29 +00:00
Shay
ec07062e31
Update installation docs to indicate that we support Python 3.10 (#11820) 2022-01-26 16:05:29 -08:00
Brendan Abolivier
2d3bd9aa67
Add a module callback to set username at registration (#11790)
This is in the context of mainlining the Tchap fork of Synapse. Currently in Tchap usernames are derived from the user's email address (extracted from the UIA results, more specifically the m.login.email.identity step).
This change also exports the check_username method from the registration handler as part of the module API, so that a module can check if the username it's trying to generate is correct and doesn't conflict with an existing one, and fallback gracefully if not.

Co-authored-by: David Robertson <davidr@element.io>
2022-01-26 14:21:13 +00:00
Brendan Abolivier
95b3f952fa
Add a config flag to inhibit M_USER_IN_USE during registration (#11743)
This is mostly motivated by the tchap use case, where usernames are automatically generated from the user's email address (in a way that allows figuring out the email address from the username). Therefore, it's an issue if we respond to requests on /register and /register/available with M_USER_IN_USE, because it can potentially leak email addresses (which include the user's real name and place of work).

This commit adds a flag to inhibit the M_USER_IN_USE errors that are raised both by /register/available, and when providing a username early into the registration process. This error will still be raised if the user completes the registration process but the username conflicts. This is particularly useful when using modules (https://github.com/matrix-org/synapse/pull/11790 adds a module callback to set the username of users at registration) or SSO, since they can ensure the username is unique.

More context is available in the PR that introduced this behaviour to synapse-dinsic: matrix-org/synapse-dinsic#48 - as well as the issue in the matrix-dinsic repo: matrix-org/matrix-dinsic#476
2022-01-26 13:02:54 +01:00
Dirk Klimpel
6a72c910f1
Add admin API to get a list of federated rooms (#11658) 2022-01-25 16:11:40 +00:00
Richard van der Hoff
fc8598bc87
Minor updates, and docs, for schema delta files (#11823)
* Make functions in python deltas optional

It's annoying to always have to write stubs for these.

* Documentation for delta files

* changelog
2022-01-25 14:11:13 +00:00
Forest Johnson
4210143f53
Docs: add missing PR submission process how-tos (#11821)
* Docs: add missing PR submission process how-tos

The documentation says that in order to submit a pull request you have to run the linter and links to [Run the linters](https://matrix-org.github.io/synapse/latest/development/contributing_guide.html#run-the-linters).  IMO "Run the linters" should explain that development dependencies are a pre-requisite.

I also included `pip install wheel`  which I had to run inside my virtual environment on ubuntu before I `pip install -e ".[all,dev]"` would succeed.
2022-01-25 14:09:56 +00:00
Dirk Klimpel
0d6cfea9b8
Add admin API to reset connection timeouts for remote server (#11639)
* Fix get federation status of destination if no error occured
2022-01-25 12:06:29 +00:00
reivilibre
df54c8485a
Remove account data (including client config, push rules and ignored users) upon user deactivation. (#11621)
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2022-01-24 13:37:00 +00:00
Shay
9006ee36d1
Drop support for and remove references to EOL Python 3.6 (#11683)
* remove reference in comments to python3.6

* upgrade tox python env in script

* bump python version in example for completeness

* upgrade python version requirement in setup doc

* upgrade necessary python version in __init__.py

* upgrade python version in setup.py

* newsfragment

* drops refs to bionic and replace with focal

* bump refs to postgres 9.6 to 10

* fix hanging ci

* try installing tzdata first

* revert change made in b979f336

* ignore new random mypy error while debugging other error

* fix lint error for temporary workaround

* revert change to install list

* try passing env var

* export debian frontend var?

* move line and add comment

* bump pillow dependency

* bump lxml depenency

* install libjpeg-dev for pillow

* bump automat version to one compatible with py3.8

* add libwebp for pillow

* bump twisted trunk python version

* change suffix of newsfragment

* remove redundant python 3.7 checks

* lint
2022-01-21 14:23:26 -08:00
Richard van der Hoff
56834ab779
installation.md: drop python 3.6 support (#11781)
#11595 dropped support for python 3.6, but forgot to update this doc.
2022-01-20 14:37:11 +00:00
Patrick Cloke
91221b6961
Add deprecation warnings for webclient listener and non-HTTP(S) web_client_location. (#11774)
This changes the behaviour of the root endpoint to redirect
directly to the configuration of `web_client_location` if it is
given an HTTP(S) URL.
2022-01-20 14:21:06 +00:00
Philippe Daouadi
15ffc4143c
Fix preview of imgur and Tenor URLs. (#11669)
By scraping Open Graph information from the HTML even
when an autodiscovery endpoint is found. The results are
then combined to capture as much information as possible
from the page.
2022-01-18 13:20:24 -05:00
AndrewFerr
cefd4b87a3
Warn against using Let's Encrypt certs for encrypted TURN (#11686)
* Warn against using Let's Encrypt certs for encrypted TURN

This helps to avoid client-side issues:
* https://github.com/vector-im/element-android/issues/1533
* https://github.com/vector-im/element-ios/issues/2712

Signed-off-by: Andrew Ferrazzutti <fair@miscworks.net>
2022-01-17 15:13:09 +00:00
Richard van der Hoff
86615aa965
Fix up docs for track_puppeted_user_ips (again) (#11757)
Fixes #11741
2022-01-17 14:55:30 +00:00
Richard van der Hoff
b0352f9c08
Update documentation for configuring facebook login (#11755)
... and a minor thinko fix in the sample config.
2022-01-17 12:35:00 +00:00
Andrew Morgan
18862f20b5
Remove the 'password_hash' from the Users Admin API endpoint response dictionary (#11576) 2022-01-14 14:53:33 +00:00
Jason Robinson
904bb04409
Fix sample_config.yaml in regards track_puppeted_user_ips (#11749)
* Fix sample_config.yaml in regards track_puppeted_user_ips

Closes #11741

Signed-off-by: Jason Robinson <jasonr@matrix.org>
2022-01-14 14:11:55 +00:00
qwertyforce
b9632046fb
update room spec url in config files (#11739)
* change spec url in config files

* Create 11739.txt

* .txt -> .doc
2022-01-13 17:09:15 +00:00
Andy Balaam
5ff5f17377
Mention python3-venv and libpq-dev dependencies in contribution guide (#11740) 2022-01-13 16:33:37 +00:00
Jason Robinson
2560b1b6b2
Allow tracking puppeted users for MAU (#11561)
Currently when puppeting another user, the user doing the puppeting is
tracked for client IPs and MAU (if configured).

When tracking MAU is important, it becomes necessary to be possible to
also track the client IPs and MAU of puppeted users. As an example a
client that manages user creation and creation of tokens via the Synapse
admin API, passing those tokens for the client to use.

This PR adds optional configuration to enable tracking of puppeted users
into monthly active users. The default behaviour stays the same.

Signed-off-by: Jason Robinson <jasonr@matrix.org>
2022-01-12 16:09:36 +00:00
haslersn
99ba5ae7b7
Fix documentation of supported PostgreSQL version (#11725)
Signed-off-by: Sebastian Hasler <sebastian.hasler@stuvus.uni-stuttgart.de>
2022-01-12 14:43:48 +00:00
Andrew Morgan
7c3408d1a8
Document the SYNAPSE_TEST_PERSIST_SQLITE_DB unit test env var (#11715) 2022-01-10 17:06:42 +00:00
Olivier Wilkinson (reivilibre)
ffd71029ab Add support removal notice to the upgrade notes 2022-01-05 14:14:16 +00:00
Philipp Matthias Schäfer
eedb4527f1
Fix link from generated configuration file to documentation (#11678)
Co-authored-by: reivilibre <olivier@librepush.net>
Co-authored-by: reivilibre <oliverw@matrix.org>
2022-01-05 12:16:52 +00:00
Dirk Klimpel
7a1cefc6e3
Add admin API to get users' account data (#11664)
Co-authored-by: reivilibre <olivier@librepush.net>
2022-01-05 11:49:06 +00:00
reivilibre
84d790a32e
Clarify SSO mapping provider documentation by writing def or async def before the names of methods, as appropriate. (#11681) 2022-01-05 11:25:32 +00:00
Donny Johnson
0715e77b06
Correct Synapse install command for FreeBSD. (#11267)
Co-authored-by: reivilibre <olivier@librepush.net>
2022-01-05 10:38:51 +00:00
Richard van der Hoff
79f6d3550a
update ngnix reverse-proxy example (#11680)
this should not be a case-insensitive match.
2022-01-04 22:31:45 +00:00
Patrick Cloke
8422a7f7f6
Include the topic event in the prejoin state, per MSC3173. (#11666)
Invites and knocks will now include the topic in the stripped state
send to clients before joining the room.
2022-01-04 11:08:08 -05:00
AndrewFerr
c500bf37d6
Add details for how to set up TURN behind NAT (#11553)
Signed-off-by: Andrew Ferrazzutti <fair@miscworks.net>
2021-12-23 10:42:50 +00:00
Richard van der Hoff
144b9ede89 Merge branch 'master' into develop 2021-12-21 18:13:05 +00:00
Richard van der Hoff
57ca8ab10f Add notes about dropping support for Python 3.6 and Postgres 9.6. 2021-12-21 12:06:31 +00:00
reivilibre
83a74d9350
Document the usage of refresh tokens. (#11427)
Co-authored-by: David Robertson <davidr@element.io>
2021-12-08 15:31:17 +00:00
reivilibre
2f053f3f82
Stabilise support for MSC2918 refresh tokens as they have now been merged into the Matrix specification. (#11435) 2021-12-06 19:11:43 +00:00
Dirk Klimpel
8b4b153c9e
Add admin API to get some information about federation status (#11407) 2021-12-06 16:59:50 +00:00
Eric Eastwood
4eb77965cd
Update backward extremity docs to make it clear that it does not indicate whether we have fetched an events' prev_events (#11469)
Spawning from https://github.com/matrix-org/synapse/pull/9445#discussion_r758958181

Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2021-12-03 18:25:04 -06:00
Shay
49e1356ee3
Minor cleanup on recently ported doc pages (#11466)
* move wiki pages to synapse/docs and add a few titles where necessary

* update SUMMARY.md with added pages

* add changelog

* move incorrectly located newsfragment

* update changelog number

* snake case added files and update summary.md accordingly

* update issue/pr links

* update relative links to docs

* update changelog to indicate that we moved wiki pages to the docs and state reasoning

* requested changes to admin_faq.md

* requested changes to database_maintenance_tools.md

* requested changes to understanding_synapse_through_graphana_graphs.md

* add changelog

* fix leftover merge errata

* fix unwanted changes from merge

* use two spaces between entries

* outdent code blocks
2021-12-02 09:46:20 -08:00
Shay
84dc50e160
Port wiki pages to documentation website (#11402)
* move wiki pages to synapse/docs and add a few titles where necessary

* update SUMMARY.md with added pages

* add changelog

* move incorrectly located newsfragment

* update changelog number

* snake case added files and update summary.md accordingly

* update issue/pr links

* update relative links to docs

* update changelog to indicate that we moved wiki pages to the docs and state reasoning

* revert unintentional change to CHANGES.md

* add link

Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>

* Update CHANGES.md

Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>

Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2021-12-01 10:12:19 -08:00
Brendan Abolivier
153194c771
Link background update controller docs to summary (#11475) 2021-12-01 14:13:01 +00:00
Richard van der Hoff
b9fef1a7cd
Update openid.md
fix header level
2021-12-01 10:57:55 +00:00
Etienne Dysli Metref
b0eb64ff7b
Remove mention of OIDC certification from Dex (#11470)
Dex isn't yet [1,2] a certified OpenID Provider implementation. As of today, it's not on the list maintained by the OpenID Foundation. [3]

[1] https://github.com/dexidp/dex/issues/42
[2] https://github.com/dexidp/dex/issues/262
[3] https://openid.net/certification/
2021-12-01 09:40:51 +00:00
Shay
f1795463bf
Add a note about huge pages to our Postgres doc (#11467)
* Add note to postgres doc about hugepages

* Newsfragment
2021-11-30 19:05:20 -08:00
Marcus
a9481223d1
Improved push typing (#11409)
Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
2021-11-30 11:49:20 +00:00
Patrick Cloke
a4521ce0a8
Support the stable /hierarchy endpoint from MSC2946 (#11329)
This also makes additional updates where the implementation
had drifted from the approved MSC.

Unstable endpoints will be removed at a later data.
2021-11-29 14:32:20 -05:00
Erik Johnston
d08ef6f155
Make background updates controllable via a plugin (#11306)
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
2021-11-29 17:57:06 +01:00
Sean Quah
7564b8e118
Update the media repository documentation (#11415) 2021-11-29 15:37:56 +00:00
Kostas
1035663833
Add config for customizing the claim used for JWT logins. (#11361)
Allows specifying a different claim (from the default "sub") to use
when calculating the localpart of the Matrix ID used during the
JWT login.
2021-11-22 13:01:03 -05:00
Shay
d9e9771d6b
Update README.md 2021-11-19 14:01:55 -08:00
Dirk Klimpel
ea20937084
Add an admin API to run background jobs. (#11352)
Instead of having admins poke into the database directly.

Can currently run jobs to populate stats and to populate
the user directory.
2021-11-19 19:39:46 +00:00
Dirk Klimpel
81b18fe5c0
Add dedicated admin API for blocking a room (#11324) 2021-11-18 17:43:49 +00:00
Nicolai Søborg
e2dabec996
Docs: Quote wildcard federation_certificate_verification_whitelist (#11381)
Otherwise I get this beautiful stacktrace:

```
python3 -m synapse.app.homeserver --config-path /etc/matrix/homeserver.yaml
Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/root/synapse/synapse/app/homeserver.py", line 455, in <module>
    main()
  File "/root/synapse/synapse/app/homeserver.py", line 445, in main
    hs = setup(sys.argv[1:])
  File "/root/synapse/synapse/app/homeserver.py", line 345, in setup
    config = HomeServerConfig.load_or_generate_config(
  File "/root/synapse/synapse/config/_base.py", line 671, in load_or_generate_config
    config_dict = read_config_files(config_files)
  File "/root/synapse/synapse/config/_base.py", line 717, in read_config_files
    yaml_config = yaml.safe_load(file_stream)
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/__init__.py", line 125, in safe_load
    return load(stream, SafeLoader)
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/__init__.py", line 81, in load
    return loader.get_single_data()
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/constructor.py", line 49, in get_single_data
    node = self.get_single_node()
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/composer.py", line 36, in get_single_node
    document = self.compose_document()
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/composer.py", line 55, in compose_document
    node = self.compose_node(None, None)
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/composer.py", line 84, in compose_node
    node = self.compose_mapping_node(anchor)
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/composer.py", line 133, in compose_mapping_node
    item_value = self.compose_node(node, item_key)
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/composer.py", line 82, in compose_node
    node = self.compose_sequence_node(anchor)
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/composer.py", line 110, in compose_sequence_node
    while not self.check_event(SequenceEndEvent):
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/parser.py", line 98, in check_event
    self.current_event = self.state()
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/parser.py", line 379, in parse_block_sequence_first_entry
    return self.parse_block_sequence_entry()
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/parser.py", line 384, in parse_block_sequence_entry
    if not self.check_token(BlockEntryToken, BlockEndToken):
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/scanner.py", line 116, in check_token
    self.fetch_more_tokens()
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/scanner.py", line 227, in fetch_more_tokens
    return self.fetch_alias()
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/scanner.py", line 610, in fetch_alias
    self.tokens.append(self.scan_anchor(AliasToken))
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/scanner.py", line 922, in scan_anchor
    raise ScannerError("while scanning an %s" % name, start_mark,
yaml.scanner.ScannerError: while scanning an alias
  in "/etc/matrix/homeserver.yaml", line 614, column 5
expected alphabetic or numeric character, but found '.'
  in "/etc/matrix/homeserver.yaml", line 614, column 6
```

Signed-off-by: Nicolai Søborg <git@xn--sb-lka.org>
2021-11-18 12:24:40 +00:00
Aaron R
d993c3bb1e
Add support for /_matrix/media/v3 APIs (#11371)
* Add support for `/_matrix/media/v3` APIs

Signed-off-by: Aaron Raimist <aaron@raim.ist>

* Update `workers.md` to use v3 client and media APIs

Signed-off-by: Aaron Raimist <aaron@raim.ist>

* Add changelog

Signed-off-by: Aaron Raimist <aaron@raim.ist>
2021-11-17 15:30:24 +00:00
David Robertson
b76337fdf8
Merge branch 'master' into develop 2021-11-17 14:19:56 +00:00
David Robertson
077b74929f
Merge remote-tracking branch 'origin/release-v1.47' 2021-11-17 14:19:27 +00:00
Patrick Cloke
24b61f379a
Add ability to un-shadow-ban via the admin API. (#11347) 2021-11-16 12:43:53 +00:00
Ashwin Nair
e72135b9d3
change 'Home Server' to one word 'homeserver' (#11320)
Signed-off-by: Ashwin S. Nair <58840757+Ashwin-exe@users.noreply.github.com>
2021-11-16 10:21:01 +00:00
reivilibre
2fffcb24d8
Suggest using /etc/matrix-synapse/conf.d/ for configuration with Debian packages (#11281) 2021-11-12 13:17:06 +00:00
reivilibre
4ad5ee9996
Correct target of link to the modules page from the Password Auth Providers page (#11309) 2021-11-12 12:58:39 +00:00
Dirk Klimpel
8840a7b7f1
Convert delete room admin API to async endpoint (#11223)
Signed-off-by: Dirk Klimpel dirk@klimpel.org
2021-11-12 12:35:31 +00:00
Andrew Morgan
5f277ffe89
Add documentation page stubs for Single Sign-On, SAML and CAS pages (#11298) 2021-11-10 17:54:56 +00:00
Stanislav Motylkov
b09d90cac9
Fix typos in the username_available admin API documentation. (#11286) 2021-11-09 21:11:05 +00:00
David Robertson
b6f4d122ef
Allow admins to proactively block rooms (#11228)
Co-authored-by: Dirk Klimpel <5740567+dklimpel@users.noreply.github.com>
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2021-11-09 13:11:47 +00:00
Erik Johnston
4ee71b9637
Add some background update admin APIs (#11263)
Fixes #11259
2021-11-08 16:08:02 +00:00
Richard van der Hoff
86a497efaa
Default value for public_baseurl (#11210)
We might as well use a default value for `public_baseurl` based on
`server_name` - in many cases, it will be correct.
2021-11-08 14:13:10 +00:00
jmcparland
02742fd058
Wrong DTLS port in "Troubleshooting" (#11268)
Port 5349, not 5479.
2021-11-08 10:34:39 +00:00
Dan Callahan
9799c569bb
Minor cleanup to Debian packaging (#11269)
* Remove unused Vagrant scripts

* Change package Architecture to any

* Preinstall the wheel package when building venvs.

Addresses the following warnings during Debian builds:

    Using legacy 'setup.py install' for jaeger-client, since package 'wheel' is not installed.
    Using legacy 'setup.py install' for matrix-synapse-ldap3, since package 'wheel' is not installed.
    Using legacy 'setup.py install' for opentracing, since package 'wheel' is not installed.
    Using legacy 'setup.py install' for psycopg2, since package 'wheel' is not installed.
    Using legacy 'setup.py install' for systemd-python, since package 'wheel' is not installed.
    Using legacy 'setup.py install' for pympler, since package 'wheel' is not installed.
    Using legacy 'setup.py install' for threadloop, since package 'wheel' is not installed.
    Using legacy 'setup.py install' for thrift, since package 'wheel' is not installed.

* Allow /etc/default/matrix-synapse to be missing

Per the systemd.exec manpage, prefixing an EnvironmentFile with "-":

> indicates that if the file does not exist, it will not be read and no
> error or warning message is logged.

Signed-off-by: Dan Callahan <danc@element.io>
2021-11-07 21:18:33 +00:00
Julian
09cb441a04
Add doc to integrate synapse with LemonLDAP OIDC (#11257)
Co-authored-by: David Robertson <david.m.robertson1@gmail.com>
Co-authored-by: Julian Vanden Broeck <julian.vandenbroeck@dalibo.com>
2021-11-05 12:08:02 +00:00
reivilibre
499c44d696
Make minor correction to type of auth_checkers callbacks (#11253) 2021-11-04 17:10:11 +00:00
Andrew Morgan
d688a6dee5 fix a small typo in the delete room api docs 2021-11-03 11:09:00 +00:00
Dirk Klimpel
7537201840
Add search by room ID and room alias to List Room admin API (#11099)
Fixes: #10874
Signed-off-by: Dirk Klimpel dirk@klimpel.org
2021-11-02 10:01:13 +00:00
reivilibre
69ab3dddbc
Make check_event_allowed module API callback not fail open (accept events) when an exception is raised (#11033) 2021-11-01 15:45:56 +00:00
Dirk Klimpel
66bdca3e31
Remove deprecated delete room admin API (#11213)
Remove deprecated delete room admin API,
`POST /_synapse/admin/v1/rooms/<room_id>/delete`
2021-11-01 15:11:24 +00:00
Richard van der Hoff
71f9966f27
Support for serving server well-known files (#11211)
Fixes https://github.com/matrix-org/synapse/issues/8308
2021-11-01 15:10:16 +00:00
Brett Bethke
2014098d01
Add domain specific matching for haproxy config (#11128) 2021-11-01 14:16:02 +00:00
Sumner Evans
ece84f2c45
Improve code formatting and fix a few typos in docs (#11221)
* Labeled a lot more code blocks with the appropriate type
* Fixed a couple of minor typos (missing/extraneous commas)

Signed-off-by: Sumner Evans <me@sumnerevans.com>
2021-11-01 11:35:55 +00:00
Brendan Abolivier
e320f5dba3
Deprecate user_may_create_room_with_invites (#11206) 2021-11-01 10:46:08 +01:00
Sean Quah
3ed17ff651
Clarify lack of Windows support in documentation (#11198) 2021-10-29 14:03:58 +01:00
Skyler Mäntysaari
a19bf32a03
docs/openid: Add Authentik documentation. (#11151) 2021-10-28 10:31:22 +00:00
Brendan Abolivier
c7a5e49664
Implement an on_new_event callback (#11126)
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2021-10-26 15:17:36 +02:00
Jason Robinson
63cbdd8af0
Enable changing user type via users admin API (#11174)
Users admin API can now also modify user
type in addition to allowing it to be
set on user creation.

Signed-off-by: Jason Robinson <jasonr@matrix.org>
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
2021-10-26 09:01:06 +00:00
Dirk Klimpel
b0f03aeb6a
Add more information what happens when a user is deactivated (#11083) 2021-10-22 09:00:28 +01:00
Aaron R
2c61a318cc
Show error when timestamp in seconds is provided to the /purge_media_cache API (#11101) 2021-10-20 14:41:48 +00:00
Travis Ralston
106d99b8cd
Remove false warning about copying the log config to a homeserver.yaml (#11092)
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2021-10-20 11:48:15 +00:00
Brendan Abolivier
78d5896d19
Document the version of Synapse each module callback was introduced in (#11132)
* Mention callbacks introduced in v1.37.0

According to the documentation introduced in https://github.com/matrix-org/synapse/pull/10062

* Mention callbacks introduced in v1.39.0

According to https://github.com/matrix-org/synapse/pull/10386 and https://github.com/matrix-org/synapse/pull/9884

* Mention callbacks introduced in v1.42.0

According to https://github.com/matrix-org/synapse/pull/10524

* Mention callbacks introduced in v1.44.0 and v1.45.0

As per https://github.com/matrix-org/synapse/pull/10898, https://github.com/matrix-org/synapse/pull/10910 and https://github.com/matrix-org/synapse/pull/10894

* Mention callbacks introduced in v1.46.0

According to https://github.com/matrix-org/synapse/pull/10548
2021-10-20 11:04:27 +00:00
Andrew Morgan
5e0e683541
Fix instances of [example]{.title-ref} in the upgrade notes (#11118) 2021-10-19 14:13:56 +01:00
Brendan Abolivier
73743b8ad1
Document Synapse's behaviour when dealing with multiple modules (#11096)
Document Synapse's behaviour when multiple modules register the same
callback/web resource/etc.

Co-authored-by: reivilibre <oliverw@matrix.org>
2021-10-18 18:26:52 +02:00
Patrick Cloke
5573133348
Move experimental & retention config out of the server module. (#11070) 2021-10-15 14:30:48 +00:00
Dirk Klimpel
013e0f9cae
Update doc of the allowed characters for registration tokens (#11093)
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
2021-10-15 09:56:39 +00:00
David Robertson
404444260a
Fix upgrade dead links (#11069) 2021-10-13 14:32:43 +01:00
Azrenbeth
cdd308845b
Port the Password Auth Providers module interface to the new generic interface (#10548)
Co-authored-by: Azrenbeth <7782548+Azrenbeth@users.noreply.github.com>
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
2021-10-13 11:21:52 +00:00
Brendan Abolivier
9e13cd98af
Update upgrade notes 2021-10-12 11:23:51 +01:00
reivilibre
5e29d417fc
Include the requirements for [mypy,lint] in [dev] (#11034) 2021-10-11 16:34:31 +01:00
Olivier Wilkinson (reivilibre)
593eeac19e Revert accidental push to develop. 2021-10-08 17:17:22 +01:00
Olivier Wilkinson (reivilibre)
9f23ff78da Update contributing guide to use [all,dev] 2021-10-08 17:11:32 +01:00
Andrew Morgan
e564bdd127
Add content to the Synapse documentation intro page (#10990)
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2021-10-06 17:09:35 +00:00
Brendan Abolivier
4e51621064
Add a spamchecker method to allow or deny 3pid invites (#10894)
This is in the context of creating new module callbacks that modules in https://github.com/matrix-org/synapse-dinsic can use, in an effort to reconcile the spam checker API in synapse-dinsic with the one in mainline.

Note that a module callback already exists for 3pid invites (https://matrix-org.github.io/synapse/develop/modules/third_party_rules_callbacks.html#check_threepid_can_be_invited) but it doesn't check whether the sender of the invite is allowed to send it.
2021-10-06 17:18:13 +02:00
Brendan Abolivier
829f2a82b0
Add a spamchecker callback to allow or deny room joins (#10910)
Co-authored-by: Erik Johnston <erik@matrix.org>
2021-10-06 14:32:16 +00:00
Max Kratz
38b7db5885
Updated development doc on samling environment for testing. (#10973) 2021-10-06 07:20:41 -04:00
Max Kratz
6744273f0b
Remove "reference" wording according Synapse homeserver (#10971) 2021-10-06 10:05:07 +00:00
Dirk Klimpel
7036a7a60a
Update links to MSCs in documentation (#10991)
Based on matrix-doc switching from master -> main and
MSCs being merged.
2021-10-05 11:35:19 +00:00
Richard van der Hoff
e32b9f44ee
Update installation instructions (#10919)
Various updates to the install docs.
2021-09-29 10:57:53 +00:00
Sean Quah
f7768f62cb
Avoid storing URL cache files in storage providers (#10911)
URL cache files are short-lived and it does not make sense to offload
them (eg. to the cloud) or back them up.
2021-09-27 12:55:27 +01:00
Brendan Abolivier
b10257e879
Add a spamchecker callback to allow or deny room creation based on invites (#10898)
This is in the context of creating new module callbacks that modules in https://github.com/matrix-org/synapse-dinsic can use, in an effort to reconcile the spam checker API in synapse-dinsic with the one in mainline.

This adds a callback that's fairly similar to user_may_create_room except it also allows processing based on the invites sent at room creation.
2021-09-24 16:38:23 +02:00
David Robertson
ea01d4c2de
Update postgresql testing script (#10906)
- Use sytest:bionic. Sytest:latest is two years old (do we want
  CI to push out latest at all?) and comes with Python 3.5, which we
  explictly no longer support. The script now runs under PostgreSQL 10
  as a result.
- Advertise script in the docs
- Move pg testing script to scripts-dev directory
- Write to host as the script's exector, not root

A few changes to make it speedier to re-run the tests:

- Create blank DB in the container, not the script, so we don't have to
  `initdb` each time
- Use a named volume to persist the tox environment, so we don't have to
  fetch and install a bunch of packages from PyPI each time

Co-authored-by: reivilibre <olivier@librepush.net>
2021-09-24 14:27:09 +00:00
Patrick Cloke
52913d56a5
Add documentation for experimental feature flags. (#10865) 2021-09-22 13:41:42 +00:00
Patrick Cloke
ba7a91aea5
Refactor oEmbed previews (#10814)
The major change is moving the decision of whether to use oEmbed
further up the call-stack. This reverts the _download_url method to
being a "dumb" functionwhich takes a single URL and downloads it
(as it was before #7920).

This also makes more minor refactorings:

* Renames internal variables for clarity.
* Factors out shared code between the HTML and rich oEmbed
  previews.
* Fixes tests to preview an oEmbed image.
2021-09-21 16:09:57 +00:00
David Robertson
60453315bd
Always add local users to the user directory (#10796)
It's a simplification, but one that'll help make the user directory logic easier
to follow with the other changes upcoming. It's not strictly required for those
changes, but this will help simplify the resulting logic that listens for
`m.room.member` events and generally make the logic easier to follow.

This means the config option `search_all_users` ends up controlling the
search query only, and not the data we store. The cost of doing so is an
extra row in the `user_directory` and `user_directory_search` tables for
each local user which

- belongs to no public rooms
- belongs to no private rooms of size ≥ 2

I think the cost of this will be marginal (since they'll already have entries
 in `users` and `profiles` anyway).

As a small upside, a homeserver whose directory was built with this
change can toggle `search_all_users` without having to rebuild their
directory.

Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2021-09-21 12:02:34 +00:00
Charles Wright
6b6bb81b23
Fix #10837 by adding JSON encoding/decoding to the Module API example… (#10845) 2021-09-17 18:04:37 +01:00
BramvdnHeuvel
9f111075e8
Fix copy-paste error in the password section of the sample-config. (#10804) 2021-09-13 08:58:34 -04:00
reivilibre
ceab5a4bfa
Fix 2 typos in docs/log_contexts.md (#10795) 2021-09-10 16:33:36 +01:00
David Robertson
318162f5de
Easy refactors of the user directory (#10789)
No functional changes here. This came out as I was working to tackle #5677
2021-09-10 10:54:38 +01:00
Brendan Abolivier
03caba6577
Improve the modules doc (#10758)
* Split up the documentation in several files rather than one huge one
* Add examples for each callback category
* Other niceties like fixing https://github.com/matrix-org/synapse/issues/10632
* Add titles to callbacks so they're easier to find in the navigation panels and link to
2021-09-08 17:14:54 +00:00
Patrick Cloke
1fdf2cf8e8
Document that /spaces & /hierarchy can be routed to workers. (#10648)
Also refactors some of the registration of endpoints on workers.
2021-09-08 11:02:31 -04:00
Patrick Cloke
89ba834818
Use attrs internally for the URL preview code & add documentation. (#10753) 2021-09-07 13:10:34 +00:00
Azrenbeth
6e895366ea
Add config option to use non-default manhole password and keys (#10643) 2021-09-06 16:08:03 +01:00
David Teller
e1641b46d1
Doc: Clarifying undoing room shutdowns (#10480) 2021-09-06 14:24:31 +01:00
cuttingedge1109
00640ee71a
Fix documentation of directory name for remote thumbnails (#10556) 2021-09-02 14:07:53 +01:00
Olivier Wilkinson (reivilibre)
e059094119 Synapse 1.42.0rc1 (2021-09-01)
==============================
 
 Server administrators are reminded to read [the upgrade notes](docs/upgrade.md#upgrading-to-v1420).
 
 Features
 --------
 
 - Add support for [MSC3231](https://github.com/matrix-org/matrix-doc/pull/3231): Token authenticated registration. Users can be required to submit a token during registration to authenticate themselves. Contributed by Callum Brown. ([\#10142](https://github.com/matrix-org/synapse/issues/10142))
 - Add support for [MSC3283](https://github.com/matrix-org/matrix-doc/pull/3283): Expose `enable_set_displayname` in capabilities. ([\#10452](https://github.com/matrix-org/synapse/issues/10452))
 - Port the `PresenceRouter` module interface to the new generic interface. ([\#10524](https://github.com/matrix-org/synapse/issues/10524))
 - Add pagination to the spaces summary based on updates to [MSC2946](https://github.com/matrix-org/matrix-doc/pull/2946). ([\#10613](https://github.com/matrix-org/synapse/issues/10613), [\#10725](https://github.com/matrix-org/synapse/issues/10725))
 
 Bugfixes
 --------
 
 - Validate new `m.room.power_levels` events. Contributed by @aaronraimist. ([\#10232](https://github.com/matrix-org/synapse/issues/10232))
 - Display an error on User-Interactive Authentication fallback pages when authentication fails. Contributed by Callum Brown. ([\#10561](https://github.com/matrix-org/synapse/issues/10561))
 - Remove pushers when deleting an e-mail address from an account. Pushers for old unlinked emails will also be deleted. ([\#10581](https://github.com/matrix-org/synapse/issues/10581), [\#10734](https://github.com/matrix-org/synapse/issues/10734))
 - Reject Client-Server `/keys/query` requests which provide `device_ids` incorrectly. ([\#10593](https://github.com/matrix-org/synapse/issues/10593))
 - Rooms with unsupported room versions are no longer returned via `/sync`. ([\#10644](https://github.com/matrix-org/synapse/issues/10644))
 - Enforce the maximum length for per-room display names and avatar URLs. ([\#10654](https://github.com/matrix-org/synapse/issues/10654))
 - Fix a bug which caused the `synapse_user_logins_total` Prometheus metric not to be correctly initialised on restart. ([\#10677](https://github.com/matrix-org/synapse/issues/10677))
 - Improve `ServerNoticeServlet` to avoid duplicate requests and add unit tests. ([\#10679](https://github.com/matrix-org/synapse/issues/10679))
 - Fix long-standing issue which caused an error when a thumbnail is requested and there are multiple thumbnails with the same quality rating. ([\#10684](https://github.com/matrix-org/synapse/issues/10684))
 - Fix a regression introduced in v1.41.0 which affected the performance of concurrent fetches of large sets of events, in extreme cases causing the process to hang. ([\#10703](https://github.com/matrix-org/synapse/issues/10703))
 - Fix a regression introduced in Synapse 1.41 which broke email transmission on Systems using older versions of the Twisted library. ([\#10713](https://github.com/matrix-org/synapse/issues/10713))
 
 Improved Documentation
 ----------------------
 
 - Add documentation on how to connect Django with Synapse using OpenID Connect and django-oauth-toolkit. Contributed by @HugoDelval. ([\#10192](https://github.com/matrix-org/synapse/issues/10192))
 - Advertise https://matrix-org.github.io/synapse documentation in the `README` and `CONTRIBUTING` files. ([\#10595](https://github.com/matrix-org/synapse/issues/10595))
 - Fix some of the titles not rendering in the OpenID Connect documentation. ([\#10639](https://github.com/matrix-org/synapse/issues/10639))
 - Minor clarifications to the documentation for reverse proxies. ([\#10708](https://github.com/matrix-org/synapse/issues/10708))
 - Remove table of contents from the top of installation and contributing documentation pages. ([\#10711](https://github.com/matrix-org/synapse/issues/10711))
 
 Deprecations and Removals
 -------------------------
 
 - Remove deprecated Shutdown Room and Purge Room Admin API. ([\#8830](https://github.com/matrix-org/synapse/issues/8830))
 
 Internal Changes
 ----------------
 
 - Improve type hints for the proxy agent and SRV resolver modules. Contributed by @dklimpel. ([\#10608](https://github.com/matrix-org/synapse/issues/10608))
 - Clean up some of the federation event authentication code for clarity. ([\#10614](https://github.com/matrix-org/synapse/issues/10614), [\#10615](https://github.com/matrix-org/synapse/issues/10615), [\#10624](https://github.com/matrix-org/synapse/issues/10624), [\#10640](https://github.com/matrix-org/synapse/issues/10640))
 - Add a comment asking developers to leave a reason when bumping the database schema version. ([\#10621](https://github.com/matrix-org/synapse/issues/10621))
 - Remove not needed database updates in modify user admin API. ([\#10627](https://github.com/matrix-org/synapse/issues/10627))
 - Convert room member storage tuples to `attrs` classes. ([\#10629](https://github.com/matrix-org/synapse/issues/10629), [\#10642](https://github.com/matrix-org/synapse/issues/10642))
 - Use auto-attribs for the attrs classes used in sync. ([\#10630](https://github.com/matrix-org/synapse/issues/10630))
 - Make `backfill` and `get_missing_events` use the same codepath. ([\#10645](https://github.com/matrix-org/synapse/issues/10645))
 - Improve the performance of the `/hierarchy` API (from [MSC2946](https://github.com/matrix-org/matrix-doc/pull/2946)) by caching responses received over federation. ([\#10647](https://github.com/matrix-org/synapse/issues/10647))
 - Run a nightly CI build against Twisted trunk. ([\#10651](https://github.com/matrix-org/synapse/issues/10651), [\#10672](https://github.com/matrix-org/synapse/issues/10672))
 - Do not print out stack traces for network errors when fetching data over federation. ([\#10662](https://github.com/matrix-org/synapse/issues/10662))
 - Simplify tests for device admin rest API. ([\#10664](https://github.com/matrix-org/synapse/issues/10664))
 - Add missing type hints to REST servlets. ([\#10665](https://github.com/matrix-org/synapse/issues/10665), [\#10666](https://github.com/matrix-org/synapse/issues/10666), [\#10674](https://github.com/matrix-org/synapse/issues/10674))
 - Flatten the `tests.synapse.rests` package by moving the contents of `v1` and `v2_alpha` into the parent. ([\#10667](https://github.com/matrix-org/synapse/issues/10667))
 - Update `complement.sh` to rebuild the base Docker image when run with workers. ([\#10686](https://github.com/matrix-org/synapse/issues/10686))
 - Split the event-processing methods in `FederationHandler` into a separate `FederationEventHandler`. ([\#10692](https://github.com/matrix-org/synapse/issues/10692))
 - Remove unused `compare_digest` function. ([\#10706](https://github.com/matrix-org/synapse/issues/10706))
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCgAdFiEE8SRSDO7gYkSP4chELS76LzL74EcFAmEvd3cACgkQLS76LzL7
 4EcF0hAAvOjDeH2hn0eI4YX03/BTCesJ5pCTbEJeC9Hsui3SbWamyYarH0Ae0gM6
 BP9x5/XkCdsZhKT2OJedtZQUQ0cUws5FdJP87NnE1m7KBW9djbTxmkue3JKaHNTT
 ajvEoLe8UjOFbjvyMc+188uWVgtKQ6KLcetwllTpYY3FJ3+yk9CQP1ydYzwfHcHe
 hXW3b20WoRdAmH0B27mj1oQzCeXclbj1kCthcBcxH5RytA8bBc1itBtpAP2FFaeS
 Ml6paoXSs0wZaXXL5T1MhQtcVOOG5Yh7EPsemdyfqkwhIk9QtV3nmGfC7K5iEeq0
 DldTgBNE0wLO+ycdgEZ2oCvitCLT4WDoKkVqKPA91wpuvCMeK5lnDdqbFUSN3hfd
 l8Oz+ydhjsCx7+pBBopnsyGrPIHNaN/vySYGF3lPa3ANKS2J0IVlAaJ59bZqzg2C
 vzdyMBCnn0fhnXMkADPLnIYqtebvC09JlPBRGo5BKmwmlM8Ea1dLQ+Lcy5vXalcw
 +GaYw3U805c03EvsWQxWchjwq6IwMeP06YpK/1ruMjlrCWYJp3V7f/4hXjlTdYm5
 OEWc/kHOnlLfH2cA4aS0sV5WsnEuftotQIzBXuXWw5hSMdFmH7eAviPg0vXgfeA/
 i0FOkOgEYR8O3RMh4Cip8+h44t8SPNjYUFJK9ip3DmNateeUCH0=
 =XxMc
 -----END PGP SIGNATURE-----

Merge tag 'v1.42.0rc1' into develop

Synapse 1.42.0rc1 (2021-09-01)
==============================

Server administrators are reminded to read [the upgrade notes](docs/upgrade.md#upgrading-to-v1420).

Features
--------

- Add support for [MSC3231](https://github.com/matrix-org/matrix-doc/pull/3231): Token authenticated registration. Users can be required to submit a token during registration to authenticate themselves. Contributed by Callum Brown. ([\#10142](https://github.com/matrix-org/synapse/issues/10142))
- Add support for [MSC3283](https://github.com/matrix-org/matrix-doc/pull/3283): Expose `enable_set_displayname` in capabilities. ([\#10452](https://github.com/matrix-org/synapse/issues/10452))
- Port the `PresenceRouter` module interface to the new generic interface. ([\#10524](https://github.com/matrix-org/synapse/issues/10524))
- Add pagination to the spaces summary based on updates to [MSC2946](https://github.com/matrix-org/matrix-doc/pull/2946). ([\#10613](https://github.com/matrix-org/synapse/issues/10613), [\#10725](https://github.com/matrix-org/synapse/issues/10725))

Bugfixes
--------

- Validate new `m.room.power_levels` events. Contributed by @aaronraimist. ([\#10232](https://github.com/matrix-org/synapse/issues/10232))
- Display an error on User-Interactive Authentication fallback pages when authentication fails. Contributed by Callum Brown. ([\#10561](https://github.com/matrix-org/synapse/issues/10561))
- Remove pushers when deleting an e-mail address from an account. Pushers for old unlinked emails will also be deleted. ([\#10581](https://github.com/matrix-org/synapse/issues/10581), [\#10734](https://github.com/matrix-org/synapse/issues/10734))
- Reject Client-Server `/keys/query` requests which provide `device_ids` incorrectly. ([\#10593](https://github.com/matrix-org/synapse/issues/10593))
- Rooms with unsupported room versions are no longer returned via `/sync`. ([\#10644](https://github.com/matrix-org/synapse/issues/10644))
- Enforce the maximum length for per-room display names and avatar URLs. ([\#10654](https://github.com/matrix-org/synapse/issues/10654))
- Fix a bug which caused the `synapse_user_logins_total` Prometheus metric not to be correctly initialised on restart. ([\#10677](https://github.com/matrix-org/synapse/issues/10677))
- Improve `ServerNoticeServlet` to avoid duplicate requests and add unit tests. ([\#10679](https://github.com/matrix-org/synapse/issues/10679))
- Fix long-standing issue which caused an error when a thumbnail is requested and there are multiple thumbnails with the same quality rating. ([\#10684](https://github.com/matrix-org/synapse/issues/10684))
- Fix a regression introduced in v1.41.0 which affected the performance of concurrent fetches of large sets of events, in extreme cases causing the process to hang. ([\#10703](https://github.com/matrix-org/synapse/issues/10703))
- Fix a regression introduced in Synapse 1.41 which broke email transmission on Systems using older versions of the Twisted library. ([\#10713](https://github.com/matrix-org/synapse/issues/10713))

Improved Documentation
----------------------

- Add documentation on how to connect Django with Synapse using OpenID Connect and django-oauth-toolkit. Contributed by @HugoDelval. ([\#10192](https://github.com/matrix-org/synapse/issues/10192))
- Advertise https://matrix-org.github.io/synapse documentation in the `README` and `CONTRIBUTING` files. ([\#10595](https://github.com/matrix-org/synapse/issues/10595))
- Fix some of the titles not rendering in the OpenID Connect documentation. ([\#10639](https://github.com/matrix-org/synapse/issues/10639))
- Minor clarifications to the documentation for reverse proxies. ([\#10708](https://github.com/matrix-org/synapse/issues/10708))
- Remove table of contents from the top of installation and contributing documentation pages. ([\#10711](https://github.com/matrix-org/synapse/issues/10711))

Deprecations and Removals
-------------------------

- Remove deprecated Shutdown Room and Purge Room Admin API. ([\#8830](https://github.com/matrix-org/synapse/issues/8830))

Internal Changes
----------------

- Improve type hints for the proxy agent and SRV resolver modules. Contributed by @dklimpel. ([\#10608](https://github.com/matrix-org/synapse/issues/10608))
- Clean up some of the federation event authentication code for clarity. ([\#10614](https://github.com/matrix-org/synapse/issues/10614), [\#10615](https://github.com/matrix-org/synapse/issues/10615), [\#10624](https://github.com/matrix-org/synapse/issues/10624), [\#10640](https://github.com/matrix-org/synapse/issues/10640))
- Add a comment asking developers to leave a reason when bumping the database schema version. ([\#10621](https://github.com/matrix-org/synapse/issues/10621))
- Remove not needed database updates in modify user admin API. ([\#10627](https://github.com/matrix-org/synapse/issues/10627))
- Convert room member storage tuples to `attrs` classes. ([\#10629](https://github.com/matrix-org/synapse/issues/10629), [\#10642](https://github.com/matrix-org/synapse/issues/10642))
- Use auto-attribs for the attrs classes used in sync. ([\#10630](https://github.com/matrix-org/synapse/issues/10630))
- Make `backfill` and `get_missing_events` use the same codepath. ([\#10645](https://github.com/matrix-org/synapse/issues/10645))
- Improve the performance of the `/hierarchy` API (from [MSC2946](https://github.com/matrix-org/matrix-doc/pull/2946)) by caching responses received over federation. ([\#10647](https://github.com/matrix-org/synapse/issues/10647))
- Run a nightly CI build against Twisted trunk. ([\#10651](https://github.com/matrix-org/synapse/issues/10651), [\#10672](https://github.com/matrix-org/synapse/issues/10672))
- Do not print out stack traces for network errors when fetching data over federation. ([\#10662](https://github.com/matrix-org/synapse/issues/10662))
- Simplify tests for device admin rest API. ([\#10664](https://github.com/matrix-org/synapse/issues/10664))
- Add missing type hints to REST servlets. ([\#10665](https://github.com/matrix-org/synapse/issues/10665), [\#10666](https://github.com/matrix-org/synapse/issues/10666), [\#10674](https://github.com/matrix-org/synapse/issues/10674))
- Flatten the `tests.synapse.rests` package by moving the contents of `v1` and `v2_alpha` into the parent. ([\#10667](https://github.com/matrix-org/synapse/issues/10667))
- Update `complement.sh` to rebuild the base Docker image when run with workers. ([\#10686](https://github.com/matrix-org/synapse/issues/10686))
- Split the event-processing methods in `FederationHandler` into a separate `FederationEventHandler`. ([\#10692](https://github.com/matrix-org/synapse/issues/10692))
- Remove unused `compare_digest` function. ([\#10706](https://github.com/matrix-org/synapse/issues/10706))
2021-09-01 14:58:14 +01:00
Olivier Wilkinson (reivilibre)
6b2aca473a 1.42.0rc1 2021-09-01 11:47:24 +01:00
Patrick Cloke
e2481dbe93
Allow configuration of the oEmbed URLs. (#10714)
This adds configuration options (under an `oembed` section) to
configure which URLs are matched to use oEmbed for URL
previews.
2021-08-31 18:37:07 -04:00
David Robertson
46ff99ef95
Advertise matrix-org.github.io/synapse docs (#10595)
Point to the book where possible, and use hyperlinks to github to refer to files not included in the book.

Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2021-08-31 11:29:27 +01:00
Azrenbeth
54aa7047eb
Removed page summaries from the top of installation and contributing doc pages (#10711)
- Removed page summaries from CONTRIBUTING and installation pages as 
this information was already in the table of contents on the right hand side
- Fixed some broken links in CONTRIBUTING
- Added margin-right tag for when table of contents is being shown
(otherwise the text in the page sometimes overlaps with it)
2021-08-27 14:19:17 +00:00
Richard van der Hoff
051ddac53b
Clarifications to reverse_proxy.md (#10708)
* Update reverse_proxy.md

* Create 10708.doc
2021-08-27 12:54:21 +01:00
Azrenbeth
ad17fbd20e
Remove pushers when deleting 3pid from account (#10581)
When a user deletes an email from their account it will
now also remove all pushers for that email and that user
(even if these pushers were created by a different client)
2021-08-26 13:53:57 +01:00
Hugo DELVAL
86415f162d
doc: add django-oauth-toolkit to oidc doc (#10192)
Signed-off-by: Hugo Delval <hugo.delval@gmail.com>
2021-08-23 17:12:36 +00:00
Andrew Morgan
3e83f97154
Fix the titles in the OIDC documentation (#10639)
* Fix the titles in the OIDC documentation

Having them as links broke the table-of-contents rendering in mdbook.
Plus there's no reason for only some of the provider titles to be links.

* Changelog

* Add link to google idp docs
2021-08-23 14:58:31 +01:00
Callum Brown
947dbbdfd1
Implement MSC3231: Token authenticated registration (#10142)
Signed-off-by: Callum Brown <callum@calcuode.com>

This is part of my GSoC project implementing [MSC3231](https://github.com/matrix-org/matrix-doc/pull/3231).
2021-08-21 22:14:43 +01:00
Dirk Klimpel
220f901229
Remove not needed database updates in modify user admin API (#10627) 2021-08-19 10:25:05 +01:00
Erik Johnston
78a70a2e0b Merge branch 'release-v1.41' into develop 2021-08-18 17:02:47 +01:00
Erik Johnston
b9c35586a4 Update docs/upgrade.md with new version 2021-08-18 16:59:36 +01:00
Andrew Morgan
3692f7fd33
Mount /_synapse/admin/v1/users/{userId}/media admin API on media workers only (#10628)
Co-authored-by: Patrick Cloke <patrickc@matrix.org>
2021-08-18 13:25:12 +01:00
Callum Brown
6e613a10d0
Display an error page during failure of fallback UIA. (#10561) 2021-08-18 08:13:35 -04:00
Dirk Klimpel
430241a1e9
Remove deprecated Shutdown Room and Purge Room Admin API (#8830) 2021-08-17 21:19:13 +01:00
Andrew Morgan
703e3a9e85
Allow /createRoom to be run on workers (#10564)
Fixes https://github.com/matrix-org/synapse/issues/7867
2021-08-17 14:33:16 +01:00
Azrenbeth
1a9f531c79
Port the PresenceRouter module interface to the new generic interface (#10524)
Port the PresenceRouter module interface to the new generic interface introduced in v1.37.0
2021-08-17 13:22:45 +00:00
Brendan Abolivier
5639759980
Centralise the custom template directory (#10596)
Several configuration sections are using separate settings for custom template directories, which can be confusing. This PR adds a new top-level configuration for a custom template directory which is then used for every module. The only exception is the consent templates, since the consent template directory require a specific hierarchy, so it's probably better that it stays separate from everything else.
2021-08-17 14:45:24 +02:00
reivilibre
5f7b1e1f27
Make PeriodicallyFlushingMemoryHandler the default logging handler. (#10518) 2021-08-17 13:13:11 +01:00
Dirk Klimpel
3bcd525b46
Allow to edit external_ids by Edit User admin API (#10598)
Signed-off-by: Dirk Klimpel dirk@klimpel.org
2021-08-17 11:56:11 +01:00
Will Hunt
a933c2c7d8
Add an admin API to check if a username is available (#10578)
This adds a new API GET /_synapse/admin/v1/username_available?username=foo to check if a username is available. It is the counterpart to https://matrix.org/docs/spec/client_server/r0.6.0#get-matrix-client-r0-register-available, except that it works even if registration is disabled.
2021-08-17 10:52:38 +01:00
reivilibre
19e51b14d2
Manhole: wrap coroutines in defer.ensureDeferred automatically (#10602) 2021-08-16 18:11:48 +01:00
Dirk Klimpel
915b37e5ef
Admin API to delete media for a specific user (#10558) 2021-08-11 19:29:59 +00:00
Dirk Klimpel
339c3918e1
support federation queries through http connect proxy (#10475)
Signed-off-by: Marcus Hoffmann <bubu@bubu1.eu>
Signed-off-by: Dirk Klimpel dirk@klimpel.org
2021-08-11 15:34:59 +01:00
Drew Short
189c055eb6
Moved homeserver documentation above reverse proxy examples (#10551)
Signed-off-by: Drew Short <warrick@sothr.com>
2021-08-09 15:12:53 +00:00
Richard van der Hoff
74d7336686
Add a setting to disable TLS for sending email (#10546)
This is mostly useful in case the server offers TLS, but doesn't present a valid certificate.
2021-08-06 10:13:34 +00:00
Richard van der Hoff
0cb4274dbf Merge remote-tracking branch 'origin/release-v1.40' into develop 2021-08-05 19:01:03 +01:00
Dirk Klimpel
4578531002
fix broken links in upgrade.md (#10543)
Signed-off-by: Dirk Klimpel dirk@klimpel.org
2021-08-05 19:00:44 +01:00
Dirk Klimpel
834cdc3606
Add documentation for configuring a forward proxy. (#10443) 2021-08-05 07:20:05 -04:00
Kento Okamoto
72935b7c50
Add warnings to ip_range_blacklist usage with proxies (#10129)
Per issue #9812 using `url_preview_ip_range_blacklist` with a proxy via `HTTPS_PROXY` or `HTTP_PROXY` environment variables has some inconsistent bahavior than mentioned. This PR changes the following:

- Changes the Sample Config file to include a note mentioning that `url_preview_ip_range_blacklist` and `ip_range_blacklist` is ignored when using a proxy
- Changes some logic in synapse/config/repository.py to send a warning when both `*ip_range_blacklist` configs and a proxy environment variable are set and but no longer throws an error.

Signed-off-by: Kento Okamoto <kentokamoto@protonmail.com>
2021-08-03 18:13:34 +00:00
Richard van der Hoff
4b10880da3
Make sync response cache time configurable. (#10513) 2021-08-03 14:45:04 +01:00
reivilibre
f4ac934afe
Revert use of PeriodicallyFlushingMemoryHandler by default (#10515) 2021-08-03 11:30:39 +01:00
Eric Eastwood
2bae2c632f
Add developer documentation to explain room DAG concepts like outliers and state_groups (#10464) 2021-08-03 10:08:57 +00:00
Toni Spets
ba5287f5e8
Allow setting transaction limit for db connections (#10440)
Setting the value will help PostgreSQL free up memory by recycling
the connections in the connection pool.

Signed-off-by: Toni Spets <toni.spets@iki.fi>
2021-08-02 13:24:43 +00:00
reivilibre
e16eab29d6
Add a PeriodicallyFlushingMemoryHandler to prevent logging silence (#10407)
Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
2021-07-27 14:32:05 +01:00
Dirk Klimpel
89c4ca81bb
Add creation_ts to list users admin API (#10448)
Signed-off-by: Dirk Klimpel dirk@klimpel.org
2021-07-22 16:05:16 +02:00
Eric Eastwood
d518b05a86
Move dev/ docs to development/ (#10453) 2021-07-22 12:58:24 +02:00
Patrick Cloke
5b68816de9
Fix the hierarchy of OpenID providers in the docs. (#10445) 2021-07-21 13:48:06 -04:00
Brendan Abolivier
a743bf4694
Port the ThirdPartyEventRules module interface to the new generic interface (#10386)
Port the third-party event rules interface to the generic module interface introduced in v1.37.0
2021-07-20 12:39:46 +02:00
Brendan Abolivier
36dc15412d
Add a module type for account validity (#9884)
This adds an API for third-party plugin modules to implement account validity, so they can provide this feature instead of Synapse. The module implementing the current behaviour for this feature can be found at https://github.com/matrix-org/synapse-email-account-validity.

To allow for a smooth transition between the current feature and the new module, hooks have been added to the existing account validity endpoints to allow their behaviours to be overridden by a module.
2021-07-16 18:11:53 +02:00
Brendan Abolivier
9f497024aa Merge branch 'master' into develop 2021-07-15 14:54:45 +01:00
Brendan Abolivier
3fffb71254
Make deprecation notice of the spam checker doc more obvious (#10395) 2021-07-15 15:54:22 +02:00
Luke Walsh
23a90a6a5c
Updating install prerequisites for newer macOS & ARM Macs. (#9971) 2021-07-15 13:18:58 +01:00
Andrew Morgan
c141455049
Docs: Use something other than the document name to describe a page (#10399)
Our documentation has a history of using a document's name as a way to link to it, such as "See [workers.md]() for details". This makes sense when you're traversing a directory of files, but less sense when the files are abstracted away - as they are on the documentation website.

This PR changes the links to various documentation pages to something that fits better into the surrounding sentence, as you would when making any hyperlink on the web.
2021-07-15 12:47:55 +01:00
Brendan Abolivier
5ecad4e7a5
Update the logcontext doc (#10353)
By referring to awaitables instead of deferreds.
2021-07-15 11:38:05 +01:00
Moritz Dietz
0ae95b3847
doc: Add delegation example to the caddy reverse proxy section (#10368) 2021-07-14 18:50:30 +02:00
Richard van der Hoff
28ffff73c1
Instructions on installing RC debs (#10396) 2021-07-14 17:12:01 +01:00
Dirk Klimpel
e938f69697
Fix some links in docs and contrib (#10370) 2021-07-13 11:55:48 +01:00
Erik Johnston
251cfc4e09 Synapse 1.38.0rc2 (2021-07-09)
==============================
 
 Bugfixes
 --------
 
 - Fix bug where inbound federation in a room could be delayed due to not correctly dropping a lock. Introduced in v1.37.1. ([\#10336](https://github.com/matrix-org/synapse/issues/10336))
 
 Improved Documentation
 ----------------------
 
 - Update links to documentation in the sample config. Contributed by @dklimpel. ([\#10287](https://github.com/matrix-org/synapse/issues/10287))
 - Fix broken links in [INSTALL.md](INSTALL.md). Contributed by @dklimpel. ([\#10331](https://github.com/matrix-org/synapse/issues/10331))
 -----BEGIN PGP SIGNATURE-----
 
 iQFEBAABCgAuFiEEBTGR3/RnAzBGUif3pULk7RsPrAkFAmDoH+4QHGVyaWtAbWF0
 cml4Lm9yZwAKCRClQuTtGw+sCXxYCACneuRvkdvYqiH+PhPe8tXqhhJIifH1LecY
 FlJqp4OJPR2VFzio1btsgpRPQyLBLHZkJ9pgWsXAETbYOO+hSeOc4nIHsyqlSJhe
 v01sCUE4sle3DBrw15fG4XpercsiM3TFMyR9pV9laq9nIn8j+CY5K6W5t12/mYGy
 asHS0IKilCMhJlFwgE3eBr6P6fywi0JoIrr8EpfIs4eC2qDFpUlsrAQSkbE1JvdP
 O4BGZJKVysg3a6WYSWdJytqLYe942k8qUF4B4h4VmQi0xbuKSsTLiK/cFC8ohRMv
 E+O5O/KgwqwE/XOcukbsjlHxuiiFZTq6154PwLxXUpNnsMNn2/ph
 =6iBw
 -----END PGP SIGNATURE-----

Merge tag 'v1.38.0rc2' into develop

Synapse 1.38.0rc2 (2021-07-09)
==============================

Bugfixes
--------

- Fix bug where inbound federation in a room could be delayed due to not correctly dropping a lock. Introduced in v1.37.1. ([\#10336](https://github.com/matrix-org/synapse/issues/10336))

Improved Documentation
----------------------

- Update links to documentation in the sample config. Contributed by @dklimpel. ([\#10287](https://github.com/matrix-org/synapse/issues/10287))
- Fix broken links in [INSTALL.md](INSTALL.md). Contributed by @dklimpel. ([\#10331](https://github.com/matrix-org/synapse/issues/10331))
2021-07-09 11:26:17 +01:00
Brendan Abolivier
33ae301fee
Fix formatting in the logcontext doc (#10337) 2021-07-08 18:16:30 +02:00
Cristina
f6767abc05
Remove functionality associated with unused historical stats tables (#9721)
Fixes #9602
2021-07-08 16:57:13 +01:00
Dirk Klimpel
974261cd81
Fix broken links in INSTALL.md (#10331)
Signed-off-by: Dirk Klimpel dirk@klimpel.org
2021-07-08 15:46:13 +01:00
Richard van der Hoff
189652b2fe
Fix a broken link in the admin api docs (#10322)
* Fix a broken link in the admin api docs

* Rename 10321.doc to 10321.docs

* Rename 10321.docs to 10322.doc
2021-07-07 12:54:57 +01:00
Richard van der Hoff
240b3ce253 Merge remote-tracking branch 'origin/release-v1.38' into develop
merge @dklimpel's fixes to the sample config
2021-07-07 12:37:10 +01:00
Dirk Klimpel
56fd5fa8e1
Update links to documentation in sample config (#10287)
Signed-off-by: Dirk Klimpel dirk@klimpel.org
2021-07-07 12:35:45 +01:00
Dirk Klimpel
2d044667cf
Simplify structure of room admin API docs (#10313) 2021-07-07 12:18:36 +01:00
Richard van der Hoff
d7a94a7dcc
Add upgrade notes about disk space for events migration (#10314) 2021-07-06 11:00:05 +01:00
Erik Johnston
7a5873277e
Add support for evicting cache entries based on last access time. (#10205) 2021-07-05 16:32:12 +01:00
Brendan Abolivier
10671da05b
Fix bad link in modules documentation (#10302)
Fix link in modules doc to point at instructions on registering a callback instead of ones on registering a web resource.
2021-07-02 13:20:43 +02:00