Commit Graph

1098 Commits

Author SHA1 Message Date
Dirk Klimpel
ea20937084
Add an admin API to run background jobs. (#11352)
Instead of having admins poke into the database directly.

Can currently run jobs to populate stats and to populate
the user directory.
2021-11-19 19:39:46 +00:00
Patrick Cloke
7ae559944a
Fix checking whether a room can be published on creation. (#11392)
If `room_list_publication_rules` was configured with a rule with a
non-wildcard alias and a room was created with an alias then an
internal server error would have been thrown.

This fixes the error and properly applies the publication rules
during room creation.
2021-11-19 15:19:32 +00:00
Patrick Cloke
4d6d38ac2f
Remove dead code from acme support. (#11393) 2021-11-19 07:07:22 -05:00
Shay
92b75388f5
Remove legacy code related to deprecated trust_identity_server_for_password_resets config flag (#11333)
* remove code legacy code related to deprecated config flag "trust_identity_server_for_password_resets" from synapse/config/emailconfig.py

* remove legacy code supporting depreciated config flag "trust_identity_server_for_password_resets" from synapse/config/registration.py

* remove legacy code supporting depreciated config flag "trust_identity_server_for_password_resets" from synapse/handlers/identity.py

* add tests to ensure config error is thrown and synapse refuses to start when depreciated config flag is found

* add changelog

* slightly change behavior to only check for deprecated flag if set to 'true'

* Update changelog.d/11333.misc

Co-authored-by: reivilibre <oliverw@matrix.org>

Co-authored-by: reivilibre <oliverw@matrix.org>
2021-11-18 10:56:32 -08:00
Nicolai Søborg
e2dabec996
Docs: Quote wildcard federation_certificate_verification_whitelist (#11381)
Otherwise I get this beautiful stacktrace:

```
python3 -m synapse.app.homeserver --config-path /etc/matrix/homeserver.yaml
Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/root/synapse/synapse/app/homeserver.py", line 455, in <module>
    main()
  File "/root/synapse/synapse/app/homeserver.py", line 445, in main
    hs = setup(sys.argv[1:])
  File "/root/synapse/synapse/app/homeserver.py", line 345, in setup
    config = HomeServerConfig.load_or_generate_config(
  File "/root/synapse/synapse/config/_base.py", line 671, in load_or_generate_config
    config_dict = read_config_files(config_files)
  File "/root/synapse/synapse/config/_base.py", line 717, in read_config_files
    yaml_config = yaml.safe_load(file_stream)
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/__init__.py", line 125, in safe_load
    return load(stream, SafeLoader)
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/__init__.py", line 81, in load
    return loader.get_single_data()
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/constructor.py", line 49, in get_single_data
    node = self.get_single_node()
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/composer.py", line 36, in get_single_node
    document = self.compose_document()
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/composer.py", line 55, in compose_document
    node = self.compose_node(None, None)
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/composer.py", line 84, in compose_node
    node = self.compose_mapping_node(anchor)
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/composer.py", line 133, in compose_mapping_node
    item_value = self.compose_node(node, item_key)
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/composer.py", line 82, in compose_node
    node = self.compose_sequence_node(anchor)
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/composer.py", line 110, in compose_sequence_node
    while not self.check_event(SequenceEndEvent):
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/parser.py", line 98, in check_event
    self.current_event = self.state()
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/parser.py", line 379, in parse_block_sequence_first_entry
    return self.parse_block_sequence_entry()
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/parser.py", line 384, in parse_block_sequence_entry
    if not self.check_token(BlockEntryToken, BlockEndToken):
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/scanner.py", line 116, in check_token
    self.fetch_more_tokens()
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/scanner.py", line 227, in fetch_more_tokens
    return self.fetch_alias()
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/scanner.py", line 610, in fetch_alias
    self.tokens.append(self.scan_anchor(AliasToken))
  File "/root/synapse/env/lib/python3.8/site-packages/yaml/scanner.py", line 922, in scan_anchor
    raise ScannerError("while scanning an %s" % name, start_mark,
yaml.scanner.ScannerError: while scanning an alias
  in "/etc/matrix/homeserver.yaml", line 614, column 5
expected alphabetic or numeric character, but found '.'
  in "/etc/matrix/homeserver.yaml", line 614, column 6
```

Signed-off-by: Nicolai Søborg <git@xn--sb-lka.org>
2021-11-18 12:24:40 +00:00
Richard van der Hoff
86a497efaa
Default value for public_baseurl (#11210)
We might as well use a default value for `public_baseurl` based on
`server_name` - in many cases, it will be correct.
2021-11-08 14:13:10 +00:00
Nick Barrett
af54167516
Enable passing typing stream writers as a list. (#11237)
This makes the typing stream writer config match the other stream writers
that only currently support a single worker.
2021-11-03 14:25:47 +00:00
Richard van der Hoff
71f9966f27
Support for serving server well-known files (#11211)
Fixes https://github.com/matrix-org/synapse/issues/8308
2021-11-01 15:10:16 +00:00
Sean Quah
2b82ec425f
Add type hints for most HomeServer parameters (#11095) 2021-10-22 18:15:41 +01:00
Jason Robinson
b9ce53e878
Fix synapse.config module "read" command (#11145)
`synapse.config.__main__` has the possibility to read a config item. This can be used to conveniently also validate the config is valid before trying to start Synapse.

 The "read" command broke in https://github.com/matrix-org/synapse/pull/10916 as it now requires passing in "server.server_name" for example.

 Also made the read command optional so one can just call this with just the confirm file reference and get a "Config parses OK" if things are ok.

Signed-off-by: Jason Robinson <jasonr@matrix.org>
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
2021-10-22 12:00:52 +02:00
Patrick Cloke
ba00e20234
Add a thread relation type per MSC3440. (#11088)
Adds experimental support for MSC3440's `io.element.thread` relation
type (and the aggregation for it).
2021-10-21 14:39:16 -04:00
Patrick Cloke
5573133348
Move experimental & retention config out of the server module. (#11070) 2021-10-15 14:30:48 +00:00
Azrenbeth
cdd308845b
Port the Password Auth Providers module interface to the new generic interface (#10548)
Co-authored-by: Azrenbeth <7782548+Azrenbeth@users.noreply.github.com>
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
2021-10-13 11:21:52 +00:00
David Robertson
797ee7812d
Relax ignore-missing-imports for modules that have stubs now and update mypy (#11006)
Updating mypy past version 0.9 means that third-party stubs are no-longer distributed with typeshed. See http://mypy-lang.blogspot.com/2021/06/mypy-0900-released.html for details.
We therefore pull in stub packages in setup.py

Additionally, some modules that we were previously ignoring import failures for now have stubs. So let's use them.

The rest of this change consists of fixups to make the newer mypy + stubs pass CI.

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2021-10-08 14:49:41 +01:00
Patrick Cloke
f4b1a9a527
Require direct references to configuration variables. (#10985)
This removes the magic allowing accessing configurable
variables directly from the config object. It is now required
that a specific configuration class is used (e.g. `config.foo`
must be replaced with `config.server.foo`).
2021-10-06 10:47:41 -04:00
Patrick Cloke
94b620a5ed
Use direct references for configuration variables (part 6). (#10916) 2021-09-29 06:44:15 -04:00
Patrick Cloke
eb2c7e51c4
Clean-up type hints in server config (#10915)
By using attrs instead of dicts to store configuration.

Also updates some of the attrs classes to use proper type
hints and auto_attribs.
2021-09-28 09:24:40 -04:00
Sean Quah
6c83c27107
Fix race conditions when creating media store and config directories (#10913) 2021-09-27 11:29:23 +01:00
Patrick Cloke
bb7fdd821b
Use direct references for configuration variables (part 5). (#10897) 2021-09-24 07:25:21 -04:00
Patrick Cloke
47854c71e9
Use direct references for configuration variables (part 4). (#10893) 2021-09-23 12:03:01 -04:00
Patrick Cloke
e584534403
Use direct references for some configuration variables (part 3) (#10885)
This avoids the overhead of searching through the various
configuration classes by directly referencing the class that
the attributes are in.

It also improves type hints since mypy can now resolve the
types of the configuration variables.
2021-09-23 07:13:34 -04:00
Andrew Morgan
aa2c027792
Remove unnecessary parentheses around tuples returned from methods (#10889) 2021-09-23 11:59:07 +01:00
David Robertson
60453315bd
Always add local users to the user directory (#10796)
It's a simplification, but one that'll help make the user directory logic easier
to follow with the other changes upcoming. It's not strictly required for those
changes, but this will help simplify the resulting logic that listens for
`m.room.member` events and generally make the logic easier to follow.

This means the config option `search_all_users` ends up controlling the
search query only, and not the data we store. The cost of doing so is an
extra row in the `user_directory` and `user_directory_search` tables for
each local user which

- belongs to no public rooms
- belongs to no private rooms of size ≥ 2

I think the cost of this will be marginal (since they'll already have entries
 in `users` and `profiles` anyway).

As a small upside, a homeserver whose directory was built with this
change can toggle `search_all_users` without having to rebuild their
directory.

Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2021-09-21 12:02:34 +00:00
Patrick Cloke
b3590614da
Require type hints in the handlers module. (#10831)
Adds missing type hints to methods in the synapse.handlers
module and requires all methods to have type hints there.

This also removes the unused construct_auth_difference method
from the FederationHandler.
2021-09-20 08:56:23 -04:00
Patrick Cloke
01c88a09cd
Use direct references for some configuration variables (#10798)
Instead of proxying through the magic getter of the RootConfig
object. This should be more performant (and is more explicit).
2021-09-13 13:07:12 -04:00
BramvdnHeuvel
9f111075e8
Fix copy-paste error in the password section of the sample-config. (#10804) 2021-09-13 08:58:34 -04:00
reivilibre
524b8ead77
Add types to synapse.util. (#10601) 2021-09-10 17:03:18 +01:00
Sean
273b6861f2
Remove unstable MSC2858 API, including experimental.msc2858_enabled config option (#10693)
Signed-off-by: Sean Quah <seanq@element.io>
2021-09-09 17:59:59 +01:00
Patrick Cloke
580a15e039
Request JSON for oEmbed requests (and ignore XML only providers). (#10759)
This adds the format to the request arguments / URL to
ensure that JSON data is returned (which is all that
Synapse supports).

This also adds additional error checking / filtering to the
configuration file to ignore XML-only providers.
2021-09-08 07:17:52 -04:00
Brendan Abolivier
287108fb2e Merge tag 'v1.42.0rc2' into develop
Synapse 1.42.0rc2 (2021-09-06)
==============================

This version of Synapse removes deprecated room-management admin APIs, removes out-of-date
email pushers, and improves error handling for fallback templates for user-interactive
authentication. For more information on these points, server administrators are
encouraged to read [the upgrade notes](docs/upgrade.md#upgrading-to-v1420).

Features
--------

- Support room version 9 from [MSC3375](https://github.com/matrix-org/matrix-doc/pull/3375). ([\#10747](https://github.com/matrix-org/synapse/issues/10747))

Internal Changes
----------------

- Print a warning when using one of the deprecated `template_dir` settings. ([\#10768](https://github.com/matrix-org/synapse/issues/10768))
2021-09-06 16:29:59 +01:00
Azrenbeth
6e895366ea
Add config option to use non-default manhole password and keys (#10643) 2021-09-06 16:08:03 +01:00
Brendan Abolivier
8c9e723fe0
Add a warning when using deprecated template_dir settings (#10768)
The deprecation itself happened in #10596 which shipped with Synapse v1.41.0. However, it doesn't seem fair to suddenly drop support for these settings in ~4-6w without being more vocal about said deprecation.
2021-09-06 16:23:50 +02:00
Patrick Cloke
e2481dbe93
Allow configuration of the oEmbed URLs. (#10714)
This adds configuration options (under an `oembed` section) to
configure which URLs are matched to use oEmbed for URL
previews.
2021-08-31 18:37:07 -04:00
Erik Johnston
cd22fb568a Merge branch 'master' into develop 2021-08-24 17:13:29 +01:00
Callum Brown
947dbbdfd1
Implement MSC3231: Token authenticated registration (#10142)
Signed-off-by: Callum Brown <callum@calcuode.com>

This is part of my GSoC project implementing [MSC3231](https://github.com/matrix-org/matrix-doc/pull/3231).
2021-08-21 22:14:43 +01:00
Patrick Cloke
5cda75fede
Set room version 8 as preferred for restricted rooms. (#10571) 2021-08-20 07:17:50 -04:00
Dirk Klimpel
b5fef6054a
Support MSC3283: Expose enable_set_displayname in capabilities (#10452) 2021-08-19 09:40:40 +00:00
Erik Johnston
78a70a2e0b Merge branch 'release-v1.41' into develop 2021-08-18 17:02:47 +01:00
Azrenbeth
1a9f531c79
Port the PresenceRouter module interface to the new generic interface (#10524)
Port the PresenceRouter module interface to the new generic interface introduced in v1.37.0
2021-08-17 13:22:45 +00:00
Brendan Abolivier
5639759980
Centralise the custom template directory (#10596)
Several configuration sections are using separate settings for custom template directories, which can be confusing. This PR adds a new top-level configuration for a custom template directory which is then used for every module. The only exception is the consent templates, since the consent template directory require a specific hierarchy, so it's probably better that it stays separate from everything else.
2021-08-17 14:45:24 +02:00
reivilibre
5f7b1e1f27
Make PeriodicallyFlushingMemoryHandler the default logging handler. (#10518) 2021-08-17 13:13:11 +01:00
Brendan Abolivier
ae2714c1f3
Allow using several custom template directories (#10587)
Allow using several directories in read_templates.
2021-08-17 10:23:14 +00:00
Michael Telatynski
0ace38b7b3
Experimental support for MSC3266 Room Summary API. (#10394) 2021-08-16 14:49:12 +00:00
Richard van der Hoff
74d7336686
Add a setting to disable TLS for sending email (#10546)
This is mostly useful in case the server offers TLS, but doesn't present a valid certificate.
2021-08-06 10:13:34 +00:00
Kento Okamoto
72935b7c50
Add warnings to ip_range_blacklist usage with proxies (#10129)
Per issue #9812 using `url_preview_ip_range_blacklist` with a proxy via `HTTPS_PROXY` or `HTTP_PROXY` environment variables has some inconsistent bahavior than mentioned. This PR changes the following:

- Changes the Sample Config file to include a note mentioning that `url_preview_ip_range_blacklist` and `ip_range_blacklist` is ignored when using a proxy
- Changes some logic in synapse/config/repository.py to send a warning when both `*ip_range_blacklist` configs and a proxy environment variable are set and but no longer throws an error.

Signed-off-by: Kento Okamoto <kentokamoto@protonmail.com>
2021-08-03 18:13:34 +00:00
Richard van der Hoff
4b10880da3
Make sync response cache time configurable. (#10513) 2021-08-03 14:45:04 +01:00
reivilibre
f4ac934afe
Revert use of PeriodicallyFlushingMemoryHandler by default (#10515) 2021-08-03 11:30:39 +01:00
Toni Spets
ba5287f5e8
Allow setting transaction limit for db connections (#10440)
Setting the value will help PostgreSQL free up memory by recycling
the connections in the connection pool.

Signed-off-by: Toni Spets <toni.spets@iki.fi>
2021-08-02 13:24:43 +00:00
Šimon Brandner
c3b037795a
Support for MSC2285 (hidden read receipts) (#10413)
Implementation of matrix-org/matrix-doc#2285
2021-07-28 10:05:11 +02:00
reivilibre
e16eab29d6
Add a PeriodicallyFlushingMemoryHandler to prevent logging silence (#10407)
Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>
2021-07-27 14:32:05 +01:00