Commit Graph

896 Commits

Author SHA1 Message Date
reivilibre
526f84bc2e
Fix Prometheus recording rules to not use legacy metric names. () 2022-09-08 15:01:42 +01:00
Patrick Cloke
32fc3b7ba4
Remove configuration options for direct TCP replication. ()
Removes the ability to configure legacy direct TCP replication. Workers now require Redis to run.
2022-09-06 07:50:02 +00:00
Brendan Abolivier
898fef2789
Share some metrics between the Prometheus exporter and the phone home stats () 2022-09-05 10:26:43 +00:00
reivilibre
be4250c7a8
Add experimental configuration option to allow disabling legacy Prometheus metric names. ()
Co-authored-by: David Robertson <davidr@element.io>
2022-08-24 11:35:54 +00:00
David Robertson
956e015413
Drop support for delegating email validation, round 2 () 2022-08-23 11:40:00 +00:00
Ayush Anand
22ea51faf9
Add support for compression to federation responses ()
Closes .

Signed-off-by: Ayush Anand <iamayushanand@gmail.com>
2022-08-18 15:14:47 +01:00
Sean Quah
f4ab6a4a96
Refuse to start when faster_joins is enabled on a worker deployment ()
Synapse does not currently support faster room joins on deployments with
workers.

Signed-off-by: Sean Quah <seanq@matrix.org>
2022-08-16 16:21:55 +01:00
David Robertson
19e5d44886
Revert "Update locked versions of mypy and mypy-zope ()"
This reverts commit f383b9b3ec. Other PRs
were seeing mypy failures that looked to be related to mypy-zope.
Confusingly, we didn't see this on .

Revert this for now and investigate later.
2022-08-15 14:51:05 +01:00
David Robertson
f383b9b3ec
Update locked versions of mypy and mypy-zope () 2022-08-15 11:32:30 +01:00
3nprob
98fb610cc0
Revert "Drop support for delegating email validation ()" ()
Reverts commit fa71bb18b5, and tweaks documentation.

Signed-off-by: 3nprob <git@3n.anonaddy.com>
2022-07-29 10:29:23 +00:00
Nick Mills-Barrett
86e366a46e
Remove old empty/redundant slaved stores. () 2022-07-21 17:56:45 +00:00
Nick Mills-Barrett
190f49d8ab
Use cache store remove base slaved ()
This comes from two identical definitions in each of the base stores, and means the base slaved store is now empty and can be removed.
2022-07-21 11:51:30 +01:00
Richard van der Hoff
fa71bb18b5
Drop support for delegating email validation ()
* Drop support for delegating email validation

Delegating email validation to an IS is insecure (since it allows the owner of
the IS to do a password reset on your HS), and has long been deprecated. It
will now cause a config error at startup.

* Update unit test which checks for email verification

Give it an `email` config instead of a threepid delegate

* Remove unused method `requestEmailToken`

* Simplify config handling for email verification

Rather than an enum and a boolean, all we need here is a single bool, which
says whether we are or are not doing email verification.

* update docs

* changelog

* upgrade.md: fix typo

* update version number

this will be in 1.64, not 1.63

* update version number

this one too
2022-07-12 19:18:53 +01:00
Erik Johnston
e610128c50
Add a filter_event_for_clients_with_state function () 2022-07-11 14:14:09 +01:00
reivilibre
9667bad55d
Improve startup times in Complement test runs against workers, particularly in CPU-constrained environments. ()
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2022-06-30 11:58:12 +00:00
Brendan Abolivier
417f4cf40b
Don't use keyword arguments when initialising modules () 2022-06-15 15:36:16 +02:00
reivilibre
5b645ae2ad
Refactor entry points so that they all have a main function. () 2022-06-14 17:41:06 +00:00
David Robertson
f30bcbd84a
Fix Synapse git info missing in version strings () 2022-06-07 15:24:11 +01:00
Patrick Cloke
cf05258f76
Remove groups replication code. ()
The replication logic for groups is no longer used, so the message
passing infrastructure can be removed.
2022-05-31 13:04:08 -04:00
Patrick Cloke
a8db8c6eba
Remove user-visible groups/communities code ()
Makes it so that groups/communities no longer exist from a user-POV. E.g. we remove:

* All API endpoints (including Client-Server, Server-Server, and admin).
* Documented configuration options (and the experimental flag, which is now unused).
* Special handling during room upgrades.
* The `groups` section of the `/sync` response.
2022-05-25 07:53:40 -04:00
Nick Mills-Barrett
1f9013ce60
Add the batch_send endpoint to generic workers () 2022-05-25 09:51:07 +00:00
Nick Mills-Barrett
2e5f88b5e6
Add the /account/whoami endpoint to generic workers () 2022-05-25 10:41:41 +01:00
David Robertson
d38d242411
Reload cache factors from disk on SIGHUP () 2022-05-11 13:43:22 +00:00
reivilibre
699192fc1a
Add the update_user_directory_from_worker configuration option (superseding update_user_directory) to allow a generic worker to be designated as the worker to update the user directory. ()
Co-authored-by: Shay <hillerys@element.io>
2022-05-10 11:08:45 +01:00
David Robertson
fa0eab9c8e
Use ParamSpec in a few places () 2022-05-09 10:27:39 +00:00
reivilibre
c2d50e9f6c
Add the notify_appservices_from_worker configuration option (superseding notify_appservices) to allow a generic worker to be designated as the worker to send traffic to Application Services. () 2022-05-06 11:43:53 +01:00
David Robertson
6463244375
Remove unused # type: ignores ()
Over time we've begun to use newer versions of mypy, typeshed, stub
packages---and of course we've improved our own annotations. This makes
some type ignore comments no longer necessary. I have removed them.

There was one exception: a module that imports `select.epoll`. The
ignore is redundant on Linux, but I've kept it ignored for those of us
who work on the source tree using not-Linux. ()

I'm more interested in the config line which enforces this. I want
unused ignores to be reported, because I think it's useful feedback when
annotating to know when you've fixed a problem you had to previously
ignore.

* Installing extras before typechecking

Lacking an easy way to install all extras generically, let's bite the bullet and
make install the hand-maintained `all` extra before typechecking.

Now that https://github.com/matrix-org/backend-meta/pull/6 is merged to
the release/v1 branch.
2022-04-27 14:03:44 +01:00
Patrick Cloke
4586119f0b
Add missing type hints to config classes. () 2022-04-11 12:07:23 -04:00
reivilibre
f871222880
Move update_client_ip background job from the main process to the background worker. () 2022-04-01 13:08:55 +01:00
Shay
3c41d87b67
Add restrictions by default to open registration in Synapse () 2022-03-25 10:11:01 -07:00
Nick Mills-Barrett
1530cef192
Make it possible to enable compression for the metrics HTTP resource ()
* Make it possible to enable compression for the metrics HTTP resource

This can provide significant bandwidth savings pulling metrics from
synapse instances.

* Add changelog file.

* Fix type hint
2022-03-21 11:52:10 +00:00
Patrick Cloke
54f674f7a9
Deprecate the groups/communities endpoints and add an experimental configuration flag. () 2022-03-12 13:23:37 -05:00
Patrick Cloke
3e4af36bc8
Rename get_tcp_replication to get_replication_command_handler. ()
Since the object it returns is a ReplicationCommandHandler.

This is clean-up from adding support to Redis where the command handler
was added as an additional layer of abstraction from the TCP protocol.
2022-03-10 13:01:56 +00:00
David Robertson
313581e4e9
Use importlib.metadata to read requirements ()
* Pull runtime dep checks into their own module
* Reimplement `check_requirements` using `importlib`

I've tried to make this clearer. We start by working out which of
Synapse's requirements we need to be installed here and now. I was
surprised that there wasn't an easier way to see which packages were
installed by a given extra.

I've pulled out the error messages into functions that deal with "is
this for an extra or not". And I've rearranged the loop over two
different sets of requirements into one loop with a "must be instaled"
flag.

I hope you agree that this is clearer.

* Test cases
2022-03-01 17:44:41 +00:00
Richard van der Hoff
e24ff8ebe3
Remove HomeServer.get_datastore() ()
The presence of this method was confusing, and mostly present for backwards
compatibility. Let's get rid of it.

Part of 
2022-02-23 11:04:02 +00:00
David Robertson
4ae956c8bb
Use version string helper from matrix-common ()
* Require latest matrix-common
* Use the common function
2022-02-14 13:12:22 +00:00
Patrick Cloke
380c3d40f4
Return JSON errors for unknown resources under /matrix/client. ()
Re-applies the changes from 3e0cfd447e (),
reverting d93ec0a0ba () now that the conflict
with the webclient listener was fixed in 119edf51eb ().
2022-02-08 07:06:25 -05:00
Patrick Cloke
119edf51eb
Remove support for the webclient listener. ()
Also remove support for non-HTTP(S) web_client_location.
2022-02-03 18:36:49 +00:00
Shay
b8bf600700
Check that gc method is available before using in synapse/app/_base ()
* add check that gc.freeze is available before calling

* newsfragment

* lint

* Update comment

Co-authored-by: Dan Callahan <danc@element.io>

Co-authored-by: Dan Callahan <danc@element.io>
2022-01-25 10:35:18 -08:00
Brendan Abolivier
1d5f7b2cc6
Log modules at startup () 2022-01-25 15:35:35 +01:00
Shay
9006ee36d1
Drop support for and remove references to EOL Python 3.6 ()
* remove reference in comments to python3.6

* upgrade tox python env in script

* bump python version in example for completeness

* upgrade python version requirement in setup doc

* upgrade necessary python version in __init__.py

* upgrade python version in setup.py

* newsfragment

* drops refs to bionic and replace with focal

* bump refs to postgres 9.6 to 10

* fix hanging ci

* try installing tzdata first

* revert change made in b979f336

* ignore new random mypy error while debugging other error

* fix lint error for temporary workaround

* revert change to install list

* try passing env var

* export debian frontend var?

* move line and add comment

* bump pillow dependency

* bump lxml depenency

* install libjpeg-dev for pillow

* bump automat version to one compatible with py3.8

* add libwebp for pillow

* bump twisted trunk python version

* change suffix of newsfragment

* remove redundant python 3.7 checks

* lint
2022-01-21 14:23:26 -08:00
Patrick Cloke
d09099642e
Fix redirecting to the webclient for non-HTTP(S) web_client_location. ()
To not change the behaviour during the deprecation period.

Follow-up to .
2022-01-20 15:34:45 +00:00
Patrick Cloke
91221b6961
Add deprecation warnings for webclient listener and non-HTTP(S) web_client_location. ()
This changes the behaviour of the root endpoint to redirect
directly to the configuration of `web_client_location` if it is
given an HTTP(S) URL.
2022-01-20 14:21:06 +00:00
Andrew Morgan
9eab71aa93 Merge branch 'master' into develop 2022-01-18 16:46:39 +00:00
Andrew Morgan
d93ec0a0ba
Partially revert to prevent webclient overriding client resource () 2022-01-18 16:03:56 +00:00
Richard van der Hoff
20c6d85c6e
Simplify GC prometheus metrics ()
Rather than hooking into the reactor loop, just add a timed task that runs every 100 ms to do the garbage collection.

Part 1 of a quest to simplify the reactor monkey-patching.
2022-01-13 14:35:52 +00:00
Patrick Cloke
3e0cfd447e
Return JSON errors for unknown resources under /matrix/client. ()
Instead of returning 404 errors with HTML bodies when an unknown
prefix was requested (e.g. /matrix/client/v1 before Synapse v1.49.0).
2021-12-20 16:00:13 +00:00
Patrick Cloke
f44d729d4c
Additional type hints for config module. ()
This adds some misc. type hints to helper methods used
in the `synapse.config` module.
2021-12-01 07:28:23 -05:00
Shay
70cbb1a5e3
Don't start Synapse master process if worker_app is set ()
* Add check to catch syanpse master process starting when workers are configured

* add test to verify that starting master process with worker config raises error

* newsfragment

* specify config.worker.worker_app in check

* update test

* report specific config option that triggered the error

Co-authored-by: reivilibre <oliverw@matrix.org>

* clarify error message

Co-authored-by: reivilibre <oliverw@matrix.org>

Co-authored-by: reivilibre <oliverw@matrix.org>
2021-11-30 10:12:18 -08:00
Maximilian Bosch
42bf020463
Expose worker & homeserver as entrypoints in setup.py ()
Co-authored-by: reivilibre <oliverw@matrix.org>
2021-11-30 16:52:45 +00:00