Commit Graph

45 Commits

Author SHA1 Message Date
Patrick Cloke
32fc3b7ba4
Remove configuration options for direct TCP replication. ()
Removes the ability to configure legacy direct TCP replication. Workers now require Redis to run.
2022-09-06 07:50:02 +00:00
Shay
493c2fc44a
Remove code generating comments in configuration file () 2022-06-14 07:53:42 -07:00
reivilibre
699192fc1a
Add the update_user_directory_from_worker configuration option (superseding update_user_directory) to allow a generic worker to be designated as the worker to update the user directory. ()
Co-authored-by: Shay <hillerys@element.io>
2022-05-10 11:08:45 +01:00
reivilibre
c2d50e9f6c
Add the notify_appservices_from_worker configuration option (superseding notify_appservices) to allow a generic worker to be designated as the worker to send traffic to Application Services. () 2022-05-06 11:43:53 +01:00
Patrick Cloke
4586119f0b
Add missing type hints to config classes. () 2022-04-11 12:07:23 -04:00
Patrick Cloke
10a88ba91c
Use auto_attribs/native type hints for attrs classes. () 2022-01-13 13:49:28 +00:00
Patrick Cloke
f44d729d4c
Additional type hints for config module. ()
This adds some misc. type hints to helper methods used
in the `synapse.config` module.
2021-12-01 07:28:23 -05:00
Nick Barrett
af54167516
Enable passing typing stream writers as a list. ()
This makes the typing stream writer config match the other stream writers
that only currently support a single worker.
2021-11-03 14:25:47 +00:00
Erik Johnston
9d25a0ae65
Split presence out of master () 2021-04-23 12:21:55 +01:00
Jonathan de Jong
4b965c862d
Remove redundant "coding: utf-8" lines ()
Part of 

Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now.

`Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`
2021-04-14 15:34:27 +01:00
Erik Johnston
2927921942
Clean up ShardedWorkerHandlingConfig ()
* Split ShardedWorkerHandlingConfig

This is so that we have a type level understanding of when it is safe to
call `get_instance(..)` (as opposed to `should_handle(..)`).

* Remove special cases in ShardedWorkerHandlingConfig.

`ShardedWorkerHandlingConfig` tried to handle the various different ways
it was possible to configure federation senders and pushers. This led to
special cases that weren't hit during testing.

To fix this the handling of the different cases is moved from there and
`generic_worker` into the worker config class. This allows us to have
the logic in one place and allows the rest of the code to ignore the
different cases.
2021-02-24 13:23:18 +00:00
Eric Eastwood
0a00b7ff14
Update black, and run auto formatting over the codebase ()
- Update black version to the latest
 - Run black auto formatting over the codebase
    - Run autoformatting according to [`docs/code_style.md
`](80d6dc9783/docs/code_style.md)
 - Update `code_style.md` docs around installing black to use the correct version
2021-02-16 22:32:34 +00:00
Erik Johnston
6633a4015a
Allow moving account data and receipts streams off master () 2021-01-18 15:47:59 +00:00
Erik Johnston
b530eaa262
Allow running sendToDevice on workers () 2021-01-07 20:19:26 +00:00
Patrick Cloke
96358cb424
Add authentication to replication endpoints. ()
Authentication is done by checking a shared secret provided
in the Synapse configuration file.
2020-12-04 10:56:28 -05:00
Patrick Cloke
62894673e6
Allow background tasks to be run on a separate worker. () 2020-10-02 08:23:15 -04:00
Erik Johnston
04cc249b43
Add experimental support for sharding event persister. Again. ()
This is *not* ready for production yet. Caveats:

1. We should write some tests...
2. The stream token that we use for events can get stalled at the minimum position of all writers. This means that new events may not be processed and e.g. sent down sync streams if a writer isn't writing or is slow.
2020-09-14 10:16:41 +01:00
Brendan Abolivier
9f8abdcc38
Revert "Add experimental support for sharding event persister. ()" ()
* Revert "Add experimental support for sharding event persister. ()"

This reverts commit 82c1ee1c22.

* Changelog
2020-09-04 10:19:42 +01:00
Erik Johnston
82c1ee1c22
Add experimental support for sharding event persister. ()
This is *not* ready for production yet. Caveats:

1. We should write some tests...
2. The stream token that we use for events can get stalled at the minimum position of all writers. This means that new events may not be processed and e.g. sent down sync streams if a writer isn't writing or is slow.
2020-09-02 15:48:37 +01:00
Erik Johnston
2c1b9d6763
Update worker docs with recent enhancements () 2020-07-29 23:22:13 +01:00
Erik Johnston
f2e38ca867
Allow moving typing off master () 2020-07-16 15:12:54 +01:00
Richard van der Hoff
03619324fc
Create a ListenerConfig object ()
This ended up being a bit more invasive than I'd hoped for (not helped by
generic_worker duplicating some of the code from homeserver), but hopefully
it's an improvement.

The idea is that, rather than storing unstructured `dict`s in the config for
the listener configurations, we instead parse it into a structured
`ListenerConfig` object.
2020-06-16 12:44:07 +01:00
Erik Johnston
e5c67d04db
Add option to move event persistence off master () 2020-05-22 16:11:35 +01:00
Erik Johnston
1de36407d1
Add instance_map config and route replication calls () 2020-05-14 14:00:58 +01:00
Amber Brown
f743108a94
Refactor HomeserverConfig so it can be typechecked () 2019-10-10 09:39:35 +01:00
Matthew Hodgson
7ef319aefe fix broken copyrights 2019-09-23 12:28:01 +01:00
Amber Brown
0d0f6d12bc Fix logging in workers ()
This also adds a worker blacklist.
2019-07-22 16:05:00 +01:00
Amber Brown
6a85cb5ef7
Remove non-dedicated logging options and command line arguments () 2019-07-19 01:40:08 +10:00
Richard van der Hoff
c3c6b00d95
Pass config_dir_path and data_dir_path into Config.read_config. ()
* Pull config_dir_path and data_dir_path calculation out of read_config_files

* Pass config_dir_path and data_dir_path into read_config
2019-06-24 11:34:45 +01:00
Richard van der Hoff
6cda36777b Drop support for cpu_affinity ()
This has no useful purpose on python3, and is generally a source of confusion.
2019-06-22 11:01:55 +10:00
Amber Brown
32e7c9e7f2
Run Black. () 2019-06-20 19:32:02 +10:00
Erik Johnston
9ad448c1e5 Correctly handle all command line options 2019-03-14 13:32:14 +00:00
Erik Johnston
72bfaf746d Allow passing --daemonize to workers 2019-03-13 17:33:54 +00:00
Erik Johnston
24dd73028a Add replication http endpoint for event sending 2018-02-07 10:32:32 +00:00
Erik Johnston
f4d93ae424 Actually make it work 2018-01-12 10:39:27 +00:00
Richard van der Hoff
10d8b701a1 Allow configuration of CPU affinity
Make it possible to set the CPU affinity in the config file, so that we don't
need to remember to do it manually every time.
2017-08-15 17:08:28 +01:00
Erik Johnston
1fc4a962e4 Add a frontend proxy 2017-07-07 18:19:46 +01:00
Erik Johnston
d1605794ad Remove unused worker config option 2017-04-04 11:17:00 +01:00
Erik Johnston
36c28bc467 Update all the workers and master to use TCP replication 2017-04-03 15:35:52 +01:00
Erik Johnston
97efe99ae9 Make worker listener config backwards compat 2017-01-20 11:45:29 +00:00
Mark Haines
a352b68acf Use worker_ prefixes for worker config, use existing support for multiple config files 2016-06-16 17:29:50 +01:00
Mark Haines
364d616792 Access the event_cache_size directly from the server object.
This means that the workers can override the event_cache_size
directly without clobbering the value in the main synapse config.
2016-06-16 12:53:15 +01:00
Mark Haines
bde13833cb Access replication_url from the worker config directly 2016-06-16 12:44:40 +01:00
Mark Haines
80a1bc7db5 Comment on what's going on in clobber_with_worker_config 2016-06-16 11:29:45 +01:00
Mark Haines
dbb5a39b64 Add worker config module 2016-06-16 11:09:15 +01:00