Matrix-Mirrors/synapse-product
1
0
Fork 0
mirror of https://git.anonymousland.org/anonymousland/synapse-product.git synced 2024-12-25 10:09:22 -05:00
Code Issues Packages Projects Releases Wiki Activity

Fixup worker doc (again) (#8000)

Browse Source
This commit is contained in:
Erik Johnston 2020-08-06 10:35:59 +01:00 committed by GitHub
parent a7bdf98d01
commit 079bc3c8e3
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
3 changed files with 43 additions and 14 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
changelog.d/8000.doc Normal file
View File

@ -0,0 +1 @@
Improve workers docs.

2
docs/systemd-with-workers/workers/federation_reader.yaml
View File

@ -1,7 +1,7 @@
worker_app: synapse.app.federation_reader worker_app: synapse.app.federation_reader
worker_name: federation_reader1
worker_replication_host: 127.0.0.1 worker_replication_host: 127.0.0.1
worker_replication_port: 9092
worker_replication_http_port: 9093 worker_replication_http_port: 9093
worker_listeners: worker_listeners:

54
docs/workers.md
View File

@ -23,7 +23,7 @@ The processes communicate with each other via a Synapse-specific protocol called
feeds streams of newly written data between processes so they can be kept in feeds streams of newly written data between processes so they can be kept in
sync with the database state. sync with the database state.
When configured to do so, Synapse uses a When configured to do so, Synapse uses a
[Redis pub/sub channel](https://redis.io/topics/pubsub) to send the replication [Redis pub/sub channel](https://redis.io/topics/pubsub) to send the replication
stream between all configured Synapse processes. Additionally, processes may stream between all configured Synapse processes. Additionally, processes may
make HTTP requests to each other, primarily for operations which need to wait make HTTP requests to each other, primarily for operations which need to wait
@ -66,23 +66,31 @@ https://hub.docker.com/r/matrixdotorg/synapse/.
To make effective use of the workers, you will need to configure an HTTP To make effective use of the workers, you will need to configure an HTTP
reverse-proxy such as nginx or haproxy, which will direct incoming requests to reverse-proxy such as nginx or haproxy, which will direct incoming requests to
the correct worker, or to the main synapse instance. See the correct worker, or to the main synapse instance. See
[reverse_proxy.md](reverse_proxy.md) for information on setting up a reverse [reverse_proxy.md](reverse_proxy.md) for information on setting up a reverse
proxy. proxy.
To enable workers you should create a configuration file for each worker When using workers, each worker process has its own configuration file which
process. Each worker configuration file inherits the configuration of the shared contains settings specific to that worker, such as the HTTP listener that it
homeserver configuration file. You can then override configuration specific to provides (if any), logging configuration, etc.
that worker, e.g. the HTTP listener that it provides (if any); logging
configuration; etc. You should minimise the number of overrides though to Normally, the worker processes are configured to read from a shared
maintain a usable config. configuration file as well as the worker-specific configuration files. This
makes it easier to keep common configuration settings synchronised across all
the processes.
The main process is somewhat special in this respect: it does not normally
need its own configuration file and can take all of its configuration from the
shared configuration file.
### Shared Configuration ### Shared configuration
Normally, only a couple of changes are needed to make an existing configuration
file suitable for use with workers. First, you need to enable an "HTTP replication
listener" for the main process; and secondly, you need to enable redis-based
replication. For example:
Next you need to add both a HTTP replication listener, used for HTTP requests
between processes, and redis config to the shared Synapse configuration file
(`homeserver.yaml`). For example:
```yaml ```yaml
# extend the existing `listeners` section. This defines the ports that the # extend the existing `listeners` section. This defines the ports that the
@ -105,7 +113,7 @@ Under **no circumstances** should the replication listener be exposed to the
public internet; it has no authentication and is unencrypted. public internet; it has no authentication and is unencrypted.
### Worker Configuration ### Worker configuration
In the config file for each worker, you must specify the type of worker In the config file for each worker, you must specify the type of worker
application (`worker_app`), and you should specify a unqiue name for the worker application (`worker_app`), and you should specify a unqiue name for the worker
@ -145,6 +153,9 @@ plain HTTP endpoint on port 8083 separately serving various endpoints, e.g.
Obviously you should configure your reverse-proxy to route the relevant Obviously you should configure your reverse-proxy to route the relevant
endpoints to the worker (`localhost:8083` in the above example). endpoints to the worker (`localhost:8083` in the above example).
### Running Synapse with workers
Finally, you need to start your worker processes. This can be done with either Finally, you need to start your worker processes. This can be done with either
`synctl` or your distribution's preferred service manager such as `systemd`. We `synctl` or your distribution's preferred service manager such as `systemd`. We
recommend the use of `systemd` where available: for information on setting up recommend the use of `systemd` where available: for information on setting up
@ -407,6 +418,23 @@ all these to be folded into the `generic_worker` app and to use config to define
which processes handle the various proccessing such as push notifications. which processes handle the various proccessing such as push notifications.
## Migration from old config
There are two main independent changes that have been made: introducing Redis
support and merging apps into `synapse.app.generic_worker`. Both these changes
are backwards compatible and so no changes to the config are required, however
server admins are encouraged to plan to migrate to Redis as the old style direct
TCP replication config is deprecated.
To migrate to Redis add the `redis` config as above, and optionally remove the
TCP `replication` listener from master and `worker_replication_port` from worker
config.
To migrate apps to use `synapse.app.generic_worker` simply update the
`worker_app` option in the worker configs, and where worker are started (e.g.
in systemd service files, but not required for synctl).
## Architectural diagram ## Architectural diagram
The following shows an example setup using Redis and a reverse proxy: The following shows an example setup using Redis and a reverse proxy: