This commit is contained in:
deathrow 2022-11-01 20:33:33 -04:00
commit 4657d2e9f1
No known key found for this signature in database
GPG Key ID: FF39D67A22069F73
45 changed files with 2469 additions and 0 deletions

6
.env.sample Normal file
View File

@ -0,0 +1,6 @@
POSTGRES_USER=user
POSTGRES_PASSWORD=password
POSTGRES_DB=db
SWAG_URL=example.tld
SWAG_SUBDOMAINS=matrix
SWAG_EMAIL=email@example.tld

300
README.md Normal file
View File

@ -0,0 +1,300 @@
## Synapse Docker Deployement
A Synapse Docker deployment with:
- Hardened Synapse Image
- Hardened Worker Images
- Mjolnir & Mjolnir Module
- Multi-threaded Synapse Process via Workers
- Privacy-respecting Registration Captcha
- Manage Docker Variables inside of `.env`
- Manage `state` with the state compressor
- Manage Server via `synadm`
- Images Built Locally
### Getting Started
Dependencies: `cargo docker docker-compose git python `
Clone the repository:
```
git clone https://codeberg.org/deathrow/synapse-docker-deployement
```
CD into the repository:
```
cd synapse-docker-deployment
```
Execute the init script to:
- `git clone` the docker images
- Build the docker images
- Build `auto-state-compressor`
- Install `synadm`
*Will take a long time!*
```
bash init.sh
```
Modify variables inside `.env.sample` and move to `.env`
Run this command to generate the Synapse configuration file:
``
docker-compose run --rm -e SYNAPSE_SERVER_NAME=example.tld -e SYNAPSE_REPORT_STATS=no synapse generate
``
### Synapse Configuration
The Synapse config file will be located at `./files/homeserver.yaml`.
Modify the following:
*You will need to uncomment (#) these*
``web_client_location: https://element.example.tld``
``public_baseurl: https://matrix.example.tld``
``serve_server_wellknown: true``
Under the `listeners:` section, add the following:
```
- port: 9093
type: http
resources:
- names: [replication]
```
Under the `retention:` section, you are able to set retention of messages.
Uncomment `enabled: false` if you wish to keep messages indefinitely. *(will take up more disk space)*
For the `purge_jobs:` section, add:
```
purge_jobs:
- longest_max_lifetime: 1h
interval: 30m
- shortest_max_lifetime: 1h
longest_max_lifetime: 12h
interval: 1h
- shortest_max_lifetime: 12h
longest_max_lifetime: 1d
interval: 12h
- shortest_max_lifetime: 1d
longest_max_lifetime: 10y
interval: 24h
```
For `caches:` set the following:
```
caches:
global_factor: 2.0
per_cache_factors:
get_users_who_share_room_with_user: 5.0
sync_response_cache_duration: 2m
```
Under the `databases:` section, remove the default database and add the following:
*(change with the postgres values set inside `.env`)*
Keep the host set to `postgres` as this is the name specified in the `docker-compose.yml`
```
database:
name: psycopg2
txn_limit: 10000
args:
user: user
password: password
database: db
host: postgres
port: 5432
cp_min: 5
cp_max: 10
```
Under the ``## Ratelimiting ##`` section, add the following:
```
rc_federation:
window_size: 1000
sleep_limit: 10
sleep_delay: 500
reject_limit: 50
concurrent: 3
federation_rr_transactions_per_room_per_second: 50
```
Uncomment the `url_preview_enabled: true` and the setting to go with it:
```
url_preview_ip_range_blacklist:
- '127.0.0.0/8'
- '10.0.0.0/8'
- '172.16.0.0/12'
- '192.168.0.0/16'
- '100.64.0.0/10'
- '192.0.0.0/24'
- '169.254.0.0/16'
- '192.88.99.0/24'
- '198.18.0.0/15'
- '192.0.2.0/24'
- '198.51.100.0/24'
- '203.0.113.0/24'
- '224.0.0.0/4'
- '::1/128'
- 'fe80::/10'
- 'fc00::/7'
- '2001:db8::/32'
- 'ff00::/8'
- 'fec0::/10'
```
If you wish to use the `url_preview_url_blacklist:` to blacklist certain URLs from being previewed, you can use the following settings:
```
# blacklist all *.google.com URLs
- netloc: 'google.com'
- netloc: '*.google.com'
# blacklist all plain HTTP URLs
- scheme: 'http'
# blacklist any URL with a literal IPv4 address
- netloc: '^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+$'
```
If you wish to change the number of rounds used to generate a password hash, you may modify the ``bcrypt_rounds:`` setting.
Uncomment ``inhibit_user_in_use_error: true``
Uncomment ``suppress_key_server_warning: true``
Uncomment ``send_federation: false`` and add the following:
```
federation_sender_instances:
- federation1
- federation2
- federation3
```
Under the `redis:` section, uncomment `` enabled: true`` and add the following settings:
```
host: redis
port: 6379
```
### Nginx
The path for NGINX is `/swag/nginx`.
Ensure to review each file before you use it, some variables may need changed such as the `matrix.example.tld` and such.
### Start the server
To start the server, type:
`docker-compose up -d`, you may wish to omit the `-d` on the first run to ensure there are no errors.
### Pantalaimon
Modify the `pantalaimon_data/pantalaimon.conf` to change the `matrix.example.tld`
### Mjolnir
Create a new user on your server with the username `mjonlir`.
[Mjolnir Configuration](https://github.com/matrix-org/mjolnir/blob/main/config/default.yaml)
Inside of `mjolnir/config/production.yaml` modify:
Set `homeserverUrl: "http://pantalaimon:8008"`,
Under `pantalaimon:` set `use: true` with the username `mjolnir` and `password:`
Create a new encrypted room on your server and copy the ID and set it as ``managementRoom: !123:example.tld``
Under `web:` set `enabled: true`
Set `displayReports: true`
In `homeserver.yaml` add the following `modules:`
```
modules:
- module: mjolnir.Module
config:
# Prevent servers/users in the ban lists from inviting users on this
# server to rooms. Default true.
block_invites: true
# Flag messages sent by servers/users in the ban lists as spam. Currently
# this means that spammy messages will appear as empty to users. Default
# false.
block_messages: true
# Remove users from the user directory search by filtering matrix IDs and
# display names by the entries in the user ban list. Default false.
block_usernames: true
# The room IDs of the ban lists to honour. Unlike other parts of Mjolnir,
# this list cannot be room aliases or permalinks. This server is expected
# to already be joined to the room - Mjolnir will not automatically join
# these rooms.
ban_lists:
- "!123:example.tld"
- "!456:example.tld"
message_max_length:
# Limit the characters in a message (event body) that a client can send in an event on this server.
# By default there is no limit (beyond the the limit the spec enforces on event size).
# Uncomment if you want messages to be limited to 510 characters.
#threshold: 510
# Limit messages only in certain rooms rooms.
# By default all rooms will enforce the limit.
# Uncomment if you want messages to only be subject to character limits in certain rooms.
rooms:
- "!123:localhost:9999"
- "!456:localhost:9999"
# Also hide messages from remote servers that are over the `message_limit`.
# By default only events from this server will be limited.
# WARNING: Remote users on other servers will still be able to messages over the limit.
# Uncomment to enforce the `message_limit` on events from remote servers.
remote_servers: false
```
### Captcha
The [synapse-captcha](https://codeberg.org/deathrow/synapse-captcha) is included with this deployment. Refer to this for configuration.
### Additional
To bypass ratelimits for certain users:
``
docker exec -it postgres psql insert into ratelimit_override values ('@user:example.tld', 0, 0);
``
Your `mjolnir` and any other admin accounts should be set in the example above.
For synapse state compressor:
``
./synapse_auto_compressor -p postgresql://user:password@localhost/db -c 500 -n 100
``
### Links
- [Synapse-Docker-Compose](https://github.com/tommytran732/Synapse-Docker-Compose)
- [Matrix-org Synapse Docker Compose Workers](https://github.com/matrix-org/synapse/tree/develop/contrib/docker_compose_workers)
- [matrix-conf](https://git.envs.net/envs/matrix-conf/)

444
docker-compose.yml Normal file
View File

@ -0,0 +1,444 @@
version: '3'
services:
synapse:
build: ./images/synapse-docker
restart: always
container_name: synapse
user: "991:991"
environment:
- SYNAPSE_CONFIG_DIR=/data
- SYNAPSE_CONFIG_PATH=/data/homeserver.yaml
volumes:
- ./files:/data:Z
depends_on:
- postgres
networks:
- matrix
security_opt:
- no-new-privileges:true
cap_drop:
- ALL
extra_hosts:
- "host.docker.internal:host-gateway"
postgres:
image: docker.io/postgres:14.5-alpine
container_name: postgres
user: "70:70"
restart: always
environment:
- POSTGRES_USER=${POSTGRES_USER}
- POSTGRES_PASSWORD=${POSTGRES_PASSWORD}
- POSTGRES_DB=${POSTGRES_DB}
- POSTGRES_INITDB_ARGS=--encoding=UTF-8 --lc-collate=C --lc-ctype=C
volumes:
- ./schemas:/var/lib/postgresql/data:Z
ports:
- 127.0.0.1:5432:5432
networks:
- matrix
read_only: true
security_opt:
- no-new-privileges:true
cap_drop:
- ALL
tmpfs:
- /var/run/postgresql:size=50M,mode=0770,uid=70,gid=70,noexec,nosuid,nodev
swag:
image: ghcr.io/linuxserver/swag
restart: always
container_name: swag
cap_add:
- NET_ADMIN
environment:
- PUID=1000
- PGID=1000
- URL=${SWAG_URL}
- SUBDOMAINS=${SWAG_SUBDOMAINS}
- VALIDATION=http
- EMAIL=${SWAG_EMAIL}
- ONLY_SUBDOMAINS=true
volumes:
- ./swag:/config:Z
- /etc/localtime:/etc/localtime:ro
ports:
- 443:443
- 80:80
networks:
- matrix
- registration
- adminer
pantalaimon:
build: ./images/pantalaimon
container_name: pantalaimon
restart: always
volumes:
- ./pantalaimon_data:/data:Z
depends_on:
- synapse
networks:
- matrix
read_only: true
security_opt:
- no-new-privileges:true
cap_drop:
- ALL
mjolnir:
build: ./images/mjolnir
container_name: mjolnir
restart: always
volumes:
- ./mjolnir:/data:Z
depends_on:
- pantalaimon
networks:
- matrix
read_only: true
security_opt:
- no-new-privileges:true
cap_drop:
- ALL
maubot:
image: dock.mau.dev/maubot/maubot:latest
container_name: maubot
restart: always
volumes:
- ./maubot:/data:Z
networks:
- matrix
adminer:
image: adminer
container_name: adminer
restart: unless-stopped
networks:
- matrix
- adminer
matrix-registration:
build: ./images/synapse-captcha
container_name: matrix-registration
restart: always
command: [
"--config-path=/data/config.yaml",
"serve"
]
ports:
- 127.0.0.1:5000:5000
volumes:
- ./matrix-registration_data:/data:Z
networks:
- matrix
redis:
image: redis:latest
container_name: redis
restart: always
networks:
- matrix
# WORKERS
sync3:
build: ./images/synapse-worker-docker
container_name: sync3
user: "991:991"
environment:
SYNAPSE_WORKER: synapse.app.generic_worker
restart: unless-stopped
entrypoint: ["/start.py", "run", "--config-path=/data/homeserver.yaml", "--config-path=/data/workers/sync3.yaml"]
volumes:
- ./files:/data:Z
- ./bridges:/bridges:Z
depends_on:
- synapse
ports:
- 127.0.0.1:8092:8081
security_opt:
- no-new-privileges:true
cap_drop:
- ALL
sync2:
build: ./images/synapse-worker-docker
container_name: sync2
user: "991:991"
restart: unless-stopped
environment:
SYNAPSE_WORKER: synapse.app.generic_worker
entrypoint: ["/start.py", "run", "--config-path=/data/homeserver.yaml", "--config-path=/data/workers/sync2.yaml"]
volumes:
- ./files:/data:Z
- ./bridges:/bridges:Z
depends_on:
- synapse
ports:
- 127.0.0.1:8091:8081
security_opt:
- no-new-privileges:true
cap_drop:
- ALL
sync1:
build: ./images/synapse-worker-docker
container_name: sync1
user: "991:991"
environment:
SYNAPSE_WORKER: synapse.app.generic_worker
restart: unless-stopped
entrypoint: ["/start.py", "run", "--config-path=/data/homeserver.yaml", "--config-path=/data/workers/sync1.yaml"]
volumes:
- ./files:/data:Z
- ./bridges:/bridges:Z
depends_on:
- synapse
ports:
- 127.0.0.1:8090:8081
security_opt:
- no-new-privileges:true
cap_drop:
- ALL
additional2:
build: ./images/workers/synapse
container_name: additional2
user: "991:991"
restart: unless-stopped
environment:
SYNAPSE_WORKER: synapse.app.generic_worker
entrypoint: ["/start.py", "run", "--config-path=/data/homeserver.yaml", "--config-path=/data/workers/additional2.yaml"]
volumes:
- ./files:/data:Z
- ./bridges:/bridges:Z
depends_on:
- synapse
ports:
- 127.0.0.1:8086:8034
security_opt:
- no-new-privileges:true
cap_drop:
- ALL
additional1:
build: ./images/workers/synapse
container_name: additional1
user: "991:991"
restart: unless-stopped
environment:
SYNAPSE_WORKER: synapse.app.generic_worker
entrypoint: ["/start.py", "run", "--config-path=/data/homeserver.yaml", "--config-path=/data/workers/additional1.yaml"]
volumes:
- ./files:/data:Z
- ./bridges:/bridges:Z
depends_on:
- synapse
networks:
- matrix
ports:
- 127.0.0.1:8085:8034
security_opt:
- no-new-privileges:true
cap_drop:
- ALL
event1:
build: ./images/workers/synapse
container_name: event1
user: "991:991"
restart: unless-stopped
environment:
SYNAPSE_WORKER: synapse.app.generic_worker
entrypoint: ["/start.py", "run", "--config-path=/data/homeserver.yaml", "--config-path=/data/workers/event1.yaml"]
volumes:
- ./files:/data:Z
- ./bridges:/bridges:Z
depends_on:
- synapse
networks:
- matrix
ports:
- 127.0.0.1:8087:8034
security_opt:
- no-new-privileges:true
cap_drop:
- ALL
event2:
build: ./images/workers/synapse
container_name: event2
user: "991:991"
restart: unless-stopped
environment:
SYNAPSE_WORKER: synapse.app.generic_worker
entrypoint: ["/start.py", "run", "--config-path=/data/homeserver.yaml", "--config-path=/data/workers/event2.yaml"]
volumes:
- ./files:/data:Z
- ./bridges:/bridges:Z
depends_on:
- synapse
networks:
- matrix
ports:
- 127.0.0.1:8088:8034
security_opt:
- no-new-privileges:true
cap_drop:
- ALL
synapse-generic-worker-3:
build: ./images/workers/synapse
container_name: synapse-generic-worker-3
user: "991:991"
restart: unless-stopped
environment:
SYNAPSE_WORKER: synapse.app.generic_worker
entrypoint: ["/start.py", "run", "--config-path=/data/homeserver.yaml", "--config-path=/data/workers/worker3.yaml"]
volumes:
- ./files:/data:Z
- ./bridges:/bridges:Z
depends_on:
- synapse
networks:
- matrix
ports:
- 127.0.0.1:8082:8081
security_opt:
- no-new-privileges:true
cap_drop:
- ALL
synapse-generic-worker-2:
build: ./images/synapse-worker-docker
container_name: synapse-generic-worker-2
user: "991:991"
restart: unless-stopped
environment:
SYNAPSE_WORKER: synapse.app.generic_worker
entrypoint: ["/start.py", "run", "--config-path=/data/homeserver.yaml", "--config-path=/data/workers/worker2.yaml"]
volumes:
- ./files:/data:Z
- ./bridges:/bridges:Z
depends_on:
- synapse
networks:
- matrix
ports:
- 127.0.0.1:8081:8081
security_opt:
- no-new-privileges:true
cap_drop:
- ALL
synapse-generic-worker-1:
build: ./images/synapse-worker-docker
container_name: synapse-generic-worker-1
user: "991:991"
restart: unless-stopped
environment:
SYNAPSE_WORKER: synapse.app.generic_worker
entrypoint: ["/start.py", "run", "--config-path=/data/homeserver.yaml", "--config-path=/data/workers/worker1.yaml"]
volumes:
- ./files:/data:Z
- ./bridges:/bridges:Z
depends_on:
- synapse
networks:
- matrix
ports:
- 127.0.0.1:8080:8081
security_opt:
- no-new-privileges:true
cap_drop:
- ALL
synapse-federation-sender-4:
build: ./images/workers/synapse
container_name: synapse-federation-sender-4
user: "991:991"
restart: unless-stopped
environment:
SYNAPSE_WORKER: synapse.app.federation_sender
entrypoint: ["/start.py", "run", "--config-path=/data/homeserver.yaml", "--config-path=/data/workers/federation4.yaml"]
volumes:
- ./files:/data:Z
- ./bridges:/bridges:Z
depends_on:
- synapse
networks:
- matrix
security_opt:
- no-new-privileges:true
cap_drop:
- ALL
synapse-federation-sender-3:
build: ./images/workers/synapse
container_name: synapse-federation-sender-3
user: "991:991"
restart: unless-stopped
environment:
SYNAPSE_WORKER: synapse.app.federation_sender
entrypoint: ["/start.py", "run", "--config-path=/data/homeserver.yaml", "--config-path=/data/workers/federation3.yaml"]
volumes:
- ./files:/data:Z
- ./bridges:/bridges:Z
depends_on:
- synapse
networks:
- matrix
security_opt:
- no-new-privileges:true
cap_drop:
- ALL
synapse-federation-sender-2:
build: ./images/workers/synapse
container_name: synapse-federation-sender-2
user: "991:991"
restart: unless-stopped
environment:
SYNAPSE_WORKER: synapse.app.federation_sender
entrypoint: ["/start.py", "run", "--config-path=/data/homeserver.yaml", "--config-path=/data/workers/federation2.yaml"]
volumes:
- ./files:/data:Z
- ./bridges:/bridges:Z
depends_on:
- synapse
networks:
- matrix
security_opt:
- no-new-privileges:true
cap_drop:
- ALL
synapse-federation-sender-1:
build: ./images/workers/synapse
container_name: synapse-federation-sender-1
user: "991:991"
restart: unless-stopped
environment:
SYNAPSE_WORKER: synapse.app.federation_sender
entrypoint: ["/start.py", "run", "--config-path=/data/homeserver.yaml", "--config-path=/data/workers/federation1.yaml"]
volumes:
- ./files:/data:Z
- ./bridges:/bridges:Z
depends_on:
- synapse
networks:
- matrix
security_opt:
- no-new-privileges:true
cap_drop:
- ALL
networks:
matrix:
registration:
adminer:

View File

@ -0,0 +1,84 @@
# Log configuration for Synapse.
#
# This is a YAML file containing a standard Python logging configuration
# dictionary. See [1] for details on the valid settings.
#
# Synapse also supports structured logging for machine readable logs which can
# be ingested by ELK stacks. See [2] for details.
#
# [1]: https://docs.python.org/3.7/library/logging.config.html#configuration-dictionary-schema
# [2]: https://matrix-org.github.io/synapse/latest/structured_logging.html
version: 1
formatters:
precise:
format: '%(asctime)s - %(name)s - %(lineno)d - %(levelname)s - %(request)s - %(message)s'
handlers:
file:
class: logging.handlers.TimedRotatingFileHandler
formatter: precise
filename: /data/workers/log/additional1.log
when: midnight
backupCount: 2 # Does not include the current log file.
encoding: utf8
# Default to buffering writes to log file for efficiency.
# WARNING/ERROR logs will still be flushed immediately, but there will be a
# delay (of up to `period` seconds, or until the buffer is full with
# `capacity` messages) before INFO/DEBUG logs get written.
buffer:
class: synapse.logging.handlers.PeriodicallyFlushingMemoryHandler
target: file
# The capacity is the maximum number of log lines that are buffered
# before being written to disk. Increasing this will lead to better
# performance, at the expensive of it taking longer for log lines to
# be written to disk.
# This parameter is required.
capacity: 10
# Logs with a level at or above the flush level will cause the buffer to
# be flushed immediately.
# Default value: 40 (ERROR)
# Other values: 50 (CRITICAL), 30 (WARNING), 20 (INFO), 10 (DEBUG)
flushLevel: 30 # Flush immediately for WARNING logs and higher
# The period of time, in seconds, between forced flushes.
# Messages will not be delayed for longer than this time.
# Default value: 5 seconds
period: 5
# A handler that writes logs to stderr. Unused by default, but can be used
# instead of "buffer" and "file" in the logger handlers.
console:
class: logging.StreamHandler
formatter: precise
loggers:
synapse.storage.SQL:
# beware: increasing this to DEBUG will make synapse log sensitive
# information such as access tokens.
level: WARN
twisted:
# We send the twisted logging directly to the file handler,
# to work around https://github.com/matrix-org/synapse/issues/3471
# when using "buffer" logger. Use "console" to log to stderr instead.
handlers: [file]
propagate: false
root:
level: WARN
# Write logs to the `buffer` handler, which will buffer them together in memory,
# then write them to a file.
#
# Replace "buffer" with "console" to log to stderr instead. (Note that you'll
# also need to update the configuration for the `twisted` logger above, in
# this case.)
#
handlers: [buffer]
disable_existing_loggers: false

View File

@ -0,0 +1,84 @@
# Log configuration for Synapse.
#
# This is a YAML file containing a standard Python logging configuration
# dictionary. See [1] for details on the valid settings.
#
# Synapse also supports structured logging for machine readable logs which can
# be ingested by ELK stacks. See [2] for details.
#
# [1]: https://docs.python.org/3.7/library/logging.config.html#configuration-dictionary-schema
# [2]: https://matrix-org.github.io/synapse/latest/structured_logging.html
version: 1
formatters:
precise:
format: '%(asctime)s - %(name)s - %(lineno)d - %(levelname)s - %(request)s - %(message)s'
handlers:
file:
class: logging.handlers.TimedRotatingFileHandler
formatter: precise
filename: /data/workers/log/additional2.log
when: midnight
backupCount: 2 # Does not include the current log file.
encoding: utf8
# Default to buffering writes to log file for efficiency.
# WARNING/ERROR logs will still be flushed immediately, but there will be a
# delay (of up to `period` seconds, or until the buffer is full with
# `capacity` messages) before INFO/DEBUG logs get written.
buffer:
class: synapse.logging.handlers.PeriodicallyFlushingMemoryHandler
target: file
# The capacity is the maximum number of log lines that are buffered
# before being written to disk. Increasing this will lead to better
# performance, at the expensive of it taking longer for log lines to
# be written to disk.
# This parameter is required.
capacity: 10
# Logs with a level at or above the flush level will cause the buffer to
# be flushed immediately.
# Default value: 40 (ERROR)
# Other values: 50 (CRITICAL), 30 (WARNING), 20 (INFO), 10 (DEBUG)
flushLevel: 30 # Flush immediately for WARNING logs and higher
# The period of time, in seconds, between forced flushes.
# Messages will not be delayed for longer than this time.
# Default value: 5 seconds
period: 5
# A handler that writes logs to stderr. Unused by default, but can be used
# instead of "buffer" and "file" in the logger handlers.
console:
class: logging.StreamHandler
formatter: precise
loggers:
synapse.storage.SQL:
# beware: increasing this to DEBUG will make synapse log sensitive
# information such as access tokens.
level: WARN
twisted:
# We send the twisted logging directly to the file handler,
# to work around https://github.com/matrix-org/synapse/issues/3471
# when using "buffer" logger. Use "console" to log to stderr instead.
handlers: [file]
propagate: false
root:
level: WARN
# Write logs to the `buffer` handler, which will buffer them together in memory,
# then write them to a file.
#
# Replace "buffer" with "console" to log to stderr instead. (Note that you'll
# also need to update the configuration for the `twisted` logger above, in
# this case.)
#
handlers: [buffer]
disable_existing_loggers: false

84
files/log/event1.yaml Normal file
View File

@ -0,0 +1,84 @@
# Log configuration for Synapse.
#
# This is a YAML file containing a standard Python logging configuration
# dictionary. See [1] for details on the valid settings.
#
# Synapse also supports structured logging for machine readable logs which can
# be ingested by ELK stacks. See [2] for details.
#
# [1]: https://docs.python.org/3.7/library/logging.config.html#configuration-dictionary-schema
# [2]: https://matrix-org.github.io/synapse/latest/structured_logging.html
version: 1
formatters:
precise:
format: '%(asctime)s - %(name)s - %(lineno)d - %(levelname)s - %(request)s - %(message)s'
handlers:
file:
class: logging.handlers.TimedRotatingFileHandler
formatter: precise
filename: /data/workers/log/event1.log
when: midnight
backupCount: 2 # Does not include the current log file.
encoding: utf8
# Default to buffering writes to log file for efficiency.
# WARNING/ERROR logs will still be flushed immediately, but there will be a
# delay (of up to `period` seconds, or until the buffer is full with
# `capacity` messages) before INFO/DEBUG logs get written.
buffer:
class: synapse.logging.handlers.PeriodicallyFlushingMemoryHandler
target: file
# The capacity is the maximum number of log lines that are buffered
# before being written to disk. Increasing this will lead to better
# performance, at the expensive of it taking longer for log lines to
# be written to disk.
# This parameter is required.
capacity: 10
# Logs with a level at or above the flush level will cause the buffer to
# be flushed immediately.
# Default value: 40 (ERROR)
# Other values: 50 (CRITICAL), 30 (WARNING), 20 (INFO), 10 (DEBUG)
flushLevel: 30 # Flush immediately for WARNING logs and higher
# The period of time, in seconds, between forced flushes.
# Messages will not be delayed for longer than this time.
# Default value: 5 seconds
period: 5
# A handler that writes logs to stderr. Unused by default, but can be used
# instead of "buffer" and "file" in the logger handlers.
console:
class: logging.StreamHandler
formatter: precise
loggers:
synapse.storage.SQL:
# beware: increasing this to DEBUG will make synapse log sensitive
# information such as access tokens.
level: WARN
twisted:
# We send the twisted logging directly to the file handler,
# to work around https://github.com/matrix-org/synapse/issues/3471
# when using "buffer" logger. Use "console" to log to stderr instead.
handlers: [file]
propagate: false
root:
level: WARN
# Write logs to the `buffer` handler, which will buffer them together in memory,
# then write them to a file.
#
# Replace "buffer" with "console" to log to stderr instead. (Note that you'll
# also need to update the configuration for the `twisted` logger above, in
# this case.)
#
handlers: [buffer]
disable_existing_loggers: false

84
files/log/event2.yaml Normal file
View File

@ -0,0 +1,84 @@
# Log configuration for Synapse.
#
# This is a YAML file containing a standard Python logging configuration
# dictionary. See [1] for details on the valid settings.
#
# Synapse also supports structured logging for machine readable logs which can
# be ingested by ELK stacks. See [2] for details.
#
# [1]: https://docs.python.org/3.7/library/logging.config.html#configuration-dictionary-schema
# [2]: https://matrix-org.github.io/synapse/latest/structured_logging.html
version: 1
formatters:
precise:
format: '%(asctime)s - %(name)s - %(lineno)d - %(levelname)s - %(request)s - %(message)s'
handlers:
file:
class: logging.handlers.TimedRotatingFileHandler
formatter: precise
filename: /data/workers/log/event2.log
when: midnight
backupCount: 2 # Does not include the current log file.
encoding: utf8
# Default to buffering writes to log file for efficiency.
# WARNING/ERROR logs will still be flushed immediately, but there will be a
# delay (of up to `period` seconds, or until the buffer is full with
# `capacity` messages) before INFO/DEBUG logs get written.
buffer:
class: synapse.logging.handlers.PeriodicallyFlushingMemoryHandler
target: file
# The capacity is the maximum number of log lines that are buffered
# before being written to disk. Increasing this will lead to better
# performance, at the expensive of it taking longer for log lines to
# be written to disk.
# This parameter is required.
capacity: 10
# Logs with a level at or above the flush level will cause the buffer to
# be flushed immediately.
# Default value: 40 (ERROR)
# Other values: 50 (CRITICAL), 30 (WARNING), 20 (INFO), 10 (DEBUG)
flushLevel: 30 # Flush immediately for WARNING logs and higher
# The period of time, in seconds, between forced flushes.
# Messages will not be delayed for longer than this time.
# Default value: 5 seconds
period: 5
# A handler that writes logs to stderr. Unused by default, but can be used
# instead of "buffer" and "file" in the logger handlers.
console:
class: logging.StreamHandler
formatter: precise
loggers:
synapse.storage.SQL:
# beware: increasing this to DEBUG will make synapse log sensitive
# information such as access tokens.
level: WARN
twisted:
# We send the twisted logging directly to the file handler,
# to work around https://github.com/matrix-org/synapse/issues/3471
# when using "buffer" logger. Use "console" to log to stderr instead.
handlers: [file]
propagate: false
root:
level: WARN
# Write logs to the `buffer` handler, which will buffer them together in memory,
# then write them to a file.
#
# Replace "buffer" with "console" to log to stderr instead. (Note that you'll
# also need to update the configuration for the `twisted` logger above, in
# this case.)
#
handlers: [buffer]
disable_existing_loggers: false

84
files/log/federation1.yaml Executable file
View File

@ -0,0 +1,84 @@
# Log configuration for Synapse.
#
# This is a YAML file containing a standard Python logging configuration
# dictionary. See [1] for details on the valid settings.
#
# Synapse also supports structured logging for machine readable logs which can
# be ingested by ELK stacks. See [2] for details.
#
# [1]: https://docs.python.org/3.7/library/logging.config.html#configuration-dictionary-schema
# [2]: https://matrix-org.github.io/synapse/latest/structured_logging.html
version: 1
formatters:
precise:
format: '%(asctime)s - %(name)s - %(lineno)d - %(levelname)s - %(request)s - %(message)s'
handlers:
file:
class: logging.handlers.TimedRotatingFileHandler
formatter: precise
filename: /data/workers/log/federation1.log
when: midnight
backupCount: 2 # Does not include the current log file.
encoding: utf8
# Default to buffering writes to log file for efficiency.
# WARNING/ERROR logs will still be flushed immediately, but there will be a
# delay (of up to `period` seconds, or until the buffer is full with
# `capacity` messages) before INFO/DEBUG logs get written.
buffer:
class: synapse.logging.handlers.PeriodicallyFlushingMemoryHandler
target: file
# The capacity is the maximum number of log lines that are buffered
# before being written to disk. Increasing this will lead to better
# performance, at the expensive of it taking longer for log lines to
# be written to disk.
# This parameter is required.
capacity: 10
# Logs with a level at or above the flush level will cause the buffer to
# be flushed immediately.
# Default value: 40 (ERROR)
# Other values: 50 (CRITICAL), 30 (WARNING), 20 (INFO), 10 (DEBUG)
flushLevel: 30 # Flush immediately for WARNING logs and higher
# The period of time, in seconds, between forced flushes.
# Messages will not be delayed for longer than this time.
# Default value: 5 seconds
period: 5
# A handler that writes logs to stderr. Unused by default, but can be used
# instead of "buffer" and "file" in the logger handlers.
console:
class: logging.StreamHandler
formatter: precise
loggers:
synapse.storage.SQL:
# beware: increasing this to DEBUG will make synapse log sensitive
# information such as access tokens.
level: WARN
twisted:
# We send the twisted logging directly to the file handler,
# to work around https://github.com/matrix-org/synapse/issues/3471
# when using "buffer" logger. Use "console" to log to stderr instead.
handlers: [file]
propagate: false
root:
level: WARN
# Write logs to the `buffer` handler, which will buffer them together in memory,
# then write them to a file.
#
# Replace "buffer" with "console" to log to stderr instead. (Note that you'll
# also need to update the configuration for the `twisted` logger above, in
# this case.)
#
handlers: [buffer]
disable_existing_loggers: false

84
files/log/federation2.yaml Executable file
View File

@ -0,0 +1,84 @@
# Log configuration for Synapse.
#
# This is a YAML file containing a standard Python logging configuration
# dictionary. See [1] for details on the valid settings.
#
# Synapse also supports structured logging for machine readable logs which can
# be ingested by ELK stacks. See [2] for details.
#
# [1]: https://docs.python.org/3.7/library/logging.config.html#configuration-dictionary-schema
# [2]: https://matrix-org.github.io/synapse/latest/structured_logging.html
version: 1
formatters:
precise:
format: '%(asctime)s - %(name)s - %(lineno)d - %(levelname)s - %(request)s - %(message)s'
handlers:
file:
class: logging.handlers.TimedRotatingFileHandler
formatter: precise
filename: /data/workers/log/federation2.log
when: midnight
backupCount: 2 # Does not include the current log file.
encoding: utf8
# Default to buffering writes to log file for efficiency.
# WARNING/ERROR logs will still be flushed immediately, but there will be a
# delay (of up to `period` seconds, or until the buffer is full with
# `capacity` messages) before INFO/DEBUG logs get written.
buffer:
class: synapse.logging.handlers.PeriodicallyFlushingMemoryHandler
target: file
# The capacity is the maximum number of log lines that are buffered
# before being written to disk. Increasing this will lead to better
# performance, at the expensive of it taking longer for log lines to
# be written to disk.
# This parameter is required.
capacity: 10
# Logs with a level at or above the flush level will cause the buffer to
# be flushed immediately.
# Default value: 40 (ERROR)
# Other values: 50 (CRITICAL), 30 (WARNING), 20 (INFO), 10 (DEBUG)
flushLevel: 30 # Flush immediately for WARNING logs and higher
# The period of time, in seconds, between forced flushes.
# Messages will not be delayed for longer than this time.
# Default value: 5 seconds
period: 5
# A handler that writes logs to stderr. Unused by default, but can be used
# instead of "buffer" and "file" in the logger handlers.
console:
class: logging.StreamHandler
formatter: precise
loggers:
synapse.storage.SQL:
# beware: increasing this to DEBUG will make synapse log sensitive
# information such as access tokens.
level: WARN
twisted:
# We send the twisted logging directly to the file handler,
# to work around https://github.com/matrix-org/synapse/issues/3471
# when using "buffer" logger. Use "console" to log to stderr instead.
handlers: [file]
propagate: false
root:
level: WARN
# Write logs to the `buffer` handler, which will buffer them together in memory,
# then write them to a file.
#
# Replace "buffer" with "console" to log to stderr instead. (Note that you'll
# also need to update the configuration for the `twisted` logger above, in
# this case.)
#
handlers: [buffer]
disable_existing_loggers: false

84
files/log/federation3.yaml Executable file
View File

@ -0,0 +1,84 @@
# Log configuration for Synapse.
#
# This is a YAML file containing a standard Python logging configuration
# dictionary. See [1] for details on the valid settings.
#
# Synapse also supports structured logging for machine readable logs which can
# be ingested by ELK stacks. See [2] for details.
#
# [1]: https://docs.python.org/3.7/library/logging.config.html#configuration-dictionary-schema
# [2]: https://matrix-org.github.io/synapse/latest/structured_logging.html
version: 1
formatters:
precise:
format: '%(asctime)s - %(name)s - %(lineno)d - %(levelname)s - %(request)s - %(message)s'
handlers:
file:
class: logging.handlers.TimedRotatingFileHandler
formatter: precise
filename: /data/workers/log/federation3.log
when: midnight
backupCount: 2 # Does not include the current log file.
encoding: utf8
# Default to buffering writes to log file for efficiency.
# WARNING/ERROR logs will still be flushed immediately, but there will be a
# delay (of up to `period` seconds, or until the buffer is full with
# `capacity` messages) before INFO/DEBUG logs get written.
buffer:
class: synapse.logging.handlers.PeriodicallyFlushingMemoryHandler
target: file
# The capacity is the maximum number of log lines that are buffered
# before being written to disk. Increasing this will lead to better
# performance, at the expensive of it taking longer for log lines to
# be written to disk.
# This parameter is required.
capacity: 10
# Logs with a level at or above the flush level will cause the buffer to
# be flushed immediately.
# Default value: 40 (ERROR)
# Other values: 50 (CRITICAL), 30 (WARNING), 20 (INFO), 10 (DEBUG)
flushLevel: 30 # Flush immediately for WARNING logs and higher
# The period of time, in seconds, between forced flushes.
# Messages will not be delayed for longer than this time.
# Default value: 5 seconds
period: 5
# A handler that writes logs to stderr. Unused by default, but can be used
# instead of "buffer" and "file" in the logger handlers.
console:
class: logging.StreamHandler
formatter: precise
loggers:
synapse.storage.SQL:
# beware: increasing this to DEBUG will make synapse log sensitive
# information such as access tokens.
level: WARN
twisted:
# We send the twisted logging directly to the file handler,
# to work around https://github.com/matrix-org/synapse/issues/3471
# when using "buffer" logger. Use "console" to log to stderr instead.
handlers: [file]
propagate: false
root:
level: WARN
# Write logs to the `buffer` handler, which will buffer them together in memory,
# then write them to a file.
#
# Replace "buffer" with "console" to log to stderr instead. (Note that you'll
# also need to update the configuration for the `twisted` logger above, in
# this case.)
#
handlers: [buffer]
disable_existing_loggers: false

84
files/log/sync1.yaml Normal file
View File

@ -0,0 +1,84 @@
# Log configuration for Synapse.
#
# This is a YAML file containing a standard Python logging configuration
# dictionary. See [1] for details on the valid settings.
#
# Synapse also supports structured logging for machine readable logs which can
# be ingested by ELK stacks. See [2] for details.
#
# [1]: https://docs.python.org/3.7/library/logging.config.html#configuration-dictionary-schema
# [2]: https://matrix-org.github.io/synapse/latest/structured_logging.html
version: 1
formatters:
precise:
format: '%(asctime)s - %(name)s - %(lineno)d - %(levelname)s - %(request)s - %(message)s'
handlers:
file:
class: logging.handlers.TimedRotatingFileHandler
formatter: precise
filename: /data/workers/log/worker_generic_sync1.log
when: midnight
backupCount: 2 # Does not include the current log file.
encoding: utf8
# Default to buffering writes to log file for efficiency.
# WARNING/ERROR logs will still be flushed immediately, but there will be a
# delay (of up to `period` seconds, or until the buffer is full with
# `capacity` messages) before INFO/DEBUG logs get written.
buffer:
class: synapse.logging.handlers.PeriodicallyFlushingMemoryHandler
target: file
# The capacity is the maximum number of log lines that are buffered
# before being written to disk. Increasing this will lead to better
# performance, at the expensive of it taking longer for log lines to
# be written to disk.
# This parameter is required.
capacity: 10
# Logs with a level at or above the flush level will cause the buffer to
# be flushed immediately.
# Default value: 40 (ERROR)
# Other values: 50 (CRITICAL), 30 (WARNING), 20 (INFO), 10 (DEBUG)
flushLevel: 30 # Flush immediately for WARNING logs and higher
# The period of time, in seconds, between forced flushes.
# Messages will not be delayed for longer than this time.
# Default value: 5 seconds
period: 5
# A handler that writes logs to stderr. Unused by default, but can be used
# instead of "buffer" and "file" in the logger handlers.
console:
class: logging.StreamHandler
formatter: precise
loggers:
synapse.storage.SQL:
# beware: increasing this to DEBUG will make synapse log sensitive
# information such as access tokens.
level: WARN
twisted:
# We send the twisted logging directly to the file handler,
# to work around https://github.com/matrix-org/synapse/issues/3471
# when using "buffer" logger. Use "console" to log to stderr instead.
handlers: [file]
propagate: false
root:
level: WARN
# Write logs to the `buffer` handler, which will buffer them together in memory,
# then write them to a file.
#
# Replace "buffer" with "console" to log to stderr instead. (Note that you'll
# also need to update the configuration for the `twisted` logger above, in
# this case.)
#
handlers: [buffer]
disable_existing_loggers: false

84
files/log/sync2.yaml Normal file
View File

@ -0,0 +1,84 @@
# Log configuration for Synapse.
#
# This is a YAML file containing a standard Python logging configuration
# dictionary. See [1] for details on the valid settings.
#
# Synapse also supports structured logging for machine readable logs which can
# be ingested by ELK stacks. See [2] for details.
#
# [1]: https://docs.python.org/3.7/library/logging.config.html#configuration-dictionary-schema
# [2]: https://matrix-org.github.io/synapse/latest/structured_logging.html
version: 1
formatters:
precise:
format: '%(asctime)s - %(name)s - %(lineno)d - %(levelname)s - %(request)s - %(message)s'
handlers:
file:
class: logging.handlers.TimedRotatingFileHandler
formatter: precise
filename: /data/workers/log/worker_generic_sync2.log
when: midnight
backupCount: 2 # Does not include the current log file.
encoding: utf8
# Default to buffering writes to log file for efficiency.
# WARNING/ERROR logs will still be flushed immediately, but there will be a
# delay (of up to `period` seconds, or until the buffer is full with
# `capacity` messages) before INFO/DEBUG logs get written.
buffer:
class: synapse.logging.handlers.PeriodicallyFlushingMemoryHandler
target: file
# The capacity is the maximum number of log lines that are buffered
# before being written to disk. Increasing this will lead to better
# performance, at the expensive of it taking longer for log lines to
# be written to disk.
# This parameter is required.
capacity: 10
# Logs with a level at or above the flush level will cause the buffer to
# be flushed immediately.
# Default value: 40 (ERROR)
# Other values: 50 (CRITICAL), 30 (WARNING), 20 (INFO), 10 (DEBUG)
flushLevel: 30 # Flush immediately for WARNING logs and higher
# The period of time, in seconds, between forced flushes.
# Messages will not be delayed for longer than this time.
# Default value: 5 seconds
period: 5
# A handler that writes logs to stderr. Unused by default, but can be used
# instead of "buffer" and "file" in the logger handlers.
console:
class: logging.StreamHandler
formatter: precise
loggers:
synapse.storage.SQL:
# beware: increasing this to DEBUG will make synapse log sensitive
# information such as access tokens.
level: WARN
twisted:
# We send the twisted logging directly to the file handler,
# to work around https://github.com/matrix-org/synapse/issues/3471
# when using "buffer" logger. Use "console" to log to stderr instead.
handlers: [file]
propagate: false
root:
level: WARN
# Write logs to the `buffer` handler, which will buffer them together in memory,
# then write them to a file.
#
# Replace "buffer" with "console" to log to stderr instead. (Note that you'll
# also need to update the configuration for the `twisted` logger above, in
# this case.)
#
handlers: [buffer]
disable_existing_loggers: false

84
files/log/sync3.yaml Normal file
View File

@ -0,0 +1,84 @@
# Log configuration for Synapse.
#
# This is a YAML file containing a standard Python logging configuration
# dictionary. See [1] for details on the valid settings.
#
# Synapse also supports structured logging for machine readable logs which can
# be ingested by ELK stacks. See [2] for details.
#
# [1]: https://docs.python.org/3.7/library/logging.config.html#configuration-dictionary-schema
# [2]: https://matrix-org.github.io/synapse/latest/structured_logging.html
version: 1
formatters:
precise:
format: '%(asctime)s - %(name)s - %(lineno)d - %(levelname)s - %(request)s - %(message)s'
handlers:
file:
class: logging.handlers.TimedRotatingFileHandler
formatter: precise
filename: /data/workers/log/worker_generic_sync3.log
when: midnight
backupCount: 2 # Does not include the current log file.
encoding: utf8
# Default to buffering writes to log file for efficiency.
# WARNING/ERROR logs will still be flushed immediately, but there will be a
# delay (of up to `period` seconds, or until the buffer is full with
# `capacity` messages) before INFO/DEBUG logs get written.
buffer:
class: synapse.logging.handlers.PeriodicallyFlushingMemoryHandler
target: file
# The capacity is the maximum number of log lines that are buffered
# before being written to disk. Increasing this will lead to better
# performance, at the expensive of it taking longer for log lines to
# be written to disk.
# This parameter is required.
capacity: 10
# Logs with a level at or above the flush level will cause the buffer to
# be flushed immediately.
# Default value: 40 (ERROR)
# Other values: 50 (CRITICAL), 30 (WARNING), 20 (INFO), 10 (DEBUG)
flushLevel: 30 # Flush immediately for WARNING logs and higher
# The period of time, in seconds, between forced flushes.
# Messages will not be delayed for longer than this time.
# Default value: 5 seconds
period: 5
# A handler that writes logs to stderr. Unused by default, but can be used
# instead of "buffer" and "file" in the logger handlers.
console:
class: logging.StreamHandler
formatter: precise
loggers:
synapse.storage.SQL:
# beware: increasing this to DEBUG will make synapse log sensitive
# information such as access tokens.
level: WARN
twisted:
# We send the twisted logging directly to the file handler,
# to work around https://github.com/matrix-org/synapse/issues/3471
# when using "buffer" logger. Use "console" to log to stderr instead.
handlers: [file]
propagate: false
root:
level: WARN
# Write logs to the `buffer` handler, which will buffer them together in memory,
# then write them to a file.
#
# Replace "buffer" with "console" to log to stderr instead. (Note that you'll
# also need to update the configuration for the `twisted` logger above, in
# this case.)
#
handlers: [buffer]
disable_existing_loggers: false

84
files/log/worker1.yaml Normal file
View File

@ -0,0 +1,84 @@
# Log configuration for Synapse.
#
# This is a YAML file containing a standard Python logging configuration
# dictionary. See [1] for details on the valid settings.
#
# Synapse also supports structured logging for machine readable logs which can
# be ingested by ELK stacks. See [2] for details.
#
# [1]: https://docs.python.org/3.7/library/logging.config.html#configuration-dictionary-schema
# [2]: https://matrix-org.github.io/synapse/latest/structured_logging.html
version: 1
formatters:
precise:
format: '%(asctime)s - %(name)s - %(lineno)d - %(levelname)s - %(request)s - %(message)s'
handlers:
file:
class: logging.handlers.TimedRotatingFileHandler
formatter: precise
filename: /data/workers/log/worker1.log
when: midnight
backupCount: 2 # Does not include the current log file.
encoding: utf8
# Default to buffering writes to log file for efficiency.
# WARNING/ERROR logs will still be flushed immediately, but there will be a
# delay (of up to `period` seconds, or until the buffer is full with
# `capacity` messages) before INFO/DEBUG logs get written.
buffer:
class: synapse.logging.handlers.PeriodicallyFlushingMemoryHandler
target: file
# The capacity is the maximum number of log lines that are buffered
# before being written to disk. Increasing this will lead to better
# performance, at the expensive of it taking longer for log lines to
# be written to disk.
# This parameter is required.
capacity: 10
# Logs with a level at or above the flush level will cause the buffer to
# be flushed immediately.
# Default value: 40 (ERROR)
# Other values: 50 (CRITICAL), 30 (WARNING), 20 (INFO), 10 (DEBUG)
flushLevel: 30 # Flush immediately for WARNING logs and higher
# The period of time, in seconds, between forced flushes.
# Messages will not be delayed for longer than this time.
# Default value: 5 seconds
period: 5
# A handler that writes logs to stderr. Unused by default, but can be used
# instead of "buffer" and "file" in the logger handlers.
console:
class: logging.StreamHandler
formatter: precise
loggers:
synapse.storage.SQL:
# beware: increasing this to DEBUG will make synapse log sensitive
# information such as access tokens.
level: WARN
twisted:
# We send the twisted logging directly to the file handler,
# to work around https://github.com/matrix-org/synapse/issues/3471
# when using "buffer" logger. Use "console" to log to stderr instead.
handlers: [file]
propagate: false
root:
level: WARN
# Write logs to the `buffer` handler, which will buffer them together in memory,
# then write them to a file.
#
# Replace "buffer" with "console" to log to stderr instead. (Note that you'll
# also need to update the configuration for the `twisted` logger above, in
# this case.)
#
handlers: [buffer]
disable_existing_loggers: false

84
files/log/worker2.yaml Normal file
View File

@ -0,0 +1,84 @@
# Log configuration for Synapse.
#
# This is a YAML file containing a standard Python logging configuration
# dictionary. See [1] for details on the valid settings.
#
# Synapse also supports structured logging for machine readable logs which can
# be ingested by ELK stacks. See [2] for details.
#
# [1]: https://docs.python.org/3.7/library/logging.config.html#configuration-dictionary-schema
# [2]: https://matrix-org.github.io/synapse/latest/structured_logging.html
version: 1
formatters:
precise:
format: '%(asctime)s - %(name)s - %(lineno)d - %(levelname)s - %(request)s - %(message)s'
handlers:
file:
class: logging.handlers.TimedRotatingFileHandler
formatter: precise
filename: /data/workers/log/worker2.log
when: midnight
backupCount: 2 # Does not include the current log file.
encoding: utf8
# Default to buffering writes to log file for efficiency.
# WARNING/ERROR logs will still be flushed immediately, but there will be a
# delay (of up to `period` seconds, or until the buffer is full with
# `capacity` messages) before INFO/DEBUG logs get written.
buffer:
class: synapse.logging.handlers.PeriodicallyFlushingMemoryHandler
target: file
# The capacity is the maximum number of log lines that are buffered
# before being written to disk. Increasing this will lead to better
# performance, at the expensive of it taking longer for log lines to
# be written to disk.
# This parameter is required.
capacity: 10
# Logs with a level at or above the flush level will cause the buffer to
# be flushed immediately.
# Default value: 40 (ERROR)
# Other values: 50 (CRITICAL), 30 (WARNING), 20 (INFO), 10 (DEBUG)
flushLevel: 30 # Flush immediately for WARNING logs and higher
# The period of time, in seconds, between forced flushes.
# Messages will not be delayed for longer than this time.
# Default value: 5 seconds
period: 5
# A handler that writes logs to stderr. Unused by default, but can be used
# instead of "buffer" and "file" in the logger handlers.
console:
class: logging.StreamHandler
formatter: precise
loggers:
synapse.storage.SQL:
# beware: increasing this to DEBUG will make synapse log sensitive
# information such as access tokens.
level: WARN
twisted:
# We send the twisted logging directly to the file handler,
# to work around https://github.com/matrix-org/synapse/issues/3471
# when using "buffer" logger. Use "console" to log to stderr instead.
handlers: [file]
propagate: false
root:
level: WARN
# Write logs to the `buffer` handler, which will buffer them together in memory,
# then write them to a file.
#
# Replace "buffer" with "console" to log to stderr instead. (Note that you'll
# also need to update the configuration for the `twisted` logger above, in
# this case.)
#
handlers: [buffer]
disable_existing_loggers: false

84
files/log/worker3.yaml Normal file
View File

@ -0,0 +1,84 @@
# Log configuration for Synapse.
#
# This is a YAML file containing a standard Python logging configuration
# dictionary. See [1] for details on the valid settings.
#
# Synapse also supports structured logging for machine readable logs which can
# be ingested by ELK stacks. See [2] for details.
#
# [1]: https://docs.python.org/3.7/library/logging.config.html#configuration-dictionary-schema
# [2]: https://matrix-org.github.io/synapse/latest/structured_logging.html
version: 1
formatters:
precise:
format: '%(asctime)s - %(name)s - %(lineno)d - %(levelname)s - %(request)s - %(message)s'
handlers:
file:
class: logging.handlers.TimedRotatingFileHandler
formatter: precise
filename: /data/workers/log/worker3.log
when: midnight
backupCount: 2 # Does not include the current log file.
encoding: utf8
# Default to buffering writes to log file for efficiency.
# WARNING/ERROR logs will still be flushed immediately, but there will be a
# delay (of up to `period` seconds, or until the buffer is full with
# `capacity` messages) before INFO/DEBUG logs get written.
buffer:
class: synapse.logging.handlers.PeriodicallyFlushingMemoryHandler
target: file
# The capacity is the maximum number of log lines that are buffered
# before being written to disk. Increasing this will lead to better
# performance, at the expensive of it taking longer for log lines to
# be written to disk.
# This parameter is required.
capacity: 10
# Logs with a level at or above the flush level will cause the buffer to
# be flushed immediately.
# Default value: 40 (ERROR)
# Other values: 50 (CRITICAL), 30 (WARNING), 20 (INFO), 10 (DEBUG)
flushLevel: 30 # Flush immediately for WARNING logs and higher
# The period of time, in seconds, between forced flushes.
# Messages will not be delayed for longer than this time.
# Default value: 5 seconds
period: 5
# A handler that writes logs to stderr. Unused by default, but can be used
# instead of "buffer" and "file" in the logger handlers.
console:
class: logging.StreamHandler
formatter: precise
loggers:
synapse.storage.SQL:
# beware: increasing this to DEBUG will make synapse log sensitive
# information such as access tokens.
level: WARN
twisted:
# We send the twisted logging directly to the file handler,
# to work around https://github.com/matrix-org/synapse/issues/3471
# when using "buffer" logger. Use "console" to log to stderr instead.
handlers: [file]
propagate: false
root:
level: WARN
# Write logs to the `buffer` handler, which will buffer them together in memory,
# then write them to a file.
#
# Replace "buffer" with "console" to log to stderr instead. (Note that you'll
# also need to update the configuration for the `twisted` logger above, in
# this case.)
#
handlers: [buffer]
disable_existing_loggers: false

View File

@ -0,0 +1,22 @@
worker_app: synapse.app.generic_worker
worker_name: additional1
# The replication listener on the synapse to talk to.
worker_replication_host: synapse
worker_replication_http_port: 9093
worker_main_http_uri: http://synapse:8008/
worker_log_config: /data/log/additional1.yaml
worker_listeners:
- type: http
port: 8081
tls: false
x_forwarded: true
resources:
- names: [client, federation]
- type: http
port: 8034
resources:
- names: [replication]

View File

@ -0,0 +1,22 @@
worker_app: synapse.app.generic_worker
worker_name: additional2
# The replication listener on the synapse to talk to.
worker_replication_host: synapse
worker_replication_http_port: 9093
worker_main_http_uri: http://synapse:8008/
worker_log_config: /data/log/additional2.yaml
worker_listeners:
- type: http
port: 8081
tls: false
x_forwarded: true
resources:
- names: [client, federation]
- type: http
port: 8034
resources:
- names: [replication]

14
files/workers/event1.yaml Normal file
View File

@ -0,0 +1,14 @@
worker_app: synapse.app.generic_worker
worker_name: event1
# The replication listener on the synapse to talk to.
worker_replication_host: synapse
worker_replication_http_port: 9093
worker_log_config: /data/log/event1.yaml
worker_listeners:
- type: http
port: 8034
resources:
- names: [replication]

14
files/workers/event2.yaml Normal file
View File

@ -0,0 +1,14 @@
worker_app: synapse.app.generic_worker
worker_name: event2
# The replication listener on the synapse to talk to.
worker_replication_host: synapse
worker_replication_http_port: 9093
worker_log_config: /data/log/event2.yaml
worker_listeners:
- type: http
port: 8034
resources:
- names: [replication]

View File

@ -0,0 +1,8 @@
worker_app: synapse.app.federation_sender
worker_name: federation1
# The replication listener on the main synapse process.
worker_replication_host: synapse
worker_replication_http_port: 9093
worker_log_config: /data/log/federation1.yaml

View File

@ -0,0 +1,8 @@
worker_app: synapse.app.federation_sender
worker_name: federation2
# The replication listener on the main synapse process.
worker_replication_host: synapse
worker_replication_http_port: 9093
worker_log_config: /data/log/federation2.yaml

View File

@ -0,0 +1,8 @@
worker_app: synapse.app.federation_sender
worker_name: federation3
# The replication listener on the main synapse process.
worker_replication_host: synapse
worker_replication_http_port: 9093
worker_log_config: /data/log/federation3.yaml

18
files/workers/sync1.yaml Normal file
View File

@ -0,0 +1,18 @@
worker_app: synapse.app.generic_worker
worker_name: sync1
# The replication listener on the synapse to talk to.
worker_replication_host: synapse
worker_replication_http_port: 9093
worker_main_http_uri: http://synapse:8008/
worker_log_config: /data/log/sync1.yaml
worker_listeners:
- type: http
port: 8081
x_forwarded: true
resources:
- names: [client, federation]
compress: false

17
files/workers/sync2.yaml Normal file
View File

@ -0,0 +1,17 @@
worker_app: synapse.app.generic_worker
worker_name: sync2
# The replication listener on the synapse to talk to.
worker_replication_host: synapse
worker_replication_http_port: 9093
worker_main_http_uri: http://synapse:8008/
worker_log_config: /data/log/sync2.yaml
worker_listeners:
- type: http
port: 8081
x_forwarded: true
resources:
- names: [client, federation]

17
files/workers/sync3.yaml Normal file
View File

@ -0,0 +1,17 @@
worker_app: synapse.app.generic_worker
worker_name: sync3
# The replication listener on the synapse to talk to.
worker_replication_host: synapse
worker_replication_http_port: 9093
worker_main_http_uri: http://synapse:8008/
worker_log_config: /data/log/sync3.yaml
worker_listeners:
- type: http
port: 8081
x_forwarded: true
resources:
- names: [client, federation]

View File

@ -0,0 +1,17 @@
worker_app: synapse.app.generic_worker
worker_name: worker1
# The replication listener on the synapse to talk to.
worker_replication_host: synapse
worker_replication_http_port: 9093
worker_main_http_uri: http://synapse:8008/
worker_log_config: /data/log/worker1.yaml
worker_listeners:
- type: http
port: 8081
x_forwarded: true
resources:
- names: [client, federation]

View File

@ -0,0 +1,17 @@
worker_app: synapse.app.generic_worker
worker_name: worker2
# The replication listener on the synapse to talk to.
worker_replication_host: synapse
worker_replication_http_port: 9093
worker_main_http_uri: http://synapse:8008/
worker_log_config: /data/log/worker2.yaml
worker_listeners:
- type: http
port: 8081
x_forwarded: true
resources:
- names: [client, federation]

View File

@ -0,0 +1,17 @@
worker_app: synapse.app.generic_worker
worker_name: worker3
# The replication listener on the synapse to talk to.
worker_replication_host: synapse
worker_replication_http_port: 9093
worker_main_http_uri: http://synapse:8008/
worker_log_config: /data/log/worker3.yaml
worker_listeners:
- type: http
port: 8081
x_forwarded: true
resources:
- names: [client, federation]

7
images/build.sh Normal file
View File

@ -0,0 +1,7 @@
#!/bin/bash
docker build ./synapse-captcha/
docker build ./mjolnir/
docker build ./pantalaimon/
docker build ./synapse-docker/
docker build ./synapse-worker-docker/

33
init.sh Normal file
View File

@ -0,0 +1,33 @@
#!/bin/bash
echo "===== Cloning Docker Images ====="
cd images/
git clone https://github.com/matrix-org/mjolnir
git clone https://github.com/matrix-org/pantalaimon/
git clone https://codberg.org/deathrow/synapse-worker-docker
git clone https://codeberg.org/deathrow/synapse-docker
git clone https://codeberg.org/deathrow/synapse-captcha
echo "===== Building Docker Images ====="
bash build.sh
echo "===== Building Tools ====="
cd ../tools/
git clone https://github.com/matrix-org/rust-synapse-compress-state
cd rust-synapse-compress-state/synapse-auto-compressor
cargo build
mv ../target/debug/synapse_auto_compressor ../../synapse_auto_compressor
cd ../../
git clone https://github.com/joj0/synadm
cd synadm
sudo python3 setup.py install
cd ..
echo "===== Pulling Docker Images ====="
docker-compose pull
echo "===== Complete ====="

View File

View File

@ -0,0 +1,12 @@
[Default]
LogLevel = Debug
SSL = True
[local-matrix]
Homeserver = https://matrix.example.tld
ListenAddress = 0.0.0.0
ListenPort = 8008
SSL = False
UseKeyring = False
IgnoreVerification = True

View File

@ -0,0 +1,85 @@
## Client API requests
location ~ ^/_matrix/client/(api/v1|r0|v3|unstable)/createRoom$ {
include /config/nginx/include.d/synapse-proxy.conf;
proxy_pass http://generic_worker_lc;
}
location ~ ^/_matrix/client/(api/v1|r0|v3|unstable)/publicRooms$ {
include /config/nginx/include.d/synapse-proxy.conf;
proxy_pass http://generic_worker_lc;
}
location ~ ^/_matrix/client/(api/v1|r0|v3|unstable)/rooms/.*/joined_members$ {
include /config/nginx/include.d/synapse-proxy.conf;
proxy_pass http://generic_worker_lc;
}
location ~ ^/_matrix/client/(api/v1|r0|v3|unstable)/rooms/.*/context/.*$ {
include /config/nginx/include.d/synapse-proxy.conf;
proxy_pass http://generic_worker_lc;
}
location ~ ^/_matrix/client/(api/v1|r0|v3|unstable)/rooms/.*/members$ {
include /config/nginx/include.d/synapse-proxy.conf;
proxy_pass http://generic_worker_lc;
}
location ~ ^/_matrix/client/(api/v1|r0|v3|unstable)/rooms/.*/state$ {
include /config/nginx/include.d/synapse-proxy.conf;
proxy_pass http://generic_worker_lc;
}
location ~ ^/_matrix/client/v1/rooms/.*/hierarchy$ {
include /config/nginx/include.d/synapse-proxy.conf;
proxy_pass http://generic_worker_lc;
}
location ~ ^/_matrix/client/unstable/org.matrix.msc2716/rooms/.*/batch_send$ {
include /config/nginx/include.d/synapse-proxy.conf;
proxy_pass http://generic_worker_lc;
}
location ~ ^/_matrix/client/unstable/im.nheko.summary/rooms/.*/summary$ {
include /config/nginx/include.d/synapse-proxy.conf;
proxy_pass http://generic_worker_lc;
}
location ~ ^/_matrix/client/(r0|v3|unstable)/account/3pid$ {
include /config/nginx/include.d/synapse-proxy.conf;
proxy_pass http://generic_worker_lc;
}
location ~ ^/_matrix/client/(r0|v3|unstable)/account/whoami$ {
include /config/nginx/include.d/synapse-proxy.conf;
proxy_pass http://generic_worker_lc;
}
location ~ ^/_matrix/client/(r0|v3|unstable)/devices$ {
include /config/nginx/include.d/synapse-proxy.conf;
proxy_pass http://generic_worker_lc;
}
location ~ ^/_matrix/client/versions$ {
include /config/nginx/include.d/synapse-proxy.conf;
proxy_pass http://generic_worker_lc;
}
location ~ ^/_matrix/client/(api/v1|r0|v3|unstable)/voip/turnServer$ {
include /config/nginx/include.d/synapse-proxy.conf;
proxy_pass http://generic_worker_lc;
}
location ~ ^/_matrix/client/(api/v1|r0|v3|unstable)/rooms/.*/event/ {
include /config/nginx/include.d/synapse-proxy.conf;
proxy_pass http://generic_worker_lc;
}
location ~ ^/_matrix/client/(api/v1|r0|v3|unstable)/joined_rooms$ {
include /config/nginx/include.d/synapse-proxy.conf;
proxy_pass http://generic_worker_lc;
}
location ~ ^/_matrix/client/(api/v1|r0|v3|unstable)/search$ {
include /config/nginx/include.d/synapse-proxy.conf;
proxy_pass http://generic_worker_lc;
}

View File

@ -0,0 +1,26 @@
## Encryption requests
location ~ ^/_matrix/client/(r0|v3|unstable)/keys/query$ {
include /config/nginx/include.d/synapse-proxy.conf;
proxy_pass http://generic_worker_lc;
}
location ~ ^/_matrix/client/(r0|v3|unstable)/keys/changes$ {
include /config/nginx/include.d/synapse-proxy.conf;
proxy_pass http://generic_worker_lc;
}
location ~ ^/_matrix/client/(r0|v3|unstable)/keys/claim$ {
include /config/nginx/include.d/synapse-proxy.conf;
proxy_pass http://generic_worker_lc;
}
location ~ ^/_matrix/client/(r0|v3|unstable)/room_keys/ {
include /config/nginx/include.d/synapse-proxy.conf;
proxy_pass http://generic_worker_lc;
}
# Note that ^/_matrix/client/(r0|v3|unstable)/keys/upload/ requires `worker_main_http_uri`
location ~ ^/_matrix/client/(r0|v3|unstable)/keys/upload/ {
include /config/nginx/include.d/synapse-proxy.conf;
proxy_pass http://generic_worker_lc;
}

View File

@ -0,0 +1,91 @@
## Federation requests
location ~ ^/_matrix/federation/v1/event/ {
include /config/nginx/include.d/synapse-proxy.conf;
proxy_pass http://generic_worker_lc;
}
location ~ ^/_matrix/federation/v1/state/ {
include /config/nginx/include.d/synapse-proxy.conf;
proxy_pass http://generic_worker_lc;
}
location ~ ^/_matrix/federation/v1/state_ids/ {
include /config/nginx/include.d/synapse-proxy.conf;
proxy_pass http://generic_worker_lc;
}
location ~ ^/_matrix/federation/v1/backfill/ {
include /config/nginx/include.d/synapse-proxy.conf;
proxy_pass http://generic_worker_lc;
}
location ~ ^/_matrix/federation/v1/get_missing_events/ {
include /config/nginx/include.d/synapse-proxy.conf;
proxy_pass http://generic_worker_lc;
}
location ~ ^/_matrix/federation/v1/publicRooms {
include /config/nginx/include.d/synapse-proxy.conf;
proxy_pass http://generic_worker_lc;
}
location ~ ^/_matrix/federation/v1/query/ {
include /config/nginx/include.d/synapse-proxy.conf;
proxy_pass http://generic_worker_lc;
}
location ~ ^/_matrix/federation/v1/make_join/ {
include /config/nginx/include.d/synapse-proxy.conf;
proxy_pass http://generic_worker_lc;
}
location ~ ^/_matrix/federation/v1/make_leave/ {
include /config/nginx/include.d/synapse-proxy.conf;
proxy_pass http://generic_worker_lc;
}
location ~ ^/_matrix/federation/(v1|v2)/send_join/ {
include /config/nginx/include.d/synapse-proxy.conf;
proxy_pass http://generic_worker_lc;
}
location ~ ^/_matrix/federation/(v1|v2)/send_leave/ {
include /config/nginx/include.d/synapse-proxy.conf;
proxy_pass http://generic_worker_lc;
}
location ~ ^/_matrix/federation/(v1|v2)/invite/ {
include /config/nginx/include.d/synapse-proxy.conf;
proxy_pass http://generic_worker_lc;
}
location ~ ^/_matrix/federation/v1/event_auth/ {
include /config/nginx/include.d/synapse-proxy.conf;
proxy_pass http://generic_worker_lc;
}
location ~ ^/_matrix/federation/v1/exchange_third_party_invite/ {
include /config/nginx/include.d/synapse-proxy.conf;
proxy_pass http://generic_worker_lc;
}
location ~ ^/_matrix/federation/v1/user/devices/ {
include /config/nginx/include.d/synapse-proxy.conf;
proxy_pass http://generic_worker_lc;
}
location ~ ^/_matrix/key/v2/query {
include /config/nginx/include.d/synapse-proxy.conf;
proxy_pass http://generic_worker_lc;
}
location ~ ^/_matrix/federation/v1/hierarchy/ {
include /config/nginx/include.d/synapse-proxy.conf;
proxy_pass http://generic_worker_lc;
}
# Inbound federation transaction request
location ~ ^/_matrix/federation/v1/send/ {
include /config/nginx/include.d/synapse-proxy.conf;
proxy_pass http://generic_worker_ih;
}

View File

@ -0,0 +1,12 @@
location /_matrix/maubot/v1/logs {
proxy_pass http://maubot:29316;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
proxy_set_header X-Forwarded-For $remote_addr;
}
location /_matrix/maubot {
proxy_pass http://maubot:29316;
proxy_set_header X-Forwarded-For $remote_addr;
}

View File

@ -0,0 +1,19 @@
location ~ ^/_matrix/client/r0/rooms/([^/]*)/report/(.*)$ {
# Abuse reports should be sent to Mjölnir.
include /config/nginx/proxy.conf;
include /config/nginx/resolver.conf;
include /config/nginx/include.d/synapse-proxy.conf;
set $upstream_app mjolnir;
set $upstream_port 8081;
set $upstream_proto http;
# Add CORS, otherwise a browser will refuse this request.
add_header 'Access-Control-Allow-Credentials' 'true' always;
add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS' always;
add_header 'Access-Control-Allow-Headers' 'Authorization,Content-Type,Accept,Origin,User-Agent,DNT,Cache-Control,X-Mx-ReqToken,Keep-Alive,X-Requested-With,If-Modified-Since' always;
add_header 'Access-Control-Expose-Headers' 'Content-Length,Content-Range' always;
add_header 'Access-Control-Max-Age' 1728000;
# Alias the regexps, to ensure that they're not rewritten.
set $room_id $1;
set $event_id $2;
proxy_pass $upstream_proto://$upstream_app:$upstream_port/api/1/report/$room_id/$event_id;
}

View File

@ -0,0 +1,9 @@
location /register {
include /config/nginx/proxy.conf;
include /config/nginx/resolver.conf;
include /config/nginx/include.d/synapse-proxy.conf;
set $upstream_app matrix-registration;
set $upstream_port 5000;
set $upstream_proto http;
proxy_pass $upstream_proto://$upstream_app:$upstream_port;
}

View File

@ -0,0 +1,6 @@
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header X-Forwarded-Proto $scheme;
port_in_redirect off;

View File

@ -0,0 +1,9 @@
location /_synapse/admin/v1/event_reports {
include /config/nginx/proxy.conf;
include /config/nginx/resolver.conf;
set $upstream_app synapse;
set $upstream_port 8008;
set $upstream_proto http;
proxy_pass $upstream_proto://$upstream_app:$upstream_port;
}

View File

@ -0,0 +1,24 @@
upstream generic_worker_ih {
ip_hash;
server synapse-generic-worker-1:8080;
server synapse-generic-worker-2:8081;
server sync1:8090;
server sync2:8091;
server sync3:8092;
}
upstream generic_worker_lc {
least_conn;
server synapse-generic-worker-1:8080;
server synapse-generic-worker-2:8081;
server sync1:8090;
server sync2:8091;
server sync3:8092;
}
upstream sync_worker {
ip_hash;
server sync1:8090;
server sync2:8091;
server sync3:8092;
}

View File

@ -0,0 +1,17 @@
server {
listen 443 ssl http2;
listen [::]:443 ssl http2;
server_name adminer.*;
include /config/nginx/ssl.conf;
location / {
include /config/nginx/proxy.conf;
include /config/nginx/resolver.conf;
set $upstream_app adminer;
set $upstream_port 8080;
set $upstream_proto http;
proxy_pass $upstream_proto://$upstream_app:$upstream_port;
}
}

View File

@ -0,0 +1,61 @@
#Workers
include /config/nginx/include.d/upstream_workers.conf;
server {
listen 443 ssl http2;
listen [::]:443 ssl http2;
listen 8448 ssl http2 default_server;
listen [::]:8448 ssl http2 default_server;
server_name matrix.*;
include /config/nginx/ssl.conf;
client_max_body_size 1024M;
# Well-Known
location /.well-known/matrix/server {
return 200 '{"m.server": "matrix.example.tld:443"}';
}
# Mjolnir Reports
include /config/nginx/include.d/mjolnir-reports.conf;
# Maubot
include /config/nginx/include.d/maubot.conf;
# Synapse Reports
include /config/nginx/include.d/synapse-reports.conf;
# Federation_Worker
include /config/nginx/include.d/federation_worker.conf;
# Encryption_Worker
include /config/nginx/include.d/encryption_worker.conf;
# Client_Worker
include /config/nginx/include.d/client_worker.conf;
location ~ ^(/_matrix|/_synapse/client) {
include /config/nginx/proxy.conf;
include /config/nginx/resolver.conf;
include /config/nginx/include.d/synapse-proxy.conf;
set $upstream_app synapse;
set $upstream_port 8008;
set $upstream_proto http;
proxy_pass $upstream_proto://$upstream_app:$upstream_port;
}
# Register
include /config/nginx/include.d/register.conf;
location / {
include /config/nginx/proxy.conf;
include /config/nginx/resolver.conf;
include /config/nginx/include.d/synapse-proxy.conf;
set $upstream_app synapse;
set $upstream_port 8008;
set $upstream_proto http;
proxy_pass $upstream_proto://$upstream_app:$upstream_port;
}
}

View File

@ -0,0 +1,17 @@
server {
listen 443 ssl http2;
listen [::]:443 ssl http2;
server_name maubot.*;
include /config/nginx/ssl.conf;
location / {
include /config/nginx/proxy.conf;
include /config/nginx/resolver.conf;
set $upstream_app maubot;
set $upstream_port 29316;
set $upstream_proto http;
proxy_pass $upstream_proto://$upstream_app:$upstream_port;
}
}