Add the ability to set the log level using the SYNAPSE_TEST_LOG_LEVEL environment when using complement.sh. (#13152)

This commit is contained in:
reivilibre 2022-07-05 10:46:20 +01:00 committed by GitHub
parent e514495465
commit 65e675504f
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
6 changed files with 44 additions and 6 deletions

1
changelog.d/13152.misc Normal file
View File

@ -0,0 +1 @@
Add the ability to set the log level using the `SYNAPSE_TEST_LOG_LEVEL` environment when using `complement.sh`.

View File

@ -67,6 +67,13 @@ The following environment variables are supported in `generate` mode:
* `UID`, `GID`: the user id and group id to use for creating the data * `UID`, `GID`: the user id and group id to use for creating the data
directories. If unset, and no user is set via `docker run --user`, defaults directories. If unset, and no user is set via `docker run --user`, defaults
to `991`, `991`. to `991`, `991`.
* `SYNAPSE_LOG_LEVEL`: the log level to use (one of `DEBUG`, `INFO`, `WARNING` or `ERROR`).
Defaults to `INFO`.
* `SYNAPSE_LOG_SENSITIVE`: if set and the log level is set to `DEBUG`, Synapse
will log sensitive information such as access tokens.
This should not be needed unless you are a developer attempting to debug something
particularly tricky.
## Postgres ## Postgres

View File

@ -49,11 +49,17 @@ handlers:
class: logging.StreamHandler class: logging.StreamHandler
formatter: precise formatter: precise
{% if not SYNAPSE_LOG_SENSITIVE %}
{#
If SYNAPSE_LOG_SENSITIVE is unset, then override synapse.storage.SQL to INFO
so that DEBUG entries (containing sensitive information) are not emitted.
#}
loggers: loggers:
synapse.storage.SQL: synapse.storage.SQL:
# beware: increasing this to DEBUG will make synapse log sensitive # beware: increasing this to DEBUG will make synapse log sensitive
# information such as access tokens. # information such as access tokens.
level: INFO level: INFO
{% endif %}
root: root:
level: {{ SYNAPSE_LOG_LEVEL or "INFO" }} level: {{ SYNAPSE_LOG_LEVEL or "INFO" }}

View File

@ -29,6 +29,10 @@
# * SYNAPSE_USE_EXPERIMENTAL_FORKING_LAUNCHER: Whether to use the forking launcher, # * SYNAPSE_USE_EXPERIMENTAL_FORKING_LAUNCHER: Whether to use the forking launcher,
# only intended for usage in Complement at the moment. # only intended for usage in Complement at the moment.
# No stability guarantees are provided. # No stability guarantees are provided.
# * SYNAPSE_LOG_LEVEL: Set this to DEBUG, INFO, WARNING or ERROR to change the
# log level. INFO is the default.
# * SYNAPSE_LOG_SENSITIVE: If unset, SQL and SQL values won't be logged,
# regardless of the SYNAPSE_LOG_LEVEL setting.
# #
# NOTE: According to Complement's ENTRYPOINT expectations for a homeserver image (as defined # NOTE: According to Complement's ENTRYPOINT expectations for a homeserver image (as defined
# in the project's README), this script may be run multiple times, and functionality should # in the project's README), this script may be run multiple times, and functionality should
@ -38,7 +42,7 @@ import os
import subprocess import subprocess
import sys import sys
from pathlib import Path from pathlib import Path
from typing import Any, Dict, List, Mapping, MutableMapping, NoReturn, Set from typing import Any, Dict, List, Mapping, MutableMapping, NoReturn, Optional, Set
import yaml import yaml
from jinja2 import Environment, FileSystemLoader from jinja2 import Environment, FileSystemLoader
@ -552,13 +556,17 @@ def generate_worker_log_config(
Returns: the path to the generated file Returns: the path to the generated file
""" """
# Check whether we should write worker logs to disk, in addition to the console # Check whether we should write worker logs to disk, in addition to the console
extra_log_template_args = {} extra_log_template_args: Dict[str, Optional[str]] = {}
if environ.get("SYNAPSE_WORKERS_WRITE_LOGS_TO_DISK"): if environ.get("SYNAPSE_WORKERS_WRITE_LOGS_TO_DISK"):
extra_log_template_args["LOG_FILE_PATH"] = "{dir}/logs/{name}.log".format( extra_log_template_args["LOG_FILE_PATH"] = f"{data_dir}/logs/{worker_name}.log"
dir=data_dir, name=worker_name
) extra_log_template_args["SYNAPSE_LOG_LEVEL"] = environ.get("SYNAPSE_LOG_LEVEL")
extra_log_template_args["SYNAPSE_LOG_SENSITIVE"] = environ.get(
"SYNAPSE_LOG_SENSITIVE"
)
# Render and write the file # Render and write the file
log_config_filepath = "/conf/workers/{name}.log.config".format(name=worker_name) log_config_filepath = f"/conf/workers/{worker_name}.log.config"
convert( convert(
"/conf/log.config", "/conf/log.config",
log_config_filepath, log_config_filepath,

View File

@ -309,6 +309,10 @@ The above will run a monolithic (single-process) Synapse with SQLite as the data
- Passing `POSTGRES=1` as an environment variable to use the Postgres database instead. - Passing `POSTGRES=1` as an environment variable to use the Postgres database instead.
- Passing `WORKERS=1` as an environment variable to use a workerised setup instead. This option implies the use of Postgres. - Passing `WORKERS=1` as an environment variable to use a workerised setup instead. This option implies the use of Postgres.
To increase the log level for the tests, set `SYNAPSE_TEST_LOG_LEVEL`, e.g:
```sh
SYNAPSE_TEST_LOG_LEVEL=DEBUG COMPLEMENT_DIR=../complement ./scripts-dev/complement.sh -run TestImportHistoricalMessages
```
### Prettier formatting with `gotestfmt` ### Prettier formatting with `gotestfmt`

View File

@ -145,6 +145,18 @@ else
test_tags="$test_tags,faster_joins" test_tags="$test_tags,faster_joins"
fi fi
if [[ -n "$SYNAPSE_TEST_LOG_LEVEL" ]]; then
# Set the log level to what is desired
export PASS_SYNAPSE_LOG_LEVEL="$SYNAPSE_TEST_LOG_LEVEL"
# Allow logging sensitive things (currently SQL queries & parameters).
# (This won't have any effect if we're not logging at DEBUG level overall.)
# Since this is just a test suite, this is fine and won't reveal anyone's
# personal information
export PASS_SYNAPSE_LOG_SENSITIVE=1
fi
# Run the tests! # Run the tests!
echo "Images built; running complement" echo "Images built; running complement"
cd "$COMPLEMENT_DIR" cd "$COMPLEMENT_DIR"