Allow enabling the asyncio reactor in complement (#14858)

Signed-off-by: Jason Little realtyem@gmail.com
This commit is contained in:
realtyem 2023-02-01 17:42:45 -06:00 committed by GitHub
parent 1d3a54aa30
commit 58214dbb9b
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
6 changed files with 42 additions and 4 deletions

View File

@ -541,8 +541,11 @@ jobs:
- run: | - run: |
set -o pipefail set -o pipefail
POSTGRES=${{ (matrix.database == 'Postgres') && 1 || '' }} WORKERS=${{ (matrix.arrangement == 'workers') && 1 || '' }} COMPLEMENT_DIR=`pwd`/complement synapse/scripts-dev/complement.sh -json 2>&1 | synapse/.ci/scripts/gotestfmt COMPLEMENT_DIR=`pwd`/complement synapse/scripts-dev/complement.sh -json 2>&1 | synapse/.ci/scripts/gotestfmt
shell: bash shell: bash
env:
POSTGRES: ${{ (matrix.database == 'Postgres') && 1 || '' }}
WORKERS: ${{ (matrix.arrangement == 'workers') && 1 || '' }}
name: Run Complement Tests name: Run Complement Tests
cargo-test: cargo-test:

1
changelog.d/14858.misc Normal file
View File

@ -0,0 +1 @@
Run the integration test suites with the asyncio reactor enabled in CI.

View File

@ -6,7 +6,7 @@ set -e
echo "Complement Synapse launcher" echo "Complement Synapse launcher"
echo " Args: $@" echo " Args: $@"
echo " Env: SYNAPSE_COMPLEMENT_DATABASE=$SYNAPSE_COMPLEMENT_DATABASE SYNAPSE_COMPLEMENT_USE_WORKERS=$SYNAPSE_COMPLEMENT_USE_WORKERS" echo " Env: SYNAPSE_COMPLEMENT_DATABASE=$SYNAPSE_COMPLEMENT_DATABASE SYNAPSE_COMPLEMENT_USE_WORKERS=$SYNAPSE_COMPLEMENT_USE_WORKERS SYNAPSE_COMPLEMENT_USE_ASYNCIO_REACTOR=$SYNAPSE_COMPLEMENT_USE_ASYNCIO_REACTOR"
function log { function log {
d=$(date +"%Y-%m-%d %H:%M:%S,%3N") d=$(date +"%Y-%m-%d %H:%M:%S,%3N")
@ -76,6 +76,17 @@ else
fi fi
if [[ -n "$SYNAPSE_COMPLEMENT_USE_ASYNCIO_REACTOR" ]]; then
if [[ -n "$SYNAPSE_USE_EXPERIMENTAL_FORKING_LAUNCHER" ]]; then
export SYNAPSE_COMPLEMENT_FORKING_LAUNCHER_ASYNC_IO_REACTOR="1"
else
export SYNAPSE_ASYNC_IO_REACTOR="1"
fi
else
export SYNAPSE_ASYNC_IO_REACTOR="0"
fi
# Add Complement's appservice registration directory, if there is one # Add Complement's appservice registration directory, if there is one
# (It can be absent when there are no application services in this test!) # (It can be absent when there are no application services in this test!)
if [ -d /complement/appservice ]; then if [ -d /complement/appservice ]; then

View File

@ -332,6 +332,7 @@ The above will run a monolithic (single-process) Synapse with SQLite as the data
[here](https://github.com/matrix-org/synapse/blob/develop/docker/configure_workers_and_start.py#L54). [here](https://github.com/matrix-org/synapse/blob/develop/docker/configure_workers_and_start.py#L54).
A safe example would be `WORKER_TYPES="federation_inbound, federation_sender, synchrotron"`. A safe example would be `WORKER_TYPES="federation_inbound, federation_sender, synchrotron"`.
See the [worker documentation](../workers.md) for additional information on workers. See the [worker documentation](../workers.md) for additional information on workers.
- Passing `ASYNCIO_REACTOR=1` as an environment variable to use the Twisted asyncio reactor instead of the default one.
To increase the log level for the tests, set `SYNAPSE_TEST_LOG_LEVEL`, e.g: To increase the log level for the tests, set `SYNAPSE_TEST_LOG_LEVEL`, e.g:
```sh ```sh

View File

@ -228,6 +228,11 @@ else
test_tags="$test_tags,msc2716" test_tags="$test_tags,msc2716"
fi fi
if [[ -n "$ASYNCIO_REACTOR" ]]; then
# Enable the Twisted asyncio reactor
export PASS_SYNAPSE_COMPLEMENT_USE_ASYNCIO_REACTOR=true
fi
if [[ -n "$SYNAPSE_TEST_LOG_LEVEL" ]]; then if [[ -n "$SYNAPSE_TEST_LOG_LEVEL" ]]; then
# Set the log level to what is desired # Set the log level to what is desired

View File

@ -110,6 +110,8 @@ def _worker_entrypoint(
and then kick off the worker's main() function. and then kick off the worker's main() function.
""" """
from synapse.util.stringutils import strtobool
sys.argv = args sys.argv = args
# reset the custom signal handlers that we installed, so that the children start # reset the custom signal handlers that we installed, so that the children start
@ -117,9 +119,24 @@ def _worker_entrypoint(
for sig, handler in _original_signal_handlers.items(): for sig, handler in _original_signal_handlers.items():
signal.signal(sig, handler) signal.signal(sig, handler)
# Install the asyncio reactor if the
# SYNAPSE_COMPLEMENT_FORKING_LAUNCHER_ASYNC_IO_REACTOR is set to 1. The
# SYNAPSE_ASYNC_IO_REACTOR variable would be used, but then causes
# synapse/__init__.py to also try to install an asyncio reactor.
if strtobool(
os.environ.get("SYNAPSE_COMPLEMENT_FORKING_LAUNCHER_ASYNC_IO_REACTOR", "0")
):
import asyncio
from twisted.internet.asyncioreactor import AsyncioSelectorReactor
reactor = AsyncioSelectorReactor(asyncio.get_event_loop())
proxy_reactor._install_real_reactor(reactor)
else:
from twisted.internet.epollreactor import EPollReactor from twisted.internet.epollreactor import EPollReactor
proxy_reactor._install_real_reactor(EPollReactor()) proxy_reactor._install_real_reactor(EPollReactor())
func() func()