Commit Graph

220 Commits

Author SHA1 Message Date
Dirk Klimpel
ecbe0ddbe7
Add support for knocking to workers. () 2023-03-02 12:59:53 -05:00
David Robertson
c4f4dc35cd
Dockerfile-workers: spell out when config isn't generated ()
* Complement: Spell out when config isn't generated

* Changelog
2023-03-02 15:55:26 +00:00
reivilibre
682d31c702
Allow use of the /filter Client-Server APIs on workers. () 2023-02-28 16:37:19 +00:00
realtyem
58214dbb9b
Allow enabling the asyncio reactor in complement ()
Signed-off-by: Jason Little realtyem@gmail.com
2023-02-01 23:42:45 +00:00
reivilibre
22cc93afe3
Enable Faster Remote Room Joins against worker-mode Synapse. ()
* Enable Complement tests for Faster Remote Room Joins on worker-mode

* (dangerous) Add an override to allow Complement to use FRRJ under workers

* Newsfile

Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>

* Fix race where we didn't send out replication notification

* MORE HACKS

* Fix get_un_partial_stated_rooms_token to take instance_name

* Fix bad merge

* Remove warning

* Correctly advance un_partial_stated_room_stream

* Fix merge

* Add another notify_replication

* Fixups

* Create a separate ReplicationNotifier

* Fix test

* Fix portdb

* Create a separate ReplicationNotifier

* Fix test

* Fix portdb

* Fix presence test

* Newsfile

* Apply suggestions from code review

* Update changelog.d/14752.misc

Co-authored-by: Erik Johnston <erik@matrix.org>

* lint

Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>
Co-authored-by: Erik Johnston <erik@matrix.org>
2023-01-22 21:10:11 +00:00
katlol
cf18fea9e1
Dockerfile: Bump Python version from 3.9 to 3.11 ()
Closes https://github.com/matrix-org/synapse/issues/13234

Signed-off-by: Katia Esposito <1695469+katlol@users.noreply.github.com>

Signed-off-by: Katia Esposito <1695469+katlol@users.noreply.github.com>
2023-01-20 12:07:13 +00:00
Andrew Morgan
a7b54ca8d8
Implement MSC3930: polls push rules () 2023-01-19 12:47:10 +00:00
David Robertson
b88cfe6d41
Require poetry>=1.3.2 ()
* Upgrade to new lockfile format

Now requires poetry >= 1.2.2 to read and poetry >= 1.3.0 to write.

Cheat sheet:

```
poetry --version
poetry show > scratch/before
pipx upgrade poetry
poetry --version
poetry show > scratch/after
diff scratch{before,after} && echo "no change!"
```

* Use Poetry 1.3.2 when reading or writing lockfile

* Remove unneeded(?) poetry dep for cibuildwheel

* Update docs

* Remove redundant call to setup-python

* Remove outdated comments related to Poetry 1.x

* Remove outdated docs line

was fixed in 

* Minor improvements to poetry cheat sheet

* Invoke setup-python-poetry with explicit version

Not sure about this. It's hardcoding versions everywhere.

* Changelog

* Check the lockfile is version 2.0

Might one day incorporate other checks like 

* Typo fixes, thanks Sean

Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>

Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
2023-01-17 18:04:44 +00:00
David Robertson
5b3af1c7d0
Stabilise serving partial join responses ()
Serving partial join responses is no longer experimental. They will only be served under the stable identifier if the the undocumented config flag experimental.msc3706_enabled is set to true.

Synapse continues to request a partial join only if the undocumented config flag experimental.faster_joins is set to true; this setting remains present and unaffected.
2023-01-17 12:44:15 +00:00
Andrew Morgan
54cd90ea60
Implement MSC3890: Remotely silence local notifications () 2023-01-13 19:32:10 +00:00
Andrew Morgan
e787fb776c
Switch to our fork of dh-virtualenv for compatibility with Python 3.11 () 2023-01-04 16:26:29 +00:00
Andrew Morgan
c4456114e1
Add experimental support for MSC3391: deleting account data () 2023-01-01 03:40:46 +00:00
Mathieu Velten
eb2defc2f7
Add release note and update doc regarding ICU ()
Fixes .

Signed-off-by: Mathieu Velten <mathieuv@matrix.org>
2022-12-20 16:06:26 +00:00
Brendan Abolivier
2a3cd59dd0
Add optional ICU support for user search ()
Fixes 

This change uses ICU (International Components for Unicode) to improve boundary detection in user search.

This change also adds a new dependency on libicu-dev and pkg-config for the Debian packages, which are available in all supported distros.
2022-12-12 13:21:17 +01:00
reivilibre
be3a8a85e3
Add --editable flag to complement.sh which uses an editable install of Synapse for faster turn-around times whilst developing iteratively. ()
Co-authored-by: Mathieu Velten <mathieuv@matrix.org>
2022-12-07 15:45:31 +00:00
Mathieu Velten
22e91b8019
docker: remove useless cargo install with apt ()
Signed-off-by: Mathieu Velten <mathieuv@matrix.org>
2022-12-07 15:29:32 +01:00
Eric Eastwood
8f10c8b054
Move MSC3030 /timestamp_to_event endpoint to stable v1 location ()
Fix https://github.com/matrix-org/synapse/issues/14390

 - Client API: `/_matrix/client/unstable/org.matrix.msc3030/rooms/<roomID>/timestamp_to_event?ts=<timestamp>&dir=<direction>` -> `/_matrix/client/v1/rooms/<roomID>/timestamp_to_event?ts=<timestamp>&dir=<direction>`
 - Federation API: `/_matrix/federation/unstable/org.matrix.msc3030/timestamp_to_event/<roomID>?ts=<timestamp>&dir=<direction>` -> `/_matrix/federation/v1/timestamp_to_event/<roomID>?ts=<timestamp>&dir=<direction>`

Complement test changes: https://github.com/matrix-org/complement/pull/559
2022-11-28 15:54:18 -06:00
realtyem
c15e9a0edb
Remove need for worker_main_http_uri setting to use /keys/upload. () 2022-11-16 22:16:25 +00:00
realtyem
e9cbddc8e7
Modernize configure_workers_and_start.py bootstrapping script for Dockerfile-workers. () 2022-11-09 12:02:15 +00:00
realtyem
d85cba1aa0
Add all Stream Writer worker types to configure_workers_and_start.py ()
Co-authored-by: reivilibre <oliverw@matrix.org>
2022-11-08 13:14:00 +00:00
realtyem
69814eb282
Allow override for requesting specific worker types for Complement on command line. ()
* Expose getting SYNAPSE_WORKER_TYPES from external, allowing override of workers requested.

* Add WORKER_TYPES variable option to complement.sh script that passes requested workers into start_for_complement.sh entrypoint.

* Update docs to reflect this new ability.

* Changelog

* Don't rely on soft wrapping to format long strings

Good idea dklimpel. Thanks for catching that.

Co-authored-by: Dirk Klimpel <5740567+dklimpel@users.noreply.github.com>

* Small nits just noticed in docs.

* Fixup new line in docs.

Co-authored-by: Dirk Klimpel <5740567+dklimpel@users.noreply.github.com>
2022-11-08 12:34:09 +00:00
Patrick Cloke
59ca73006c
Enable testing MSC3874 in complement. () 2022-11-01 13:26:28 -04:00
Jonathan de Jong
e440f9674a
Enable URL previews in complement homeserver config. () 2022-10-18 09:52:23 -04:00
realtyem
6c5082f3e0
Flush stdout/err in Dockerfile-workers before replacing the current process ()
Also update `subprocess.check_output` to the slightly newer `subprocess.run`.

Signed-off-by: Jason Little <realtyem@gmail.com>
2022-10-18 11:56:20 +00:00
Erik Johnston
6fee2f49f3
Cache Rust build cache when building docker images () 2022-10-17 17:21:14 +00:00
realtyem
c75836fe77
Strip whitespace from worker types in Dockerfile-workers () 2022-10-14 19:38:04 +00:00
Patrick Cloke
bc2bd92b93 Merge remote-tracking branch 'origin/release-v1.69' into develop 2022-10-14 14:11:27 -04:00
realtyem
c7446906bd
Set LD_PRELOAD to load jemalloc in Dockerfile-workers. () 2022-10-14 13:29:49 +01:00
Patrick Cloke
c3e4edb4d6
Stabilize the threads API. ()
Stabilize the threads API (MSC3856) by supporting (only) the v1
path for the endpoint.

This also marks the API as safe for workers since it is a read-only
API.
2022-10-14 07:16:50 -04:00
Erik Johnston
29ee4b6698
Fix docker build OOMing in CI for arm64 builds ()
Co-authored-by: David Robertson <davidr@element.io>
2022-10-13 18:16:21 +00:00
Patrick Cloke
c604d2c218
Mark /relations endpoint as usable on workers. ()
Co-authored-by: Eric Eastwood <erice@element.io>
2022-10-12 06:46:13 -04:00
David Robertson
3f057e4c54
Use minimal Rust installation in docker images and CI () 2022-10-12 10:47:02 +01:00
Erik Johnston
d94bcbced3
Fix pinning Rust deps in docker images () 2022-10-11 10:53:34 +00:00
David Robertson
0c853e0970
Fix broken links to README () 2022-10-06 22:49:51 +00:00
Richard van der Hoff
51436c8dd5
Complement test image: capture logs from nginx ()
Have nginx send its logs to stderr/out, so that we can debug
https://github.com/matrix-org/synapse/issues/13334.
2022-10-05 17:37:34 +01:00
Mathieu Velten
42dd992bb7
Dockerfile for tests: align comments with current behavior ()
Signed-off-by: Mathieu Velten <mathieuv@matrix.org>
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2022-09-27 15:55:43 +02:00
Mathieu Velten
d5292b8017
Fix Docker build when Rust .so has been build locally first ()
Signed-off-by: Mathieu Velten <mathieuv@matrix.org>
2022-09-16 15:38:54 +00:00
Mathieu Velten
384dca53d6
complement: init postgres DB directly inside the target image ()
Doing so in the base postgres image doesn't work with buildah because
changes in a declared VOLUME in the Dockerfile is supposed to be
discarded, cf https://docs.docker.com/engine/reference/builder/#volume

Signed-off-by: Mathieu Velten <mathieuv@matrix.org>
2022-09-16 17:12:45 +02:00
Erik Johnston
c9b7e97355
Add a stub Rust crate () 2022-09-06 19:01:37 +01:00
Erik Johnston
18e4092801
Bump docker image to use stable poetry version () 2022-09-01 13:46:47 +01:00
reivilibre
2c5e2ae898
Document that the DOCKER_BUILDKIT=1 flag is needed to build the docker image. () 2022-08-12 14:25:47 +01:00
Richard van der Hoff
166fafdf8d
synapse-workers docker: copy nginx and redis in from base images ()
Part of my continuing quest to make the docker images build quicker: copy nginx and redis in from base docker images, rather than apt installing each time.
2022-08-04 12:59:27 +01:00
Jasper Spaans
503a95804e
Install cryptography build dependencies in requirements image. () 2022-08-03 11:16:32 +01:00
reivilibre
05aeeb3a80
Enable Complement CI tests in the 'latest deps' test run. ()
Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
2022-08-01 10:55:31 +00:00
Brendan Abolivier
10e4093839
Call out buildkit is required when building test docker images ()
Co-authored-by: David Robertson <davidr@element.io>
2022-07-21 14:29:58 +02:00
David Robertson
b977867358
Rate limit joins per-room () 2022-07-19 11:45:17 +00:00
David Robertson
e9ce4d089b
Use and recommend poetry 1.1.14, up from 1.1.12 () 2022-07-15 16:18:47 +01:00
Richard van der Hoff
512486bbeb
Docker: copy postgres from base image ()
When building the docker images for complement testing, copy a preinstalled
complement over from a base image, rather than apt installing it. This avoids
network traffic and is much faster.
2022-07-15 11:13:40 +01:00
reivilibre
65e675504f
Add the ability to set the log level using the SYNAPSE_TEST_LOG_LEVEL environment when using complement.sh. () 2022-07-05 09:46:20 +00:00
Till
fa10468eb4
[Complement] Allow device_name lookup over federation () 2022-07-04 12:34:50 +00:00