Commit Graph

69 Commits

Author SHA1 Message Date
David Robertson
4e13743738
Poetry: select olddeps using poetry (#12407)
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2022-04-12 10:50:11 +00:00
Richard van der Hoff
5f72ea1bde
Move complement setup stuff into the Synapse repo (#12404)
Fixes matrix-org/complement#330 (or it will, once we remove the old files).

It's not quite a lift-and-shift: I've also taken the opportunity to get rid of the custom CA that we used to use to sign the TLS certs, which has been superceded by the CA exposed by Complement.
2022-04-11 11:39:28 +01:00
David Robertson
98ec375b26
CI: Fix the export-data job to run under poetry (#12418)
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2022-04-08 19:18:54 +00:00
David Robertson
7851a2c62f
CI: Get the portdb job to run under poetry (#12412) 2022-04-08 12:00:05 +01:00
Andrew Morgan
163fd686b5
Update mdbook to 0.4.17 (#12339)
Update mdbook (the tool used to render the documentation website) to version 0.4.17.
2022-04-05 17:21:58 +01:00
David Robertson
9a3f1f5383
Nuke the tox packaging job (#12334) 2022-03-31 11:01:26 +01:00
Andrew Morgan
19a1d6a42a
Remove references to "msc2403" (#12165) 2022-03-30 11:15:47 +00:00
kegsay
8b7b371ff6
Re-enable test concurrency on Complement (#12283) 2022-03-24 15:59:06 -04:00
reivilibre
bfa7d6b035
Fix CI not attaching source distributions and wheels to the GitHub releases. (#12131) 2022-03-08 15:11:50 +00:00
Richard van der Hoff
0211f18d65
Switch the tests-done job to an Action (#12161)
I've factored it out for easier use in other workflows.
2022-03-07 12:24:06 +00:00
David Robertson
36071d39f7
Changelog (#12153) 2022-03-04 12:01:51 +00:00
Eric Eastwood
a511a890d7
Enable MSC2716 Complement tests in Synapse (#12145)
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
2022-03-03 11:19:20 +00:00
Eric Eastwood
31b125ccec
Enable MSC3030 Complement tests in Synapse (#12144)
The Complement tests for MSC3030 are now merged, https://github.com/matrix-org/complement/pull/178

Synapse implmentation: https://github.com/matrix-org/synapse/pull/9445
2022-03-03 11:45:23 +01:00
David Robertson
1fbe0316a9
Add suffices to scripts in scripts-dev (#12137)
* Rename scripts-dev to have suffices

* Update references to `scripts-dev`

* Changelog

* These scripts don't pass mypy
2022-03-02 18:00:26 +00:00
David Robertson
5f62a094de
Detox, part 1 of N (#12119)
* Don't use `tox` for `check-sampleconfig`
* Don't use `tox` for check-newsfragment
2022-03-01 19:47:02 +00:00
David Robertson
af2c1e3d2a
Tidy the building of sdists and wheels (#12051)
* Don't build distribution pkgs in tests.yml
* Run `release-artifacts` on release branches
* Use backend-meta workflow for packaging
2022-02-22 11:33:37 +00:00
Richard van der Hoff
bb98c593a5
Prepare for rename of default complement branch (#11971)
use `HEAD` rather than hardcoding `master`
2022-02-11 14:43:34 +00:00
kegsay
a3865ed525
Run Complement tests sequentially (#11910)
Since #11811 there has been general Complement flakiness around networking.
It seems like tests are hitting the wrong containers. In an effort to diagnose
the cause of this, as well as reduce its impact on this project, set the
parallelsim to 1 (no parallelism) when running tests.

If this fixes the flakiness then this indicates the cause and I can diagnose
this further. If this doesn't fix the flakiness then that implies some kind
of test pollution which also helps to diagnose this further.
2022-02-04 13:04:57 +00:00
kegsay
0938f32e93
CI: run Complement on the VM, not inside Docker (#11811)
* CI: run Complement on the VM, not inside Docker

This requires https://github.com/matrix-org/complement/pull/289

We now run Complement on the VM instead of inside a Docker container.
This is to allow Complement to bind to any high-numbered port when it
starts up its own federation servers. We want to do this to allow for
more concurrency when running complement tests. Previously, Complement
only ever bound to `:8448` when running its own federation server. This
prevented multiple federation tests running at the same time as they would
fight each other on the port. This did however allow Complement to run
in Docker, as the host could just port forward `:8448` to allow homeserver
containers to communicate to Complement. Now that we are using random
ports however, we cannot use Docker to run Complement. This ends up
being a good thing because:
 - Running Complement tests locally is closer to how they run in CI.
 - Allows the `CI` env var to be removed in Complement.
 - Slightly speeds up runs as we don't need to pull down the Complement
   image prior to running tests. This assumes GHA caches actions sensibly.

* Changelog

* Full stop

* Update .github/workflows/tests.yml

Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>

* Review comments

* Update .github/workflows/tests.yml

Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>

Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2022-01-25 15:05:22 +00:00
Richard van der Hoff
2d327d25bf
Skip the initial amd64-only Docker build (#11810)
PyNaCl's recent 1.5.0 release on PyPi includes arm64 wheels, which means our
arm64 docker images now build in a sensible amount of time, so we can skip the
amd64-only build.
2022-01-24 18:31:23 +00:00
Shay
9006ee36d1
Drop support for and remove references to EOL Python 3.6 (#11683)
* remove reference in comments to python3.6

* upgrade tox python env in script

* bump python version in example for completeness

* upgrade python version requirement in setup doc

* upgrade necessary python version in __init__.py

* upgrade python version in setup.py

* newsfragment

* drops refs to bionic and replace with focal

* bump refs to postgres 9.6 to 10

* fix hanging ci

* try installing tzdata first

* revert change made in b979f336

* ignore new random mypy error while debugging other error

* fix lint error for temporary workaround

* revert change to install list

* try passing env var

* export debian frontend var?

* move line and add comment

* bump pillow dependency

* bump lxml depenency

* install libjpeg-dev for pillow

* bump automat version to one compatible with py3.8

* add libwebp for pillow

* bump twisted trunk python version

* change suffix of newsfragment

* remove redundant python 3.7 checks

* lint
2022-01-21 14:23:26 -08:00
Richard van der Hoff
d41c4654db
Use buildkit's cache feature to speed up docker builds (#11691)
Having spent much of the last week attempting to run complement tests from somewhere with damp string instead of internet... something had to be done.
2022-01-12 10:37:57 +00:00
kegsay
3be63654e4
Prettier complement logs (#11707)
* Prettier complement logs

* Changelog
2022-01-10 11:46:40 +00:00
Shay
87da37374a
Drop EOL python 3.6 from CI (#11595)
* remove python 3.6 and postgres 9.6 from github workflow

* remove python 3.6 env from tox

* newsfragment

* correct postgres version

* add py310 to tox env list
2021-12-21 11:37:04 -08:00
Patrick Cloke
a4521ce0a8
Support the stable /hierarchy endpoint from MSC2946 (#11329)
This also makes additional updates where the implementation
had drifted from the approved MSC.

Unstable endpoints will be removed at a later data.
2021-11-29 14:32:20 -05:00
Patrick Cloke
9d1971a5c4
Return the stable event field from /send_join per MSC3083. (#11413)
This does not remove the unstable field and still parses both.
Handling of the unstable field will need to be removed in the
future.
2021-11-29 15:43:20 +00:00
Richard van der Hoff
e2e9bea1ce
Publish a develop docker image (#11380)
I'd find it helpful to have a docker image corresponding to current develop,
without having to build my own.
2021-11-19 10:56:59 +00:00
Hillery Shay
7d70582eb0
Fix broken export-data admin command and add a test for it to CI (#11078)
Fix broken export-data admin command and add a test for it to CI
2021-10-18 08:14:12 -07:00
Patrick Cloke
8711e15734
Always dump logs from trial during CI. (#11068)
Instead of only dumping them if trial passes.
2021-10-12 20:09:49 +00:00
Erik Johnston
0b4d5ce5e3
Fix CI to run the unit tests without optional deps (#11017)
This also turns off calculating code coverage, as we didn't use it and it was a lot of noise
2021-10-08 10:05:48 +01:00
David Robertson
6f6e956338
Run CI with Python 3.10 and Postgres 14 (#10992) 2021-10-05 12:43:04 +01:00
Patrick Cloke
ebd8baf61f
Clear our destination directories before copying files to GitHub pages. (#10869)
This should fix stale deleted files being still accessible.
2021-09-21 16:32:46 +00:00
David Robertson
f455b0e420
GHA: reintroduce an env var for $GITHUB_HEAD_REF (#10659)
This should ensure GHA runs synapse against the same-named sytest branch
2021-09-20 17:35:16 +01:00
Dan Callahan
2efc838f05
Avoid duplicate issues from Twisted trunk failures (#10672)
Setting `update_existing: true` in the `create-an-issue` GitHub Action
will avoid opening duplicate issues if an open issue already exists with
an identical title.

If no open issues match the title, then a new issue will be created.

This helps avoid spamming our issue tracker should there be a failure
when testing against Twisted's trunk.

This PR also pins the SHA of the `create-an-issue` action to mitigate
the risk of a malicious actor gaining access to JasonEtco's account.

See GitHub's page on security hardening third party actions for more:
https://docs.github.com/en/actions/learn-github-actions/security-hardening-for-github-actions#using-third-party-actions

Signed-off-by: Dan Callahan <danc@element.io>
2021-08-23 13:06:49 +00:00
Richard van der Hoff
4db65f911a
Run a nightly CI build against Twisted trunk. (#10651)
This creates a GHA workflow which runs at 8am every day, and runs mypy, trial and sytest against Twisted's current trunk. If any of the jobs fail, it opens an issue.
2021-08-23 11:12:45 +01:00
David Robertson
878528913d Remove buildkite-era comment 2021-08-12 11:48:36 +01:00
David Robertson
74fcd5aab9 portdb also uses coverage, so provide $TOP there 2021-08-12 10:41:01 +01:00
David Robertson
cb5976ebd7 set TOP in sytest containers 2021-08-11 20:08:48 +01:00
David Robertson
33ef86aa25 Rename ci to .ci 2021-08-11 19:59:57 +01:00
David Robertson
58e5da5aa0 Remove buildkite from portdb CI tests 2021-08-10 16:38:13 +01:00
David Robertson
c5988a8eb7 Remove unused BUILDKITE_BRANCH env var 2021-08-10 16:38:13 +01:00
David Robertson
3d67b8c82b Move sytest worker-blacklist to ci directory 2021-08-10 16:38:13 +01:00
David Robertson
03fb99a5c8 check-newsfragment: pass pr number explicitly
use PULL_REQUEST_NUMBER instead of BUILDKITE_PULL_REQUEST
remove the other user of BUILDKITE_PULL_REQUEST, namely merge_base_branch.sh
2021-08-10 16:38:13 +01:00
David Robertson
8da9e3cb69 Move test_old_deps.sh to new ci dir 2021-08-10 16:38:13 +01:00
Erik Johnston
e33f14e8d5
Don't fail CI when lint-newfile job was skipped (#10529) 2021-08-05 11:22:27 +01:00
Richard van der Hoff
a6ea32a798
Fix the tests-done github actions step, again (#10512) 2021-08-02 21:06:34 +01:00
Patrick Cloke
13944678c3
Use new go test running syntax for complement. (#10488)
Updates CI and the helper script t ensures all tests are run (in parallel).
2021-07-27 12:08:51 +00:00
Richard van der Hoff
f22252d4f9
Enable docker image caching for the deb build (#10431) 2021-07-26 11:36:01 +01:00
Erik Johnston
ab82fd6ed1 Merge branch 'release-v1.39' into develop 2021-07-23 09:19:24 +01:00
Richard van der Hoff
4c3fdfc808
Fix an error in the docker workflow (#10461) 2021-07-22 21:50:30 +01:00