Commit Graph

112 Commits

Author SHA1 Message Date
TC
b1ff6b643b Merge branch 'add_nightly_builds' into 'main'
Another tweak to GitLab secure storage

See merge request veilid/veilid!300
2024-07-19 20:50:48 +00:00
TC Johnson
ab582b8d3e
Another tweak to GitLab secure storage 2024-07-19 15:49:29 -05:00
TC
b776accb94 Merge branch 'add_nightly_builds' into 'main'
Further adventures in nightly release automation

See merge request veilid/veilid!297
2024-07-19 19:11:41 +00:00
TC Johnson
c9d2b354f7
Further adventures in nightly release automation
Pushing this up in order to end-to-end test automated nightly
releases.
 Please enter the commit message for your changes. Lines starting
2024-07-19 14:08:25 -05:00
TC
0d95aa0859 Merge branch 'add_nightly_builds' into 'main'
Fixed CICD missing params

See merge request veilid/veilid!295
2024-07-15 03:58:58 +00:00
TC Johnson
ae46f32187
Fixed CICD missing params 2024-07-14 22:58:26 -05:00
TC
d062d60bf5 Merge branch 'add_nightly_builds' into 'main'
Manually set nightly to true in Earthfile to test rest of process

See merge request veilid/veilid!293
2024-07-15 03:14:49 +00:00
TC Johnson
59a69e7839
Manually set nightly to true in Earthfile to test rest of process 2024-07-14 22:13:53 -05:00
TC
34c1bcc08c Merge branch 'add_nightly_builds' into 'main'
Found Earthly doc saying ENVAR should be in quotes

See merge request veilid/veilid!292
2024-07-15 02:15:43 +00:00
TC Johnson
579abedcab
Found Earthly doc saying ENVAR should be in quotes 2024-07-14 21:15:02 -05:00
TC
efe0d28cf1 Update .gitlab-ci.yml file 2024-07-14 23:19:53 +00:00
TC
bf6d21e1f1 Update .gitlab-ci.yml file 2024-07-14 23:17:29 +00:00
TC
e22d00b92d The CI yaml doesn't accept a bool as a custom variable. I've switched it to a string to pass linting but I think this is going to break stuff. 2024-07-14 22:43:24 +00:00
TC Johnson
fc7debba3f
Adding automatic nightly release builds
- Restructured linux repo directory structure to include Stable and Nightly releases
- Reworked Earthfile to pass IS_NIGHTLY boolean to package build scripts for use in
file names
- Added nightly pipeline to CICD config yaml. Pipeline is triggered by IS_NIGHTLY
variable
- Two distribution scripts now exist, one for Stable and one for Nightly
- Reworked RPM build spec files to create appropriate file names
- Reworked debian release generation scripts
- RPM repo directory structure is prepped for arm64 builds
- Switched from scp to rsync for moving the completed repo from ochestration
to repo server
- Created schedule in Gitlab UI to trigger nightly release at 2300 Central Time
2024-07-14 17:22:50 -05:00
Δ ǀ Ξ ȼ
33141776f3
Add support for no container repository (i.e. a new fork) 2024-05-22 23:07:57 +01:00
Δ ǀ Ξ ȼ
35b1c7f847
Rebuild container cache if Earthfile is newer than cache 2024-05-06 21:55:27 +01:00
TC
bcacaeb300 [ci dryrun] 2024-04-30 00:44:21 +00:00
TC
4242e6179d [ci dryrun] 2024-04-30 00:22:44 +00:00
TC
6a7afe9017 Uncommented crates.io section. 2024-04-30 00:04:30 +00:00
TC
c3035a0406 Uncommented crates.io section. 2024-04-30 00:02:59 +00:00
TC
59d261b899 [ci dryrun] 2024-04-29 22:34:52 +00:00
TC
058df24933 [ci dryrun] 2024-04-29 22:16:38 +00:00
TC
31347f3ce0 [ci dryrun] 2024-04-29 22:15:19 +00:00
TC Johnson
4d9787fb86
Fixes scp from build machines to orchestrator
Each machine needed a script this specified that machine's arch
and OS type. Also added a rule on the cache and test jobs to not
run if the dry run trigger is present. [ci dryrun]
2024-04-29 17:07:40 -05:00
TC Johnson
37dbe70e05
Testing dry run trigger
The trigger is [ci dryrun]
2024-04-29 16:26:13 -05:00
TC Johnson
bd4600576f
Switched to regex trigger
The previous trigger condition wasn't liked by Gitlab's
yaml parser. I'm switching to trying a commit message
based regex trigger. This message does not contain the
trigger and so should not fire the dry run.
2024-04-29 16:22:58 -05:00
TC Johnson
0a1a37f048
Replaced missing 'if:' on line 249. 2024-04-29 16:12:37 -05:00
TC Johnson
fa1ca77d35
Further dry run trigger refinements
The dry run should now be configured if changes to .gitlab-ci.yml,
Earthfile, contents of scripts/cicd, or contents of package arrive
as the result of a push OR merge but NOT when a tag is present. I
think that will prevent the dry run executing alongside the actual
release process when a new release tag is pushed but that also
contains changes to the above mentioned files. <fingers crossed emoji>
2024-04-29 16:03:38 -05:00
TC Johnson
c33f0f19da
Added Earthfile condition and RPM spec fix
Dry run will now trigger for Earthfile changes. I also reordered the
changelog in veilid-server.spec to descending to correct that error.

Commented out the crates.io publishing dry run. This branch does not
have access to the protected variables.
2024-04-29 15:26:03 -05:00
TC Johnson
038f4d2121
Added CICD dry run
This is the first attempt at setting up a dry run pipeline
to test changes to the CICD config without actually publishing
the compiled binaries and packages built by the process.

The dry run should be triggered by any changes to .gitlab-ci.yml
or changes to any of the scripts under scripts/cicd/.
2024-04-29 14:40:07 -05:00
TC Johnson
6c6be00feb
Calling bash explicitly
The paths work now, but the scripts are not set with execute permission.
I don't know if that permission will survive the transfer from my machine
across the git and runner stuff so I added an explicit call for bash instead.
2024-04-28 20:39:18 -05:00
TC Johnson
2821e4ee18
It was a typo in the script name
Was missing the r in ctrl then spread via copy paste. Fixed the typo and
reverted the paths to relative because that should work just fine.
2024-04-28 19:24:45 -05:00
TC Johnson
4644b7a401
Hardcoded runner IDs into script paths
The runner ID CI variable resolved to the runners' numeric ID whereas
the project path uses the runners' alpha-numeric formatted ID. Both
IDs show in Gitlab's settings->cicd->runners. For the time being, I've
hardcoded the alpha-numeric ID into the script paths in the CI config.
If we have to retate runners, we will need to update these entries.
2024-04-28 18:58:28 -05:00
TC Johnson
2a73946246
Modified path to CICD scripts in CI config
I thought CICD's working directory was in the project root but the release
failed to find the scripts. I've changed the script executions to absolute
paths. There's a directory is named for the runner's ID, is different on
each machine, and changes if the runner is replaced. There's a variable
that should overcome this, CI_RUNNER_ID, which I've used in the asbolute
paths. Fingers crossed, let's try it again.
2024-04-28 18:21:39 -05:00
TC Johnson
06f344ebb4
Migrating CICD scripts to Veilid repo
Copied CICD scripts into the repository so that the community can make
contributions to the build system. Wrote a brief description of the
build and distribute process. Modified the CICD config to use the repo
hosted scripts. [ci skip]
2024-04-21 13:37:45 -05:00
TC
feeafe290a moved crates and pypi publishing to build_packages stage 2024-04-05 00:53:28 +00:00
TC
e3f396f93d Correcting distribute stage order of execution 2024-04-05 00:46:47 +00:00
TC
30ff93e8df Moved crate and python publishing to a dev machine. The orchestration machine doesn't have the resources for the cargo build steps. 2024-04-04 23:22:10 +00:00
TC
7431a5380a Added crates.io and Pypi release steps to build_repositories job. Maybe it works, maybe it doesn't. We'll find out at next release. 2024-03-03 21:19:21 +00:00
Salvatore Testa
dca57abe16
Use $CI_REGISTRY_IMAGE for the registry path
GitLab will return an error if you have an upper case letter at the
start of your username.

```
invalid reference format: repository name must be lowercase
```

The built-in `$CI_REGISTRY_IMAGE` variable does the right thing.

Closes https://gitlab.com/veilid/veilid/-/issues/352
2023-12-25 14:02:40 -08:00
TC
5b5a568f2f Removed the manual trigger from tests -- YOLO 2023-11-03 01:48:07 +00:00
Δ ǀ Ξ ȼ
d38f3cb42f
Refactored Earthfile to improve caching
Targets have been parallelized so that initial push of the container cache should build the whole build a little quicker, plus the container should now use the cache for more of the build and so speed up normal builds to just the compilation and test of the code that has changed

The `build_cache` target now builds a `build-cache:latest` container that is stored in the GitLab project Container Registry, and then used (if it exists) by the `test_build` target.  The `build_cache` task runs under 3 conditions, 1. the container does not exist, 2. if scheduled, 3. if run manually from the Pipelines page in the GitLab interface.

It is recommended that the build is set up to run on a weekly schedule via the `Pipeline schedules` page in GitLab with the schedule of `0 2 * * 6`.
2023-10-31 11:17:13 +00:00
TC
b931a5b87e
Switch unit tests back to manual execution 2023-09-04 14:41:34 +00:00
TC Johnson
6f191cb8a8
CI config and Changelog tweaks
Fixed a few typos before releasing v0.2.1
2023-09-03 19:43:46 -05:00
TC Johnson
d0850e95fb
Refine MR unit tests and contribution guide
Changed the name of the unit tests CI job to accuratly reflect
that both AMD64 and ARM64 tests are executed.

Modified the contribution guide to specify that contributors
should work inside their own fork of the project.
2023-08-28 18:58:46 -05:00
Δ ǀ Ξ ȼ
ab51f68c4d Earthly Build Caching in GitLab Container Registry 2023-08-27 17:17:29 +00:00
TC
b77beeb3be Tests passed. This commit is a clean up of .gitlab-ci.yml 2023-08-22 21:49:55 +00:00
TC
9429d3de6e Update .gitlab-ci.yml file 2023-08-22 19:59:59 +00:00
TC
5d714dcf58 Update .gitlab-ci.yml file 2023-08-22 19:40:31 +00:00
TC Johnson
c525a757fd
Move tests off of Digital Ocean
Experimenting with using GitLab SaaS runners to do test CI stages.
2023-08-22 14:36:33 -05:00