Commit Graph

196 Commits

Author SHA1 Message Date
Daniel Weiße
c11631ec11
logging: reduce grpc logging noise (#3329)
* Normalize gRPC logs to print at warn level only
* Fix grpcLogger level enablement

---------

Signed-off-by: Daniel Weiße <dw@edgeless.systems>
2024-08-29 10:44:22 +02:00
renovate[bot]
fe96153507
deps: update bazel (modules) (#3304)
* deps: update bazel (modules)
* Set std=c++14
* deps: tidy all modules

---------

Signed-off-by: Daniel Weiße <dw@edgeless.systems>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Daniel Weiße <dw@edgeless.systems>
Co-authored-by: edgelessci <edgelessci@users.noreply.github.com>
Co-authored-by: Markus Rudy <mr@edgeless.systems>
2024-08-09 11:00:22 +02:00
Markus Rudy
ab94456455
debugd: daemonize logstash containers (#3216)
* debugd: daemonize logstash containers

* fixup! debugd: daemonize logstash containers

remove in-process logging
2024-06-28 14:56:46 +02:00
Markus Rudy
8aed4bb0fe debugd: provide filebeat with systemd libs (#3205) 2024-06-27 19:35:02 +02:00
Markus Rudy
3b64e654d1 debugd: use runc as podman runtime (#3205) 2024-06-27 19:35:02 +02:00
renovate[bot]
e71819eb62
deps: update Go dependencies (#3185)
* deps: update Go dependencies
* deps: tidy all modules
* Replace deprecated `grpc.DialContext` with `grpc.NewClient`

---------

Signed-off-by: Daniel Weiße <dw@edgeless.systems>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: edgelessci <edgelessci@users.noreply.github.com>
Co-authored-by: Daniel Weiße <dw@edgeless.systems>
2024-06-21 10:05:57 +02:00
Daniel Weiße
c1e4da3ea1
debugd: reset unit failed status before restarting (#3183)
Signed-off-by: Daniel Weiße <dw@edgeless.systems>
2024-06-20 11:22:37 +02:00
Daniel Weiße
c682558940
debugd: return systemd logs if restarting a unit fails (#3159)
* Wait until logcollection containers are ready
* Add systemd unit journal entries to error message

---------

Signed-off-by: Daniel Weiße <dw@edgeless.systems>
2024-06-12 08:01:54 +02:00
Daniel Weiße
32588339f3
cdbg: better error reporting on deploy failure (#3155)
* Don't report `UPLOAD_FILES_START_FAILED` as unknown error
* Return error to `cdbg` if present

---------

Signed-off-by: Daniel Weiße <dw@edgeless.systems>
2024-06-07 15:46:37 +02:00
Daniel Weiße
9c2aef88ba
debugd: remove --debug flag from bootstrapper service created by debugd (#3150)
Signed-off-by: Daniel Weiße <dw@edgeless.systems>
2024-06-06 10:25:51 +02:00
Malte Poll
2c8a16294e bazel: migrate rules_proto to bzlmod 2024-05-23 09:48:04 +02:00
Malte Poll
d960121cba bazel: update BUILD files for rules_go bzlmod migration 2024-05-23 09:48:04 +02:00
Moritz Sanft
5c3a7a5580
image: update to Fedora 40 (#3104)
* deps: upgrade OS to Fedora 40

* image: measure uki sections uname and sbat for systemd >= 254

* deps: update mainline kernel for Fedora 40

* image: update kernel to 6.6.30

* image: update upload docs

---------

Co-authored-by: Malte Poll <1780588+malt3@users.noreply.github.com>
2024-05-16 09:10:09 +02:00
renovate[bot]
10a2bca9f9 deps: update fedora:38 Docker digest to b9ff6f2 2024-05-13 09:37:54 +02:00
Daniel Weiße
9def35ed06
deps: update all Go dependencies (#3071)
* Upgrade Go dependencies

Signed-off-by: Daniel Weiße <dw@edgeless.systems>

* Group Go dependency upgrades

Signed-off-by: Daniel Weiße <dw@edgeless.systems>

* Remove usage of deprecated docker types

Signed-off-by: Daniel Weiße <dw@edgeless.systems>

* Fix usage of invalid validation tags

Signed-off-by: Daniel Weiße <dw@edgeless.systems>

* Regenerate bazel files

Signed-off-by: Daniel Weiße <dw@edgeless.systems>

* Keep github.com/bazelbuild/buildtools at old version to not break other dependencies

Signed-off-by: Daniel Weiße <dw@edgeless.systems>

---------

Signed-off-by: Daniel Weiße <dw@edgeless.systems>
2024-05-08 17:31:47 +02:00
Malte Poll
281c7c320c deps: update protobuf to v1.33.0 2024-03-06 14:50:01 +01:00
Malte Poll
522f2858c6 proto: update generated protobuf sources 2024-02-21 18:40:16 +01:00
Malte Poll
71c8a27539 deps: replace use of deprecated module azsecrets
github.com/Azure/azure-sdk-for-go/sdk/keyvault/azsecrets -> github.com/Azure/azure-sdk-for-go/sdk/security/keyvault/azsecrets
2024-02-21 18:40:16 +01:00
Markus Rudy
fe85877679
debugd: enable debug logging for systemd units (#2923) 2024-02-20 14:44:14 +01:00
miampf
54cce77bab
deps: convert zap to slog (#2825) 2024-02-08 14:20:01 +00:00
Daniel Weiße
65d28f913f Allow starting e2e tests based on attestation variant instead of csp
Signed-off-by: Daniel Weiße <dw@edgeless.systems>
2024-01-26 17:06:28 +01:00
Malte Poll
3a5753045e goleak: ignore rules_go SIGTERM handler
rules_go added a SIGTERM handler that has a goroutine that survives the scope of the goleak check.
Currently, the best known workaround is to ignore this goroutine.

https://github.com/uber-go/goleak/issues/119
https://github.com/bazelbuild/rules_go/pull/3749
https://github.com/bazelbuild/rules_go/pull/3827#issuecomment-1894002120
2024-01-22 13:11:58 +01:00
Markus Rudy
711dff37a4
ci: fetch OpenSearch password from e2e test project (#2818) 2024-01-15 13:25:15 +01:00
Markus Rudy
c23aef344d
ci: don't export e2e metrics to OpenSearch (#2794)
* ci: don't export e2e metrics to OpenSearch
* debugd: don't export metrics
2024-01-05 10:15:53 +01:00
renovate[bot]
ced03202a9
deps: update fedora:38 Docker digest to 3f01c8f (#2749)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2023-12-20 18:07:20 +01:00
Daniel Weiße
724ee44466
ci: Terraform provider e2e tests (#2712)
* Refactor selfManagedInfra input to clusterCreation in e2e tests
* Run e2e test using terraform provider
* Allow insecure measurement fetching in Terraform provider
* Run Terraform provider test instead of module test in weekly runs

---------

Signed-off-by: Daniel Weiße <dw@edgeless.systems>
2023-12-15 10:37:29 +01:00
Malte Poll
45879c7360 bazel: use pure Go platform where possible
Before, we specified that the platform has glibc 2.23 under /usr/lib.
This is technically not important for statically linked Go binaries.
2023-12-01 09:35:33 +01:00
Markus Rudy
69c64d6c69
docs: improve developer documentation for folks new to the codebase
* Correct example invocation of aws cli

* Add warning to Helm Intellisense recommendation

* Link code conventions in PR guidelines

* Tighten debugd README

* cmake is not used for building debugd anymore, remove references to it

* make the debug-cluster workflow the authoritative source for cdbg usage - don't replicate the same instructions in different places

* Document that Bazel eats a lot of RAM
2023-11-17 19:16:52 +01:00
Moritz Sanft
9a282df846
ci: separate logs and metrics indices (#2544)
* separate logs and metrics indices

* tidy
2023-10-31 12:09:27 +01:00
Moritz Sanft
0c03076181
add missing field (#2542) 2023-10-31 10:12:49 +01:00
Moritz Sanft
9a2587cfd8
increase period for Metricbeat (#2540) 2023-10-30 15:50:27 +01:00
Daniel Weiße
f4bfbe3564
docs: refer to apply command instead of init or upgrade apply (#2487)
Signed-off-by: Daniel Weiße <dw@edgeless.systems>
2023-10-27 08:30:59 +02:00
renovate[bot]
b2f3f72488
deps: update fedora:38 Docker digest to 8285246 (#2467)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2023-10-17 10:47:10 +02:00
Moritz Sanft
005e865a13
cli: use state file on init and upgrade (#2395)
* [wip] use state file in CLI

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

tidy

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* use state file in CLI

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

take clusterConfig from IDFile for compat

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

various fixes

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

wip

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* add GCP-specific values in Helm loader test

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* remove unnecessary pointer

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* write ClusterValues in one step

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* move stub to test file

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* remove mention of id-file

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* move output to `migrateTerraform`

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* unconditional assignments converting from idFile

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* move require block in go modules file

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* fall back to id file on upgrade

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* tidy

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* fix linter check

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* add notice to remove Terraform state check on manual migration

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* add `name` field

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

fix name tests

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* return early if no Terraform diff

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* tidy

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* return infrastructure state even if no diff exists

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* add TODO to remove comment

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* use state-file in miniconstellation

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* cli: remove id-file (#2402)

* remove id-file from `constellation create`

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* add file renaming to handler

* rename id-file after upgrade

* use idFile on `constellation init`

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* remove id-file from `constellation verify`

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* linter fixes

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* remove id-file from `constellation mini`

* remove id-file from `constellation recover`

* linter fixes

* remove id-file from `constellation terminate`

* fix initSecret type

* fix recover argument precedence

* fix terminate test

* generate

* add TODO to remove id-file removal

* Update cli/internal/cmd/init.go

Co-authored-by: Adrian Stobbe <stobbe.adrian@gmail.com>

* fix verify arg parse logic

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* add version test

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* remove id-file from docs

* add file not found log

* use state-file in miniconstellation

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* remove id-file from `constellation iam destroy`

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* remove id-file from `cdbg deploy`

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

---------

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
Co-authored-by: Adrian Stobbe <stobbe.adrian@gmail.com>

* use state-file in CI

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* update orchestration docs

---------

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
Co-authored-by: Adrian Stobbe <stobbe.adrian@gmail.com>
2023-10-09 13:04:29 +02:00
Malte Poll
0979a483b4 debugd: package as tar 2023-09-27 17:58:19 +02:00
Moritz Sanft
f4b2d02194
ci: collect cluster metrics to OpenSearch (#2347)
* add Metricbeat deployment to debugd

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* set metricbeat debugd image version

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* fix k8s deployment

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* use 2 separate deployments

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* only deploy via k8s in non-debug-images

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* add missing tilde

* remove k8s metrics

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* unify flag

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* add cloud metadata processor to filebeat

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* ci: fix debugd logcollection (#2355)

* add missing keyvault access role

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* bump logstash image version

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* bump filebeat / metricbeat image version

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* log used image version

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* use debugging image versions

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* increase wait timeout for image upload

* add cloud metadata processor to filebeat

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* fix template locations in container

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* fix image version typo

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* add filebeat / metricbeat users

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* remove user additions

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* update workflow step name

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* only mount config files

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* document potential rc

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* fix IAM permissions in workflow

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* fix AWS permissions

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* tidy

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* add missing workflow input

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* rename action

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* pin image versions

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* remove unnecessary workflow inputs

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

---------

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* add refStream input

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* remove inputs.yml dep

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* increase system metric period

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* fix linkchecker

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

---------

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
2023-09-27 16:17:31 +02:00
renovate[bot]
8f03213a9f
deps: update fedora:38 Docker digest to 6fc00f8 (#2371)
Co-authored-by: Paul Meyer <49727155+katexochen@users.noreply.github.com>
2023-09-26 13:20:43 +02:00
Daniel Weiße
7aba42baa5
ci: add more filters to e2e failure OpenSearch links (#2358)
Signed-off-by: Daniel Weiße <dw@edgeless.systems>
2023-09-26 13:17:59 +02:00
3u13r
548bb2dfa6
debugd: send requests over lb (#2346) 2023-09-19 16:10:22 +02:00
Moritz Sanft
60bf770e62
ci: logcollection to OpenSearch in non-debug clusters (#2080)
* refactor `debugd` file structure

* create `hack`-tool to deploy logcollection to non-debug clusters

* integrate changes into CI

* update fields

* update workflow input names

* use `working-directory`

* add opensearch creds to upgrade workflow

* make template func generic

* make templating func generic

* linebreaks

* remove magic defaults

* move `os.Exit` to main package

* make logging index configurable

* make templating generic

* remove excess brace

* update fields

* copy fields

* fix flag name

* fix linter warnings

Signed-off-by: Paul Meyer <49727155+katexochen@users.noreply.github.com>

* remove unused workflow inputs

* remove makefiles

* fix command

* bazel: fix output paths of container

This fixes the output paths of builds within the container by mounting
directories to paths that exist on the host. We also explicitly set the
output path in a .bazelrc to the user specific path. The rc file is
mounted into the container and overrides the host rc.
Also adding automatic stop in case start is called and a containers
is already running.
Sym links like bazel-out and paths bazel outputs should generally work
with this change.

Signed-off-by: Paul Meyer <49727155+katexochen@users.noreply.github.com>

* tabs -> spaces

---------

Signed-off-by: Paul Meyer <49727155+katexochen@users.noreply.github.com>
Co-authored-by: Paul Meyer <49727155+katexochen@users.noreply.github.com>
2023-08-21 08:01:33 +02:00
renovate[bot]
a5d6e80dd8
deps: update fedora:38 Docker digest to 61f921e (#2196)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2023-08-10 10:56:35 +02:00
Malte Poll
bd26e6bae7
cdbg: implement workspace / "-C" flag and "--bindir" (#2170) 2023-08-07 11:40:48 +02:00
Daniel Weiße
d1ace13713
cli: add --workspace flag to set base directory for Constellation workspace (#2148)
* Remove `--config` and `--master-secret` falgs

* Add `--workspace` flag

* In CLI, only work on files with paths created from `cli/internal/cmd`

* Properly print values for GCP on IAM create when not directly updating the config

---------

Signed-off-by: Daniel Weiße <dw@edgeless.systems>
2023-08-04 13:53:51 +02:00
Malte Poll
8da6a23aa5
bootstrapper: add fallback endpoint and custom endpoint to SAN field (#2108)
terraform: collect apiserver cert SANs and support custom endpoint

constants: add new constants for cluster configuration and custom endpoint

cloud: support apiserver cert sans and prepare for endpoint migration on AWS

config: add customEndpoint field

bootstrapper: use per-CSP apiserver cert SANs

cli: route customEndpoint to terraform and add migration for apiserver cert SANs

bootstrapper: change interface of GetLoadBalancerEndpoint to return host and port separately
2023-07-21 16:43:51 +02:00
renovate[bot]
49cff0aabb
deps: update module github.com/sigstore/rekor to v1.2.2 (#2033)
Co-authored-by: Paul Meyer <49727155+katexochen@users.noreply.github.com>
2023-07-06 15:41:14 +02:00
renovate[bot]
c58b97de76
deps: update fedora:38 Docker digest to a134743 (#2003)
Co-authored-by: Paul Meyer <49727155+katexochen@users.noreply.github.com>
2023-07-04 14:35:40 +02:00
Adrian Stobbe
7dcd8c3dab
dev-docs: refactor and add information for newbies (#1912)
* refactor dev-docs structure and add information

* improve doc

* Update dev-docs/workflows/create-debug-cluster.md

Co-authored-by: Daniel Weiße <66256922+daniel-weisse@users.noreply.github.com>

* Update dev-docs/workflows/create-debug-cluster.md

Co-authored-by: Daniel Weiße <66256922+daniel-weisse@users.noreply.github.com>

* pr feedback daniel

* Update dev-docs/README.md

Co-authored-by: Daniel Weiße <66256922+daniel-weisse@users.noreply.github.com>

* move to howto again

* split up dev-setup and pull-request into sep files

* fix backticks

* add writing style convention + testing repo

* remove OSS cluster + reduce plugins vs code

* update bazel pre-pr doc

* ghcr img private hint

* add fetch measurement + provider sub-directory hint

* add label doc + pr title check in template

* add OSS build comment

* Update CONTRIBUTING.md

Co-authored-by: Thomas Tendyck <51411342+thomasten@users.noreply.github.com>

* Update CONTRIBUTING.md

Co-authored-by: Thomas Tendyck <51411342+thomasten@users.noreply.github.com>

* Update dev-docs/README.md

Co-authored-by: Thomas Tendyck <51411342+thomasten@users.noreply.github.com>

* Update dev-docs/workflows/dev-setup.md

Co-authored-by: Thomas Tendyck <51411342+thomasten@users.noreply.github.com>

* thomas feedback

* add go proverb mention

---------

Co-authored-by: Daniel Weiße <66256922+daniel-weisse@users.noreply.github.com>
Co-authored-by: Thomas Tendyck <51411342+thomasten@users.noreply.github.com>
2023-06-19 17:39:43 +02:00
Malte Poll
537cdbcfad bazel: trim path to *.pb.go files embedded in go libraries
See https://github.com/bazelbuild/rules_go/issues/3581 for context.
2023-06-16 16:30:47 +02:00
Malte Poll
264b2df902
deps: upgrade to Fedora 38 (#1909)
* image: upgrade mkosi distro version to Fedora 38
* image: remove downgrade of GCP kernel
* ci: upgrade expected measurements for Fedora 38
* deps: upgrade bazeldnf packages to Fedora 38
* deps: upgrade container images to Fedora 38
2023-06-15 16:50:35 +02:00
3u13r
a2c98eb1d5
Correctly deploy the AWS CCM (#1853)
* aws: stop using the imds api for tags

* aws: disable tags in imds api

* aws: only tag instances with non-lecagy tag

* bootstrapper: always let coredns run before cilium

* debugd: make debugd less noisy

* fixup fix aws imds test

* fixup unsued context

* move getting instance id to readInstanceTag
2023-06-13 09:58:39 +02:00