Commit Graph

117 Commits

Author SHA1 Message Date
Moritz Sanft
c15e4efef6
terraform: Azure Marketplace image support (#2651)
* terraform: add Azure marketplace variable

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* config: add Azure marketplace variable

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* cli: use Terraform variables from config

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* terraform: pass down marketplace variable

* image: pad Azure images to 1GiB

* terraform: add version attribute to marketplace image

* semver: allow versions to be exported without prefix

* cli: boolean var to use marketplace images

* config: remove dive key

* dev-docs: add instructions on how to use marketplace images

* terraform: fix unit test

* terraform: only fetch image for non-marketplace images

* mpimage: refactor image selection

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* [remove] increase minor version for image build

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* terraform: ignore changes to source_image_reference on upgrade

* operator: add support for parsing Azure marketplace images

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* upgrade: fix imagefetcher call

* docs: add info about azure marketplace

* image: ensure more than 1GiB in size

* image: test to pad to 2GiB

* version: change back to v2.14.0-pre

* image: GPT-conformant image size padding

* [remove] increase version

* mpimage: inline prefix func

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* ci: add marketplace image e2e test

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* [remove] register workflow

* ci: fix workflow name

* ci: only allow azure test

* cli: add marketplace image input to interface

* cli: fix argument passing

* version: roll back to v2.14.0

* ci: add force-flag support

* Update docs/docs/overview/license.md

* Update dev-docs/workflows/marketplace-images.md

Co-authored-by: Moritz Eckert <m1gh7ym0@gmail.com>

---------

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
Co-authored-by: Moritz Eckert <m1gh7ym0@gmail.com>
Co-authored-by: Thomas Tendyck <51411342+thomasten@users.noreply.github.com>
2023-12-08 14:40:31 +01:00
Daniel Weiße
35abc3c354
cli: use apply command to start mini cluster (#2551)
Signed-off-by: Daniel Weiße <dw@edgeless.systems>
2023-11-20 12:10:16 +01:00
Leonard Cohnen
cfcc0898b2 helm: remove konnectivity from control-planes
This is the first step in our migration off of
konnectivity. Before node-to-node encryption
we used konnectivity to route some KubeAPI
to kubelet traffic over the pod network which then
would be encrypted.

Since we enabled node-to-node encryption this has no
security upsides anymore. Note that we still deploy
the konnectivity agents via helm and still have the
load balancer for konnectivity.

In the following releases we will remove both.
2023-11-15 19:27:33 +01:00
Adrian Stobbe
cea6204b37
terraform: Terraform module for AWS (#2503) 2023-11-08 19:10:01 +01:00
Daniel Weiße
ec424b260d
cli: refactor terraform code to be update/create agnostic (#2501)
* Move upgrade specific functions out of Terraform module
* Always allow overwriting Terraform files
* Ensure constellation-terraform dir does not exist on create

---------

Signed-off-by: Daniel Weiße <dw@edgeless.systems>
2023-10-26 10:55:50 +02:00
Daniel Weiße
ab8a17e535
cli: remove old migration steps and id-file references (#2440)
* Remove old migration steps and id-file references
* Update codeowners file

---------

Signed-off-by: Daniel Weiße <dw@edgeless.systems>
2023-10-13 10:21:21 +02:00
Moritz Sanft
a5021c52d3
joinservice: cache certificates for Azure SEV-SNP attestation (#2336)
* add ASK caching in joinservice

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* use cached ASK in Azure SEV-SNP attestation

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* update test charts

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* fix linter

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* fix typ

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* make caching mechanism less provider-specific

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* update buildfiles

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* add `omitempty` flag

Co-authored-by: Daniel Weiße <66256922+daniel-weisse@users.noreply.github.com>

* frontload certificate getter

Co-authored-by: Daniel Weiße <66256922+daniel-weisse@users.noreply.github.com>

* rename frontloaded function

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* pass cached certificates to constructor

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* fix race condition

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* fix marshalling of empty certs

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* fix validator usage

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* [wip] add certcache tests

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* add certcache tests

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* tidy

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* fix validator test

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* remove unused fields in validator

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* fix certificate precedence

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* use separate context

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* tidy

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* linter fixes

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* linter fixes

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* Remove unnecessary comment

Co-authored-by: Thomas Tendyck <51411342+thomasten@users.noreply.github.com>

* use background context

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* Use error format directive

Co-authored-by: Thomas Tendyck <51411342+thomasten@users.noreply.github.com>

* `azure` -> `Azure`

Co-authored-by: Thomas Tendyck <51411342+thomasten@users.noreply.github.com>

* improve error messages

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* add x509 -> PEM util function

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* use crypto util functions

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* fix certificate replacement logic

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* only require ASK from certcache

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* tidy

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* fix comment typo

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

---------

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
Co-authored-by: Daniel Weiße <66256922+daniel-weisse@users.noreply.github.com>
Co-authored-by: Thomas Tendyck <51411342+thomasten@users.noreply.github.com>
2023-09-29 14:29:50 +02:00
Adrian Stobbe
322c4aad10
cli: write infrastructure to new state file (#2321)
Co-authored-by: 3u13r <lc@edgeless.systems>
2023-09-25 16:19:43 +02:00
Daniel Weiße
89b342900f Move workspace path functions to sub-package of cmd
Signed-off-by: Daniel Weiße <dw@edgeless.systems>
2023-08-09 15:42:24 +02:00
Daniel Weiße
d1ace13713
cli: add --workspace flag to set base directory for Constellation workspace (#2148)
* Remove `--config` and `--master-secret` falgs

* Add `--workspace` flag

* In CLI, only work on files with paths created from `cli/internal/cmd`

* Properly print values for GCP on IAM create when not directly updating the config

---------

Signed-off-by: Daniel Weiße <dw@edgeless.systems>
2023-08-04 13:53:51 +02:00
Malte Poll
15bb3b31fd config: add nodeGroups 2023-08-04 12:36:45 +02:00
Otto Bittner
1d5a8283e0
cli: use Semver type to represent microservice versions (#2125)
Previously we used strings to pass microservice versions. This invited
bugs due to missing input validation.
2023-07-25 14:20:25 +02:00
Adrian Stobbe
a87b7894db
aws: use new LB controller to fix SecurityGroup cleanup on K8s service deletion (#2090)
* add current chart

add current helm chart

* disable service controller for aws ccm

* add new iam roles

* doc AWS internet LB + add to LB test

* pass clusterName to helm for AWS LB

* fix update-aws-lb chart to also include .helmignore

* move chart outside services

* working state

* add subnet tags for AWS subnet discovery

* fix .helmignore load rule with file in subdirectory

* upgrade iam profile

* revert new loader impl since cilium is not correctly loaded

* install chart if not already present during `upgrade apply`

* cleanup PR + fix build + add todos

cleanup PR + add todos

* shared helm pkg for cli install and bootstrapper

* add link to eks docs

* refactor iamMigrationCmd

* delete unused helm.symwallk

* move iammigrate to upgrade pkg

* fixup! delete unused helm.symwallk

* add to upgradecheck

* remove nodeSelector from go code (Otto)

* update iam docs and sort permission + remove duplicate roles

* fix bug in `upgrade check`

* better upgrade check output when svc version upgrade not possible

* pr feedback

* remove force flag in upgrade_test

* use upgrader.GetUpgradeID instead of extra type

* remove todos + fix check

* update doc lb (leo)

* remove bootstrapper helm package

* Update cli/internal/cmd/upgradecheck.go

Co-authored-by: Daniel Weiße <66256922+daniel-weisse@users.noreply.github.com>

* final nits

* add docs for e2e upgrade test setup

* Apply suggestions from code review

Co-authored-by: Daniel Weiße <66256922+daniel-weisse@users.noreply.github.com>

* Update cli/internal/helm/loader.go

Co-authored-by: Daniel Weiße <66256922+daniel-weisse@users.noreply.github.com>

* Update cli/internal/cmd/tfmigrationclient.go

Co-authored-by: Daniel Weiße <66256922+daniel-weisse@users.noreply.github.com>

* fix daniel review

* link to the iam permissions instead of manually updating them (agreed with leo)

* disable iam upgrade in upgrade apply

---------

Co-authored-by: Daniel Weiße <66256922+daniel-weisse@users.noreply.github.com>
Co-authored-by: Malte Poll
2023-07-24 10:30:53 +02:00
Malte Poll
8da6a23aa5
bootstrapper: add fallback endpoint and custom endpoint to SAN field (#2108)
terraform: collect apiserver cert SANs and support custom endpoint

constants: add new constants for cluster configuration and custom endpoint

cloud: support apiserver cert sans and prepare for endpoint migration on AWS

config: add customEndpoint field

bootstrapper: use per-CSP apiserver cert SANs

cli: route customEndpoint to terraform and add migration for apiserver cert SANs

bootstrapper: change interface of GetLoadBalancerEndpoint to return host and port separately
2023-07-21 16:43:51 +02:00
Otto Bittner
cf822f7eee
cli: unify terraform variable creation (#2119)
Before we defined the variables twice.
Once for upgrades, once for create.
Also move default node group names into a constant
2023-07-21 10:04:29 +02:00
Moritz Sanft
5f71934f56
cli: write Terraform migration output directly to constellation-id.json (#2107)
* backup `constellation-id.json` before upgrade

* remove superfluous `file.Handler` arguments

* merge `constellation-id.json` on upgrade

* fix typo
2023-07-18 09:33:42 +02:00
Otto Bittner
7388240943
Revert "attestation: add SNP-based attestation for aws-sev-snp (#1916)" (#1957)
This reverts commit c7d12055d1.
2023-06-22 17:08:44 +02:00
Otto Bittner
c7d12055d1
attestation: add SNP-based attestation for aws-sev-snp (#1916)
* config: move AMD root key to global constant
* attestation: add SNP based attestation for aws
* Always enable SNP, regardless of attestation type.
* Make AWSNitroTPM default again

There exists a bug in AWS SNP implementation where sometimes
a host might not be able to produce valid SNP reports.
Since we have to wait for AWS to fix this we are merging SNP
attestation as opt-in feature.
2023-06-21 14:19:55 +02:00
Otto Bittner
30f2b332b3
api: restructure api pkg (#1851)
* api: rename AttestationVersionRepo to Client
* api: move client into separate subpkg for
clearer import paths.
* api: rename configapi -> attestationconfig
* api: rename versionsapi -> versions
* api: rename sut to client
* api: split versionsapi client and make it public
* api: split versionapi fetcher and make it public
* config: move attestationversion type to config
* api: fix attestationconfig client test

Co-authored-by: Adrian Stobbe <stobbe.adrian@gmail.com>
2023-06-02 09:19:23 +02:00
Adrian Stobbe
b51cc52945
config: sign Azure versions on upload & verify on fetch (#1836)
* add SignContent() + integrate into configAPI

* use static client for upload versions tool; fix staticupload calleeReference bug

* use version to get proper cosign pub key.

* mock fetcher in CLI tests

* only provide config.New constructor with fetcher

Co-authored-by: Otto Bittner <cobittner@posteo.net>
Co-authored-by: Daniel Weiße <66256922+daniel-weisse@users.noreply.github.com>
2023-06-01 13:55:46 +02:00
Malte Poll
8a851c8f39 cli: dynamically select signature validation pubkey for release and pre-release artifacts 2023-05-31 14:00:00 +02:00
miampf
8686c5e7e2
bootstrapper: collect journald logs on failure (#1618) 2023-05-30 11:47:36 +00:00
Malte Poll
e5b394db87 cli: image measurements (v2) 2023-05-25 15:01:15 +02:00
Moritz Sanft
c69e6777bd
cli: Terraform migrations on upgrade (#1685)
* add terraform planning

* overwrite terraform files in upgrade workspace

* Revert "overwrite terraform files in upgrade workspace"

This reverts commit 8bdacfb8bef23ef2cdbdb06bad0855b3bbc42df0.

* prepare terraform workspace

* test upgrade integration

* print upgrade abort

* rename plan file

* write output to file

* add show plan test

* add upgrade tf workdir

* fix workspace preparing

* squash to 1 command

* test

* bazel build

* plan test

* register flag manually

* bazel tidy

* fix linter

* remove MAA variable

* fix workdir

* accept tf variables

* variable fetching

* fix resource indices

* accept Terraform targets

* refactor upgrade command

* Terraform migration apply unit test

* pass down image fetcher to test

* use new flags in e2e test

* move file name to constant

* update buildfiles

* fix version constant

* conditionally create MAA

* move interface down

* upgrade dir

* update buildfiles

* fix interface

* fix createMAA check

* fix imports

* update buildfiles

* wip: workspace backup

* copy utils

* backup upgrade workspace

* remove debug print

* replace old state after upgrade

* check if flag exists

* prepare test workspace

* remove prefix

Co-authored-by: Otto Bittner <cobittner@posteo.net>

* respect file permissions

* refactor tf upgrader

* check workspace before upgrades

* remove temp upgrade dir after completion

* clean up workspace after abortion

* fix upgrade apply test

* fix linter

---------

Co-authored-by: Otto Bittner <cobittner@posteo.net>
2023-05-22 13:31:20 +02:00
Daniel Weiße
d7a2ddd939
config: add separate option for handling attestation parameters (#1623)
* Add attestation options to config

* Add join-config migration path for clusters with old measurement format

* Always create MAA provider for Azure SNP clusters

* Remove confidential VM option from provider in favor of attestation options

* cli: add config migrate command to handle config migration (#1678)

---------

Signed-off-by: Daniel Weiße <dw@edgeless.systems>
2023-05-03 11:11:53 +02:00
Moritz Sanft
1d0ee796e8
cli: add Terraform log support (#1620)
* add Terraform logging

* add TF logging to CLI

* fix path

* only create file if logging is enabled

* update bazel files

* register persistent flags manually

* clidocgen

* move logging code to separate file

* reword yes flag parsing error

* update bazel buildfile

* factor out log level setting
2023-04-14 14:15:07 +02:00
Otto Bittner
cac43a1dd0 ci: add e2e-upgrade test
The test is implemented as a go test.
It can be executed as a bazel target.
The general workflow is to setup a cluster,
point the test to the workspace in which to
find the kubeconfig and the constellation config
and specify a target image, k8s and
service version. The test will succeed
if it detects all target versions in the cluster
within the configured timeout.
The CI automates the above steps.
A separate workflow is introduced as there
are multiple input fields to the test.
Adding all of these to the manual e2e test
seemed confusing.

Co-authored-by: Fabian Kammel <fk@edgeless.systems>
2023-03-23 14:57:38 +01:00
Daniel Weiße
5a0234b3f2
attestation: add option for MAA fallback to verify azure's snp-sev id key digest (#1257)
* Convert enforceIDKeyDigest setting to enum

* Use MAA fallback in Azure SNP attestation

* Only create MAA provider if MAA fallback is enabled

---------

Signed-off-by: Daniel Weiße <dw@edgeless.systems>
Co-authored-by: Thomas Tendyck <tt@edgeless.systems>
2023-03-21 12:46:49 +01:00
Moritz Sanft
f2ce9518a3
cli: support custom attestation policies for maa (#1375)
* create and update maa attestation policy

* use interface to allow unit testing

* fix test csp

* http request for policy patch

* go mod tidy

* remove hyphen

* go mod tidy

* wip: adapt to feedback

* linting fixes

* remove csp from tf call

* fix type assertion

* Add MAA URL to instance tags (#1409)

Signed-off-by: Daniel Weiße <dw@edgeless.systems>

* conditionally create maa provider

* only set instance tag when maa is created

* fix azure unit test

* bazel tidy

* remove AzureCVM const

Co-authored-by: Thomas Tendyck <51411342+thomasten@users.noreply.github.com>

* encode policy at runtime

* remove policy arg

* fix unit test

---------

Signed-off-by: Daniel Weiße <dw@edgeless.systems>
Co-authored-by: Daniel Weiße <66256922+daniel-weisse@users.noreply.github.com>
Co-authored-by: Thomas Tendyck <51411342+thomasten@users.noreply.github.com>
2023-03-20 13:33:04 +01:00
Daniel Weiße
6ea5588bdc
config: add attestation variant (#1413)
* Add attestation type to config (optional for now)

* Get attestation variant from config in CLI

* Set attestation variant for Constellation services in helm deployments

* Remove AzureCVM variable from helm deployments

---------

Signed-off-by: Daniel Weiße <dw@edgeless.systems>
2023-03-14 11:46:27 +01:00
Thomas Tendyck
64e1f553d1 cli: remove Edition in version command, which contains duplicate info 2023-03-10 11:36:44 +01:00
Nils Hanke
dc4769d0a0 constants: use "Enterprise" for enterprise build 2023-03-09 17:32:50 +01:00
Malte Poll
bdba9d8ba6
bazel: add build files for go (#1186)
* build: correct toolchain order
* build: gazelle-update-repos
* build: use pregenerated proto for dependencies
* update bazeldnf
* deps: tpm simulator
* Update Google trillian module
* cli: add stamping as alternative build info source
* bazel: add go_test wrappers, mark special tests and select testing deps
* deps: add libvirt deps
* deps: go-libvirt patches
* deps: cloudflare circl patches
* bazel: add go_test wrappers, mark special tests and select testing deps
* bazel: keep gazelle overrides
* bazel: cleanup bazelrc
* bazel: switch CMakeLists.txt to use bazel
* bazel: fix injection of version information via stamping
* bazel: commit all build files
* dev-docs: document bazel usage
* deps: upgrade zig-cc for go 1.20
* bazel: update Perl for macOS arm64 & Linux arm64 support
* bazel: use static perl toolchain for OpenSSL
* bazel: use static protobuf (protoc) toolchain
* deps: add git and go to nix deps

Co-authored-by: Paul Meyer <49727155+katexochen@users.noreply.github.com>
2023-03-09 15:23:42 +01:00
Daniel Weiße
5bad5f768b
attestation: create issuer based on kernel cmd line (#1355)
Signed-off-by: Daniel Weiße <dw@edgeless.systems>
2023-03-09 09:47:28 +01:00
Paul Meyer
630016d1b3 openstack: use password to authenticate in cluster
Signed-off-by: Paul Meyer <49727155+katexochen@users.noreply.github.com>
2023-03-08 09:04:57 -05:00
Malte Poll
8aa42e30ad
cli: set OpenStack service account credentials (#1328) 2023-03-03 10:10:36 +01:00
Malte Poll
fc33a74c78
constants: make VersionInfo readonly (#1316)
The variable VersionInfo is supposed to be set by `go build -X ...` during link time but should not be modified at runtime.
This change ensures the underlying var is private and can only be accessed by a public getter.
2023-03-01 11:55:12 +01:00
Fabian Kammel
5e7dc0d7db
Option to disable spinner via environment variable. (#1207)
Signed-off-by: Fabian Kammel <fk@edgeless.systems>
2023-02-16 15:43:19 +01:00
Otto Bittner
50646b2a10 cli: refactor upgrade apply cmd to match name
* `upgrade apply` will try to make the locally configured and
actual version in the cluster match by appling necessary
upgrades.
* Skip image or kubernetes upgrades if one is already
in progress.
* Skip downgrades/equal-as-running versions
* Move NodeVersionResourceName constant from operators
to internal as its needed in the CLI.
2023-02-15 16:44:47 +01:00
Paul Meyer
84a787b538
cli: add name of build type to version cmd output (#1179)
Signed-off-by: Paul Meyer <49727155+katexochen@users.noreply.github.com>
2023-02-14 14:30:10 +01:00
Daniel Weiße
c29107f5be
init: create kubeconfig file with unique user/cluster name (#1133)
* Generate kubeconfig with unique name

* Move create name flag to config

* Add name validation to config

* Move name flag in e2e tests to config generation

* Remove name flag from create

* Update ascii cinema flow

---------

Signed-off-by: Daniel Weiße <dw@edgeless.systems>
2023-02-10 13:27:22 +01:00
Malte Poll
2d326ea3f0
cli: set placeholder uid for QEMU / MiniConstellation (#1069) 2023-01-25 14:42:52 +01:00
Paul Meyer
a8cbfd848f
keyservice: use dash in container name (#1016)
Co-authored-by: Otto Bittner <cobittner@posteo.net>
2023-01-20 18:51:06 +01:00
3u13r
632090c21b
azure: allow a set of idkeydigest values (#991) 2023-01-18 16:49:55 +01:00
Thomas Tendyck
f0f109a1ea verify: use fixed user data 2023-01-17 16:14:00 +01:00
Otto Bittner
90b88e1cf9 kms: rename kms to keyservice
In the light of extending our eKMS support it will be helpful
to have a tighter use of the word "KMS".
KMS should refer to the actual component that manages keys.
The keyservice, also called KMS in the constellation code,
does not manage keys itself. It talks to a KMS backend,
which in turn does the actual key management.
2023-01-16 11:56:34 +01:00
Leonard Cohnen
2700d5182b operator: reconcile kubernetesClusterVersion 2023-01-09 12:16:54 +01:00
Leonard Cohnen
8c5e41b865 constants: rename components annotation key 2023-01-09 12:16:54 +01:00
Leonard Cohnen
25c3a8a1f3 init: add cluster version to kubernetes components 2023-01-05 14:52:09 +01:00
3u13r
f14af0c3eb
upgrade: support Kubernetes components (#839)
* upgrade: add Kubernetes components to NodeVersion

* update rfc
2023-01-03 12:09:53 +01:00