* add current chart
add current helm chart
* disable service controller for aws ccm
* add new iam roles
* doc AWS internet LB + add to LB test
* pass clusterName to helm for AWS LB
* fix update-aws-lb chart to also include .helmignore
* move chart outside services
* working state
* add subnet tags for AWS subnet discovery
* fix .helmignore load rule with file in subdirectory
* upgrade iam profile
* revert new loader impl since cilium is not correctly loaded
* install chart if not already present during `upgrade apply`
* cleanup PR + fix build + add todos
cleanup PR + add todos
* shared helm pkg for cli install and bootstrapper
* add link to eks docs
* refactor iamMigrationCmd
* delete unused helm.symwallk
* move iammigrate to upgrade pkg
* fixup! delete unused helm.symwallk
* add to upgradecheck
* remove nodeSelector from go code (Otto)
* update iam docs and sort permission + remove duplicate roles
* fix bug in `upgrade check`
* better upgrade check output when svc version upgrade not possible
* pr feedback
* remove force flag in upgrade_test
* use upgrader.GetUpgradeID instead of extra type
* remove todos + fix check
* update doc lb (leo)
* remove bootstrapper helm package
* Update cli/internal/cmd/upgradecheck.go
Co-authored-by: Daniel Weiße <66256922+daniel-weisse@users.noreply.github.com>
* final nits
* add docs for e2e upgrade test setup
* Apply suggestions from code review
Co-authored-by: Daniel Weiße <66256922+daniel-weisse@users.noreply.github.com>
* Update cli/internal/helm/loader.go
Co-authored-by: Daniel Weiße <66256922+daniel-weisse@users.noreply.github.com>
* Update cli/internal/cmd/tfmigrationclient.go
Co-authored-by: Daniel Weiße <66256922+daniel-weisse@users.noreply.github.com>
* fix daniel review
* link to the iam permissions instead of manually updating them (agreed with leo)
* disable iam upgrade in upgrade apply
---------
Co-authored-by: Daniel Weiße <66256922+daniel-weisse@users.noreply.github.com>
Co-authored-by: Malte Poll
terraform: collect apiserver cert SANs and support custom endpoint
constants: add new constants for cluster configuration and custom endpoint
cloud: support apiserver cert sans and prepare for endpoint migration on AWS
config: add customEndpoint field
bootstrapper: use per-CSP apiserver cert SANs
cli: route customEndpoint to terraform and add migration for apiserver cert SANs
bootstrapper: change interface of GetLoadBalancerEndpoint to return host and port separately
* Add common backend for interacting with cryptsetup
* Use common cryptsetup backend in bootstrapper
* Use common cryptsetup backend in disk-mapper
* Use common cryptsetup backend in csi lib
---------
Signed-off-by: Daniel Weiße <dw@edgeless.systems>
* cli: add "--skip-helm-wait" flag
This flag can be used to disable the atomic and wait flags during helm install.
This is useful when debugging a failing constellation init, since the user gains access to
the cluster even when one of the deployments is never in a ready state.
* init
* make zone flag mandatory again
* add info about zone update + refactor
* add comment in docs about zone update
* Update cli/internal/cmd/iamcreate_test.go
Co-authored-by: Thomas Tendyck <51411342+thomasten@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Thomas Tendyck <51411342+thomasten@users.noreply.github.com>
* thomas feedback
* add format check to config validation
* remove TODO
* Update docs/docs/workflows/config.md
Co-authored-by: Thomas Tendyck <51411342+thomasten@users.noreply.github.com>
* thomas nit
---------
Co-authored-by: Thomas Tendyck <51411342+thomasten@users.noreply.github.com>
This change is required to ensure we have not tls handshake errors when connecting to the kubernetes api.
Currently, the certificates used by kube-apiserver pods contain a SAN field with the (single) public ip of the loadbalancer.
If we would allow multiple loadbalancer frontend ips, we could encounter cases where the certificate is only valid for one public ip,
while we try to connect to a different ip.
To prevent this, we consciously disable support for the multi-zone loadbalancer frontend on AWS for now.
This will be re-enabled in the future.
* Fix usage of errors.As in upgrade command implementation
* Use struct pointers when working with custom errors
---------
Signed-off-by: Daniel Weiße <dw@edgeless.systems>
* Delete helm chart on failure before retrying installation
* Add chart name to debug output
* Remove now unused wait flag from helm Release struct
---------
Signed-off-by: Daniel Weiße <dw@edgeless.systems>
* fail on version mismatch
* rename to validateCLIandConstellationVersionAreEqual
* fix test
* image version must only be major,minor patch equal (ignore suffix)
* add version support doc
* fix: do not check patch version equality for image and cli
* skip validate on force
* docs: describe SEV-SNP support on AWS
* config: remove launchMeasurement
awsSEVSNP attestation config should not have this value.
It doesn't have a function yet.
* init
* migration working
* make tf variables with default value optional in go through ptr type
* fix CI build
* pr feedback
* add azure targets tf
* skip migration for empty targets
* make instance_count optional
* change role naming to dashed + add validation
* make node_group.zones optional
* Update cli/internal/terraform/terraform/azure/main.tf
Co-authored-by: Malte Poll <1780588+malt3@users.noreply.github.com>
* malte feedback
---------
Co-authored-by: Malte Poll <1780588+malt3@users.noreply.github.com>
* config: move AMD root key to global constant
* attestation: add SNP based attestation for aws
* Always enable SNP, regardless of attestation type.
* Make AWSNitroTPM default again
There exists a bug in AWS SNP implementation where sometimes
a host might not be able to produce valid SNP reports.
Since we have to wait for AWS to fix this we are merging SNP
attestation as opt-in feature.
* invalidate app client id field for azure and provide info
* remove TestNewWithDefaultOptions case
* fix test
* remove appClientID field
* remove client secret + rename err
* remove from docs
* otto feedback
* update docs
* delete env test in cfg since no envs set anymore
* Update dev-docs/workflows/github-actions.md
Co-authored-by: Otto Bittner <cobittner@posteo.net>
* WARNING to stderr
* fix check
---------
Co-authored-by: Otto Bittner <cobittner@posteo.net>
* aws: stop using the imds api for tags
* aws: disable tags in imds api
* aws: only tag instances with non-lecagy tag
* bootstrapper: always let coredns run before cilium
* debugd: make debugd less noisy
* fixup fix aws imds test
* fixup unsued context
* move getting instance id to readInstanceTag
* variant: move into internal/attestation
* attesation: move aws attesation into subfolder nitrotpm
* config: add aws-sev-snp variant
* cli: add tf option to enable AWS SNP
For now the implementations in aws/nitrotpm and aws/snp
are identical. They both contain the aws/nitrotpm impl.
A separate commit will add the actual attestation logic.
* fetch latest version when older than 2 weeks
* extend hack upload tool to pass an upload date
* Revert "config: disable user-facing version Azure SEV SNP fetch for v2.8 (#1882)"
This reverts commit c7b22d314a.
* fix tests
* use NewAzureSEVSNPVersionList for type guarantees
* Revert "use NewAzureSEVSNPVersionList for type guarantees"
This reverts commit 942566453f4b4a2b6dc16f8689248abf1dc47db4.
* assure list is sorted
* improve root.go style
* daniel feedback
* rename to attestationconfigapi + put client and fetcher inside pkg
* rename api/version to versionsapi and put fetcher + client inside pkg
* rename AttestationConfigAPIFetcher to Fetcher
* config: disable user-facing version fetch for Azure SEV SNP
don't allow "latest" value and disable user-facing version fetcher for Azure SEV SNP
Co-authored-by: @derpsteb
* fix unittests
* attestation: getTrustedKey
---------
Co-authored-by: Otto Bittner <cobittner@posteo.net>
* client supports delete version
* rename to new attestation / fetcher naming
* add delete command to upload tool
* test client delete
* bazel update
* use general client in attestation client
* Update hack/configapi/cmd/delete.go
Co-authored-by: Daniel Weiße <66256922+daniel-weisse@users.noreply.github.com>
* daniel feedback
* unit test azure sev upload
* Update hack/configapi/cmd/delete.go
Co-authored-by: Daniel Weiße <66256922+daniel-weisse@users.noreply.github.com>
* add client integration test
* new client cmds use apiObject
---------
Co-authored-by: Daniel Weiße <66256922+daniel-weisse@users.noreply.github.com>
* sign version with release key and remove version from fetcher interface
* extend azure-reporter GH action to upload updated version values to the Attestation API
* api: rename AttestationVersionRepo to Client
* api: move client into separate subpkg for
clearer import paths.
* api: rename configapi -> attestationconfig
* api: rename versionsapi -> versions
* api: rename sut to client
* api: split versionsapi client and make it public
* api: split versionapi fetcher and make it public
* config: move attestationversion type to config
* api: fix attestationconfig client test
Co-authored-by: Adrian Stobbe <stobbe.adrian@gmail.com>
* add SignContent() + integrate into configAPI
* use static client for upload versions tool; fix staticupload calleeReference bug
* use version to get proper cosign pub key.
* mock fetcher in CLI tests
* only provide config.New constructor with fetcher
Co-authored-by: Otto Bittner <cobittner@posteo.net>
Co-authored-by: Daniel Weiße <66256922+daniel-weisse@users.noreply.github.com>
* Add attestation options to config
* Add join-config migration path for clusters with old measurement format
* Always create MAA provider for Azure SNP clusters
* Remove confidential VM option from provider in favor of attestation options
* cli: add config migrate command to handle config migration (#1678)
---------
Signed-off-by: Daniel Weiße <dw@edgeless.systems>
* cli: upgrade check show cli upgrades
* only check compatibility for valid upgrades
* use semver.Sort
* extend unit tests
* add unit test for new compatible cli versions
* adapt to feedback
* fix rebase
* rework output
* minor -> major
Co-authored-by: Otto Bittner <cobittner@posteo.net>
* minor -> major
Co-authored-by: Otto Bittner <cobittner@posteo.net>
* dynamic major version
Co-authored-by: Otto Bittner <cobittner@posteo.net>
* remove currentK8sVer argument
* bazel gen & tidy
* bazel update
---------
Co-authored-by: Otto Bittner <cobittner@posteo.net>
The new command allows checking the status of an upgrade
and which versions are installed.
Also remove the unused restclient.
And make GetConstellationVersion a function.
* support for prerelease tag / pseudoversion
* build version first
Co-authored-by: Daniel Weiße <66256922+daniel-weisse@users.noreply.github.com>
* use strings.Cut
Co-authored-by: Daniel Weiße <66256922+daniel-weisse@users.noreply.github.com>
---------
Co-authored-by: Daniel Weiße <66256922+daniel-weisse@users.noreply.github.com>
* Remove deprecated fields
* Remove warning for not setting attestationVariant
* Dont write attestationVariant to config
---------
Signed-off-by: Daniel Weiße <dw@edgeless.systems>
* Remove using measurements from the initial control-plane node for the cluster's initial measurements
* Add using measurements from the user's config for the cluster's initial measurements to align behavior with upgrade command
---------
Signed-off-by: Daniel Weiße <dw@edgeless.systems>
The test is implemented as a go test.
It can be executed as a bazel target.
The general workflow is to setup a cluster,
point the test to the workspace in which to
find the kubeconfig and the constellation config
and specify a target image, k8s and
service version. The test will succeed
if it detects all target versions in the cluster
within the configured timeout.
The CI automates the above steps.
A separate workflow is introduced as there
are multiple input fields to the test.
Adding all of these to the manual e2e test
seemed confusing.
Co-authored-by: Fabian Kammel <fk@edgeless.systems>
* Convert enforceIDKeyDigest setting to enum
* Use MAA fallback in Azure SNP attestation
* Only create MAA provider if MAA fallback is enabled
---------
Signed-off-by: Daniel Weiße <dw@edgeless.systems>
Co-authored-by: Thomas Tendyck <tt@edgeless.systems>
* create and update maa attestation policy
* use interface to allow unit testing
* fix test csp
* http request for policy patch
* go mod tidy
* remove hyphen
* go mod tidy
* wip: adapt to feedback
* linting fixes
* remove csp from tf call
* fix type assertion
* Add MAA URL to instance tags (#1409)
Signed-off-by: Daniel Weiße <dw@edgeless.systems>
* conditionally create maa provider
* only set instance tag when maa is created
* fix azure unit test
* bazel tidy
* remove AzureCVM const
Co-authored-by: Thomas Tendyck <51411342+thomasten@users.noreply.github.com>
* encode policy at runtime
* remove policy arg
* fix unit test
---------
Signed-off-by: Daniel Weiße <dw@edgeless.systems>
Co-authored-by: Daniel Weiße <66256922+daniel-weisse@users.noreply.github.com>
Co-authored-by: Thomas Tendyck <51411342+thomasten@users.noreply.github.com>
* Add attestation type to config (optional for now)
* Get attestation variant from config in CLI
* Set attestation variant for Constellation services in helm deployments
* Remove AzureCVM variable from helm deployments
---------
Signed-off-by: Daniel Weiße <dw@edgeless.systems>
* upload cli version list
* fix flag
* name
* allow cli kind for listing
* [remove] update vapi cli
* allow cli kind
* use latest versionsapi image version
* fix kind parsing
* use workflow calls in on_release action
* [remove] update container tag
* change back to latest tag
* build: correct toolchain order
* build: gazelle-update-repos
* build: use pregenerated proto for dependencies
* update bazeldnf
* deps: tpm simulator
* Update Google trillian module
* cli: add stamping as alternative build info source
* bazel: add go_test wrappers, mark special tests and select testing deps
* deps: add libvirt deps
* deps: go-libvirt patches
* deps: cloudflare circl patches
* bazel: add go_test wrappers, mark special tests and select testing deps
* bazel: keep gazelle overrides
* bazel: cleanup bazelrc
* bazel: switch CMakeLists.txt to use bazel
* bazel: fix injection of version information via stamping
* bazel: commit all build files
* dev-docs: document bazel usage
* deps: upgrade zig-cc for go 1.20
* bazel: update Perl for macOS arm64 & Linux arm64 support
* bazel: use static perl toolchain for OpenSSL
* bazel: use static protobuf (protoc) toolchain
* deps: add git and go to nix deps
Co-authored-by: Paul Meyer <49727155+katexochen@users.noreply.github.com>
This will help the measurement generation done as part of internal/attestation/measurements/measurement-generator.
It can currently replace measurements but not reformat the code (in cases where the number of measurements differs).
The libvirt container is currently not automatically upgraded (it still has version v2.2.0 before this change).
To ensure we update libvirt for this release, we manually upgrade the libvirt container image.
Applies the updated NodeVersion object with one request
instead of two. This makes sure that the first request does
not accidentially put the cluster into a "updgrade in progress"
status. Which would lead users to having to run apply twice.
* Move storage clients to separate packages
* Allow setting of client credentials for AWS S3
* Use managed identity client secret or default credentials for Azure Blob Storage
* Use credentials file to authorize GCS client
---------
Signed-off-by: Daniel Weiße <dw@edgeless.systems>
The variable VersionInfo is supposed to be set by `go build -X ...` during link time but should not be modified at runtime.
This change ensures the underlying var is private and can only be accessed by a public getter.
* add custom version type
* extend functionality
* adapt to requested changes
* move to own package
* remove duplicate tests, rename package
* not handle err
* add cli k8s compatibility api to ci
* extend versionsapi package
* rework cli info upload via ci
* join errors natively
* fix semver
* upload from hack file
* fix ci checks
* add distributionid
* setup go before running hack file
* setup go after repo checkout
* use logger instead of panic, invalidate cache
* use provided ctx
Co-authored-by: Paul Meyer <49727155+katexochen@users.noreply.github.com>
---------
Co-authored-by: Paul Meyer <49727155+katexochen@users.noreply.github.com>
To adhere to our compatibility goal of not breaking
old configs, the kubernetes patch version is automatically
extended for configs in the transistional version v2.6.
* Allow unset name key in config but print warning if unset
* Print deprecation warnings for config to os.Stderr
---------
Signed-off-by: Daniel Weiße <dw@edgeless.systems>
* `upgrade apply` will try to make the locally configured and
actual version in the cluster match by appling necessary
upgrades.
* Skip image or kubernetes upgrades if one is already
in progress.
* Skip downgrades/equal-as-running versions
* Move NodeVersionResourceName constant from operators
to internal as its needed in the CLI.
* Generate kubeconfig with unique name
* Move create name flag to config
* Add name validation to config
* Move name flag in e2e tests to config generation
* Remove name flag from create
* Update ascii cinema flow
---------
Signed-off-by: Daniel Weiße <dw@edgeless.systems>
Upgrade check is used to find updates for the current cluster.
Optionally the found upgrades can be persisted to the config
for consumption by the upgrade-execute cmd.
The old `upgrade execute` in this commit does not work with
the new `upgrade plan`.
The current versions are read from the cluster.
Supported versions are read from the cli and the versionsapi.
Adds a new config field MicroserviceVersion that will be used
by `upgrade execute` to update the service versions.
The field is optional until 2.7
A deprecation warning for the upgrade key is printed during
config validation.
Kubernetes versions now specify the patch version to make it
explicit for users if an upgrade changes the k8s version.
Version validation checks that the configured versions
are not more than one minor version below the CLI's version.
The validation can be disabled using --force.
This is necessary for now during development as the CLI
does not have a prerelease version, as our images do.
* Replace external KMS backend logic for AWS, Azure, and GCP with go-kms-wrapping
* Move kms client setup config into its own package for easier parsing
* Update kms integration flag naming
* Error if nil storage is passed to external KMS
---------
Signed-off-by: Daniel Weiße <dw@edgeless.systems>
So far the masterSecret was sent to the initial bootstrapper
on init/recovery. With this commit this information is encoded
in the kmsURI that is sent during init.
For recover, the communication with the recoveryserver is
changed. Before a streaming gRPC call was used to
exchanges UUID for measurementSecret and state disk key.
Now a standard gRPC is made that includes the same kmsURI &
storageURI that are sent during init.
In the light of extending our eKMS support it will be helpful
to have a tighter use of the word "KMS".
KMS should refer to the actual component that manages keys.
The keyservice, also called KMS in the constellation code,
does not manage keys itself. It talks to a KMS backend,
which in turn does the actual key management.
* update constellation-os to constellation-version references
* update nodeimage to nodeversion in CRD type name
Signed-off-by: Fabian Kammel <fk@edgeless.systems>
Co-authored-by: Malte Poll <mp@edgeless.systems>
These backups could be used in case an upgrade
misbehaves after helm declared it as successful.
The manual backups are required as helm-rollback
won't touch custom resources and changes to CRDs
delete resources of the old version.
* Update module github.com/talos-systems/talos/pkg/machinery to v1.3.1
* Rename talos-systems/talos to siderolabs/talos
Co-authored-by: Paul Meyer <49727155+katexochen@users.noreply.github.com>
Run with: constellation upgrade execute --helm.
This will only upgrade the helm charts. No config is needed.
Upgrades are implemented via helm's upgrade action, i.e. they
automatically roll back if something goes wrong. Releases could
still be managed via helm, even after an upgrade with constellation
has been done.
Currently not user facing as CRD/CR backups are still in progress.
These backups should be automatically created and saved to the
user's disk as updates may delete CRs. This happens implicitly
through CRD upgrades, which are part of microservice upgrades.
Switch azure default region to west us
Update find-image script to work with new API spec
Add version for every os image build
generate measurements: Use new API paths
CLI: config fetch measurements: Use image short versions to fetch measurements
CLI: allows shortnames to specify image in config
Image build pipeline: Change paths to contain "ref" and "stream"
