* terraform-provider: add usage example for Azure
* terraform-provider: add usage example for AWS
* terraform-provider: add usage example for GCP
* terraform-provider: update usage example for Azure
* terraform-provider: update generated documentation
* docs: adjust creation on Azure and link to examples
* terraform-provider: unify image in-/output (#2725)
* terraform-provider: check for returned error when converting microservices
* terraform-provider: use state values for outputs after creation
* terraform-provider: ignore invalid upgrades (#2728)
---------
Co-authored-by: Daniel Weiße <66256922+daniel-weisse@users.noreply.github.com>
Co-authored-by: Thomas Tendyck <51411342+thomasten@users.noreply.github.com>
* Replace reference to CMakeList on main branch with versioned link or Bazel
* Allow Bazel as a valid word
---------
Signed-off-by: Daniel Weiße <dw@edgeless.systems>
* terraform: add Azure marketplace variable
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* config: add Azure marketplace variable
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* cli: use Terraform variables from config
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* terraform: pass down marketplace variable
* image: pad Azure images to 1GiB
* terraform: add version attribute to marketplace image
* semver: allow versions to be exported without prefix
* cli: boolean var to use marketplace images
* config: remove dive key
* dev-docs: add instructions on how to use marketplace images
* terraform: fix unit test
* terraform: only fetch image for non-marketplace images
* mpimage: refactor image selection
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* [remove] increase minor version for image build
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* terraform: ignore changes to source_image_reference on upgrade
* operator: add support for parsing Azure marketplace images
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* upgrade: fix imagefetcher call
* docs: add info about azure marketplace
* image: ensure more than 1GiB in size
* image: test to pad to 2GiB
* version: change back to v2.14.0-pre
* image: GPT-conformant image size padding
* [remove] increase version
* mpimage: inline prefix func
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* ci: add marketplace image e2e test
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* [remove] register workflow
* ci: fix workflow name
* ci: only allow azure test
* cli: add marketplace image input to interface
* cli: fix argument passing
* version: roll back to v2.14.0
* ci: add force-flag support
* Update docs/docs/overview/license.md
* Update dev-docs/workflows/marketplace-images.md
Co-authored-by: Moritz Eckert <m1gh7ym0@gmail.com>
---------
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
Co-authored-by: Moritz Eckert <m1gh7ym0@gmail.com>
Co-authored-by: Thomas Tendyck <51411342+thomasten@users.noreply.github.com>
* Allow creation of Constellation clusters using `apply` command
* Add auto-completion for `--skip-phases` flag
* Deprecate create command
* Replace all doc references to create command with apply
---------
Signed-off-by: Daniel Weiße <dw@edgeless.systems>
* add Azure Terraform module
* add maa-patching command to cli
* refactor release process
* factor out image fetching to own action
* add CI
* generate
* fix some unnecessary changes
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* use `constellation maa-patch` in ci
* insecure flag when using debug image
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* only update maa url if existing
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* make node group zone optional on aws and gcp
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* [remove] register updated workflow
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* Revert "[remove] register updated workflow"
This reverts commit e70b9515b7eabbcbe0d41fa1296c48750cd02ace.
* create MAA
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* make maa-patching only run on azure
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* add comment
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* require node group zone for GCP and AWS
* remove unnecessary bazel action
* stamp version to correct file
* refer to `maa-patch` command in docs
* run Azure test in weekly e2e
* comment / naming improvements
* remove sa_account resource
* disable spellcheck ot use "URL"
* `create_maa` variable
* don't write maa url to config
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* default to nightly image
* use input ref and stream
* fix command check
* don't set region in weekly e2e call
* patch maa if url is not empty
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* remove `create_maa` variable
* remove binaries
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* remove undefined input
* replace invalid attestation URL error message
Co-authored-by: Thomas Tendyck <51411342+thomasten@users.noreply.github.com>
* fix punctuation
Co-authored-by: Thomas Tendyck <51411342+thomasten@users.noreply.github.com>
* skip hidden commands in clidocgen
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* enable spellcheck before code block
* move spellcheck trigger out of info block
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* fix workflow dependencies
* let image default to CLI version
---------
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
Co-authored-by: Thomas Tendyck <51411342+thomasten@users.noreply.github.com>
* Add apply command
* Mark init and upgrade apply as deprecated
* Use apply command in CI
* Add skippable phases for attestation config and cert SANs
---------
Signed-off-by: Daniel Weiße <dw@edgeless.systems>
* create state file during config generate
* use written file in `constellation create`
* document creation of state file
* remove accidentally added test
* check error when writing state file
* [wip] use state file in CLI
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
tidy
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* use state file in CLI
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
take clusterConfig from IDFile for compat
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
various fixes
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
wip
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* add GCP-specific values in Helm loader test
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* remove unnecessary pointer
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* write ClusterValues in one step
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* move stub to test file
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* remove mention of id-file
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* move output to `migrateTerraform`
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* unconditional assignments converting from idFile
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* move require block in go modules file
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* fall back to id file on upgrade
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* tidy
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* fix linter check
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* add notice to remove Terraform state check on manual migration
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* add `name` field
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
fix name tests
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* return early if no Terraform diff
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* tidy
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* return infrastructure state even if no diff exists
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* add TODO to remove comment
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* use state-file in miniconstellation
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* cli: remove id-file (#2402)
* remove id-file from `constellation create`
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* add file renaming to handler
* rename id-file after upgrade
* use idFile on `constellation init`
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* remove id-file from `constellation verify`
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* linter fixes
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* remove id-file from `constellation mini`
* remove id-file from `constellation recover`
* linter fixes
* remove id-file from `constellation terminate`
* fix initSecret type
* fix recover argument precedence
* fix terminate test
* generate
* add TODO to remove id-file removal
* Update cli/internal/cmd/init.go
Co-authored-by: Adrian Stobbe <stobbe.adrian@gmail.com>
* fix verify arg parse logic
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* add version test
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* remove id-file from docs
* add file not found log
* use state-file in miniconstellation
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* remove id-file from `constellation iam destroy`
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* remove id-file from `cdbg deploy`
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
---------
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
Co-authored-by: Adrian Stobbe <stobbe.adrian@gmail.com>
* use state-file in CI
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* update orchestration docs
---------
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
Co-authored-by: Adrian Stobbe <stobbe.adrian@gmail.com>
* add Metricbeat deployment to debugd
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* set metricbeat debugd image version
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* fix k8s deployment
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* use 2 separate deployments
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* only deploy via k8s in non-debug-images
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* add missing tilde
* remove k8s metrics
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* unify flag
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* add cloud metadata processor to filebeat
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* ci: fix debugd logcollection (#2355)
* add missing keyvault access role
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* bump logstash image version
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* bump filebeat / metricbeat image version
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* log used image version
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* use debugging image versions
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* increase wait timeout for image upload
* add cloud metadata processor to filebeat
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* fix template locations in container
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* fix image version typo
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* add filebeat / metricbeat users
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* remove user additions
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* update workflow step name
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* only mount config files
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* document potential rc
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* fix IAM permissions in workflow
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* fix AWS permissions
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* tidy
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* add missing workflow input
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* rename action
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* pin image versions
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* remove unnecessary workflow inputs
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
---------
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* add refStream input
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* remove inputs.yml dep
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* increase system metric period
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
* fix linkchecker
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
---------
Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
Co-authored-by: Daniel Weiße <66256922+daniel-weisse@users.noreply.github.com>
Co-authored-by: Thomas Tendyck <51411342+thomasten@users.noreply.github.com>
* Replace UpdateAttestationConfig with ApplyJoinConfig
* Dont set up join-config over Helm, it is now only managed by our CLI directly during init and upgrade
* Remove measurementSalt and attestationConfig parsing from helm, they were only needed for the JoinConfig
* Add migration step to remove join-config from Helm management
* Update attestation config trouble shooting tip
---------
Signed-off-by: Daniel Weiße <dw@edgeless.systems>
* Remove `--config` and `--master-secret` falgs
* Add `--workspace` flag
* In CLI, only work on files with paths created from `cli/internal/cmd`
* Properly print values for GCP on IAM create when not directly updating the config
---------
Signed-off-by: Daniel Weiße <dw@edgeless.systems>
* add new iam upgrade apply
* remove iam tf plan from upgrade apply check
* add iam migration warning to upgrade apply
* update release process
* document migration
* Apply suggestions from code review
Co-authored-by: Otto Bittner <cobittner@posteo.net>
* add iam upgrade
* remove upgrade dir check in test
* ask only without --yes
* make iam upgrade provider specific
* test without seperate logins
* remove csi and only add conditionally
* Revert "test without seperate logins"
This reverts commit 05a12e59c9.
* fix msising cred
* support iam migration for all csps
* add iam upgrade label
---------
Co-authored-by: Otto Bittner <cobittner@posteo.net>
* add current chart
add current helm chart
* disable service controller for aws ccm
* add new iam roles
* doc AWS internet LB + add to LB test
* pass clusterName to helm for AWS LB
* fix update-aws-lb chart to also include .helmignore
* move chart outside services
* working state
* add subnet tags for AWS subnet discovery
* fix .helmignore load rule with file in subdirectory
* upgrade iam profile
* revert new loader impl since cilium is not correctly loaded
* install chart if not already present during `upgrade apply`
* cleanup PR + fix build + add todos
cleanup PR + add todos
* shared helm pkg for cli install and bootstrapper
* add link to eks docs
* refactor iamMigrationCmd
* delete unused helm.symwallk
* move iammigrate to upgrade pkg
* fixup! delete unused helm.symwallk
* add to upgradecheck
* remove nodeSelector from go code (Otto)
* update iam docs and sort permission + remove duplicate roles
* fix bug in `upgrade check`
* better upgrade check output when svc version upgrade not possible
* pr feedback
* remove force flag in upgrade_test
* use upgrader.GetUpgradeID instead of extra type
* remove todos + fix check
* update doc lb (leo)
* remove bootstrapper helm package
* Update cli/internal/cmd/upgradecheck.go
Co-authored-by: Daniel Weiße <66256922+daniel-weisse@users.noreply.github.com>
* final nits
* add docs for e2e upgrade test setup
* Apply suggestions from code review
Co-authored-by: Daniel Weiße <66256922+daniel-weisse@users.noreply.github.com>
* Update cli/internal/helm/loader.go
Co-authored-by: Daniel Weiße <66256922+daniel-weisse@users.noreply.github.com>
* Update cli/internal/cmd/tfmigrationclient.go
Co-authored-by: Daniel Weiße <66256922+daniel-weisse@users.noreply.github.com>
* fix daniel review
* link to the iam permissions instead of manually updating them (agreed with leo)
* disable iam upgrade in upgrade apply
---------
Co-authored-by: Daniel Weiße <66256922+daniel-weisse@users.noreply.github.com>
Co-authored-by: Malte Poll
* cli: add "--skip-helm-wait" flag
This flag can be used to disable the atomic and wait flags during helm install.
This is useful when debugging a failing constellation init, since the user gains access to
the cluster even when one of the deployments is never in a ready state.
* helm: configure GCP cloud controller manager to search in all zones of a region
See also: d716fdd452/providers/gce/gce.go (L376-L380)
* operators: add nodeGroupName to ScalingGroup CRD
NodeGroupName is the human friendly name of the node group that will be exposed to customers via the Constellation config in the future.
* operators: support simple executor / scheduler to reconcile on non-k8s resources
* operators: add new return type for ListScalingGroups to support arbitrary node groups
* operators: ListScalingGroups should return additionally created node groups on AWS
* operators: ListScalingGroups should return additionally created node groups on Azure
* operators: ListScalingGroups should return additionally created node groups on GCP
* operators: ListScalingGroups should return additionally created node groups on unsupported CSPs
* operators: implement external scaling group reconciler
This controller scans the cloud provider infrastructure and changes k8s resources accordingly.
It creates ScaleSet resources when new node groups are created and deletes them if the node groups are removed.
* operators: no longer create scale sets when the operator starts
In the future, scale sets are created dynamically.
* operators: watch for node join/leave events using a controller
* operators: deploy new controllers
* docs: update auto scaling documentation with support for node groups
* init
* make zone flag mandatory again
* add info about zone update + refactor
* add comment in docs about zone update
* Update cli/internal/cmd/iamcreate_test.go
Co-authored-by: Thomas Tendyck <51411342+thomasten@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Thomas Tendyck <51411342+thomasten@users.noreply.github.com>
* thomas feedback
* add format check to config validation
* remove TODO
* Update docs/docs/workflows/config.md
Co-authored-by: Thomas Tendyck <51411342+thomasten@users.noreply.github.com>
* thomas nit
---------
Co-authored-by: Thomas Tendyck <51411342+thomasten@users.noreply.github.com>
* docs: describe SEV-SNP support on AWS
* config: remove launchMeasurement
awsSEVSNP attestation config should not have this value.
It doesn't have a function yet.
* config: move AMD root key to global constant
* attestation: add SNP based attestation for aws
* Always enable SNP, regardless of attestation type.
* Make AWSNitroTPM default again
There exists a bug in AWS SNP implementation where sometimes
a host might not be able to produce valid SNP reports.
Since we have to wait for AWS to fix this we are merging SNP
attestation as opt-in feature.
* invalidate app client id field for azure and provide info
* remove TestNewWithDefaultOptions case
* fix test
* remove appClientID field
* remove client secret + rename err
* remove from docs
* otto feedback
* update docs
* delete env test in cfg since no envs set anymore
* Update dev-docs/workflows/github-actions.md
Co-authored-by: Otto Bittner <cobittner@posteo.net>
* WARNING to stderr
* fix check
---------
Co-authored-by: Otto Bittner <cobittner@posteo.net>
* variant: move into internal/attestation
* attesation: move aws attesation into subfolder nitrotpm
* config: add aws-sev-snp variant
* cli: add tf option to enable AWS SNP
For now the implementations in aws/nitrotpm and aws/snp
are identical. They both contain the aws/nitrotpm impl.
A separate commit will add the actual attestation logic.
Incorporate customer feedback regarding the recommended commands when upgrading a Constellation cluster.
Showing the full command "constellation upgrade check --write-config" is important to ensure only valid, safe upgrades are applied.
Co-authored-by: Otto Bittner <cobittner@posteo.net>
