Commit Graph

239 Commits

Author SHA1 Message Date
Malte Poll
6f9020d527 cli: use pre-uploaded image on OpenStack
Before, the terraform infrastructure code would upload an image on the fly.
Now, we upload images in advance and specify the image ID instead.
2024-02-19 18:16:45 +01:00
Malte Poll
b5e848a87e terraform: provide required configuration for yawol on OpenStack 2024-02-12 13:13:48 +01:00
Adrian Stobbe
489e07677e
ref: pre v2.15 cleanup (#2871) 2024-01-29 21:32:37 +01:00
Daniel Weiße
e350ca0f57
attestation: add Azure TDX attestation (#2827)
* Implement Azure TDX attestation primitives
* Add default measurements and claims for Azure TDX
* Enable Constellation on Azure TDX

---------

Signed-off-by: Daniel Weiße <dw@edgeless.systems>
2024-01-24 15:10:15 +01:00
Malte Poll
3a5753045e goleak: ignore rules_go SIGTERM handler
rules_go added a SIGTERM handler that has a goroutine that survives the scope of the goleak check.
Currently, the best known workaround is to ignore this goroutine.

https://github.com/uber-go/goleak/issues/119
https://github.com/bazelbuild/rules_go/pull/3749
https://github.com/bazelbuild/rules_go/pull/3827#issuecomment-1894002120
2024-01-22 13:11:58 +01:00
Adrian Stobbe
9667dfff58
terraform: align infrastructure module attributes (#2703)
* all vars have snail_case

* make iam schema consistent

* infrastructure schema

* terraform: update AWS infrastructure module

* fix ci

* terraform: update AWS infrastructure module

* terraform: update AWS IAM module

* terraform: update Azure Infrastructure module inputs

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* terraform: update Azure IAM module

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* terraform: update GCP infrastructure module

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* terraform: update GCP IAM module

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* terraform: update OpenStack Infrastructure module

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* terraform: update QEMU Infrastructure module

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* terraform-module: fix input name

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* terraform: tidy

* cli: ignore whitespace in Terraform variable tests

* terraform-module: fix AWS output names

* terraform-module: fix output references

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* terraform: rename `api_server_cert_sans`

* Update terraform/infrastructure/aws/modules/public_private_subnet/variables.tf

Co-authored-by: Daniel Weiße <66256922+daniel-weisse@users.noreply.github.com>

* fix self-managed

* terraform: revert AWS modules output file renaming

* terraform: remove duplicate varable declaration

* terraform: rename Azure location field

* ci: adjust output name in self-managed e2e test

* e2e: continuously print output in upgrade test

* e2e: write to output variables

* cli: migrate IAM variable names

* cli: make `location` field optional

---------

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
Co-authored-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
Co-authored-by: Daniel Weiße <66256922+daniel-weisse@users.noreply.github.com>
2023-12-15 10:36:58 +01:00
Moritz Sanft
6ff321364d
terraform: fix Azure marketplace image defaults (#2692)
* config: default to false instead of null for Azure marketplace image

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* terraform: make Azure marketplace image default to null

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

---------

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
2023-12-11 08:15:47 +01:00
Daniel Weiße
272584a34c
constellation-lib: move state and featureset packages (#2682)
Signed-off-by: Daniel Weiße <dw@edgeless.systems>
2023-12-08 16:27:04 +01:00
Moritz Sanft
c15e4efef6
terraform: Azure Marketplace image support (#2651)
* terraform: add Azure marketplace variable

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* config: add Azure marketplace variable

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* cli: use Terraform variables from config

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* terraform: pass down marketplace variable

* image: pad Azure images to 1GiB

* terraform: add version attribute to marketplace image

* semver: allow versions to be exported without prefix

* cli: boolean var to use marketplace images

* config: remove dive key

* dev-docs: add instructions on how to use marketplace images

* terraform: fix unit test

* terraform: only fetch image for non-marketplace images

* mpimage: refactor image selection

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* [remove] increase minor version for image build

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* terraform: ignore changes to source_image_reference on upgrade

* operator: add support for parsing Azure marketplace images

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* upgrade: fix imagefetcher call

* docs: add info about azure marketplace

* image: ensure more than 1GiB in size

* image: test to pad to 2GiB

* version: change back to v2.14.0-pre

* image: GPT-conformant image size padding

* [remove] increase version

* mpimage: inline prefix func

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* ci: add marketplace image e2e test

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* [remove] register workflow

* ci: fix workflow name

* ci: only allow azure test

* cli: add marketplace image input to interface

* cli: fix argument passing

* version: roll back to v2.14.0

* ci: add force-flag support

* Update docs/docs/overview/license.md

* Update dev-docs/workflows/marketplace-images.md

Co-authored-by: Moritz Eckert <m1gh7ym0@gmail.com>

---------

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
Co-authored-by: Moritz Eckert <m1gh7ym0@gmail.com>
Co-authored-by: Thomas Tendyck <51411342+thomasten@users.noreply.github.com>
2023-12-08 14:40:31 +01:00
Moritz Sanft
968cdc1a38
cli: move cli/internal libraries (#2623)
* cli: move internal packages

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* cli: fix buildfiles

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* bazel: fix exclude dir

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* cli: move back libraries that will not be used by TF provider

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

---------

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
2023-11-22 14:52:56 +01:00
Adrian Stobbe
cea6204b37
terraform: Terraform module for AWS (#2503) 2023-11-08 19:10:01 +01:00
Adrian Stobbe
eaec73cca4
cli: fix invalid upper case name on AWS (#2546) 2023-11-03 10:09:43 +01:00
Daniel Weiße
e4d8bda792
cli: spinner for planning Terraform migrations (#2533)
* Remove mention of "changes below" for changes that are listed above the message
* Add a spinner for Terraform Plan action

---------

Signed-off-by: Daniel Weiße <dw@edgeless.systems>
2023-10-30 09:19:35 +01:00
Daniel Weiße
af36099c70 Revert "deps: update Terraform google to v5 (#2506)"
This reverts commit 37cda7f4f2.
2023-10-27 14:23:04 +02:00
Daniel Weiße
a321f839bc Revert "deps: update Terraform google-beta to v5 (#2507)"
This reverts commit cb11c8e297.
2023-10-27 14:23:04 +02:00
Daniel Weiße
ec424b260d
cli: refactor terraform code to be update/create agnostic (#2501)
* Move upgrade specific functions out of Terraform module
* Always allow overwriting Terraform files
* Ensure constellation-terraform dir does not exist on create

---------

Signed-off-by: Daniel Weiße <dw@edgeless.systems>
2023-10-26 10:55:50 +02:00
renovate[bot]
3a8296b2f3
deps: update Terraform docker to v3 (#2508)
Co-authored-by: Paul Meyer <49727155+katexochen@users.noreply.github.com>
2023-10-26 08:19:08 +02:00
renovate[bot]
cb11c8e297
deps: update Terraform google-beta to v5 (#2507)
Co-authored-by: Paul Meyer <49727155+katexochen@users.noreply.github.com>
2023-10-26 08:18:07 +02:00
renovate[bot]
37cda7f4f2
deps: update Terraform google to v5 (#2506)
Co-authored-by: Paul Meyer <49727155+katexochen@users.noreply.github.com>
2023-10-26 08:17:32 +02:00
3u13r
e053d1fa71
terraform: always output node cidr (#2481)
* terraform: always output node cidr
2023-10-23 15:06:48 +02:00
3u13r
0bfb4f7e11
align tf output vars with CLI parsing (#2475) 2023-10-19 10:03:22 +02:00
3u13r
0c89f57ac5
Support internal load balancers (#2388)
* arch: support internal lb on Azure

* arch: support internal lb on GCP

* helm: remove lb svc from verify deployment

* arch: support internal lb on AWS

* terraform: add jump hosts for internal lb

* cli: expose internalLoadBalancer in config

* ci: add e2e-manual-internal

* add in-cluster endpoint to terraform output
2023-10-17 15:46:15 +02:00
Moritz Sanft
a8605d7294
cli: use custom byte-slice marshalling for state file (#2460)
* custom byte slice marshalling

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* byte slice compatibility

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* other byte slice compat test

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* add missing dep

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* export byte type alias

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* regenerate exported type

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* test marshal and unmarshal together

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

---------

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
2023-10-17 10:35:54 +02:00
Daniel Weiße
93b70e5633
cli: revert libvirt terraform provider upgrade (#2454)
Signed-off-by: Daniel Weiße <dw@edgeless.systems>
2023-10-16 10:43:30 +02:00
renovate[bot]
0f5039f2e9
deps: update Terraform libvirt to v0.7.4 (#2434)
Co-authored-by: Paul Meyer <49727155+katexochen@users.noreply.github.com>
2023-10-11 13:47:51 +02:00
Moritz Sanft
005e865a13
cli: use state file on init and upgrade (#2395)
* [wip] use state file in CLI

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

tidy

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* use state file in CLI

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

take clusterConfig from IDFile for compat

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

various fixes

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

wip

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* add GCP-specific values in Helm loader test

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* remove unnecessary pointer

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* write ClusterValues in one step

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* move stub to test file

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* remove mention of id-file

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* move output to `migrateTerraform`

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* unconditional assignments converting from idFile

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* move require block in go modules file

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* fall back to id file on upgrade

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* tidy

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* fix linter check

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* add notice to remove Terraform state check on manual migration

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* add `name` field

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

fix name tests

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* return early if no Terraform diff

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* tidy

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* return infrastructure state even if no diff exists

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* add TODO to remove comment

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* use state-file in miniconstellation

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* cli: remove id-file (#2402)

* remove id-file from `constellation create`

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* add file renaming to handler

* rename id-file after upgrade

* use idFile on `constellation init`

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* remove id-file from `constellation verify`

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* linter fixes

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* remove id-file from `constellation mini`

* remove id-file from `constellation recover`

* linter fixes

* remove id-file from `constellation terminate`

* fix initSecret type

* fix recover argument precedence

* fix terminate test

* generate

* add TODO to remove id-file removal

* Update cli/internal/cmd/init.go

Co-authored-by: Adrian Stobbe <stobbe.adrian@gmail.com>

* fix verify arg parse logic

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* add version test

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* remove id-file from docs

* add file not found log

* use state-file in miniconstellation

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* remove id-file from `constellation iam destroy`

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* remove id-file from `cdbg deploy`

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

---------

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
Co-authored-by: Adrian Stobbe <stobbe.adrian@gmail.com>

* use state-file in CI

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* update orchestration docs

---------

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
Co-authored-by: Adrian Stobbe <stobbe.adrian@gmail.com>
2023-10-09 13:04:29 +02:00
3u13r
1452e64675
Refactor Terraform to have all ports in a list (#2409)
* terraform: aws refactoring

* terraform: gcp refactoring

* terraform: azure refactoring
2023-10-05 12:34:02 +02:00
Malte Poll
daa5b51904 terraform: disable secure boot for GCP 2023-09-27 17:58:19 +02:00
Malte Poll
4e2b9745bb terraform: disable secure boot for QEMU / MiniConstellation 2023-09-27 17:58:19 +02:00
renovate[bot]
6445c11110
deps: update Terraform aws to v5.17.0 (#2366)
Co-authored-by: Paul Meyer <49727155+katexochen@users.noreply.github.com>
2023-09-26 15:04:32 +02:00
renovate[bot]
5cfa91f3db
deps: update Terraform azurerm to v3.74.0 (#2368)
Co-authored-by: Paul Meyer <49727155+katexochen@users.noreply.github.com>
2023-09-26 13:17:17 +02:00
renovate[bot]
dffdc37004
deps: update Terraform azuread to v2.43.0 (#2367)
Co-authored-by: Paul Meyer <49727155+katexochen@users.noreply.github.com>
2023-09-26 11:48:55 +02:00
Adrian Stobbe
4680882708
cli: remove TF ApplyOutput dependency in CLI (#2323) 2023-09-25 17:10:23 +02:00
Adrian Stobbe
322c4aad10
cli: write infrastructure to new state file (#2321)
Co-authored-by: 3u13r <lc@edgeless.systems>
2023-09-25 16:19:43 +02:00
3u13r
2776e40df7
join: join over lb if available (#2348)
* join: join over lb if available
2023-09-25 10:23:35 +02:00
renovate[bot]
74fe8af8e6
deps: update Terraform google-beta to v4.83.0 new (#2345)
Co-authored-by: Leonard Cohnen <lc@edgeless.systems>
2023-09-22 10:37:19 +02:00
Moritz Sanft
dc837eddb5
cli: temporarily increase AWS ASG creation timeout (#2340)
* increase ASG timeout

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* make timeout dependent on SEV-SNP option

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

---------

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
2023-09-20 11:25:02 +02:00
renovate[bot]
82e561f139
deps: update Terraform google to v4.83.0 (#2344) 2023-09-19 15:17:21 +02:00
Moritz Sanft
95cf4bdf21
cli: perform upgrades in-place in Terraform workspace (#2317)
* perform upgrades in-place in terraform workspace

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* update buildfiles

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* add iam upgrade apply test

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* update buildfiles

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* fix linter

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* make config fetcher stubbable

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* change workspace restoring behaviour

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* allow overwriting existing Terraform files

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* allow overwrites of TF variables

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* fix iam upgrade apply

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* fix embed directive

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* make loader test less brittle

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* pass upgrade ID to user

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* naming nit

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* use upgradeDir

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* tidy

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

---------

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
2023-09-14 11:51:20 +02:00
3u13r
a25c90e9bb
remove deprecated constellation create flags (#2325)
* chore: clean-up TODOs

* cli: make OpenStack error explicit

* cli: remove deprecated flags

* config: require DeployCSIDriver field
2023-09-08 21:15:02 +02:00
Otto Bittner
6e5ba774d8 cli: disable nosmt via VMM temporarily.
AWS asked us to disable these options temporarily until they resolve
some internal issues that sometimes prevents these instances
from starting.
2023-09-05 08:23:18 +02:00
Otto Bittner
75ce11af14
cli: disable smt via cpu_options (#2291)
Disabling SMT dynamically inside the image creates problems on AWS.
The problem should be fixed by disabling smt through the VMM.
By recommendation from AWS: add idle=poll.
This should improve our launch success rate while they investigate some
upstream issues.
2023-09-01 11:26:21 +02:00
Daniel Weiße
0a911806d1
cli: remove/refactor upgrade package (#2266)
* Move IAM migration client to cloudcmd package

* Move Terraform Cluster upgrade client to cloudcmd package

* Use hcl for creating Terraform IAM variables files

* Unify terraform upgrade code

* Rename some cloudcmd files for better clarity

---------

Signed-off-by: Daniel Weiße <dw@edgeless.systems>
2023-08-23 10:35:42 +02:00
Daniel Weiße
9477999be2
cli: clean up terraform package (#2256)
* Clean up Terraform pkg

* Add note to Terraform migration functions expecting to be run on initialized workspace

---------

Signed-off-by: Daniel Weiße <dw@edgeless.systems>
2023-08-21 10:26:53 +02:00
renovate[bot]
ae7888a13f
deps: update Terraform azuread to v2.41.0 (#2254)
Co-authored-by: Paul Meyer <49727155+katexochen@users.noreply.github.com>
2023-08-17 10:29:49 +02:00
Thomas Tendyck
587ae6a575
deps: limit Terraform version to FOSS releases (#2241)
* deps: limit Terraform version to FOSS releases

* fix: enforce upper version constraint

---------

Co-authored-by: Leonard Cohnen <lc@edgeless.systems>
2023-08-16 23:25:53 +02:00
3u13r
8c321ec1ab
cli: add role to aws instance name (#2130) 2023-08-14 13:42:20 +02:00
Daniel Weiße
e30179a8aa Remove manual state migration steps for AWS
Signed-off-by: Daniel Weiße <dw@edgeless.systems>
2023-08-10 15:46:59 +02:00
Daniel Weiße
99c579b45a Add package design goals to CLI package documentation
Signed-off-by: Daniel Weiße <dw@edgeless.systems>
2023-08-09 15:42:24 +02:00
Daniel Weiße
23394ea2e2
cli: fix missing safety check in ShowIAM (#2165)
* Add missing safety check to ShowIAM

* someErr->assert.AnError

---------

Signed-off-by: Daniel Weiße <dw@edgeless.systems>
2023-08-09 15:25:59 +02:00