Commit Graph

187 Commits

Author SHA1 Message Date
Malte Poll
d0ec7a3e54 terraform: move OpenStack flavorID into nodeGroups 2023-08-04 12:36:45 +02:00
3u13r
720c48ea45
cli: refactor terraform output parsing (#2158) 2023-08-03 16:17:23 +02:00
Adrian Stobbe
70ce195a5f
cli: unify chart value setup (#2153) 2023-08-03 13:54:48 +02:00
3u13r
5119d843f1
terraform: fix uami parsing (#2155) 2023-08-03 13:22:26 +02:00
renovate[bot]
7e3123232e
deps: update Terraform azurerm to v3.67.0 (#2147)
Also update Azure terraform: 
ignore snp policy changes on resource

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: edgelessci <edgelessci@users.noreply.github.com>
Co-authored-by: Otto Bittner <cobittner@posteo.net>
2023-08-02 08:15:22 +02:00
Adrian Stobbe
26480016a9
azure: fix ccm config with correct uami client_id (#2144)
* fix azure ccm config with correct uami client_id

* fix tests
2023-08-01 08:40:44 +02:00
Adrian Stobbe
26305e8f80
cli: install helm charts in cli instead of bootstrapper (#2136)
* init

* fixup! init

* gcp working?

* fixup! fixup! init

* azure cfg for microService installation

* fixup! azure cfg for microService installation

* fixup! azure cfg for microService installation

* cleanup bootstrapper code

* cleanup helminstall code

* fixup! cleanup helminstall code

* Update internal/deploy/helm/install.go

Co-authored-by: Daniel Weiße <66256922+daniel-weisse@users.noreply.github.com>

* daniel feedback

* TODO add provider (also to CreateCluster) so we can ensure that provider specific output

* fixup! daniel feedback

* use debugLog in helm installer

* placeholderHelmInstaller

* rename to stub

---------

Co-authored-by: Daniel Weiße <66256922+daniel-weisse@users.noreply.github.com>
2023-07-31 10:53:05 +02:00
Otto Bittner
7db058f946
cli: wait for public ip update before reading fqdn (#2135) 2023-07-26 15:23:37 +02:00
Adrian Stobbe
a87b7894db
aws: use new LB controller to fix SecurityGroup cleanup on K8s service deletion (#2090)
* add current chart

add current helm chart

* disable service controller for aws ccm

* add new iam roles

* doc AWS internet LB + add to LB test

* pass clusterName to helm for AWS LB

* fix update-aws-lb chart to also include .helmignore

* move chart outside services

* working state

* add subnet tags for AWS subnet discovery

* fix .helmignore load rule with file in subdirectory

* upgrade iam profile

* revert new loader impl since cilium is not correctly loaded

* install chart if not already present during `upgrade apply`

* cleanup PR + fix build + add todos

cleanup PR + add todos

* shared helm pkg for cli install and bootstrapper

* add link to eks docs

* refactor iamMigrationCmd

* delete unused helm.symwallk

* move iammigrate to upgrade pkg

* fixup! delete unused helm.symwallk

* add to upgradecheck

* remove nodeSelector from go code (Otto)

* update iam docs and sort permission + remove duplicate roles

* fix bug in `upgrade check`

* better upgrade check output when svc version upgrade not possible

* pr feedback

* remove force flag in upgrade_test

* use upgrader.GetUpgradeID instead of extra type

* remove todos + fix check

* update doc lb (leo)

* remove bootstrapper helm package

* Update cli/internal/cmd/upgradecheck.go

Co-authored-by: Daniel Weiße <66256922+daniel-weisse@users.noreply.github.com>

* final nits

* add docs for e2e upgrade test setup

* Apply suggestions from code review

Co-authored-by: Daniel Weiße <66256922+daniel-weisse@users.noreply.github.com>

* Update cli/internal/helm/loader.go

Co-authored-by: Daniel Weiße <66256922+daniel-weisse@users.noreply.github.com>

* Update cli/internal/cmd/tfmigrationclient.go

Co-authored-by: Daniel Weiße <66256922+daniel-weisse@users.noreply.github.com>

* fix daniel review

* link to the iam permissions instead of manually updating them (agreed with leo)

* disable iam upgrade in upgrade apply

---------

Co-authored-by: Daniel Weiße <66256922+daniel-weisse@users.noreply.github.com>
Co-authored-by: Malte Poll
2023-07-24 10:30:53 +02:00
Malte Poll
8da6a23aa5
bootstrapper: add fallback endpoint and custom endpoint to SAN field (#2108)
terraform: collect apiserver cert SANs and support custom endpoint

constants: add new constants for cluster configuration and custom endpoint

cloud: support apiserver cert sans and prepare for endpoint migration on AWS

config: add customEndpoint field

bootstrapper: use per-CSP apiserver cert SANs

cli: route customEndpoint to terraform and add migration for apiserver cert SANs

bootstrapper: change interface of GetLoadBalancerEndpoint to return host and port separately
2023-07-21 16:43:51 +02:00
Otto Bittner
cf822f7eee
cli: unify terraform variable creation (#2119)
Before we defined the variables twice.
Once for upgrades, once for create.
Also move default node group names into a constant
2023-07-21 10:04:29 +02:00
Otto Bittner
c2849f4bbe
cli: ignore name changes on lb public ip resource (#2117)
Changing the name forces a recreate, which would break existing clusters.
The name change seems to be "only" about having clearer names.
2023-07-19 10:15:23 +02:00
Malte Poll
5cbdb3a519
terraform: allows cluster name length of 10 characters on AWS (#2110) 2023-07-17 17:45:41 +02:00
renovate[bot]
f364bd6b9d
deps: update Terraform google-beta to v4.72.0 (#2027)
Co-authored-by: Paul Meyer <49727155+katexochen@users.noreply.github.com>
2023-07-06 11:53:40 +02:00
renovate[bot]
3f1faead94
deps: update Terraform google to v4.72.0 (#2026)
Co-authored-by: Paul Meyer <49727155+katexochen@users.noreply.github.com>
2023-07-06 11:53:15 +02:00
renovate[bot]
ff74afa00d
deps: update Terraform azurerm to v3.63.0 (#2025)
Co-authored-by: Paul Meyer <49727155+katexochen@users.noreply.github.com>
2023-07-06 11:52:44 +02:00
renovate[bot]
bf09313dcf
deps: update Terraform aws to v5.6.2 (#2024)
Co-authored-by: Leonard Cohnen <lc@edgeless.systems>
2023-07-06 10:21:35 +02:00
renovate[bot]
e6dbb13c6c
deps: update Terraform openstack to v1.52.1 (#2028)
Co-authored-by: Paul Meyer <49727155+katexochen@users.noreply.github.com>
2023-07-05 16:52:58 +02:00
Adrian Stobbe
c39df2f7da
terraform: openstack node groups (#1966)
* openstack

* rename to base_name

* fix openstack boot vtpm

* add docs for accessing bootstrapper logs

* rename to initial count
2023-07-03 16:33:00 +02:00
Malte Poll
66f1333c31
terraform: use single zone loadbalancer frontend on AWS (#1983)
This change is required to ensure we have not tls handshake errors when connecting to the kubernetes api.
Currently, the certificates used by kube-apiserver pods contain a SAN field with the (single) public ip of the loadbalancer.
If we would allow multiple loadbalancer frontend ips, we could encounter cases where the certificate is only valid for one public ip,
while we try to connect to a different ip.
To prevent this, we consciously disable support for the multi-zone loadbalancer frontend on AWS for now.
This will be re-enabled in the future.
2023-06-30 16:56:31 +02:00
Malte Poll
5f8ea1348a
terraform: instance_count => initial_count (#1989)
Normalize naming for the "instance_count" / "initial_count" int terraform to always use "initial_count".
This is required, since there is a naming confusion on AWS.
"initial_count" is more precise, since it reflects the fact that this value is ignored when applying the terraform template
after the scaling groups already exist.
2023-06-30 10:53:00 +02:00
Malte Poll
f64e44a438 aws: support LBs in multiple zones when retrieving metadata 2023-06-28 18:13:01 +02:00
Malte Poll
3edc1c3ebb cli: manual AWS terraform state transitions
This commit is designed to be reverted in the future (AB#3248).
Terraform does not implement moved blocks with dynamic targets: https://github.com/hashicorp/terraform/issues/31335 so we have to migrate the terraform state ourselves.
2023-06-28 18:13:01 +02:00
Malte Poll
22ebdace43 terraform: aws node groups 2023-06-28 18:13:01 +02:00
Adrian Stobbe
9bb91ca447
terraform: QEMU node groups (#1961)
* init

add variables

add amount to instance_group again

fix tf validate

rollback old names

make fields optional

fix image ref mini

daniel comments

use latest

* Update cli/internal/terraform/terraform/qemu/main.tf

Co-authored-by: Malte Poll <1780588+malt3@users.noreply.github.com>

* add uid to resource name

* make machine a global variable again

* fix tf

---------

Co-authored-by: Malte Poll <1780588+malt3@users.noreply.github.com>
2023-06-28 14:42:34 +02:00
Moritz Sanft
fe0b8c1e5b
remove Terraform targets (#1970) 2023-06-27 11:27:50 +02:00
Malte Poll
92cd9c1dac
terraform: always use uniform role names (#1960) 2023-06-23 12:08:30 +02:00
Adrian Stobbe
487fa1e397
terraform: azure node groups (#1955)
* init

* migration working

* make tf variables with default value optional in go through ptr type

* fix CI build

* pr feedback

* add azure targets tf

* skip migration for empty targets

* make instance_count optional

* change role naming to dashed + add validation

* make node_group.zones optional

* Update cli/internal/terraform/terraform/azure/main.tf

Co-authored-by: Malte Poll <1780588+malt3@users.noreply.github.com>

* malte feedback

---------

Co-authored-by: Malte Poll <1780588+malt3@users.noreply.github.com>
2023-06-22 16:53:40 +02:00
Moritz Sanft
224c74f883
csi: aws csi driver policies (#1945)
* add required disk permissions

* update worker node policy for ebs

* Revert "update worker node policy for ebs"

This reverts commit 9c24d374e0b30bc8970e00978462fb36ee6acd4f.

* attach aws managed role instead

* add TODO comment

* remove duplicate role attachment

* Update cli/internal/terraform/terraform/iam/aws/main.tf

Co-authored-by: Daniel Weiße <66256922+daniel-weisse@users.noreply.github.com>

---------

Co-authored-by: Daniel Weiße <66256922+daniel-weisse@users.noreply.github.com>
2023-06-22 14:15:05 +02:00
Moritz Sanft
b25228d175
cli: store upgrade files in versioned folders (#1929)
* upgrade versioning

* dont pass upgrade kind as boolean

* whitespace

* fix godot lint check

* clarify upgrade check directory suffix

* cli: dry-run Terraform migrations on `upgrade check` (#1942)

* dry-run Terraform migrations on upgrade check

* clean whole upgrade dir

* clean up check workspace after planning

* fix parsing

* extend upgrade check test

* rename unused parameters

* exclude false positives in test
2023-06-21 09:22:32 +02:00
Malte Poll
2808012c9c
terraform: gcp node groups (#1941)
* terraform: GCP node groups

* cli: marshal GCP node groups to terraform variables

This does not have any side effects for users.
We still strictly create one control-plane and one worker group.
This is a preparation for enabling customizable node groups in the future.
2023-06-19 13:02:01 +02:00
renovate[bot]
ab52e6d4c5
fix: GCP service account creation fails sometimes (#1935)
* deps: update Terraform google to v4.69.1

* deps: tidy all modules

* add delay for service account

* deps: tidy all modules

* add delay for service account

---------

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: edgelessci <edgelessci@users.noreply.github.com>
Co-authored-by: Adrian Stobbe <stobbe.adrian@gmail.com>
2023-06-16 09:37:31 +02:00
Adrian Stobbe
07de6482b2
config: drop support for deprecated Azure's service principal authentication (#1906)
* invalidate app client id field for azure and provide info

* remove TestNewWithDefaultOptions case

* fix test

* remove appClientID field

* remove client secret + rename err

* remove from docs

* otto feedback

* update docs

* delete env test in cfg since no envs set anymore

* Update dev-docs/workflows/github-actions.md

Co-authored-by: Otto Bittner <cobittner@posteo.net>

* WARNING to stderr

* fix check

---------

Co-authored-by: Otto Bittner <cobittner@posteo.net>
2023-06-14 17:50:57 +02:00
3u13r
a2c98eb1d5
Correctly deploy the AWS CCM (#1853)
* aws: stop using the imds api for tags

* aws: disable tags in imds api

* aws: only tag instances with non-lecagy tag

* bootstrapper: always let coredns run before cilium

* debugd: make debugd less noisy

* fixup fix aws imds test

* fixup unsued context

* move getting instance id to readInstanceTag
2023-06-13 09:58:39 +02:00
Otto Bittner
8f21972aec
attestation: add awsSEVSNP as new variant (#1900)
* variant: move into internal/attestation
* attesation: move aws attesation into subfolder nitrotpm
* config: add aws-sev-snp variant
* cli: add tf option to enable AWS SNP

For now the implementations in aws/nitrotpm and aws/snp
are identical. They both contain the aws/nitrotpm impl.
A separate commit will add the actual attestation logic.
2023-06-09 15:41:02 +02:00
Adrian Stobbe
d9c604ed2c
terraform: update aws to v5.1.0 (#1891) 2023-06-09 10:37:25 +02:00
Malte Poll
439359ffbc
cli: prevent terraform apply drift when patching and re-applying existing terraform deployment (#1873)
The implementation would recreate the gcp instance template (including all instances and state disks) whenever the image tfvar changes.
Fixed by ignoring lifecycle changes on the instance templates.
Fixes 8c3b963
2023-06-05 14:52:39 +02:00
Moritz Sanft
8c3b963a3f
cli: Terraform upgrades maa patching (#1821)
* patch maa after upgrade

* buildfiles

* reword comment

* remove whitespace

* temp: log measurements URL

* temp: update import

* ignore changes to attestation policies

* add issue URL

* separate output in e2e upgrade test

* use enterprise CLI for e2e test

* remove measurements print

* add license headers
2023-06-02 10:47:44 +02:00
Adrian Stobbe
b51cc52945
config: sign Azure versions on upload & verify on fetch (#1836)
* add SignContent() + integrate into configAPI

* use static client for upload versions tool; fix staticupload calleeReference bug

* use version to get proper cosign pub key.

* mock fetcher in CLI tests

* only provide config.New constructor with fetcher

Co-authored-by: Otto Bittner <cobittner@posteo.net>
Co-authored-by: Daniel Weiße <66256922+daniel-weisse@users.noreply.github.com>
2023-06-01 13:55:46 +02:00
3u13r
e0285c122e
todo responsibilities and cleanup (#1837)
* chore: add TODO responsibilities

* chore: remove not needed TODOs

* chore: remove outdated migrations

* chore: remove resolved goleak exception

* chore: remove not needed cosign env

* config: add link to our Azure snp docs
2023-06-01 12:33:06 +02:00
Malte Poll
60b125cb59
cli: add windows amd64 build target (#1835) 2023-05-30 12:02:43 +02:00
3u13r
661f084ffa
cli: use uami for in-cluter authentication (#1820) 2023-05-26 11:45:03 +02:00
3u13r
6062b10035
cli: split image into oss and enterprise (#1788) 2023-05-23 10:49:47 +02:00
Moritz Sanft
c69e6777bd
cli: Terraform migrations on upgrade (#1685)
* add terraform planning

* overwrite terraform files in upgrade workspace

* Revert "overwrite terraform files in upgrade workspace"

This reverts commit 8bdacfb8bef23ef2cdbdb06bad0855b3bbc42df0.

* prepare terraform workspace

* test upgrade integration

* print upgrade abort

* rename plan file

* write output to file

* add show plan test

* add upgrade tf workdir

* fix workspace preparing

* squash to 1 command

* test

* bazel build

* plan test

* register flag manually

* bazel tidy

* fix linter

* remove MAA variable

* fix workdir

* accept tf variables

* variable fetching

* fix resource indices

* accept Terraform targets

* refactor upgrade command

* Terraform migration apply unit test

* pass down image fetcher to test

* use new flags in e2e test

* move file name to constant

* update buildfiles

* fix version constant

* conditionally create MAA

* move interface down

* upgrade dir

* update buildfiles

* fix interface

* fix createMAA check

* fix imports

* update buildfiles

* wip: workspace backup

* copy utils

* backup upgrade workspace

* remove debug print

* replace old state after upgrade

* check if flag exists

* prepare test workspace

* remove prefix

Co-authored-by: Otto Bittner <cobittner@posteo.net>

* respect file permissions

* refactor tf upgrader

* check workspace before upgrades

* remove temp upgrade dir after completion

* clean up workspace after abortion

* fix upgrade apply test

* fix linter

---------

Co-authored-by: Otto Bittner <cobittner@posteo.net>
2023-05-22 13:31:20 +02:00
3u13r
964775c4c2
Add autoscaling and cluster upgrade support for AWS (#1758)
* aws: autoscaling and upgrades

* docs: update scaling and upgrades for AWS

* deps: pin vuln check against release
2023-05-19 13:57:31 +02:00
3u13r
3b7bae7535
deps: bump minimum terraform version (#1797) 2023-05-18 12:59:10 +02:00
Daniel Weiße
c478df36fa Add TDX bazel files
Signed-off-by: Daniel Weiße <dw@edgeless.systems>
2023-05-17 11:37:26 +02:00
Daniel Weiße
dd2da25ebe attestation: tdx issuer/validator (#1265)
* Add TDX validator

* Add TDX issuer

---------

Signed-off-by: Daniel Weiße <dw@edgeless.systems>
2023-05-17 11:37:26 +02:00
Malte Poll
d104af6e51 image: support intel TDX direct linux boot under TDX OVMF 2023-05-17 11:37:26 +02:00
Malte Poll
79986a2b25 cli: implement qemu direct linux boot 2023-05-17 11:37:26 +02:00