constellation/.github/actions/constellation_create/action.yml

284 lines
10 KiB
YAML
Raw Normal View History

name: Constellation create
ci: add e2e test for self-managed infrastructure (#2472) * add self-managed infra e2e test * self-managed terminatio Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix upgrade test Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix indentation Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * use -r when copying dir Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add terraform variable parsing * copy constellation conf Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * remove unnecessary line breaks * add missing value Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add image fetching for CSP Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix quoting Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add missing input to internal lb test * normalize Azure URLs.. Of course * tidy Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix expressions * initsecret to hex * update hexdump cmd * add build test Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add node / pod cidr outputs Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * explicitly delete the state file Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add missing license header Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * always write all outputs Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix list output Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * remove state-file and admin-conf on destroy * dont use test payload Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * [remove] use self managed infra in manual e2e for testing Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * init: always skip infrastructure phase * patch maa in workflow Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * default to Constellation-created infra in e2e test --------- Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
2023-10-27 03:37:26 -04:00
description: Create a new Constellation cluster using the latest OS image.
inputs:
workerNodesCount:
description: "Number of worker nodes to spawn."
required: true
controlNodesCount:
description: "Number of control-plane nodes to spawn."
required: true
cloudProvider:
description: "Either 'gcp', 'aws' or 'azure'."
required: true
attestationVariant:
description: "Attestation variant to use."
required: true
machineType:
description: "Machine type of VM to spawn."
required: false
cliVersion:
description: "Version of the CLI"
required: true
osImage:
2022-11-23 03:41:42 -05:00
description: "OS image to use."
required: true
isDebugImage:
description: "Is OS img a debug img?"
required: true
kubernetesVersion:
description: "Kubernetes version to create the cluster from."
required: false
artifactNameSuffix:
description: "Suffix for artifact naming."
required: true
fetchMeasurements:
default: "false"
description: "Update measurements via the 'constellation config fetch-measurements' command."
azureSNPEnforcementPolicy:
required: false
description: "Azure SNP enforcement policy."
test:
description: "The e2e test payload."
required: true
ci: collect cluster metrics to OpenSearch (#2347) * add Metricbeat deployment to debugd Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * set metricbeat debugd image version Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix k8s deployment Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * use 2 separate deployments Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * only deploy via k8s in non-debug-images Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add missing tilde * remove k8s metrics Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * unify flag Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add cloud metadata processor to filebeat Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * ci: fix debugd logcollection (#2355) * add missing keyvault access role Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * bump logstash image version Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * bump filebeat / metricbeat image version Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * log used image version Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * use debugging image versions Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * increase wait timeout for image upload * add cloud metadata processor to filebeat Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix template locations in container Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix image version typo Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add filebeat / metricbeat users Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * remove user additions Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * update workflow step name Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * only mount config files Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * document potential rc Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix IAM permissions in workflow Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix AWS permissions Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * tidy Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add missing workflow input Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * rename action Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * pin image versions Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * remove unnecessary workflow inputs Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> --------- Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add refStream input Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * remove inputs.yml dep Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * increase system metric period Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix linkchecker Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> --------- Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
2023-09-27 10:17:31 -04:00
azureClusterCreateCredentials:
description: "Azure credentials authorized to create a Constellation cluster."
required: true
azureIAMCreateCredentials:
description: "Azure credentials authorized to create an IAM configuration."
required: true
refStream:
description: "Reference and stream of the image in use"
required: false
internalLoadBalancer:
description: "Whether to use an internal load balancer for the control plane"
required: false
clusterCreation:
description: "How to create infrastructure for the e2e test. One of [cli, terraform]."
default: "cli"
terraform: Azure Marketplace image support (#2651) * terraform: add Azure marketplace variable Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * config: add Azure marketplace variable Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * cli: use Terraform variables from config Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * terraform: pass down marketplace variable * image: pad Azure images to 1GiB * terraform: add version attribute to marketplace image * semver: allow versions to be exported without prefix * cli: boolean var to use marketplace images * config: remove dive key * dev-docs: add instructions on how to use marketplace images * terraform: fix unit test * terraform: only fetch image for non-marketplace images * mpimage: refactor image selection Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * [remove] increase minor version for image build Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * terraform: ignore changes to source_image_reference on upgrade * operator: add support for parsing Azure marketplace images Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * upgrade: fix imagefetcher call * docs: add info about azure marketplace * image: ensure more than 1GiB in size * image: test to pad to 2GiB * version: change back to v2.14.0-pre * image: GPT-conformant image size padding * [remove] increase version * mpimage: inline prefix func Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * ci: add marketplace image e2e test Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * [remove] register workflow * ci: fix workflow name * ci: only allow azure test * cli: add marketplace image input to interface * cli: fix argument passing * version: roll back to v2.14.0 * ci: add force-flag support * Update docs/docs/overview/license.md * Update dev-docs/workflows/marketplace-images.md Co-authored-by: Moritz Eckert <m1gh7ym0@gmail.com> --------- Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> Co-authored-by: Moritz Eckert <m1gh7ym0@gmail.com> Co-authored-by: Thomas Tendyck <51411342+thomasten@users.noreply.github.com>
2023-12-08 08:40:31 -05:00
marketplaceImageVersion:
description: "Marketplace OS image version. Used instead of osImage."
required: false
force:
description: "Set the force-flag on apply to ignore version mismatches."
required: false
encryptionSecret:
2023-12-20 09:17:49 -05:00
description: "The secret to use for encrypting the artifact."
required: true
outputs:
kubeconfig:
description: "The kubeconfig for the cluster."
value: ${{ steps.get-kubeconfig.outputs.KUBECONFIG }}
osImageUsed:
description: "The OS image used in the cluster."
value: ${{ steps.setImage.outputs.image }}
runs:
using: "composite"
steps:
- name: Set constellation name
shell: bash
run: |
yq eval -i "(.name) = \"e2e-test\"" constellation-conf.yaml
- name: Set Azure SNP enforcement policy
if: inputs.azureSNPEnforcementPolicy != ''
shell: bash
run: |
if [[ ${{ inputs.attestationVariant }} != 'azure-sev-snp' ]]; then
echo "SNP enforcement policy is only supported for Azure"
exit 1
fi
yq eval -i "(.attestation.azureSEVSNP.firmwareSignerConfig.enforcementPolicy) \
= \"${{ inputs.azureSNPEnforcementPolicy }}\"" constellation-conf.yaml
- name: Set image
id: setImage
shell: bash
env:
imageInput: ${{ inputs.osImage }}
run: |
if [[ -z "${imageInput}" ]]; then
echo "No image specified. Using default image from config."
image=$(yq eval ".image" constellation-conf.yaml)
echo "image=${image}" | tee -a "$GITHUB_OUTPUT"
exit 0
fi
yq eval -i "(.image) = \"${imageInput}\"" constellation-conf.yaml
echo "image=${imageInput}" | tee -a "$GITHUB_OUTPUT"
- name: Set marketplace image flag (AWS)
if: inputs.marketplaceImageVersion != '' && inputs.cloudProvider == 'aws'
shell: bash
run: |
yq eval -i "(.provider.aws.useMarketplaceImage) = true" constellation-conf.yaml
yq eval -i "(.image) = \"${{ inputs.marketplaceImageVersion }}\"" constellation-conf.yaml
terraform: Azure Marketplace image support (#2651) * terraform: add Azure marketplace variable Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * config: add Azure marketplace variable Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * cli: use Terraform variables from config Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * terraform: pass down marketplace variable * image: pad Azure images to 1GiB * terraform: add version attribute to marketplace image * semver: allow versions to be exported without prefix * cli: boolean var to use marketplace images * config: remove dive key * dev-docs: add instructions on how to use marketplace images * terraform: fix unit test * terraform: only fetch image for non-marketplace images * mpimage: refactor image selection Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * [remove] increase minor version for image build Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * terraform: ignore changes to source_image_reference on upgrade * operator: add support for parsing Azure marketplace images Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * upgrade: fix imagefetcher call * docs: add info about azure marketplace * image: ensure more than 1GiB in size * image: test to pad to 2GiB * version: change back to v2.14.0-pre * image: GPT-conformant image size padding * [remove] increase version * mpimage: inline prefix func Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * ci: add marketplace image e2e test Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * [remove] register workflow * ci: fix workflow name * ci: only allow azure test * cli: add marketplace image input to interface * cli: fix argument passing * version: roll back to v2.14.0 * ci: add force-flag support * Update docs/docs/overview/license.md * Update dev-docs/workflows/marketplace-images.md Co-authored-by: Moritz Eckert <m1gh7ym0@gmail.com> --------- Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> Co-authored-by: Moritz Eckert <m1gh7ym0@gmail.com> Co-authored-by: Thomas Tendyck <51411342+thomasten@users.noreply.github.com>
2023-12-08 08:40:31 -05:00
- name: Set marketplace image flag (Azure)
if: inputs.marketplaceImageVersion != '' && inputs.cloudProvider == 'azure'
shell: bash
run: |
yq eval -i "(.provider.azure.useMarketplaceImage) = true" constellation-conf.yaml
yq eval -i "(.image) = \"${{ inputs.marketplaceImageVersion }}\"" constellation-conf.yaml
- name: Set marketplace image flag (GCP)
if: inputs.marketplaceImageVersion != '' && inputs.cloudProvider == 'gcp'
shell: bash
run: |
yq eval -i "(.provider.gcp.useMarketplaceImage) = true" constellation-conf.yaml
yq eval -i "(.image) = \"${{ inputs.marketplaceImageVersion }}\"" constellation-conf.yaml
- name: Update measurements for non-stable images
if: inputs.fetchMeasurements
shell: bash
run: |
constellation config fetch-measurements --debug --insecure
- name: Set instanceType
if: inputs.machineType && inputs.machineType != 'default'
shell: bash
run: |
yq eval -i "(.nodeGroups[] | .instanceType) = \"${{ inputs.machineType }}\"" constellation-conf.yaml
- name: Set node count
shell: bash
run: |
yq eval -i "(.nodeGroups[] | select(.role == \"control-plane\") | .initialCount) = ${{ inputs.controlNodesCount }}" constellation-conf.yaml
yq eval -i "(.nodeGroups[] | select(.role == \"worker\") | .initialCount) = ${{ inputs.workerNodesCount }}" constellation-conf.yaml
- name: Enable debugCluster flag
if: inputs.isDebugImage == 'true'
shell: bash
run: |
yq eval -i '(.debugCluster) = true' constellation-conf.yaml
- name: Enable internalLoadBalancer flag
if: inputs.internalLoadBalancer == 'true'
shell: bash
run: |
yq eval -i '(.internalLoadBalancer) = true' constellation-conf.yaml
ci: add e2e test for self-managed infrastructure (#2472) * add self-managed infra e2e test * self-managed terminatio Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix upgrade test Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix indentation Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * use -r when copying dir Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add terraform variable parsing * copy constellation conf Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * remove unnecessary line breaks * add missing value Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add image fetching for CSP Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix quoting Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add missing input to internal lb test * normalize Azure URLs.. Of course * tidy Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix expressions * initsecret to hex * update hexdump cmd * add build test Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add node / pod cidr outputs Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * explicitly delete the state file Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add missing license header Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * always write all outputs Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix list output Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * remove state-file and admin-conf on destroy * dont use test payload Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * [remove] use self managed infra in manual e2e for testing Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * init: always skip infrastructure phase * patch maa in workflow Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * default to Constellation-created infra in e2e test --------- Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
2023-10-27 03:37:26 -04:00
- name: Show Cluster Configuration
shell: bash
run: |
echo "Creating cluster using config:"
cat constellation-conf.yaml
2022-12-13 04:07:09 -05:00
sudo sh -c 'echo "127.0.0.1 license.confidential.cloud" >> /etc/hosts' || true
ci: add e2e test for self-managed infrastructure (#2472) * add self-managed infra e2e test * self-managed terminatio Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix upgrade test Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix indentation Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * use -r when copying dir Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add terraform variable parsing * copy constellation conf Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * remove unnecessary line breaks * add missing value Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add image fetching for CSP Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix quoting Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add missing input to internal lb test * normalize Azure URLs.. Of course * tidy Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix expressions * initsecret to hex * update hexdump cmd * add build test Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add node / pod cidr outputs Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * explicitly delete the state file Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add missing license header Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * always write all outputs Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix list output Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * remove state-file and admin-conf on destroy * dont use test payload Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * [remove] use self managed infra in manual e2e for testing Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * init: always skip infrastructure phase * patch maa in workflow Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * default to Constellation-created infra in e2e test --------- Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
2023-10-27 03:37:26 -04:00
- name: Constellation create (CLI)
shell: bash
run: |
constellation apply --skip-phases=init,attestationconfig,certsans,helm,image,k8s -y --debug --tf-log=DEBUG
- name: Cdbg deploy
if: inputs.isDebugImage == 'true'
ci: collect cluster metrics to OpenSearch (#2347) * add Metricbeat deployment to debugd Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * set metricbeat debugd image version Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix k8s deployment Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * use 2 separate deployments Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * only deploy via k8s in non-debug-images Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add missing tilde * remove k8s metrics Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * unify flag Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add cloud metadata processor to filebeat Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * ci: fix debugd logcollection (#2355) * add missing keyvault access role Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * bump logstash image version Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * bump filebeat / metricbeat image version Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * log used image version Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * use debugging image versions Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * increase wait timeout for image upload * add cloud metadata processor to filebeat Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix template locations in container Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix image version typo Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add filebeat / metricbeat users Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * remove user additions Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * update workflow step name Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * only mount config files Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * document potential rc Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix IAM permissions in workflow Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix AWS permissions Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * tidy Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add missing workflow input Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * rename action Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * pin image versions Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * remove unnecessary workflow inputs Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> --------- Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add refStream input Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * remove inputs.yml dep Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * increase system metric period Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix linkchecker Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> --------- Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
2023-09-27 10:17:31 -04:00
uses: ./.github/actions/cdbg_deploy
with:
cloudProvider: ${{ inputs.cloudProvider }}
attestationVariant: ${{ inputs.attestationVariant }}
ci: collect cluster metrics to OpenSearch (#2347) * add Metricbeat deployment to debugd Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * set metricbeat debugd image version Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix k8s deployment Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * use 2 separate deployments Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * only deploy via k8s in non-debug-images Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add missing tilde * remove k8s metrics Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * unify flag Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add cloud metadata processor to filebeat Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * ci: fix debugd logcollection (#2355) * add missing keyvault access role Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * bump logstash image version Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * bump filebeat / metricbeat image version Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * log used image version Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * use debugging image versions Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * increase wait timeout for image upload * add cloud metadata processor to filebeat Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix template locations in container Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix image version typo Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add filebeat / metricbeat users Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * remove user additions Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * update workflow step name Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * only mount config files Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * document potential rc Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix IAM permissions in workflow Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix AWS permissions Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * tidy Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add missing workflow input Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * rename action Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * pin image versions Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * remove unnecessary workflow inputs Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> --------- Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add refStream input Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * remove inputs.yml dep Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * increase system metric period Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix linkchecker Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> --------- Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
2023-09-27 10:17:31 -04:00
test: ${{ inputs.test }}
azureClusterCreateCredentials: ${{ inputs.azureClusterCreateCredentials }}
azureIAMCreateCredentials: ${{ inputs.azureIAMCreateCredentials }}
refStream: ${{ inputs.refStream }}
kubernetesVersion: ${{ inputs.kubernetesVersion }}
clusterCreation: ${{ inputs.clusterCreation }}
terraform: Azure Marketplace image support (#2651) * terraform: add Azure marketplace variable Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * config: add Azure marketplace variable Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * cli: use Terraform variables from config Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * terraform: pass down marketplace variable * image: pad Azure images to 1GiB * terraform: add version attribute to marketplace image * semver: allow versions to be exported without prefix * cli: boolean var to use marketplace images * config: remove dive key * dev-docs: add instructions on how to use marketplace images * terraform: fix unit test * terraform: only fetch image for non-marketplace images * mpimage: refactor image selection Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * [remove] increase minor version for image build Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * terraform: ignore changes to source_image_reference on upgrade * operator: add support for parsing Azure marketplace images Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * upgrade: fix imagefetcher call * docs: add info about azure marketplace * image: ensure more than 1GiB in size * image: test to pad to 2GiB * version: change back to v2.14.0-pre * image: GPT-conformant image size padding * [remove] increase version * mpimage: inline prefix func Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * ci: add marketplace image e2e test Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * [remove] register workflow * ci: fix workflow name * ci: only allow azure test * cli: add marketplace image input to interface * cli: fix argument passing * version: roll back to v2.14.0 * ci: add force-flag support * Update docs/docs/overview/license.md * Update dev-docs/workflows/marketplace-images.md Co-authored-by: Moritz Eckert <m1gh7ym0@gmail.com> --------- Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> Co-authored-by: Moritz Eckert <m1gh7ym0@gmail.com> Co-authored-by: Thomas Tendyck <51411342+thomasten@users.noreply.github.com>
2023-12-08 08:40:31 -05:00
- name: Set force flag
id: set-force-flag
if: inputs.force == 'true'
shell: bash
run: |
echo "flag=--force" | tee -a $GITHUB_OUTPUT
- name: Set conformance flag
id: set-conformance-flag
if: inputs.test == 'sonobuoy conformance'
shell: bash
run: |
echo "flag=--conformance" | tee -a $GITHUB_OUTPUT
- name: Constellation apply (Terraform)
id: constellation-apply-terraform
if: inputs.clusterCreation == 'terraform'
uses: ./.github/actions/terraform_apply
with:
cloudProvider: ${{ inputs.cloudProvider }}
- name: Constellation apply
id: constellation-apply-cli
if: inputs.clusterCreation != 'terraform'
shell: bash
run: |
constellation apply --skip-phases=infrastructure --debug ${{ steps.set-force-flag.outputs.flag }} ${{ steps.set-conformance-flag.outputs.flag }}
- name: Get kubeconfig
id: get-kubeconfig
shell: bash
run: |
echo "KUBECONFIG=$(pwd)/constellation-admin.conf" | tee -a $GITHUB_OUTPUT
- name: Wait for nodes to join and become ready
shell: bash
env:
KUBECONFIG: "${{ steps.get-kubeconfig.outputs.KUBECONFIG }}"
JOINTIMEOUT: "1200" # 20 minutes timeout for all nodes to join
CONTROL_NODES_COUNT: "${{ inputs.controlNodesCount }}"
WORKER_NODES_COUNT: "${{ inputs.workerNodesCount }}"
run: ./.github/actions/constellation_create/wait-for-nodes.sh
- name: Download boot logs
if: always()
continue-on-error: true
shell: bash
env:
CSP: ${{ inputs.cloudProvider }}
run: |
echo "::group::Download boot logs"
cli: use state file on init and upgrade (#2395) * [wip] use state file in CLI Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> tidy Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * use state file in CLI Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> take clusterConfig from IDFile for compat Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> various fixes Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> wip Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add GCP-specific values in Helm loader test Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * remove unnecessary pointer Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * write ClusterValues in one step Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * move stub to test file Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * remove mention of id-file Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * move output to `migrateTerraform` Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * unconditional assignments converting from idFile Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * move require block in go modules file Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fall back to id file on upgrade Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * tidy Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix linter check Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add notice to remove Terraform state check on manual migration Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add `name` field Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> fix name tests Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * return early if no Terraform diff Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * tidy Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * return infrastructure state even if no diff exists Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add TODO to remove comment Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * use state-file in miniconstellation Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * cli: remove id-file (#2402) * remove id-file from `constellation create` Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add file renaming to handler * rename id-file after upgrade * use idFile on `constellation init` Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * remove id-file from `constellation verify` Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * linter fixes Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * remove id-file from `constellation mini` * remove id-file from `constellation recover` * linter fixes * remove id-file from `constellation terminate` * fix initSecret type * fix recover argument precedence * fix terminate test * generate * add TODO to remove id-file removal * Update cli/internal/cmd/init.go Co-authored-by: Adrian Stobbe <stobbe.adrian@gmail.com> * fix verify arg parse logic Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add version test Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * remove id-file from docs * add file not found log * use state-file in miniconstellation Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * remove id-file from `constellation iam destroy` Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * remove id-file from `cdbg deploy` Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> --------- Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> Co-authored-by: Adrian Stobbe <stobbe.adrian@gmail.com> * use state-file in CI Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * update orchestration docs --------- Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> Co-authored-by: Adrian Stobbe <stobbe.adrian@gmail.com>
2023-10-09 07:04:29 -04:00
CONSTELL_UID=$(yq '.infrastructure.uid' constellation-state.yaml)
case $CSP in
azure)
AZURE_RESOURCE_GROUP=$(yq eval ".provider.azure.resourceGroup" constellation-conf.yaml)
./.github/actions/constellation_create/az-logs.sh ${AZURE_RESOURCE_GROUP}
;;
gcp)
GCP_ZONE=$(yq eval ".provider.gcp.zone" constellation-conf.yaml)
./.github/actions/constellation_create/gcp-logs.sh ${GCP_ZONE} ${CONSTELL_UID}
;;
aws)
./.github/actions/constellation_create/aws-logs.sh us-east-2 ${CONSTELL_UID}
;;
esac
echo "::endgroup::"
- name: Upload boot logs
if: always() && !env.ACT
continue-on-error: true
2023-12-20 09:17:49 -05:00
uses: ./.github/actions/artifact_upload
with:
name: serial-logs-${{ inputs.artifactNameSuffix }}
2023-12-20 09:17:49 -05:00
path: >
!(terraform).log
encryptionSecret: ${{ inputs.encryptionSecret }}
- name: Prepare terraform state folders
if: always()
shell: bash
run: |
mkdir to-zip
cp -r constellation-terraform to-zip
cp -r constellation-iam-terraform to-zip
rm -f to-zip/constellation-terraform/plan.zip
rm -rf to-zip/constellation-terraform/.terraform to-zip/constellation-iam-terraform/.terraform
- name: Upload terraform state
if: always()
uses: ./.github/actions/artifact_upload
with:
name: terraform-state-${{ inputs.artifactNameSuffix }}
path: >
to-zip/constellation-terraform
to-zip/constellation-iam-terraform
encryptionSecret: ${{ inputs.encryptionSecret }}