constellation/.github/actions/constellation_create/action.yml

215 lines
7.5 KiB
YAML
Raw Normal View History

name: Constellation create
description: Create a new Constellation cluster using latest OS image.
inputs:
workerNodesCount:
description: "Number of worker nodes to spawn."
required: true
controlNodesCount:
description: "Number of control-plane nodes to spawn."
required: true
cloudProvider:
description: "Either 'gcp', 'aws' or 'azure'."
required: true
machineType:
description: "Machine type of VM to spawn."
required: false
cliVersion:
description: "Version of the CLI"
required: true
osImage:
2022-11-23 03:41:42 -05:00
description: "OS image to use."
required: true
isDebugImage:
description: "Is OS img a debug img?"
required: true
kubernetesVersion:
description: "Kubernetes version to create the cluster from."
required: false
artifactNameSuffix:
description: "Suffix for artifact naming."
required: true
fetchMeasurements:
default: "false"
description: "Update measurements via the 'constellation config fetch-measurements' command."
azureSNPEnforcementPolicy:
required: false
description: "Azure SNP enforcement policy."
test:
description: "The e2e test payload."
required: true
ci: collect cluster metrics to OpenSearch (#2347) * add Metricbeat deployment to debugd Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * set metricbeat debugd image version Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix k8s deployment Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * use 2 separate deployments Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * only deploy via k8s in non-debug-images Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add missing tilde * remove k8s metrics Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * unify flag Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add cloud metadata processor to filebeat Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * ci: fix debugd logcollection (#2355) * add missing keyvault access role Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * bump logstash image version Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * bump filebeat / metricbeat image version Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * log used image version Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * use debugging image versions Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * increase wait timeout for image upload * add cloud metadata processor to filebeat Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix template locations in container Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix image version typo Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add filebeat / metricbeat users Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * remove user additions Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * update workflow step name Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * only mount config files Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * document potential rc Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix IAM permissions in workflow Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix AWS permissions Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * tidy Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add missing workflow input Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * rename action Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * pin image versions Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * remove unnecessary workflow inputs Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> --------- Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add refStream input Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * remove inputs.yml dep Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * increase system metric period Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix linkchecker Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> --------- Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
2023-09-27 10:17:31 -04:00
azureClusterCreateCredentials:
description: "Azure credentials authorized to create a Constellation cluster."
required: true
azureIAMCreateCredentials:
description: "Azure credentials authorized to create an IAM configuration."
required: true
refStream:
description: "Reference and stream of the image in use"
required: false
internalLoadBalancer:
description: "Whether to use an internal load balancer for the control plane"
required: false
outputs:
kubeconfig:
description: "The kubeconfig for the cluster."
value: ${{ steps.constellation-init.outputs.KUBECONFIG }}
osImageUsed:
description: "The OS image used in the cluster."
value: ${{ steps.setImage.outputs.image }}
runs:
using: "composite"
steps:
- name: Set constellation name
shell: bash
run: |
yq eval -i "(.name) = \"e2e-test\"" constellation-conf.yaml
- name: Set Azure SNP enforcement policy
if: inputs.azureSNPEnforcementPolicy != ''
shell: bash
run: |
if [[ ${{ inputs.cloudProvider }} != 'azure' ]]; then
echo "SNP enforcement policy is only supported for Azure"
exit 1
fi
yq eval -i "(.attestation.azureSEVSNP.firmwareSignerConfig.enforcementPolicy) \
= \"${{ inputs.azureSNPEnforcementPolicy }}\"" constellation-conf.yaml
- name: Set image
id: setImage
shell: bash
env:
imageInput: ${{ inputs.osImage }}
run: |
if [[ -z "${imageInput}" ]]; then
echo "No image specified. Using default image from config."
image=$(yq eval ".image" constellation-conf.yaml)
echo "image=${image}" | tee -a "$GITHUB_OUTPUT"
exit 0
fi
yq eval -i "(.image) = \"${imageInput}\"" constellation-conf.yaml
echo "image=${imageInput}" | tee -a "$GITHUB_OUTPUT"
- name: Update measurements for non-stable images
if: inputs.fetchMeasurements
shell: bash
run: |
constellation config fetch-measurements --debug --insecure
- name: Set instanceType
if: inputs.machineType && inputs.machineType != 'default'
shell: bash
run: |
yq eval -i "(.nodeGroups[] | .instanceType) = \"${{ inputs.machineType }}\"" constellation-conf.yaml
- name: Set node count
shell: bash
run: |
yq eval -i "(.nodeGroups[] | select(.role == \"control-plane\") | .initialCount) = ${{ inputs.controlNodesCount }}" constellation-conf.yaml
yq eval -i "(.nodeGroups[] | select(.role == \"worker\") | .initialCount) = ${{ inputs.workerNodesCount }}" constellation-conf.yaml
- name: Enable debugCluster flag
if: inputs.isDebugImage == 'true'
shell: bash
run: |
yq eval -i '(.debugCluster) = true' constellation-conf.yaml
- name: Enable internalLoadBalancer flag
if: inputs.internalLoadBalancer == 'true'
shell: bash
run: |
yq eval -i '(.internalLoadBalancer) = true' constellation-conf.yaml
- name: Constellation create
shell: bash
run: |
echo "Creating cluster using config:"
cat constellation-conf.yaml
2022-12-13 04:07:09 -05:00
sudo sh -c 'echo "127.0.0.1 license.confidential.cloud" >> /etc/hosts' || true
constellation create -y --debug --tf-log=DEBUG
- name: Cdbg deploy
if: inputs.isDebugImage == 'true'
ci: collect cluster metrics to OpenSearch (#2347) * add Metricbeat deployment to debugd Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * set metricbeat debugd image version Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix k8s deployment Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * use 2 separate deployments Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * only deploy via k8s in non-debug-images Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add missing tilde * remove k8s metrics Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * unify flag Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add cloud metadata processor to filebeat Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * ci: fix debugd logcollection (#2355) * add missing keyvault access role Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * bump logstash image version Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * bump filebeat / metricbeat image version Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * log used image version Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * use debugging image versions Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * increase wait timeout for image upload * add cloud metadata processor to filebeat Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix template locations in container Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix image version typo Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add filebeat / metricbeat users Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * remove user additions Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * update workflow step name Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * only mount config files Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * document potential rc Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix IAM permissions in workflow Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix AWS permissions Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * tidy Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add missing workflow input Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * rename action Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * pin image versions Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * remove unnecessary workflow inputs Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> --------- Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add refStream input Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * remove inputs.yml dep Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * increase system metric period Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix linkchecker Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> --------- Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
2023-09-27 10:17:31 -04:00
uses: ./.github/actions/cdbg_deploy
with:
cloudProvider: ${{ inputs.cloudProvider }}
test: ${{ inputs.test }}
azureClusterCreateCredentials: ${{ inputs.azureClusterCreateCredentials }}
azureIAMCreateCredentials: ${{ inputs.azureIAMCreateCredentials }}
refStream: ${{ inputs.refStream }}
kubernetesVersion: ${{ inputs.kubernetesVersion }}
- name: Constellation init
id: constellation-init
shell: bash
run: |
constellation init --debug
echo "KUBECONFIG=$(pwd)/constellation-admin.conf" | tee -a $GITHUB_OUTPUT
- name: Wait for nodes to join and become ready
shell: bash
env:
KUBECONFIG: "${{ steps.constellation-init.outputs.KUBECONFIG }}"
JOINTIMEOUT: "1200" # 20 minutes timeout for all nodes to join
run: |
echo "::group::Wait for nodes"
NODES_COUNT=$((${{ inputs.controlNodesCount }} + ${{ inputs.workerNodesCount }}))
JOINWAIT=0
until [[ "$(kubectl get nodes -o json | jq '.items | length')" == "${NODES_COUNT}" ]] || [[ $JOINWAIT -gt $JOINTIMEOUT ]];
do
echo "$(kubectl get nodes -o json | jq '.items | length')/"${NODES_COUNT}" nodes have joined.. waiting.."
JOINWAIT=$((JOINWAIT+30))
sleep 30
done
if [[ $JOINWAIT -gt $JOINTIMEOUT ]]; then
echo "Timed out waiting for nodes to join"
exit 1
fi
echo "$(kubectl get nodes -o json | jq '.items | length')/"${NODES_COUNT}" nodes have joined"
if ! kubectl wait --for=condition=ready --all nodes --timeout=20m; then
kubectl get pods -n kube-system
kubectl get events -n kube-system
echo "::error::kubectl wait timed out before all nodes became ready"
echo "::endgroup::"
exit 1
fi
echo "::endgroup::"
- name: Download boot logs
if: always()
continue-on-error: true
shell: bash
env:
CSP: ${{ inputs.cloudProvider }}
run: |
echo "::group::Download boot logs"
cli: use state file on init and upgrade (#2395) * [wip] use state file in CLI Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> tidy Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * use state file in CLI Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> take clusterConfig from IDFile for compat Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> various fixes Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> wip Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add GCP-specific values in Helm loader test Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * remove unnecessary pointer Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * write ClusterValues in one step Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * move stub to test file Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * remove mention of id-file Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * move output to `migrateTerraform` Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * unconditional assignments converting from idFile Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * move require block in go modules file Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fall back to id file on upgrade Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * tidy Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix linter check Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add notice to remove Terraform state check on manual migration Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add `name` field Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> fix name tests Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * return early if no Terraform diff Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * tidy Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * return infrastructure state even if no diff exists Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add TODO to remove comment Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * use state-file in miniconstellation Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * cli: remove id-file (#2402) * remove id-file from `constellation create` Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add file renaming to handler * rename id-file after upgrade * use idFile on `constellation init` Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * remove id-file from `constellation verify` Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * linter fixes Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * remove id-file from `constellation mini` * remove id-file from `constellation recover` * linter fixes * remove id-file from `constellation terminate` * fix initSecret type * fix recover argument precedence * fix terminate test * generate * add TODO to remove id-file removal * Update cli/internal/cmd/init.go Co-authored-by: Adrian Stobbe <stobbe.adrian@gmail.com> * fix verify arg parse logic Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add version test Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * remove id-file from docs * add file not found log * use state-file in miniconstellation Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * remove id-file from `constellation iam destroy` Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * remove id-file from `cdbg deploy` Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> --------- Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> Co-authored-by: Adrian Stobbe <stobbe.adrian@gmail.com> * use state-file in CI Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * update orchestration docs --------- Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> Co-authored-by: Adrian Stobbe <stobbe.adrian@gmail.com>
2023-10-09 07:04:29 -04:00
CONSTELL_UID=$(yq '.infrastructure.uid' constellation-state.yaml)
case $CSP in
azure)
AZURE_RESOURCE_GROUP=$(yq eval ".provider.azure.resourceGroup" constellation-conf.yaml)
./.github/actions/constellation_create/az-logs.sh ${AZURE_RESOURCE_GROUP}
;;
gcp)
GCP_ZONE=$(yq eval ".provider.gcp.zone" constellation-conf.yaml)
./.github/actions/constellation_create/gcp-logs.sh ${GCP_ZONE} ${CONSTELL_UID}
;;
aws)
./.github/actions/constellation_create/aws-logs.sh us-east-2 ${CONSTELL_UID}
;;
esac
echo "::endgroup::"
- name: Upload boot logs
if: always() && !env.ACT
continue-on-error: true
uses: actions/upload-artifact@a8a3f3ad30e3422c9c7b888a15615d19a852ae32 # v3.1.3
with:
name: serial-logs-${{ inputs.artifactNameSuffix }}
path: |
*.log
!terraform.log