constellation/.github/actions/e2e_test/action.yml

416 lines
17 KiB
YAML
Raw Normal View History

name: e2e meta test
description: "This test does the infrastructure management and runs the e2e test of your choice."
inputs:
workerNodesCount:
description: "Number of worker nodes to spawn."
default: "2"
controlNodesCount:
description: "Number of control-plane nodes to spawn."
default: "3"
cloudProvider:
description: "Which cloud provider to use."
required: true
attestationVariant:
description: "Which attestation variant to use."
required: true
machineType:
description: "VM machine type. Make sure it matches selected cloud provider!"
osImage:
2022-11-23 03:41:42 -05:00
description: "OS image to run."
required: true
isDebugImage:
description: "Is OS img a debug img?"
required: true
cliVersion:
description: "Version of a released CLI to download, e.g. 'v2.3.0', leave empty to build it."
kubernetesVersion:
description: "Kubernetes version to create the cluster from."
refStream:
description: "RefStream of the image"
regionZone:
description: "Region or zone to use for resource creation"
required: false
gcpProject:
description: "The GCP project to deploy Constellation in."
required: true
gcpIAMCreateServiceAccount:
description: "Service account with permissions to create IAM configuration on GCP."
required: true
gcpClusterCreateServiceAccount:
description: "Service account with permissions to create a Constellation cluster on GCP."
required: true
awsOpenSearchDomain:
description: "AWS OpenSearch Endpoint Domain to upload the benchmark results."
awsOpenSearchUsers:
description: "AWS OpenSearch User to upload the benchmark results."
awsOpenSearchPwd:
description: "AWS OpenSearch Password to upload the benchmark results."
azureClusterCreateCredentials:
description: "Azure credentials authorized to create a Constellation cluster."
required: true
azureIAMCreateCredentials:
description: "Azure credentials authorized to create an IAM configuration."
required: true
test:
description: "The test to run. Can currently be one of [sonobuoy full, sonobuoy quick, autoscaling, lb, perf-bench, verify, recover, malicious join, nop, upgrade]."
required: true
sonobuoyTestSuiteCmd:
description: "The sonobuoy test suite to run."
buildBuddyApiKey:
description: "BuildBuddy API key for caching Bazel artifacts"
registry:
description: "Container registry to use"
required: true
githubToken:
description: "GitHub authorization token"
required: true
cosignPassword:
description: "The password for the cosign private key. Used for uploading to the config API"
cosignPrivateKey:
description: "The cosign private key. Used for uploading to the config API"
fetchMeasurements:
description: "Update measurements via the 'constellation config fetch-measurements' command."
default: "false"
azureSNPEnforcementPolicy:
description: "Enable security policy for the cluster."
internalLoadBalancer:
description: "Enable internal load balancer for the cluster."
clusterCreation:
description: "How to create infrastructure for the e2e test. One of [cli,, terraform]."
default: "cli"
s3AccessKey:
description: "Access key for s3proxy"
s3SecretKey:
description: "Secret key for s3proxy"
terraform: Azure Marketplace image support (#2651) * terraform: add Azure marketplace variable Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * config: add Azure marketplace variable Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * cli: use Terraform variables from config Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * terraform: pass down marketplace variable * image: pad Azure images to 1GiB * terraform: add version attribute to marketplace image * semver: allow versions to be exported without prefix * cli: boolean var to use marketplace images * config: remove dive key * dev-docs: add instructions on how to use marketplace images * terraform: fix unit test * terraform: only fetch image for non-marketplace images * mpimage: refactor image selection Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * [remove] increase minor version for image build Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * terraform: ignore changes to source_image_reference on upgrade * operator: add support for parsing Azure marketplace images Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * upgrade: fix imagefetcher call * docs: add info about azure marketplace * image: ensure more than 1GiB in size * image: test to pad to 2GiB * version: change back to v2.14.0-pre * image: GPT-conformant image size padding * [remove] increase version * mpimage: inline prefix func Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * ci: add marketplace image e2e test Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * [remove] register workflow * ci: fix workflow name * ci: only allow azure test * cli: add marketplace image input to interface * cli: fix argument passing * version: roll back to v2.14.0 * ci: add force-flag support * Update docs/docs/overview/license.md * Update dev-docs/workflows/marketplace-images.md Co-authored-by: Moritz Eckert <m1gh7ym0@gmail.com> --------- Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> Co-authored-by: Moritz Eckert <m1gh7ym0@gmail.com> Co-authored-by: Thomas Tendyck <51411342+thomasten@users.noreply.github.com>
2023-12-08 08:40:31 -05:00
marketplaceImageVersion:
description: "Marketplace OS image version. Used instead of osImage."
required: false
force:
description: "Set the force-flag on apply to ignore version mismatches."
required: false
encryptionSecret:
2023-12-20 09:17:49 -05:00
description: 'The secret to use for decrypting the artifact.'
required: true
outputs:
kubeconfig:
description: "The kubeconfig for the cluster."
value: ${{ steps.constellation-create.outputs.kubeconfig }}
namePrefix:
description: "The name prefix of the cloud resources used in the e2e test."
value: ${{ steps.create-prefix.outputs.prefix }}
runs:
using: "composite"
steps:
- name: Check input
if: (!contains(fromJson('["sonobuoy full", "sonobuoy quick", "autoscaling", "perf-bench", "verify", "lb", "recover", "malicious join", "s3proxy", "nop", "upgrade"]'), inputs.test))
shell: bash
run: |
echo "::error::Invalid input for test field: ${{ inputs.test }}"
exit 1
# Perf-bench's network benchmarks require at least two distinct worker nodes.
- name: Validate perf-bench inputs
if: inputs.test == 'perf-bench'
shell: bash
run: |
if [[ "${{ inputs.workerNodesCount }}" -lt 2 ]]; then
echo "::error::Test Perf-Bench requires at least 2 worker nodes."
exit 1
fi
- name: Validate verify input
if: inputs.test == 'verify'
shell: bash
run: |
if [[ "${{ inputs.cosignPassword }}" == '' || "${{ inputs.cosignPrivateKey }}" == '' ]]; then
echo "::error::e2e test verify requires cosignPassword and cosignPrivateKey to be set."
exit 1
fi
2022-09-14 11:41:47 -04:00
- name: Determine build target
id: determine-build-target
shell: bash
run: |
echo "hostOS=$(go env GOOS)" | tee -a "$GITHUB_OUTPUT"
echo "hostArch=$(go env GOARCH)" | tee -a "$GITHUB_OUTPUT"
2022-09-14 11:41:47 -04:00
- name: Setup bazel
2023-10-04 07:55:38 -04:00
uses: ./.github/actions/setup_bazel_nix
with:
useCache: ${{ inputs.buildBuddyApiKey != '' }}
buildBuddyApiKey: ${{ inputs.buildBuddyApiKey }}
- name: Log in to the Container registry
uses: ./.github/actions/container_registry_login
with:
registry: ${{ inputs.registry }}
username: ${{ github.actor }}
password: ${{ inputs.githubToken }}
- name: Build CLI
if: inputs.cliVersion == ''
uses: ./.github/actions/build_cli
2022-09-14 11:41:47 -04:00
with:
targetOS: ${{ steps.determine-build-target.outputs.hostOS }}
targetArch: ${{ steps.determine-build-target.outputs.hostArch }}
2023-05-30 03:08:12 -04:00
enterpriseCLI: true
outputPath: "build/constellation"
push: ${{ inputs.cliVersion == '' }}
2022-09-14 11:41:47 -04:00
- name: Download CLI
if: inputs.cliVersion != ''
shell: bash
run: |
curl -fsSL -o constellation https://github.com/edgelesssys/constellation/releases/download/${{ inputs.cliVersion }}/constellation-linux-amd64
chmod u+x constellation
echo "$(pwd)" >> $GITHUB_PATH
export PATH="$PATH:$(pwd)"
constellation version
# Do not spam license server from pipeline
sudo sh -c 'echo "127.0.0.1 license.confidential.cloud" >> /etc/hosts'
- name: Build Terraform provider binary
if: inputs.clusterCreation == 'terraform' && inputs.cliVersion == ''
uses: ./.github/actions/build_tf_provider
with:
targetOS: ${{ steps.determine-build-target.outputs.hostOS }}
targetArch: ${{ steps.determine-build-target.outputs.hostArch }}
outputPath: "build/terraform-provider-constellation"
- name: Move Terraform provider binary
if: inputs.clusterCreation == 'terraform' && inputs.cliVersion == ''
shell: bash
run: |
bazel build //bazel/settings:tag
repository_root=$(git rev-parse --show-toplevel)
out_rel=$(bazel cquery --output=files //bazel/settings:tag)
build_version=$(cat "$(realpath "${repository_root}/${out_rel}")")
terraform_provider_dir="${HOME}/.terraform.d/plugins/registry.terraform.io/edgelesssys/constellation/${build_version#v}/${{ steps.determine-build-target.outputs.hostOS }}_${{ steps.determine-build-target.outputs.hostArch }}/"
mkdir -p "${terraform_provider_dir}"
mv build/terraform-provider-constellation "${terraform_provider_dir}/terraform-provider-constellation_${build_version}"
- name: Build the bootstrapper
id: build-bootstrapper
if: inputs.isDebugImage == 'true'
uses: ./.github/actions/build_bootstrapper
2022-09-14 11:41:47 -04:00
- name: Build the upgrade-agent
id: build-upgrade-agent
if: inputs.isDebugImage == 'true'
uses: ./.github/actions/build_upgrade_agent
2022-09-14 11:41:47 -04:00
- name: Build cdbg
id: build-cdbg
if: inputs.isDebugImage == 'true'
uses: ./.github/actions/build_cdbg
2022-09-14 11:41:47 -04:00
with:
targetOS: ${{ steps.determine-build-target.outputs.hostOS }}
targetArch: ${{ steps.determine-build-target.outputs.hostArch }}
- name: Login to GCP (IAM service account)
if: inputs.cloudProvider == 'gcp'
uses: ./.github/actions/login_gcp
with:
service_account: ${{ inputs.gcpIAMCreateServiceAccount }}
- name: Login to AWS (IAM role)
if: inputs.cloudProvider == 'aws'
uses: aws-actions/configure-aws-credentials@e3dd6a429d7300a6a4c196c26e071d42e0343502 # v4.0.2
with:
role-to-assume: arn:aws:iam::795746500882:role/GithubActionsE2EIAM
aws-region: eu-central-1
# extend token expiry to 6 hours to ensure constellation can terminate
role-duration-seconds: 21600
- name: Login to Azure (IAM service principal)
if: inputs.cloudProvider == 'azure'
uses: ./.github/actions/login_azure
with:
azure_credentials: ${{ inputs.azureIAMCreateCredentials }}
- name: Create prefix
id: create-prefix
shell: bash
run: |
uuid=$(uuidgen | tr "[:upper:]" "[:lower:]")
uuid=${uuid%%-*}
echo "uuid=${uuid}" | tee -a $GITHUB_OUTPUT
echo "prefix=e2e-${{ github.run_id }}-${{ github.run_attempt }}-${uuid}" | tee -a $GITHUB_OUTPUT
2023-10-20 07:38:08 -04:00
- name: Pick a random Azure region
id: pick-az-region
uses: ./.github/actions/pick_azure_region
with:
attestationVariant: ${{ inputs.attestationVariant }}
2023-10-20 07:38:08 -04:00
- name: Create IAM configuration
id: constellation-iam-create
uses: ./.github/actions/constellation_iam_create
with:
cloudProvider: ${{ inputs.cloudProvider }}
attestationVariant: ${{ inputs.attestationVariant }}
namePrefix: ${{ steps.create-prefix.outputs.prefix }}
awsZone: ${{ inputs.regionZone || 'us-east-2c' }}
2023-10-20 07:38:08 -04:00
azureRegion: ${{ inputs.regionZone || steps.pick-az-region.outputs.region }}
gcpProjectID: ${{ inputs.gcpProject }}
gcpZone: ${{ inputs.regionZone || 'europe-west3-b' }}
kubernetesVersion: ${{ inputs.kubernetesVersion }}
- name: Login to GCP (Cluster service account)
if: inputs.cloudProvider == 'gcp'
uses: ./.github/actions/login_gcp
with:
service_account: ${{ inputs.gcpClusterCreateServiceAccount }}
- name: Login to AWS (Cluster role)
if: inputs.cloudProvider == 'aws'
uses: aws-actions/configure-aws-credentials@e3dd6a429d7300a6a4c196c26e071d42e0343502 # v4.0.2
with:
role-to-assume: arn:aws:iam::795746500882:role/GithubActionsE2ECluster
aws-region: eu-central-1
# extend token expiry to 6 hours to ensure constellation can terminate
role-duration-seconds: 21600
- name: Login to Azure (Cluster service principal)
if: inputs.cloudProvider == 'azure'
uses: ./.github/actions/login_azure
with:
azure_credentials: ${{ inputs.azureClusterCreateCredentials }}
- name: Create cluster
id: constellation-create
uses: ./.github/actions/constellation_create
with:
cloudProvider: ${{ inputs.cloudProvider }}
attestationVariant: ${{ inputs.attestationVariant }}
workerNodesCount: ${{ inputs.workerNodesCount }}
controlNodesCount: ${{ inputs.controlNodesCount }}
machineType: ${{ inputs.machineType }}
osImage: ${{ inputs.osImage }}
isDebugImage: ${{ inputs.isDebugImage }}
artifactNameSuffix: ${{ steps.create-prefix.outputs.prefix }}
fetchMeasurements: ${{ inputs.fetchMeasurements }}
cliVersion: ${{ inputs.cliVersion }}
azureSNPEnforcementPolicy: ${{ inputs.azureSNPEnforcementPolicy }}
ci: collect cluster metrics to OpenSearch (#2347) * add Metricbeat deployment to debugd Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * set metricbeat debugd image version Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix k8s deployment Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * use 2 separate deployments Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * only deploy via k8s in non-debug-images Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add missing tilde * remove k8s metrics Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * unify flag Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add cloud metadata processor to filebeat Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * ci: fix debugd logcollection (#2355) * add missing keyvault access role Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * bump logstash image version Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * bump filebeat / metricbeat image version Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * log used image version Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * use debugging image versions Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * increase wait timeout for image upload * add cloud metadata processor to filebeat Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix template locations in container Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix image version typo Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add filebeat / metricbeat users Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * remove user additions Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * update workflow step name Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * only mount config files Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * document potential rc Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix IAM permissions in workflow Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix AWS permissions Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * tidy Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add missing workflow input Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * rename action Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * pin image versions Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * remove unnecessary workflow inputs Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> --------- Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add refStream input Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * remove inputs.yml dep Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * increase system metric period Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix linkchecker Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> --------- Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
2023-09-27 10:17:31 -04:00
azureIAMCreateCredentials: ${{ inputs.azureIAMCreateCredentials }}
azureClusterCreateCredentials: ${{ inputs.azureClusterCreateCredentials }}
kubernetesVersion: ${{ inputs.kubernetesVersion }}
refStream: ${{ inputs.refStream }}
internalLoadBalancer: ${{ inputs.internalLoadBalancer }}
ci: add e2e test for self-managed infrastructure (#2472) * add self-managed infra e2e test * self-managed terminatio Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix upgrade test Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix indentation Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * use -r when copying dir Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add terraform variable parsing * copy constellation conf Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * remove unnecessary line breaks * add missing value Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add image fetching for CSP Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix quoting Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add missing input to internal lb test * normalize Azure URLs.. Of course * tidy Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix expressions * initsecret to hex * update hexdump cmd * add build test Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add node / pod cidr outputs Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * explicitly delete the state file Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add missing license header Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * always write all outputs Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix list output Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * remove state-file and admin-conf on destroy * dont use test payload Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * [remove] use self managed infra in manual e2e for testing Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * init: always skip infrastructure phase * patch maa in workflow Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * default to Constellation-created infra in e2e test --------- Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
2023-10-27 03:37:26 -04:00
test: ${{ inputs.test }}
clusterCreation: ${{ inputs.clusterCreation }}
terraform: Azure Marketplace image support (#2651) * terraform: add Azure marketplace variable Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * config: add Azure marketplace variable Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * cli: use Terraform variables from config Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * terraform: pass down marketplace variable * image: pad Azure images to 1GiB * terraform: add version attribute to marketplace image * semver: allow versions to be exported without prefix * cli: boolean var to use marketplace images * config: remove dive key * dev-docs: add instructions on how to use marketplace images * terraform: fix unit test * terraform: only fetch image for non-marketplace images * mpimage: refactor image selection Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * [remove] increase minor version for image build Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * terraform: ignore changes to source_image_reference on upgrade * operator: add support for parsing Azure marketplace images Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * upgrade: fix imagefetcher call * docs: add info about azure marketplace * image: ensure more than 1GiB in size * image: test to pad to 2GiB * version: change back to v2.14.0-pre * image: GPT-conformant image size padding * [remove] increase version * mpimage: inline prefix func Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * ci: add marketplace image e2e test Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * [remove] register workflow * ci: fix workflow name * ci: only allow azure test * cli: add marketplace image input to interface * cli: fix argument passing * version: roll back to v2.14.0 * ci: add force-flag support * Update docs/docs/overview/license.md * Update dev-docs/workflows/marketplace-images.md Co-authored-by: Moritz Eckert <m1gh7ym0@gmail.com> --------- Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> Co-authored-by: Moritz Eckert <m1gh7ym0@gmail.com> Co-authored-by: Thomas Tendyck <51411342+thomasten@users.noreply.github.com>
2023-12-08 08:40:31 -05:00
marketplaceImageVersion: ${{ inputs.marketplaceImageVersion }}
force: ${{ inputs.force }}
encryptionSecret: ${{ inputs.encryptionSecret }}
2022-09-02 06:43:04 -04:00
ci: collect cluster metrics to OpenSearch (#2347) * add Metricbeat deployment to debugd Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * set metricbeat debugd image version Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix k8s deployment Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * use 2 separate deployments Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * only deploy via k8s in non-debug-images Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add missing tilde * remove k8s metrics Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * unify flag Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add cloud metadata processor to filebeat Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * ci: fix debugd logcollection (#2355) * add missing keyvault access role Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * bump logstash image version Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * bump filebeat / metricbeat image version Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * log used image version Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * use debugging image versions Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * increase wait timeout for image upload * add cloud metadata processor to filebeat Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix template locations in container Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix image version typo Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add filebeat / metricbeat users Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * remove user additions Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * update workflow step name Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * only mount config files Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * document potential rc Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix IAM permissions in workflow Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix AWS permissions Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * tidy Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add missing workflow input Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * rename action Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * pin image versions Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * remove unnecessary workflow inputs Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> --------- Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add refStream input Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * remove inputs.yml dep Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * increase system metric period Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix linkchecker Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> --------- Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
2023-09-27 10:17:31 -04:00
- name: Deploy log- and metrics-collection (Kubernetes)
id: deploy-logcollection
ci: collect cluster metrics to OpenSearch (#2347) * add Metricbeat deployment to debugd Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * set metricbeat debugd image version Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix k8s deployment Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * use 2 separate deployments Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * only deploy via k8s in non-debug-images Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add missing tilde * remove k8s metrics Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * unify flag Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add cloud metadata processor to filebeat Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * ci: fix debugd logcollection (#2355) * add missing keyvault access role Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * bump logstash image version Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * bump filebeat / metricbeat image version Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * log used image version Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * use debugging image versions Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * increase wait timeout for image upload * add cloud metadata processor to filebeat Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix template locations in container Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix image version typo Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add filebeat / metricbeat users Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * remove user additions Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * update workflow step name Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * only mount config files Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * document potential rc Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix IAM permissions in workflow Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix AWS permissions Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * tidy Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add missing workflow input Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * rename action Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * pin image versions Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * remove unnecessary workflow inputs Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> --------- Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add refStream input Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * remove inputs.yml dep Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * increase system metric period Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix linkchecker Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> --------- Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
2023-09-27 10:17:31 -04:00
if: inputs.isDebugImage == 'false'
uses: ./.github/actions/deploy_logcollection
with:
kubeconfig: ${{ steps.constellation-create.outputs.kubeconfig }}
opensearchUser: ${{ inputs.awsOpenSearchUsers }}
opensearchPwd: ${{ inputs.awsOpenSearchPwd }}
test: ${{ inputs.test }}
provider: ${{ inputs.cloudProvider }}
attestationVariant: ${{ inputs.attestationVariant }}
isDebugImage: ${{ inputs.isDebugImage }}
kubernetesVersion: ${{ inputs.kubernetesVersion }}
refStream: ${{ inputs.refStream }}
clusterCreation: ${{ inputs.clusterCreation }}
#
# Test payloads
#
- name: Nop test payload
if: (inputs.test == 'nop') || (inputs.test == 'upgrade')
shell: bash
run: |
echo "::warning::This test has a nop payload. It doesn't run any tests."
echo "Sleeping for 30 seconds to allow logs to propagate to the log collection service."
sleep 30
- name: Run sonobuoy quick test
if: inputs.test == 'sonobuoy quick'
uses: ./.github/actions/e2e_sonobuoy
with:
sonobuoyTestSuiteCmd: "--mode quick"
kubeconfig: ${{ steps.constellation-create.outputs.kubeconfig }}
artifactNameSuffix: ${{ steps.create-prefix.outputs.prefix }}
encryptionSecret: ${{ inputs.encryptionSecret }}
- name: Run sonobuoy full test
if: inputs.test == 'sonobuoy full'
uses: ./.github/actions/e2e_sonobuoy
with:
# TODO(3u13r): Remove E2E_SKIP once AB#2174 is resolved
sonobuoyTestSuiteCmd: '--plugin e2e --plugin-env e2e.E2E_FOCUS="\[Conformance\]" --plugin-env e2e.E2E_SKIP="for service with type clusterIP|HostPort validates that there is no conflict between pods with same hostPort but different hostIP and protocol|Services should serve endpoints on same port and different protocols" --plugin https://raw.githubusercontent.com/vmware-tanzu/sonobuoy-plugins/102cd62a4091f80a795189f64ccc20738f931ef0/cis-benchmarks/kube-bench-plugin.yaml --plugin https://raw.githubusercontent.com/vmware-tanzu/sonobuoy-plugins/102cd62a4091f80a795189f64ccc20738f931ef0/cis-benchmarks/kube-bench-master-plugin.yaml'
kubeconfig: ${{ steps.constellation-create.outputs.kubeconfig }}
artifactNameSuffix: ${{ steps.create-prefix.outputs.prefix }}
encryptionSecret: ${{ inputs.encryptionSecret }}
- name: Run autoscaling test
if: inputs.test == 'autoscaling'
uses: ./.github/actions/e2e_autoscaling
with:
kubeconfig: ${{ steps.constellation-create.outputs.kubeconfig }}
- name: Run lb test
if: inputs.test == 'lb'
uses: ./.github/actions/e2e_lb
with:
kubeconfig: ${{ steps.constellation-create.outputs.kubeconfig }}
2023-02-28 05:21:26 -05:00
- name: Run Performance Benchmark
if: inputs.test == 'perf-bench'
uses: ./.github/actions/e2e_benchmark
with:
cloudProvider: ${{ inputs.cloudProvider }}
attestationVariant: ${{ inputs.attestationVariant }}
kubeconfig: ${{ steps.constellation-create.outputs.kubeconfig }}
awsOpenSearchDomain: ${{ inputs.awsOpenSearchDomain }}
awsOpenSearchUsers: ${{ inputs.awsOpenSearchUsers }}
awsOpenSearchPwd: ${{ inputs.awsOpenSearchPwd }}
encryptionSecret: ${{ inputs.encryptionSecret }}
- name: Run constellation verify test
if: inputs.test == 'verify'
2023-01-17 05:28:28 -05:00
uses: ./.github/actions/e2e_verify
with:
attestationVariant: ${{ inputs.attestationVariant }}
osImage: ${{ steps.constellation-create.outputs.osImageUsed }}
kubeconfig: ${{ steps.constellation-create.outputs.kubeconfig }}
cosignPassword: ${{ inputs.cosignPassword }}
cosignPrivateKey: ${{ inputs.cosignPrivateKey }}
- name: Run recover test
if: inputs.test == 'recover'
uses: ./.github/actions/e2e_recover
with:
controlNodesCount: ${{ inputs.controlNodesCount }}
kubeconfig: ${{ steps.constellation-create.outputs.kubeconfig }}
ci: add malicious join test (#2304) * malicious node join test Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add e2e build tag Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add namespaces to job apply Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix image and workflow Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix linter checks Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * build instructions in Dockerfile Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * only print important flags Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * use `malicious-join` namespace Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * build with bazel Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * order imports Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * test cases Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * various fixes Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add missing quotes Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix typo Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * Update e2e/malicious-join/malicious-join.go Co-authored-by: Daniel Weiße <66256922+daniel-weisse@users.noreply.github.com> * Update e2e/malicious-join/malicious-join.go Co-authored-by: Daniel Weiße <66256922+daniel-weisse@users.noreply.github.com> * use switch case Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * update image version Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix linter checks Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * wip Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * various fixes Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * update buildfiles Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * use workdir Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix linter Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add required permissions Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * remove permissions Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * remove packages: write permission at step Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * login to registry Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix typo Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix log Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * source base lib Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix sourcing order Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * export after definition Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix script header Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * dont exit after -e flag has been set Co-authored-by: Paul Meyer <49727155+katexochen@users.noreply.github.com> --------- Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> Co-authored-by: Daniel Weiße <66256922+daniel-weisse@users.noreply.github.com> Co-authored-by: Paul Meyer <49727155+katexochen@users.noreply.github.com>
2023-09-15 11:21:42 -04:00
- name: Run malicious join test
if: inputs.test == 'malicious join'
uses: ./.github/actions/e2e_malicious_join
with:
cloudProvider: ${{ inputs.cloudProvider }}
attestationVariant: ${{ inputs.attestationVariant }}
ci: add malicious join test (#2304) * malicious node join test Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add e2e build tag Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add namespaces to job apply Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix image and workflow Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix linter checks Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * build instructions in Dockerfile Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * only print important flags Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * use `malicious-join` namespace Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * build with bazel Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * order imports Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * test cases Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * various fixes Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add missing quotes Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix typo Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * Update e2e/malicious-join/malicious-join.go Co-authored-by: Daniel Weiße <66256922+daniel-weisse@users.noreply.github.com> * Update e2e/malicious-join/malicious-join.go Co-authored-by: Daniel Weiße <66256922+daniel-weisse@users.noreply.github.com> * use switch case Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * update image version Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix linter checks Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * wip Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * various fixes Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * update buildfiles Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * use workdir Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix linter Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * add required permissions Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * remove permissions Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * remove packages: write permission at step Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * login to registry Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix typo Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix log Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * source base lib Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix sourcing order Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * export after definition Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * fix script header Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> * dont exit after -e flag has been set Co-authored-by: Paul Meyer <49727155+katexochen@users.noreply.github.com> --------- Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com> Co-authored-by: Daniel Weiße <66256922+daniel-weisse@users.noreply.github.com> Co-authored-by: Paul Meyer <49727155+katexochen@users.noreply.github.com>
2023-09-15 11:21:42 -04:00
kubeconfig: ${{ steps.constellation-create.outputs.kubeconfig }}
githubToken: ${{ inputs.githubToken }}
- name: Run s3proxy e2e test
if: inputs.test == 's3proxy'
uses: ./.github/actions/e2e_s3proxy
with:
kubeconfig: ${{ steps.constellation-create.outputs.kubeconfig }}
s3AccessKey: ${{ inputs.s3AccessKey }}
s3SecretKey: ${{ inputs.s3SecretKey }}
buildBuddyApiKey: ${{ inputs.buildBuddyApiKey }}
githubToken: ${{ inputs.githubToken }}