constellation/.github/actions/deploy_logcollection/action.yml
Moritz Sanft f4b2d02194
ci: collect cluster metrics to OpenSearch (#2347)
* add Metricbeat deployment to debugd

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* set metricbeat debugd image version

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* fix k8s deployment

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* use 2 separate deployments

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* only deploy via k8s in non-debug-images

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* add missing tilde

* remove k8s metrics

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* unify flag

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* add cloud metadata processor to filebeat

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* ci: fix debugd logcollection (#2355)

* add missing keyvault access role

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* bump logstash image version

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* bump filebeat / metricbeat image version

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* log used image version

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* use debugging image versions

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* increase wait timeout for image upload

* add cloud metadata processor to filebeat

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* fix template locations in container

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* fix image version typo

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* add filebeat / metricbeat users

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* remove user additions

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* update workflow step name

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* only mount config files

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* document potential rc

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* fix IAM permissions in workflow

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* fix AWS permissions

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* tidy

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* add missing workflow input

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* rename action

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* pin image versions

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* remove unnecessary workflow inputs

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

---------

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* add refStream input

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* remove inputs.yml dep

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* increase system metric period

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

* fix linkchecker

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>

---------

Signed-off-by: Moritz Sanft <58110325+msanft@users.noreply.github.com>
2023-09-27 16:17:31 +02:00

103 lines
3.5 KiB
YAML

name: Log Collection Deployment
description: Deploy log collection functionality to the cluster.
inputs:
logstash-port:
description: "The port of the logstash service."
default: "5045"
kubeconfig:
description: "The kubeconfig of the cluster to deploy to."
required: true
opensearchUser:
description: "The username of the opensearch cluster."
required: true
opensearchPwd:
description: "The password of the opensearch cluster."
required: true
test:
description: "The e2e test payload."
required: true
provider:
description: "The CSP of the cluster."
required: true
isDebugImage:
description: "Whether the cluster is a debug cluster / uses a debug image."
required: true
refStream:
description: "Reference and stream of the image in use"
required: false
kubernetesVersion:
description: "Kubernetes version of the cluster"
required: false
runs:
using: "composite"
steps:
- name: Template Logcollection Helm Values
id: template
shell: bash
run: |
bazel run //hack/logcollector template -- \
--dir $(realpath .) \
--username ${{ inputs.opensearchUser }} \
--password ${{ inputs.opensearchPwd }} \
--port ${{ inputs.logstash-port }} \
--fields github.actor="${{ github.triggering_actor }}" \
--fields github.workflow="${{ github.workflow }}" \
--fields github.run-id="${{ github.run_id }}" \
--fields github.run-attempt="${{ github.run_attempt }}" \
--fields github.ref-name="${{ github.ref_name }}" \
--fields github.sha="${{ github.sha }}" \
--fields github.runner-os="${{ runner.os }}" \
--fields github.e2e-test-payload="${{ inputs.test }}" \
--fields github.is-debug-cluster="${{ inputs.isDebugImage }}" \
--fields github.e2e-test-provider="${{ inputs.provider }}" \
--fields github.ref-stream="${{ inputs.refStream }}" \
--fields github.kubernetes-version="${{ inputs.kubernetesVersion }}" \
--fields deployment-type="k8s"
# Make sure that helm is installed
# This is not always the case, e.g. on MacOS runners
- name: Install Helm
uses: azure/setup-helm@5119fcb9089d432beecbf79bb2c7915207344b78 # v3.5
with:
version: latest
- name: Deploy Logstash
id: deploy-logstash
shell: bash
working-directory: ./logstash
env:
KUBECONFIG: ${{ inputs.kubeconfig }}
run: |
helm repo add elastic https://helm.elastic.co
helm repo update
helm install logstash elastic/logstash \
--wait --timeout=1200s --values values.yml
- name: Deploy Filebeat
id: deploy-filebeat
shell: bash
working-directory: ./filebeat
env:
KUBECONFIG: ${{ inputs.kubeconfig }}
run: |
helm repo add elastic https://helm.elastic.co
helm repo update
helm install filebeat elastic/filebeat \
--wait --timeout=1200s --values values.yml
- name: Deploy Metricbeat
id: deploy-metricbeat
shell: bash
working-directory: ./metricbeat
env:
KUBECONFIG: ${{ inputs.kubeconfig }}
run: |
helm repo add elastic https://helm.elastic.co
helm repo update
helm install metricbeat-k8s elastic/metricbeat \
--wait --timeout=1200s --values values-control-plane.yml
helm install metricbeat-system elastic/metricbeat \
--wait --timeout=1200s --values values-all-nodes.yml