Git-Mirrors/constellation
1
0
Fork 0
mirror of https://github.com/edgelesssys/constellation.git synced 2025-07-30 18:48:39 -04:00
Code Issues Projects Releases Packages Wiki Activity

docs: add release v2.16.0

Browse source
This commit is contained in:
malt3 2024-02-29 08:59:55 +00:00 • committed by Malte Poll
parent 93eb8f0694
commit c4f27f62ee
72 changed files with 8740 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

13
docs/versioned_docs/version-2.16/workflows/cert-manager.md Normal file
View file

@ -0,0 +1,13 @@
# Install cert-manager
:::caution
If you want to use cert-manager with Constellation, pay attention to the following to avoid potential pitfalls.
:::
Constellation ships with cert-manager preinstalled.
The default installation is part of the `kube-system` namespace, as all other Constellation-managed microservices.
You are free to install more instances of cert-manager into other namespaces.
However, be aware that any new installation needs to use the same version as the one installed with Constellation or rely on the same CRD versions.
Also remember to set the `installCRDs` value to `false` when installing new cert-manager instances.
It will create problems if you have two installations of cert-manager depending on different versions of the installed CRDs.
CRDs are cluster-wide resources and cert-manager depends on specific versions of those CRDs for each release.

356
docs/versioned_docs/version-2.16/workflows/config.md Normal file
View file

@ -0,0 +1,356 @@
# Configure your cluster
:::info
This recording presents the essence of this page. It's recommended to read it in full for the motivation and all details.
:::
<asciinemaWidget src="/constellation/assets/configure-cluster.cast" rows="20" cols="112" idleTimeLimit="3" preload="true" theme="edgeless" />
---
Before you can create your cluster, you need to configure the identity and access management (IAM) for your cloud service provider (CSP) and choose machine types for the nodes.
## Creating the configuration file
You can generate a configuration file for your CSP by using the following CLI command:
<tabs groupId="csp">
<tabItem value="azure" label="Azure">
```bash
constellation config generate azure
```
</tabItem>
<tabItem value="gcp" label="GCP">
```bash
constellation config generate gcp
```
</tabItem>
<tabItem value="aws" label="AWS">
```bash
constellation config generate aws
```
</tabItem>
<tabItem value="stackit" label="STACKIT">
```bash
constellation config generate stackit
```
</tabItem>
</tabs>
This creates the file `constellation-conf.yaml` in the current directory.
## Choosing a VM type
Constellation supports the following VM types:
<tabs groupId="csp">
<tabItem value="azure" label="Azure">
By default, Constellation uses `Standard_DC4as_v5` CVMs (4 vCPUs, 16 GB RAM) to create your cluster. Optionally, you can switch to a different VM type by modifying `instanceType` in the configuration file. For CVMs, any VM type with a minimum of 4 vCPUs from the [DCasv5 & DCadsv5](https://docs.microsoft.com/en-us/azure/virtual-machines/dcasv5-dcadsv5-series) or [ECasv5 & ECadsv5](https://docs.microsoft.com/en-us/azure/virtual-machines/ecasv5-ecadsv5-series) families is supported.
You can also run `constellation config instance-types` to get the list of all supported options.
</tabItem>
<tabItem value="gcp" label="GCP">
By default, Constellation uses `n2d-standard-4` VMs (4 vCPUs, 16 GB RAM) to create your cluster. Optionally, you can switch to a different VM type by modifying `instanceType` in the configuration file. Supported are all machines with a minimum of 4 vCPUs from the [C2D](https://cloud.google.com/compute/docs/compute-optimized-machines#c2d_machine_types) or [N2D](https://cloud.google.com/compute/docs/general-purpose-machines#n2d_machines) family. You can run `constellation config instance-types` to get the list of all supported options.
</tabItem>
<tabItem value="aws" label="AWS">
By default, Constellation uses `m6a.xlarge` VMs (4 vCPUs, 16 GB RAM) to create your cluster.
Optionally, you can switch to a different VM type by modifying `instanceType` in the configuration file.
If you are using the default attestation variant `awsSEVSNP`, you can use the instance types described in [AWS's AMD SEV-SNP docs](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/snp-requirements.html).
Please mind the region restrictions mentioned in the [Getting started](../getting-started/first-steps.md#create-a-cluster) section.
If you are using the attestation variant `awsNitroTPM`, you can choose any of the [nitroTPM-enabled instance types](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/enable-nitrotpm-prerequisites.html).
The Constellation CLI can also print the supported instance types with: `constellation config instance-types`.
</tabItem>
<tabItem value="stackit" label="STACKIT">
By default, Constellation uses `m1a.4cd` VMs (4 vCPUs, 32 GB RAM) to create your cluster.
Optionally, you can switch to a different VM type by modifying `instanceType` in the configuration file.
The following instance types are known to be supported:
| name | vCPUs | GB RAM |
|----------|-------|--------|
| m1a.4cd | 4 | 32 |
| m1a.8cd | 8 | 64 |
| m1a.16cd | 16 | 120 |
| m1a.30cd | 30 | 238 |
You can choose any of the SEV-enabled instance types. You can find a list of all supported instance types in the [STACKIT documentation](https://docs.stackit.cloud/stackit/en/virtual-machine-flavors-75137231.html).
The Constellation CLI can also print the supported instance types with: `constellation config instance-types`.
</tabItem>
</tabs>
Fill the desired VM type into the `instanceType` fields in the `constellation-conf.yml` file.
## Creating additional node groups
By default, Constellation creates the node groups `control_plane_default` and `worker_default` for control-plane nodes and workers, respectively.
If you require additional control-plane or worker groups with different instance types, zone placements, or disk sizes, you can add additional node groups to the `constellation-conf.yml` file.
Each node group can be scaled individually.
Consider the following example for AWS:
```yaml
nodeGroups:
control_plane_default:
role: control-plane
instanceType: c6a.xlarge
stateDiskSizeGB: 30
stateDiskType: gp3
zone: eu-west-1c
initialCount: 3
worker_default:
role: worker
instanceType: c6a.xlarge
stateDiskSizeGB: 30
stateDiskType: gp3
zone: eu-west-1c
initialCount: 2
high_cpu:
role: worker
instanceType: c6a.24xlarge
stateDiskSizeGB: 128
stateDiskType: gp3
zone: eu-west-1c
initialCount: 1
```
This configuration creates an additional node group `high_cpu` with a larger instance type and disk.
You can use the field `zone` to specify what availability zone nodes of the group are placed in.
On Azure, this field is empty by default and nodes are automatically spread across availability zones.
STACKIT currently only offers SEV-enabled CPUs in the `eu01-1` zone.
Consult the documentation of your cloud provider for more information:
* [AWS](https://aws.amazon.com/about-aws/global-infrastructure/regions_az/)
* [Azure](https://azure.microsoft.com/en-us/explore/global-infrastructure/availability-zones)
* [GCP](https://cloud.google.com/compute/docs/regions-zones)
* [STACKIT](https://docs.stackit.cloud/stackit/en/regions-and-availability-zones-75137212.html)
## Choosing a Kubernetes version
To learn which Kubernetes versions can be installed with your current CLI, you can run `constellation config kubernetes-versions`.
See also Constellation's [Kubernetes support policy](../architecture/versions.md#kubernetes-support-policy).
## Creating an IAM configuration
You can create an IAM configuration for your cluster automatically using the `constellation iam create` command.
If you already have a Constellation configuration file, you can add the `--update-config` flag to the command. This writes the needed IAM fields into your configuration. Furthermore, the flag updates the zone/region of the configuration if it hasn't been set yet.
<tabs groupId="csp">
<tabItem value="azure" label="Azure">
You must be authenticated with the [Azure CLI](https://learn.microsoft.com/en-us/cli/azure/install-azure-cli) in the shell session with a user that has the [required permissions for IAM creation](../getting-started/install.md#set-up-cloud-credentials).
```bash
constellation iam create azure --region=westus --resourceGroup=constellTest --servicePrincipal=spTest
```
This command creates IAM configuration on the Azure region `westus` creating a new resource group `constellTest` and a new service principal `spTest`.
CVMs are available in several Azure regions. Constellation OS images are currently replicated to the following:
* `germanywestcentral`
* `westus`
* `eastus`
* `northeurope`
* `westeurope`
* `southeastasia`
If you require the OS image to be available in another region, [let us know](https://github.com/edgelesssys/constellation/issues/new?assignees=&labels=&template=feature_request.md&title=Support+new+Azure+image+region:+xx-xxxx-x).
You can find a list of all [regions in Azure's documentation](https://azure.microsoft.com/en-us/global-infrastructure/services/?products=virtual-machines&regions=all).
Paste the output into the corresponding fields of the `constellation-conf.yaml` file.
</tabItem>
<tabItem value="gcp" label="GCP">
You must be authenticated with the [GCP CLI](https://cloud.google.com/sdk/gcloud) in the shell session with a user that has the [required permissions for IAM creation](../getting-started/install.md#set-up-cloud-credentials).
```bash
constellation iam create gcp --projectID=yourproject-12345 --zone=europe-west2-a --serviceAccountID=constell-test
```
This command creates IAM configuration in the GCP project `yourproject-12345` on the GCP zone `europe-west2-a` creating a new service account `constell-test`.
Note that only regions offering CVMs of the `C2D` or `N2D` series are supported. You can find a [list of all regions in Google's documentation](https://cloud.google.com/compute/docs/regions-zones#available), which you can filter by machine type `N2D`.
Paste the output into the corresponding fields of the `constellation-conf.yaml` file.
</tabItem>
<tabItem value="aws" label="AWS">
You must be authenticated with the [AWS CLI](https://aws.amazon.com/en/cli/) in the shell session with a user that has the [required permissions for IAM creation](../getting-started/install.md#set-up-cloud-credentials).
```bash
constellation iam create aws --zone=us-east-2a --prefix=constellTest
```
This command creates IAM configuration for the AWS zone `us-east-2a` using the prefix `constellTest` for all named resources being created.
Constellation OS images are currently replicated to the following regions:
* `eu-central-1`
* `eu-west-1`
* `eu-west-3`
* `us-east-2`
* `ap-south-1`
If you require the OS image to be available in another region, [let us know](https://github.com/edgelesssys/constellation/issues/new?assignees=&labels=&template=feature_request.md&title=Support+new+AWS+image+region:+xx-xxxx-x).
You can find a list of all [regions in AWS's documentation](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html#concepts-available-regions).
Paste the output into the corresponding fields of the `constellation-conf.yaml` file.
</tabItem>
<tabItem value="stackit" label="STACKIT">
STACKIT requires manual creation and configuration of service accounts. Look at the [first steps](../getting-started/first-steps.md) for more information.
</tabItem>
</tabs>
<details>
<summary>Alternatively, you can manually create the IAM configuration on your CSP.</summary>
The following describes the configuration fields and how you obtain the required information or create the required resources.
<tabs groupId="csp">
<tabItem value="azure" label="Azure">
* **subscription**: The UUID of your Azure subscription, e.g., `8b8bd01f-efd9-4113-9bd1-c82137c32da7`.
You can view your subscription UUID via `az account show` and read the `id` field. For more information refer to [Azure's documentation](https://docs.microsoft.com/en-us/azure/azure-portal/get-subscription-tenant-id#find-your-azure-subscription).
* **tenant**: The UUID of your Azure tenant, e.g., `3400e5a2-8fe2-492a-886c-38cb66170f25`.
You can view your tenant UUID via `az account show` and read the `tenant` field. For more information refer to [Azure's documentation](https://docs.microsoft.com/en-us/azure/azure-portal/get-subscription-tenant-id#find-your-azure-ad-tenant).
* **location**: The Azure datacenter location you want to deploy your cluster in, e.g., `westus`.
CVMs are available in several Azure regions. Constellation OS images are currently replicated to the following:
* `germanywestcentral`
* `westus`
* `eastus`
* `northeurope`
* `westeurope`
* `southeastasia`
If you require the OS image to be available in another region, [let us know](https://github.com/edgelesssys/constellation/issues/new?assignees=&labels=&template=feature_request.md&title=Support+new+Azure+image+region:+xx-xxxx-x).
You can find a list of all [regions in Azure's documentation](https://azure.microsoft.com/en-us/global-infrastructure/services/?products=virtual-machines&regions=all).
* **resourceGroup**: [Create a new resource group in Azure](https://learn.microsoft.com/azure/azure-resource-manager/management/manage-resource-groups-portal) for your Constellation cluster. Set this configuration field to the name of the created resource group.
* **userAssignedIdentity**: [Create a new managed identity in Azure](https://learn.microsoft.com/azure/active-directory/managed-identities-azure-resources/how-manage-user-assigned-managed-identities). You should create the identity in a different resource group as all resources within the cluster resource group will be deleted on cluster termination.
Add three role assignments to the identity: `Owner`, `Virtual Machine Contributor`, and `Application Insights Component Contributor`. The `scope` of all three should refer to the previously created cluster resource group.
Set the configuration value to the full ID of the created identity, e.g., `/subscriptions/8b8bd01f-efd9-4113-9bd1-c82137c32da7/resourcegroups/constellation-identity/providers/Microsoft.ManagedIdentity/userAssignedIdentities/constellation-identity`. You can get it by opening the `JSON View` from the `Overview` section of the identity.
The user-assigned identity is used by instances of the cluster to access other cloud resources.
For more information about managed identities refer to [Azure's documentation](https://docs.microsoft.com/en-us/azure/active-directory/managed-identities-azure-resources/how-manage-user-assigned-managed-identities).
</tabItem>
<tabItem value="gcp" label="GCP">
* **project**: The ID of your GCP project, e.g., `constellation-129857`.
You can find it on the [welcome screen of your GCP project](https://console.cloud.google.com/welcome). For more information refer to [Google's documentation](https://support.google.com/googleapi/answer/7014113).
* **region**: The GCP region you want to deploy your cluster in, e.g., `us-west1`.
You can find a [list of all regions in Google's documentation](https://cloud.google.com/compute/docs/regions-zones#available).
* **zone**: The GCP zone you want to deploy your cluster in, e.g., `us-west1-a`.
You can find a [list of all zones in Google's documentation](https://cloud.google.com/compute/docs/regions-zones#available).
* **serviceAccountKeyPath**: To configure this, you need to create a GCP [service account](https://cloud.google.com/iam/docs/service-accounts) with the following permissions:
* `Compute Instance Admin (v1) (roles/compute.instanceAdmin.v1)`
* `Compute Network Admin (roles/compute.networkAdmin)`
* `Compute Security Admin (roles/compute.securityAdmin)`
* `Compute Storage Admin (roles/compute.storageAdmin)`
* `Service Account User (roles/iam.serviceAccountUser)`
Afterward, create and download a new JSON key for this service account. Place the downloaded file in your Constellation workspace, and set the config parameter to the filename, e.g., `constellation-129857-15343dba46cb.json`.
</tabItem>
<tabItem value="aws" label="AWS">
* **region**: The name of your chosen AWS data center region, e.g., `us-east-2`.
Constellation OS images are currently replicated to the following regions:
* `eu-central-1`
* `eu-west-1`
* `eu-west-3`
* `us-east-2`
* `ap-south-1`
If you require the OS image to be available in another region, [let us know](https://github.com/edgelesssys/constellation/issues/new?assignees=&labels=&template=feature_request.md&title=Support+new+AWS+image+region:+xx-xxxx-x).
You can find a list of all [regions in AWS's documentation](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html#concepts-available-regions).
* **zone**: The name of your chosen AWS data center availability zone, e.g., `us-east-2a`.
Learn more about [availability zones in AWS's documentation](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html#concepts-availability-zones).
* **iamProfileControlPlane**: The name of an IAM instance profile attached to all control-plane nodes.
You can create the resource with [Terraform](https://www.terraform.io/). For that, use the [provided Terraform script](https://github.com/edgelesssys/constellation/tree/release/v2.2/hack/terraform/aws/iam) to generate the necessary profile. The profile name will be provided as Terraform output value: `control_plane_instance_profile_name`.
Alternatively, you can create the AWS profile with a tool of your choice. Use the JSON policy in [main.tf](https://github.com/edgelesssys/constellation/tree/release/v2.2/hack/terraform/aws/iam/main.tf) in the resource `aws_iam_policy.control_plane_policy`.
* **iamProfileWorkerNodes**: The name of an IAM instance profile attached to all worker nodes.
You can create the resource with [Terraform](https://www.terraform.io/). For that, use the [provided Terraform script](https://github.com/edgelesssys/constellation/tree/release/v2.2/hack/terraform/aws/iam) to generate the necessary profile. The profile name will be provided as Terraform output value: `worker_nodes_instance_profile_name`.
Alternatively, you can create the AWS profile with a tool of your choice. Use the JSON policy in [main.tf](https://github.com/edgelesssys/constellation/tree/release/v2.2/hack/terraform/aws/iam/main.tf) in the resource `aws_iam_policy.worker_node_policy`.
</tabItem>
<tabItem value="stackit" label="STACKIT">
STACKIT requires manual creation and configuration of service accounts. Look at the [first steps](../getting-started/first-steps.md) for more information.
</tabItem>
</tabs>
</details>
Now that you've configured your CSP, you can [create your cluster](./create.md).
## Deleting an IAM configuration
You can keep a created IAM configuration and reuse it for new clusters. Alternatively, you can also delete it if you don't want to use it anymore.
Delete the IAM configuration by executing the following command in the same directory where you executed `constellation iam create` (the directory that contains [`constellation-iam-terraform`](../reference/terraform.md) as a subdirectory):
```bash
constellation iam destroy
```
:::caution
For Azure, deleting the IAM configuration by executing `constellation iam destroy` will delete the whole resource group created by `constellation iam create`.
This also includes any additional resources in the resource group that weren't created by Constellation.
:::

93
docs/versioned_docs/version-2.16/workflows/create.md Normal file
View file

@ -0,0 +1,93 @@
# Create your cluster
:::info
This recording presents the essence of this page. It's recommended to read it in full for the motivation and all details.
:::
<asciinemaWidget src="/constellation/assets/create-cluster.cast" rows="20" cols="112" idleTimeLimit="3" preload="true" theme="edgeless" />
---
Creating your cluster happens through multiple phases.
The most significant ones are:
1. Creating the necessary resources in your cloud environment
2. Bootstrapping the Constellation cluster and setting up a connection
3. Installing the necessary Kubernetes components
`constellation apply` handles all this in a single command.
You can use the `--skip-phases` flag to skip specific phases of the process.
For example, if you created the infrastructure manually, you can skip the cloud resource creation phase.
See the [architecture](../architecture/orchestration.md) section for details on the inner workings of this process.
:::tip
If you don't have a cloud subscription, you can also set up a [local Constellation cluster using virtualization](../getting-started/first-steps-local.md) for testing.
:::
Before you create the cluster, make sure to have a [valid configuration file](./config.md).
<tabs groupId="usage">
<tabItem value="cli" label="CLI">
```bash
constellation apply
```
`apply` stores the state of your cluster's cloud resources in a [`constellation-terraform`](../architecture/orchestration.md#cluster-creation-process) directory in your workspace.
</tabItem>
<tabItem value="self-managed" label="Self-managed">
Self-managed infrastructure allows for more flexibility in the setup, by separating the infrastructure setup from the Constellation cluster management.
This provides flexibility in DevOps and can meet potential regulatory requirements.
It's recommended to use Terraform for infrastructure management, but you can use any tool of your choice.
:::info
When using Terraform, you can use the [Constellation Terraform provider](./terraform-provider.md) to manage the entire Constellation cluster lifecycle.
:::
You can refer to the Terraform files for the selected CSP from the [Constellation GitHub repository](https://github.com/edgelesssys/constellation/tree/main/terraform/infrastructure) for a minimum Constellation cluster configuration. From this base, you can now add, edit, or substitute resources per your own requirements with the infrastructure
management tooling of your choice. You need to keep the essential functionality of the base configuration in order for your cluster to function correctly.
<!-- vale off -->
:::info
On Azure, if the enforcement policy is set to `MAAFallback` in `constellation-config.yaml`, a manual update to the MAA provider's policy is necessary.
You can apply the update with the following command after creating the infrastructure, with `<URL>` being the URL of the MAA provider (i.e., `$(terraform output attestation_url | jq -r)`, when using the minimal Terraform configuration).
```bash
constellation maa-patch <URL>
```
:::
<!-- vale on -->
Make sure all necessary resources are created, e.g., through checking your CSP's portal and retrieve the necessary values, aligned with the outputs (specified in `outputs.tf`) of the base configuration.
Fill these outputs into the corresponding fields of the `Infrastructure` block inside the `constellation-state.yaml` file. For example, fill the IP or DNS name your cluster can be reached at into the `.Infrastructure.ClusterEndpoint` field.
With the required cloud resources set up, continue with initializing your cluster.
```bash
constellation apply --skip-phases=infrastructure
```
</tabItem>
</tabs>
Finally, configure `kubectl` for your cluster:
```bash
export KUBECONFIG="$PWD/constellation-admin.conf"
```
🏁 That's it. You've successfully created a Constellation cluster.
### Troubleshooting
In case `apply` fails, the CLI collects logs from the bootstrapping instance and stores them inside `constellation-cluster.log`.

15
docs/versioned_docs/version-2.16/workflows/lb.md Normal file
View file

@ -0,0 +1,15 @@
# Expose a service
Constellation integrates the native load balancers of each CSP. Therefore, to expose a service simply [create a service of type `LoadBalancer`](https://kubernetes.io/docs/concepts/services-networking/service/#loadbalancer).
## Internet-facing LB service on AWS
To expose your application service externally you might want to use a Kubernetes Service of type `LoadBalancer`. On AWS, load-balancing is achieved through the [AWS Load Balancing Controller](https://kubernetes-sigs.github.io/aws-load-balancer-controller) as in the managed EKS.
Since recent versions, the controller deploy an internal LB by default requiring to set an annotation `service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing` to have an internet-facing LB. For more details, see the [official docs](https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.2/guide/service/nlb/).
For general information on LB with AWS see [Network load balancing on Amazon EKS](https://docs.aws.amazon.com/eks/latest/userguide/network-load-balancing.html).
:::caution
Before terminating the cluster, all LB backed services should be deleted, so that the controller can cleanup the related resources.
:::

148
docs/versioned_docs/version-2.16/workflows/recovery.md Normal file
View file

@ -0,0 +1,148 @@
# Recover your cluster
Recovery of a Constellation cluster means getting it back into a healthy state after too many concurrent node failures in the control plane.
Reasons for an unhealthy cluster can vary from a power outage, or planned reboot, to migration of nodes and regions.
Recovery events are rare, because Constellation is built for high availability and automatically and securely replaces failed nodes. When a node is replaced, Constellation's control plane first verifies the new node before it sends the node the cryptographic keys required to decrypt its [state disk](../architecture/images.md#state-disk).
Constellation provides a recovery mechanism for cases where the control plane has failed and is unable to replace nodes.
The `constellation recover` command securely connects to all nodes in need of recovery using [attested TLS](../architecture/attestation.md#attested-tls-atls) and provides them with the keys to decrypt their state disks and continue booting.
## Identify unhealthy clusters
The first step to recovery is identifying when a cluster becomes unhealthy.
Usually, this can be first observed when the Kubernetes API server becomes unresponsive.
You can check the health status of the nodes via the cloud service provider (CSP).
Constellation provides logging information on the boot process and status via serial console output.
In the following, you'll find detailed descriptions for identifying clusters stuck in recovery for each CSP.
<tabs groupId="csp">
<tabItem value="azure" label="Azure">
In the Azure portal, find the cluster's resource group.
Inside the resource group, open the control plane *Virtual machine scale set* `constellation-scale-set-controlplanes-<suffix>`.
On the left, go to **Settings** > **Instances** and check that enough members are in a *Running* state.
Second, check the boot logs of these *Instances*.
In the scale set's *Instances* view, open the details page of the desired instance.
On the left, go to **Support + troubleshooting** > **Serial console**.
In the serial console output, search for `Waiting for decryption key`.
Similar output to the following means your node was restarted and needs to decrypt the [state disk](../architecture/images.md#state-disk):
```json
{"level":"INFO","ts":"2022-09-08T09:56:41Z","caller":"cmd/main.go:55","msg":"Starting disk-mapper","version":"2.0.0","cloudProvider":"azure"}
{"level":"INFO","ts":"2022-09-08T09:56:43Z","logger":"setupManager","caller":"setup/setup.go:72","msg":"Preparing existing state disk"}
{"level":"INFO","ts":"2022-09-08T09:56:43Z","logger":"recoveryServer","caller":"recoveryserver/server.go:59","msg":"Starting RecoveryServer"}
{"level":"INFO","ts":"2022-09-08T09:56:43Z","logger":"rejoinClient","caller":"rejoinclient/client.go:65","msg":"Starting RejoinClient"}
```
The node will then try to connect to the [*JoinService*](../architecture/microservices.md#joinservice) and obtain the decryption key.
If this fails due to an unhealthy control plane, you will see log messages similar to the following:
```json
{"level":"INFO","ts":"2022-09-08T09:56:43Z","logger":"rejoinClient","caller":"rejoinclient/client.go:77","msg":"Received list with JoinService endpoints","endpoints":["10.9.0.5:30090","10.9.0.6:30090"]}
{"level":"INFO","ts":"2022-09-08T09:56:43Z","logger":"rejoinClient","caller":"rejoinclient/client.go:96","msg":"Requesting rejoin ticket","endpoint":"10.9.0.5:30090"}
{"level":"WARN","ts":"2022-09-08T09:57:03Z","logger":"rejoinClient","caller":"rejoinclient/client.go:101","msg":"Failed to rejoin on endpoint","error":"rpc error: code = Unavailable desc = connection error: desc = \"transport: Error while dialing dial tcp 10.9.0.5:30090: i/o timeout\"","endpoint":"10.9.0.5:30090"}
{"level":"INFO","ts":"2022-09-08T09:57:03Z","logger":"rejoinClient","caller":"rejoinclient/client.go:96","msg":"Requesting rejoin ticket","endpoint":"10.9.0.6:30090"}
{"level":"WARN","ts":"2022-09-08T09:57:23Z","logger":"rejoinClient","caller":"rejoinclient/client.go:101","msg":"Failed to rejoin on endpoint","error":"rpc error: code = Unavailable desc = connection error: desc = \"transport: Error while dialing dial tcp 10.9.0.6:30090: i/o timeout\"","endpoint":"10.9.0.6:30090"}
{"level":"ERROR","ts":"2022-09-08T09:57:23Z","logger":"rejoinClient","caller":"rejoinclient/client.go:110","msg":"Failed to rejoin on all endpoints"}
```
This means that you have to recover the node manually.
</tabItem>
<tabItem value="gcp" label="GCP">
First, check that the control plane *Instance Group* has enough members in a *Ready* state.
In the GCP Console, go to **Instance Groups** and check the group for the cluster's control plane `<cluster-name>-control-plane-<suffix>`.
Second, check the status of the *VM Instances*.
Go to **VM Instances** and open the details of the desired instance.
Check the serial console output of that instance by opening the **Logs** > **Serial port 1 (console)** page:
![GCP portal serial console link](../_media/recovery-gcp-serial-console-link.png)
In the serial console output, search for `Waiting for decryption key`.
Similar output to the following means your node was restarted and needs to decrypt the [state disk](../architecture/images.md#state-disk):
```json
{"level":"INFO","ts":"2022-09-08T10:21:53Z","caller":"cmd/main.go:55","msg":"Starting disk-mapper","version":"2.0.0","cloudProvider":"gcp"}
{"level":"INFO","ts":"2022-09-08T10:21:53Z","logger":"setupManager","caller":"setup/setup.go:72","msg":"Preparing existing state disk"}
{"level":"INFO","ts":"2022-09-08T10:21:53Z","logger":"rejoinClient","caller":"rejoinclient/client.go:65","msg":"Starting RejoinClient"}
{"level":"INFO","ts":"2022-09-08T10:21:53Z","logger":"recoveryServer","caller":"recoveryserver/server.go:59","msg":"Starting RecoveryServer"}
```
The node will then try to connect to the [*JoinService*](../architecture/microservices.md#joinservice) and obtain the decryption key.
If this fails due to an unhealthy control plane, you will see log messages similar to the following:
```json
{"level":"INFO","ts":"2022-09-08T10:21:53Z","logger":"rejoinClient","caller":"rejoinclient/client.go:77","msg":"Received list with JoinService endpoints","endpoints":["192.168.178.4:30090","192.168.178.2:30090"]}
{"level":"INFO","ts":"2022-09-08T10:21:53Z","logger":"rejoinClient","caller":"rejoinclient/client.go:96","msg":"Requesting rejoin ticket","endpoint":"192.168.178.4:30090"}
{"level":"WARN","ts":"2022-09-08T10:21:53Z","logger":"rejoinClient","caller":"rejoinclient/client.go:101","msg":"Failed to rejoin on endpoint","error":"rpc error: code = Unavailable desc = connection error: desc = \"transport: Error while dialing dial tcp 192.168.178.4:30090: connect: connection refused\"","endpoint":"192.168.178.4:30090"}
{"level":"INFO","ts":"2022-09-08T10:21:53Z","logger":"rejoinClient","caller":"rejoinclient/client.go:96","msg":"Requesting rejoin ticket","endpoint":"192.168.178.2:30090"}
{"level":"WARN","ts":"2022-09-08T10:22:13Z","logger":"rejoinClient","caller":"rejoinclient/client.go:101","msg":"Failed to rejoin on endpoint","error":"rpc error: code = Unavailable desc = connection error: desc = \"transport: Error while dialing dial tcp 192.168.178.2:30090: i/o timeout\"","endpoint":"192.168.178.2:30090"}
{"level":"ERROR","ts":"2022-09-08T10:22:13Z","logger":"rejoinClient","caller":"rejoinclient/client.go:110","msg":"Failed to rejoin on all endpoints"}
```
This means that you have to recover the node manually.
</tabItem>
<tabItem value="aws" label="AWS">
First, open the AWS console to view all Auto Scaling Groups (ASGs) in the region of your cluster. Select the ASG of the control plane `<cluster-name>-<UID>-control-plane` and check that enough members are in a *Running* state.
Second, check the boot logs of these *Instances*. In the ASG's *Instance management* view, select each desired instance. In the upper right corner, select **Action > Monitor and troubleshoot > Get system log**.
In the serial console output, search for `Waiting for decryption key`.
Similar output to the following means your node was restarted and needs to decrypt the [state disk](../architecture/images.md#state-disk):
```json
{"level":"INFO","ts":"2022-09-08T10:21:53Z","caller":"cmd/main.go:55","msg":"Starting disk-mapper","version":"2.0.0","cloudProvider":"gcp"}
{"level":"INFO","ts":"2022-09-08T10:21:53Z","logger":"setupManager","caller":"setup/setup.go:72","msg":"Preparing existing state disk"}
{"level":"INFO","ts":"2022-09-08T10:21:53Z","logger":"rejoinClient","caller":"rejoinclient/client.go:65","msg":"Starting RejoinClient"}
{"level":"INFO","ts":"2022-09-08T10:21:53Z","logger":"recoveryServer","caller":"recoveryserver/server.go:59","msg":"Starting RecoveryServer"}
```
The node will then try to connect to the [*JoinService*](../architecture/microservices.md#joinservice) and obtain the decryption key.
If this fails due to an unhealthy control plane, you will see log messages similar to the following:
```json
{"level":"INFO","ts":"2022-09-08T10:21:53Z","logger":"rejoinClient","caller":"rejoinclient/client.go:77","msg":"Received list with JoinService endpoints","endpoints":["192.168.178.4:30090","192.168.178.2:30090"]}
{"level":"INFO","ts":"2022-09-08T10:21:53Z","logger":"rejoinClient","caller":"rejoinclient/client.go:96","msg":"Requesting rejoin ticket","endpoint":"192.168.178.4:30090"}
{"level":"WARN","ts":"2022-09-08T10:21:53Z","logger":"rejoinClient","caller":"rejoinclient/client.go:101","msg":"Failed to rejoin on endpoint","error":"rpc error: code = Unavailable desc = connection error: desc = \"transport: Error while dialing dial tcp 192.168.178.4:30090: connect: connection refused\"","endpoint":"192.168.178.4:30090"}
{"level":"INFO","ts":"2022-09-08T10:21:53Z","logger":"rejoinClient","caller":"rejoinclient/client.go:96","msg":"Requesting rejoin ticket","endpoint":"192.168.178.2:30090"}
{"level":"WARN","ts":"2022-09-08T10:22:13Z","logger":"rejoinClient","caller":"rejoinclient/client.go:101","msg":"Failed to rejoin on endpoint","error":"rpc error: code = Unavailable desc = connection error: desc = \"transport: Error while dialing dial tcp 192.168.178.2:30090: i/o timeout\"","endpoint":"192.168.178.2:30090"}
{"level":"ERROR","ts":"2022-09-08T10:22:13Z","logger":"rejoinClient","caller":"rejoinclient/client.go:110","msg":"Failed to rejoin on all endpoints"}
```
This means that you have to recover the node manually.
</tabItem>
</tabs>
## Recover a cluster
Recovering a cluster requires the following parameters:
* The `constellation-state.yaml` file in your working directory or the cluster's endpoint
* The master secret of the cluster
A cluster can be recovered like this:
```bash
$ constellation recover --master-secret constellation-mastersecret.json
Pushed recovery key.
Pushed recovery key.
Pushed recovery key.
Recovered 3 control-plane nodes.
```
In the serial console output of the node you'll see a similar output to the following:
```json
{"level":"INFO","ts":"2022-09-08T10:26:59Z","logger":"recoveryServer","caller":"recoveryserver/server.go:93","msg":"Received recover call"}
{"level":"INFO","ts":"2022-09-08T10:26:59Z","logger":"recoveryServer","caller":"recoveryserver/server.go:125","msg":"Received state disk key and measurement secret, shutting down server"}
{"level":"INFO","ts":"2022-09-08T10:26:59Z","logger":"recoveryServer.gRPC","caller":"zap/server_interceptors.go:61","msg":"finished streaming call with code OK","grpc.start_time":"2022-09-08T10:26:59Z","system":"grpc","span.kind":"server","grpc.service":"recoverproto.API","grpc.method":"Recover","peer.address":"192.0.2.3:41752","grpc.code":"OK","grpc.time_ms":15.701}
{"level":"INFO","ts":"2022-09-08T10:27:13Z","logger":"rejoinClient","caller":"rejoinclient/client.go:87","msg":"RejoinClient stopped"}
```

58
docs/versioned_docs/version-2.16/workflows/s3proxy.md Normal file
View file

@ -0,0 +1,58 @@
# Install s3proxy
Constellation includes a transparent client-side encryption proxy for [AWS S3](https://aws.amazon.com/de/s3/) and compatible stores.
s3proxy encrypts objects before sending them to S3 and automatically decrypts them on retrieval, without requiring changes to your application.
With s3proxy, you can use S3 for storage in a confidential way without having to trust the storage provider.
## Limitations
Currently, s3proxy has the following limitations:
- Only `PutObject` and `GetObject` requests are encrypted/decrypted by s3proxy.
By default, s3proxy will block requests that may expose unencrypted data to S3 (e.g. UploadPart).
The `allow-multipart` flag disables request blocking for evaluation purposes.
- Using the [Range](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObject.html#API_GetObject_RequestSyntax) header on `GetObject` is currently not supported and will result in an error.
These limitations will be removed with future iterations of s3proxy.
If you want to use s3proxy but these limitations stop you from doing so, consider [opening an issue](https://github.com/edgelesssys/constellation/issues/new?assignees=&labels=&projects=&template=feature_request.yml).
## Deployment
You can add the s3proxy to your Constellation cluster as follows:
1. Add the Edgeless Systems chart repository:
```bash
helm repo add edgeless https://helm.edgeless.systems/stable
helm repo update
```
2. Set ACCESS_KEY and ACCESS_SECRET to valid credentials you want s3proxy to use to interact with S3.
3. Deploy s3proxy:
```bash
helm install s3proxy edgeless/s3proxy --set awsAccessKeyID="$ACCESS_KEY" --set awsSecretAccessKey="$ACCESS_SECRET"
```
If you want to run a demo application, check out the [Filestash with s3proxy](../getting-started/examples/filestash-s3proxy.md) example.
## Technical details
### Encryption
s3proxy relies on Google's [Tink Cryptographic Library](https://developers.google.com/tink) to implement cryptographic operations securely.
The used cryptographic primitives are [NIST SP 800 38f](https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-38F.pdf) for key wrapping and [AES](https://en.wikipedia.org/wiki/Advanced_Encryption_Standard)-[GCM](https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation#Galois/counter_(GCM)) with 256 bit keys for data encryption.
s3proxy uses [envelope encryption](https://cloud.google.com/kms/docs/envelope-encryption) to encrypt objects.
This means s3proxy uses a key encryption key (KEK) issued by the [KeyService](../architecture/microservices.md#keyservice) to encrypt data encryption keys (DEKs).
Each S3 object is encrypted with its own DEK.
The encrypted DEK is then saved as metadata of the encrypted object.
This enables key rotation of the KEK without re-encrypting the data in S3.
The approach also allows access to objects from different locations, as long as each location has access to the KEK.
### Traffic interception
To use s3proxy, you have to redirect your outbound S3 traffic to s3proxy.
This can either be done by modifying your client application or by changing the deployment of your application.
The necessary deployment modifications are to add DNS redirection and a trusted TLS certificate to the client's trust store.
DNS redirection can be defined for each pod, allowing you to use s3proxy for one application without changing other applications in the same cluster.
Adding a trusted TLS certificate is necessary as clients communicate with s3proxy via HTTPS.
To have your client application trust s3proxy's TLS certificate, the certificate has to be added to the client's certificate trust store.
The [Filestash with s3proxy](../getting-started/examples/filestash-s3proxy.md) example shows how to do this.

91
docs/versioned_docs/version-2.16/workflows/sbom.md Normal file
View file

@ -0,0 +1,91 @@
# Consume software bill of materials (SBOMs)
<asciinemaWidget src="/constellation/assets/check-sbom.cast" rows="20" cols="112" idleTimeLimit="3" preload="true" theme="edgeless" />
---
Constellation builds produce a [software bill of materials (SBOM)](https://www.ntia.gov/SBOM) for each generated [artifact](../architecture/microservices.md).
You can use SBOMs to make informed decisions about dependencies and vulnerabilities in a given application. Enterprises rely on SBOMs to maintain an inventory of used applications, which allows them to take data-driven approaches to managing risks related to vulnerabilities.
SBOMs for Constellation are generated using [Syft](https://github.com/anchore/syft), signed using [Cosign](https://github.com/sigstore/cosign), and stored with the produced artifact.
:::note
The public key for Edgeless Systems' long-term code-signing key is:
```
-----BEGIN PUBLIC KEY-----
MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEf8F1hpmwE+YCFXzjGtaQcrL6XZVT
JmEe5iSLvG1SyQSAew7WdMKF6o9t8e2TFuCkzlOhhlws2OHWbiFZnFWCFw==
-----END PUBLIC KEY-----
```
The public key is also available for download at https://edgeless.systems/es.pub and in the Twitter profile [@EdgelessSystems](https://twitter.com/EdgelessSystems).
Make sure the key is available in a file named `cosign.pub` to execute the following examples.
:::
## Verify and download SBOMs
The following sections detail how to work with each type of artifact to verify and extract the SBOM.
### Constellation CLI
The SBOM for Constellation CLI is made available on the [GitHub release page](https://github.com/edgelesssys/constellation/releases). The SBOM (`constellation.spdx.sbom`) and corresponding signature (`constellation.spdx.sbom.sig`) are valid for each Constellation CLI for a given version, regardless of architecture and operating system.
```bash
curl -LO https://github.com/edgelesssys/constellation/releases/download/v2.2.0/constellation.spdx.sbom
curl -LO https://github.com/edgelesssys/constellation/releases/download/v2.2.0/constellation.spdx.sbom.sig
cosign verify-blob --key cosign.pub --signature constellation.spdx.sbom.sig constellation.spdx.sbom
```
### Container Images
SBOMs for container images are [attached to the image using Cosign](https://docs.sigstore.dev/signing/other_types#sboms-software-bill-of-materials) and uploaded to the same registry.
As a consumer, use cosign to download and verify the SBOM:
```bash
# Verify and download the attestation statement
cosign verify-attestation ghcr.io/edgelesssys/constellation/verification-service@v2.2.0 --type 'https://cyclonedx.org/bom' --key cosign.pub --output-file verification-service.att.json
# Extract SBOM from attestation statement
jq -r .payload verification-service.att.json | base64 -d > verification-service.cyclonedx.sbom
```
A successful verification should result in similar output:
```shell-session
$ cosign verify-attestation ghcr.io/edgelesssys/constellation/verification-service@v2.2.0 --type 'https://cyclonedx.org/bom' --key cosign.pub --output-file verification-service.sbom
Verification for ghcr.io/edgelesssys/constellation/verification-service@v2.2.0 --
The following checks were performed on each of these signatures:
- The cosign claims were validated
- The signatures were verified against the specified public key
$ jq -r .payload verification-service.sbom | base64 -d > verification-service.cyclonedx.sbom
```
:::note
This example considers only the `verification-service`. The same approach works for all containers in the [Constellation container registry](https://github.com/orgs/edgelesssys/packages?repo_name=constellation).
:::
<!--
TODO(malt3): Once mkosi is implemented
## Operating System
-->
## Vulnerability scanning
You can use a plethora of tools to consume SBOMs. This section provides suggestions for tools that are popular and known to produce reliable results, but any tool that consumes [SPDX](https://spdx.dev/) or [CycloneDX](https://cyclonedx.org/) files should work.
Syft is able to [convert between the two formats](https://github.com/anchore/syft#format-conversion-experimental) in case you require a specific type.
### Grype
[Grype](https://github.com/anchore/grype) is a CLI tool that lends itself well for integration into CI/CD systems or local developer machines. It's also able to consume the signed attestation statement directly and does the verification in one go.
```bash
grype att:verification-service.sbom --key cosign.pub --add-cpes-if-none -q
```
### Dependency Track
[Dependency Track](https://dependencytrack.org/) is one of the oldest and most mature solutions when it comes to managing software inventory and vulnerabilities. Once imported, it continuously scans SBOMs for new vulnerabilities. It supports the CycloneDX format and provides direct guidance on how to comply with [U.S. Executive Order 14028](https://docs.dependencytrack.org/usage/executive-order-14028/).

111
docs/versioned_docs/version-2.16/workflows/scale.md Normal file
View file

@ -0,0 +1,111 @@
# Scale your cluster
Constellation provides all features of a Kubernetes cluster including scaling and autoscaling.
## Worker node scaling
### Autoscaling
Constellation comes with autoscaling disabled by default. To enable autoscaling, find the scaling group of
worker nodes:
```bash
kubectl get scalinggroups -o json | yq '.items | .[] | select(.spec.role == "Worker") | [{"name": .metadata.name, "nodeGoupName": .spec.nodeGroupName}]'
```
This will output a list of scaling groups with the corresponding cloud provider name (`name`) and the cloud provider agnostic name of the node group (`nodeGroupName`).
Then, patch the `autoscaling` field of the scaling group resource with the desired `name` to `true`:
```bash
# Replace <name> with the name of the scaling group you want to enable autoscaling for
worker_group=<name>
kubectl patch scalinggroups $worker_group --patch '{"spec":{"autoscaling": true}}' --type='merge'
kubectl get scalinggroup $worker_group -o jsonpath='{.spec}' | yq -P
```
The cluster autoscaler now automatically provisions additional worker nodes so that all pods have a place to run.
You can configure the minimum and maximum number of worker nodes in the scaling group by patching the `min` or
`max` fields of the scaling group resource:
```bash
kubectl patch scalinggroups $worker_group --patch '{"spec":{"max": 5}}' --type='merge'
kubectl get scalinggroup $worker_group -o jsonpath='{.spec}' | yq -P
```
The cluster autoscaler will now never provision more than 5 worker nodes.
If you want to see the autoscaling in action, try to add a deployment with a lot of replicas, like the
following Nginx deployment. The number of replicas needed to trigger the autoscaling depends on the size of
and count of your worker nodes. Wait for the rollout of the deployment to finish and compare the number of
worker nodes before and after the deployment:
```bash
kubectl create deployment nginx --image=nginx --replicas 150
kubectl -n kube-system get nodes
kubectl rollout status deployment nginx
kubectl -n kube-system get nodes
```
### Manual scaling
Alternatively, you can manually scale your cluster up or down:
<tabs groupId="csp">
<tabItem value="azure" label="Azure">
1. Find your Constellation resource group.
2. Select the `scale-set-workers`.
3. Go to **settings** and **scaling**.
4. Set the new **instance count** and **save**.
</tabItem>
<tabItem value="gcp" label="GCP">
1. In Compute Engine go to [Instance Groups](https://console.cloud.google.com/compute/instanceGroups/).
2. **Edit** the **worker** instance group.
3. Set the new **number of instances** and **save**.
</tabItem>
<tabItem value="aws" label="AWS">
1. Go to Auto Scaling Groups and select the worker ASG to scale up.
2. Click **Edit**
3. Set the new (increased) **Desired capacity** and **Update**.
</tabItem>
</tabs>
## Control-plane node scaling
Control-plane nodes can **only be scaled manually and only scaled up**!
To increase the number of control-plane nodes, follow these steps:
<tabs groupId="csp">
<tabItem value="azure" label="Azure">
1. Find your Constellation resource group.
2. Select the `scale-set-controlplanes`.
3. Go to **settings** and **scaling**.
4. Set the new (increased) **instance count** and **save**.
</tabItem>
<tabItem value="gcp" label="GCP">
1. In Compute Engine go to [Instance Groups](https://console.cloud.google.com/compute/instanceGroups/).
2. **Edit** the **control-plane** instance group.
3. Set the new (increased) **number of instances** and **save**.
</tabItem>
<tabItem value="aws" label="AWS">
1. Go to Auto Scaling Groups and select the control-plane ASG to scale up.
2. Click **Edit**
3. Set the new (increased) **Desired capacity** and **Update**.
</tabItem>
</tabs>
If you scale down the number of control-planes nodes, the removed nodes won't be able to exit the `etcd` cluster correctly. This will endanger the quorum that's required to run a stable Kubernetes control plane.

281
docs/versioned_docs/version-2.16/workflows/storage.md Normal file
View file

@ -0,0 +1,281 @@
# Use persistent storage
Persistent storage in Kubernetes requires cloud-specific configuration.
For abstraction of container storage, Kubernetes offers [volumes](https://kubernetes.io/docs/concepts/storage/volumes/),
allowing users to mount storage solutions directly into containers.
The [Container Storage Interface (CSI)](https://kubernetes-csi.github.io/docs/) is the standard interface for exposing arbitrary block and file storage systems into containers in Kubernetes.
Cloud service providers (CSPs) offer their own CSI-based solutions for cloud storage.
## Confidential storage
Most cloud storage solutions support encryption, such as [GCE Persistent Disks (PD)](https://cloud.google.com/kubernetes-engine/docs/how-to/using-cmek).
Constellation supports the available CSI-based storage options for Kubernetes engines in AWS, Azure, GCP, and STACKIT.
However, their encryption takes place in the storage backend and is managed by the CSP.
Thus, using the default CSI drivers for these storage types means trusting the CSP with your persistent data.
To address this, Constellation provides CSI drivers for AWS EBS, Azure Disk, GCE PD, and OpenStack Cinder, offering [encryption on the node level](../architecture/keys.md#storage-encryption). They enable transparent encryption for persistent volumes without needing to trust the cloud backend. Plaintext data never leaves the confidential VM context, offering you confidential storage.
For more details see [encrypted persistent storage](../architecture/encrypted-storage.md).
## CSI drivers
Constellation supports the following drivers, which offer node-level encryption and optional integrity protection.
<tabs groupId="csp">
<tabItem value="azure" label="Azure">
**Constellation CSI driver for Azure Disk**:
Mount Azure [Disk Storage](https://azure.microsoft.com/en-us/services/storage/disks/#overview) into your Constellation cluster.
See the instructions on how to [install the Constellation CSI driver](#installation) or check out the [repository](https://github.com/edgelesssys/constellation-azuredisk-csi-driver) for more information.
Since Azure Disks are mounted as `ReadWriteOnce`, they're only available to a single pod.
</tabItem>
<tabItem value="gcp" label="GCP">
**Constellation CSI driver for GCP Persistent Disk**:
Mount [Persistent Disk](https://cloud.google.com/persistent-disk) block storage into your Constellation cluster.
Follow the instructions on how to [install the Constellation CSI driver](#installation) or check out the [repository](https://github.com/edgelesssys/constellation-gcp-compute-persistent-disk-csi-driver) for more information.
</tabItem>
<tabItem value="aws" label="AWS">
**Constellation CSI driver for AWS Elastic Block Store**
Mount [Elastic Block Store](https://aws.amazon.com/ebs/) storage volumes into your Constellation cluster.
Follow the instructions on how to [install the Constellation CSI driver](#installation) or check out the [repository](https://github.com/edgelesssys/constellation-aws-ebs-csi-driver) for more information.
</tabItem>
<tabItem value="stackit" label="STACKIT">
**Constellation CSI driver for STACKIT / OpenStack Cinder**
Mount [Cinder](https://docs.openstack.org/cinder/latest/) block storage volumes into your Constellation cluster.
Follow the instructions on how to [install the Constellation CSI driver](#installation) or check out the [repository](https://github.com/edgelesssys/constellation-cloud-provider-openstack) for more information.
</tabItem>
</tabs>
Note that in case the options above aren't a suitable solution for you, Constellation is compatible with all other CSI-based storage options. For example, you can use [AWS EFS](https://docs.aws.amazon.com/en_en/eks/latest/userguide/efs-csi.html), [Azure Files](https://docs.microsoft.com/en-us/azure/storage/files/storage-files-introduction), or [GCP Filestore](https://cloud.google.com/filestore) with Constellation out of the box. Constellation is just not providing transparent encryption on the node level for these storage types yet.
## Installation
The Constellation CLI automatically installs Constellation's CSI driver for the selected CSP in your cluster.
If you don't need a CSI driver or wish to deploy your own, you can disable the automatic installation by setting `deployCSIDriver` to `false` in your Constellation config file.
<tabs groupId="csp">
<tabItem value="azure" label="Azure">
Azure comes with two storage classes by default.
* `encrypted-rwo`
* Uses [Standard SSDs](https://learn.microsoft.com/en-us/azure/virtual-machines/disks-types#standard-ssds)
* ext-4 filesystem
* Encryption of all data written to disk
* `integrity-encrypted-rwo`
* Uses [Premium SSDs](https://learn.microsoft.com/en-us/azure/virtual-machines/disks-types#premium-ssds)
* ext-4 filesystem
* Encryption of all data written to disk
* Integrity protection of data written to disk
For more information on encryption algorithms and key sizes, refer to [cryptographic algorithms](../architecture/encrypted-storage.md#cryptographic-algorithms).
:::info
The default storage class is set to `encrypted-rwo` for performance reasons.
If you want integrity-protected storage, set the `storageClassName` parameter of your persistent volume claim to `integrity-encrypted-rwo`.
Alternatively, you can create your own storage class with integrity protection enabled by adding `csi.storage.k8s.io/fstype: ext4-integrity` to the class `parameters`.
Or use another filesystem by specifying another file system type with the suffix `-integrity`, e.g., `csi.storage.k8s.io/fstype: xfs-integrity`.
Note that volume expansion isn't supported for integrity-protected disks.
:::
</tabItem>
<tabItem value="gcp" label="GCP">
GCP comes with two storage classes by default.
* `encrypted-rwo`
* Uses [standard persistent disks](https://cloud.google.com/compute/docs/disks#pdspecs)
* ext-4 filesystem
* Encryption of all data written to disk
* `integrity-encrypted-rwo`
* Uses [performance (SSD) persistent disks](https://cloud.google.com/compute/docs/disks#pdspecs)
* ext-4 filesystem
* Encryption of all data written to disk
* Integrity protection of data written to disk
For more information on encryption algorithms and key sizes, refer to [cryptographic algorithms](../architecture/encrypted-storage.md#cryptographic-algorithms).
:::info
The default storage class is set to `encrypted-rwo` for performance reasons.
If you want integrity-protected storage, set the `storageClassName` parameter of your persistent volume claim to `integrity-encrypted-rwo`.
Alternatively, you can create your own storage class with integrity protection enabled by adding `csi.storage.k8s.io/fstype: ext4-integrity` to the class `parameters`.
Or use another filesystem by specifying another file system type with the suffix `-integrity`, e.g., `csi.storage.k8s.io/fstype: xfs-integrity`.
Note that volume expansion isn't supported for integrity-protected disks.
:::
</tabItem>
<tabItem value="aws" label="AWS">
AWS comes with two storage classes by default.
* `encrypted-rwo`
* Uses [SSDs of `gp3` type](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-volume-types.html)
* ext-4 filesystem
* Encryption of all data written to disk
* `integrity-encrypted-rwo`
* Uses [SSDs of `gp3` type](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-volume-types.html)
* ext-4 filesystem
* Encryption of all data written to disk
* Integrity protection of data written to disk
For more information on encryption algorithms and key sizes, refer to [cryptographic algorithms](../architecture/encrypted-storage.md#cryptographic-algorithms).
:::info
The default storage class is set to `encrypted-rwo` for performance reasons.
If you want integrity-protected storage, set the `storageClassName` parameter of your persistent volume claim to `integrity-encrypted-rwo`.
Alternatively, you can create your own storage class with integrity protection enabled by adding `csi.storage.k8s.io/fstype: ext4-integrity` to the class `parameters`.
Or use another filesystem by specifying another file system type with the suffix `-integrity`, e.g., `csi.storage.k8s.io/fstype: xfs-integrity`.
Note that volume expansion isn't supported for integrity-protected disks.
:::
</tabItem>
<tabItem value="stackit" label="STACKIT">
STACKIT comes with two storage classes by default.
* `encrypted-rwo`
* Uses [disks of `storage_premium_perf1` type](https://docs.stackit.cloud/stackit/en/service-plans-blockstorage-75137974.html)
* ext-4 filesystem
* Encryption of all data written to disk
* `integrity-encrypted-rwo`
* Uses [disks of `storage_premium_perf1` type](https://docs.stackit.cloud/stackit/en/service-plans-blockstorage-75137974.html)
* ext-4 filesystem
* Encryption of all data written to disk
* Integrity protection of data written to disk
For more information on encryption algorithms and key sizes, refer to [cryptographic algorithms](../architecture/encrypted-storage.md#cryptographic-algorithms).
:::info
The default storage class is set to `encrypted-rwo` for performance reasons.
If you want integrity-protected storage, set the `storageClassName` parameter of your persistent volume claim to `integrity-encrypted-rwo`.
Alternatively, you can create your own storage class with integrity protection enabled by adding `csi.storage.k8s.io/fstype: ext4-integrity` to the class `parameters`.
Or use another filesystem by specifying another file system type with the suffix `-integrity`, e.g., `csi.storage.k8s.io/fstype: xfs-integrity`.
Note that volume expansion isn't supported for integrity-protected disks.
:::
</tabItem>
</tabs>
1. Create a [persistent volume](https://kubernetes.io/docs/concepts/storage/persistent-volumes/)
A [persistent volume claim](https://kubernetes.io/docs/concepts/storage/persistent-volumes/#persistentvolumeclaims) is a request for storage with certain properties.
It can refer to a storage class.
The following creates a persistent volume claim, requesting 20 GB of storage via the `encrypted-rwo` storage class:
```bash
cat <<EOF | kubectl apply -f -
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: pvc-example
namespace: default
spec:
accessModes:
- ReadWriteOnce
storageClassName: encrypted-rwo
resources:
requests:
storage: 20Gi
EOF
```
2. Create a Pod with persistent storage
You can assign a persistent volume claim to an application in need of persistent storage.
The mounted volume will persist restarts.
The following creates a pod that uses the previously created persistent volume claim:
```bash
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
name: web-server
namespace: default
spec:
containers:
- name: web-server
image: nginx
volumeMounts:
- mountPath: /var/lib/www/html
name: mypvc
volumes:
- name: mypvc
persistentVolumeClaim:
claimName: pvc-example
readOnly: false
EOF
```
### Change the default storage class
The default storage class is responsible for all persistent volume claims that don't explicitly request `storageClassName`.
Constellation creates a storage class with encryption enabled and sets this as the default class.
In case you wish to change it, follow the steps below:
1. List the storage classes in your cluster:
```bash
kubectl get storageclass
```
The output is similar to this:
```shell-session
NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
encrypted-rwo (default) {your-csp}.csi.confidential.cloud Delete Immediate true 1d
integrity-encrypted-rwo {your-csp}.csi.confidential.cloud Delete Immediate false 1d
```
The default storage class is marked by `(default)`.
2. Mark old default storage class as non default
If you previously used another storage class as the default, you will have to remove that annotation:
```bash
kubectl patch storageclass encrypted-rwo -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"false"}}}'
```
3. Mark new class as the default
```bash
kubectl patch storageclass integrity-encrypted-rwo -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'
```
4. Verify that your chosen storage class is default:
```bash
kubectl get storageclass
```
The output is similar to this:
```shell-session
NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
encrypted-rwo {your-csp}.csi.confidential.cloud Delete Immediate true 1d
integrity-encrypted-rwo (default) {your-csp}.csi.confidential.cloud Delete Immediate false 1d
```

60
docs/versioned_docs/version-2.16/workflows/terminate.md Normal file
View file

@ -0,0 +1,60 @@
# Terminate your cluster
:::info
This recording presents the essence of this page. It's recommended to read it in full for the motivation and all details.
:::
<asciinemaWidget src="/constellation/assets/terminate-cluster.cast" rows="20" cols="112" idleTimeLimit="3" preload="true" theme="edgeless" />
---
You can terminate your cluster using the CLI. For this, you need the Terraform state directory named [`constellation-terraform`](../reference/terraform.md) in the current directory.
:::danger
All ephemeral storage and state of your cluster will be lost. Make sure any data is safely stored in persistent storage. Constellation can recreate your cluster and the associated encryption keys, but won't backup your application data automatically.
:::
<tabs groupId="provider">
<tabItem value="cli" label="CLI">
Terminate the cluster by running:
```bash
constellation terminate
```
Or without confirmation (e.g., for automation purposes):
```bash
constellation terminate --yes
```
This deletes all resources created by Constellation in your cloud environment.
All local files created by the `apply` command are deleted as well, except for `constellation-mastersecret.json` and the configuration file.
:::caution
Termination can fail if additional resources have been created that depend on the ones managed by Constellation. In this case, you need to delete these additional
resources manually. Just run the `terminate` command again afterward to continue the termination process of the cluster.
:::
</tabItem>
<tabItem value="terraform" label="Terraform">
Terminate the cluster by running:
```bash
terraform destroy
```
Delete all files that are no longer needed:
```bash
rm constellation-state.yaml constellation-admin.conf
```
Only the `constellation-mastersecret.json` and the configuration file remain.
</tabItem>
</tabs>

118
docs/versioned_docs/version-2.16/workflows/terraform-provider.md Normal file
View file

@ -0,0 +1,118 @@
# Use the Terraform provider
The Constellation Terraform provider allows to manage the full lifecycle of a Constellation cluster (namely creation, upgrades, and deletion) via Terraform.
The provider is available through the [Terraform registry](https://registry.terraform.io/providers/edgelesssys/constellation/latest) and is released in lock-step with Constellation releases.
## Prerequisites
- a Linux / Mac operating system (ARM64/AMD64)
- a Terraform installation of version `v1.4.4` or above
## Quick setup
This example shows how to set up a Constellation cluster with the reference IAM and infrastructure setup. This setup is also used when creating a Constellation cluster through the Constellation CLI. You can either consume the IAM / infrastructure modules through a remote source (recommended) or local files. The latter requires downloading the infrastructure and IAM modules for the corresponding CSP from `terraform-modules.zip` on the [Constellation release page](https://github.com/edgelesssys/constellation/releases/latest) and placing them in the Terraform workspace directory.
1. Create a directory (workspace) for your Constellation cluster.
```bash
mkdir constellation-workspace
cd constellation-workspace
```
2. Use one of the [example configurations for using the Constellation Terraform provider](https://github.com/edgelesssys/constellation/tree/main/terraform-provider-constellation/examples/full) or create a `main.tf` file and fill it with the resources you want to create. The [Constellation Terraform provider documentation](https://registry.terraform.io/providers/edgelesssys/constellation/latest) offers thorough documentation on the resources and their attributes.
3. Initialize and apply the Terraform configuration.
<tabs groupId="csp">
<tabItem value="azure" label="Azure">
When creating a cluster on Azure, you need to manually patch the policy of the MAA provider before creating the Constellation cluster, as this feature isn't available in Azure's Terraform provider yet. The Constellation CLI provides a utility for patching, but you
can also do it manually.
```bash
terraform init
terraform apply -target module.azure_iam # adjust resource path if not using the example configuration
terraform apply -target module.azure_infrastructure # adjust resource path if not using the example configuration
constellation maa-patch $(terraform output -raw maa_url) # adjust output path / input if not using the example configuration or manually patch the resource
terraform apply -target constellation_cluster.azure_example # adjust resource path if not using the example configuration
```
Optionally, you can prefix the `terraform apply` command with `TF_LOG=INFO` to collect [Terraform logs](https://developer.hashicorp.com/terraform/internals/debugging) while applying the configuration. This may provide helpful output in debugging scenarios.
Use the following policy if manually performing the patch.
```
version= 1.0;
authorizationrules
{
[type=="x-ms-azurevm-default-securebootkeysvalidated", value==false] => deny();
[type=="x-ms-azurevm-debuggersdisabled", value==false] => deny();
// The line below was edited to use the MAA provider within Constellation. Do not edit manually.
//[type=="secureboot", value==false] => deny();
[type=="x-ms-azurevm-signingdisabled", value==false] => deny();
[type=="x-ms-azurevm-dbvalidated", value==false] => deny();
[type=="x-ms-azurevm-dbxvalidated", value==false] => deny();
=> permit();
};
issuancerules
{
};
```
</tabItem>
<tabItem value="aws" label="AWS">
Initialize the providers and apply the configuration.
```bash
terraform init
terraform apply
```
Optionally, you can prefix the `terraform apply` command with `TF_LOG=INFO` to collect [Terraform logs](https://developer.hashicorp.com/terraform/internals/debugging) while applying the configuration. This may provide helpful output in debugging scenarios.
</tabItem>
<tabItem value="gcp" label="GCP">
Initialize the providers and apply the configuration.
```bash
terraform init
terraform apply
```
Optionally, you can prefix the `terraform apply` command with `TF_LOG=INFO` to collect [Terraform logs](https://developer.hashicorp.com/terraform/internals/debugging) while applying the configuration. This may provide helpful output in debugging scenarios.
</tabItem>
</tabs>
4. Connect to the cluster.
```bash
terraform output -raw kubeconfig > constellation-admin.conf
export KUBECONFIG=$(realpath constellation-admin.conf)
```
## Bringing your own infrastructure
Instead of using the example infrastructure used in the [quick setup](#quick-setup), you can also provide your own infrastructure.
If you need a starting point for a custom infrastructure setup, you can download the infrastructure / IAM Terraform modules for the respective CSP from the Constellation [GitHub releases](https://github.com/edgelesssys/constellation/releases). You can modify and extend the modules per your requirements, while keeping the basic functionality intact.
The module contains:
- `{csp}`: cloud resources the cluster runs on
- `iam/{csp}`: IAM resources used within the cluster
When upgrading your cluster, make sure to check the Constellation release notes for potential breaking changes in the reference infrastructure / IAM modules that need to be considered.
## Cluster upgrades
:::tip
Also see the [general documentation on cluster upgrades](./upgrade.md).
:::
The steps for applying the upgrade are as follows:
1. Update the version constraint of the Constellation Terraform provider in the `required_providers` block in your Terraform configuration.
2. If you explicitly set any of the version attributes of the provider's resources and data sources (e.g. `image_version` or `constellation_microservice_version`), make sure to update them too. Refer to Constellation's [version support policy](https://github.com/edgelesssys/constellation/blob/main/dev-docs/workflows/versions-support.md) for more information on how each Constellation version and its dependencies are supported.
3. Update the IAM / infrastructure configuration.
- For [remote addresses as module sources](https://developer.hashicorp.com/terraform/language/modules/sources#fetching-archives-over-http), update the version number inside the address of the `source` field of the infrastructure / IAM module to the target version.
- For [local paths as module sources](https://developer.hashicorp.com/terraform/language/modules/sources#local-paths) or when [providing your own infrastructure](#bringing-your-own-infrastructure), see the changes made in the reference modules since the upgrade's origin version and adjust your infrastructure / IAM configuration accordingly.
4. Upgrade the Terraform module and provider dependencies and apply the targeted configuration.
```bash
terraform init -upgrade
terraform apply
```

133
docs/versioned_docs/version-2.16/workflows/troubleshooting.md Normal file
View file

@ -0,0 +1,133 @@
# Troubleshooting
This section aids you in finding problems when working with Constellation.
## Common issues
### Issues with creating new clusters
When you create a new cluster, you should always use the [latest release](https://github.com/edgelesssys/constellation/releases/latest).
If something doesn't work, check out the [known issues](https://github.com/edgelesssys/constellation/issues?q=is%3Aopen+is%3Aissue+label%3A%22known+issue%22).
### Azure: Resource Providers can't be registered
On Azure, you may receive the following error when running `apply` or `terminate` with limited IAM permissions:
```shell-session
Error: Error ensuring Resource Providers are registered.
Terraform automatically attempts to register the Resource Providers it supports to
ensure it's able to provision resources.
If you don't have permission to register Resource Providers you may wish to use the
"skip_provider_registration" flag in the Provider block to disable this functionality.
[...]
```
To continue, please ensure that the [required resource providers](../getting-started/install.md#required-permissions) have been registered in your subscription by your administrator.
Afterward, set `ARM_SKIP_PROVIDER_REGISTRATION=true` as an environment variable and either run `apply` or `terminate` again.
For example:
```bash
ARM_SKIP_PROVIDER_REGISTRATION=true constellation apply
```
Or alternatively, for `terminate`:
```bash
ARM_SKIP_PROVIDER_REGISTRATION=true constellation terminate
```
### Nodes fail to join with error `untrusted measurement value`
This error indicates that a node's [attestation statement](../architecture/attestation.md) contains measurements that don't match the trusted values expected by the [JoinService](../architecture/microservices.md#joinservice).
This may for example happen if the cloud provider updates the VM's firmware such that it influences the [runtime measurements](../architecture/attestation.md#runtime-measurements) in an unforeseen way.
A failed upgrade due to an erroneous attestation config can also cause this error.
You can change the expected measurements to resolve the failure.
:::caution
Attestation and trusted measurements are crucial for the security of your cluster.
Be extra careful when manually changing these settings.
When in doubt, check if the encountered [issue is known](https://github.com/edgelesssys/constellation/issues?q=is%3Aopen+is%3Aissue+label%3A%22known+issue%22) or [contact support](https://github.com/edgelesssys/constellation#support).
:::
:::tip
During an upgrade with modified attestation config, a backup of the current configuration is stored in the `join-config` config map in the `kube-system` namespace under the `attestationConfig_backup` key. To restore the old attestation config after a failed upgrade, replace the value of `attestationConfig` with the value from `attestationConfig_backup`:
```bash
kubectl patch configmaps -n kube-system join-config -p "{\"data\":{\"attestationConfig\":\"$(kubectl get configmaps -n kube-system join-config -o "jsonpath={.data.attestationConfig_backup}")\"}}"
```
:::
You can use the `apply` command to change measurements of a running cluster:
1. Modify the `measurements` key in your local `constellation-conf.yaml` to the expected values.
2. Run `constellation apply`.
Keep in mind that running `apply` also applies any version changes from your config to the cluster.
You can run these commands to learn about the versions currently configured in the cluster:
- Kubernetes API server version: `kubectl get nodeversion constellation-version -o json -n kube-system | jq .spec.kubernetesClusterVersion`
- image version: `kubectl get nodeversion constellation-version -o json -n kube-system | jq .spec.imageVersion`
- microservices versions: `helm list --filter 'constellation-services' -n kube-system`
### Upgrading Kubernetes resources fails
Constellation manages its Kubernetes resources using Helm.
When applying an upgrade, the charts that are about to be installed, and a values override file `overrides.yaml`,
are saved to disk in your current workspace under `constellation-upgrade/upgrade-<timestamp>/helm-charts/`.
If upgrading the charts using the Constellation CLI fails, you can review these charts and try to manually apply the upgrade.
:::caution
Changing and manually applying the charts may destroy cluster resources and can lead to broken Constellation deployments.
Proceed with caution and when in doubt,
check if the encountered [issue is known](https://github.com/edgelesssys/constellation/issues?q=is%3Aopen+is%3Aissue+label%3A%22known+issue%22) or [contact support](https://github.com/edgelesssys/constellation#support).
:::
## Diagnosing issues
### Logs
To get started on diagnosing issues with Constellation, it's often helpful to collect logs from nodes, pods, or other resources in the cluster. Most logs are available through Kubernetes' standard
[logging interfaces](https://kubernetes.io/docs/concepts/cluster-administration/logging/).
To debug issues occurring at boot time of the nodes, you can use the serial console interface of the CSP while the machine boots to get a read-only view of the boot logs.
Apart from that, Constellation also offers further [observability integrations](../architecture/observability.md).
### Node shell access
Debugging via a shell on a node is [directly supported by Kubernetes](https://kubernetes.io/docs/tasks/debug/debug-application/debug-running-pod/#node-shell-session).
1. Figure out which node to connect to:
```bash
kubectl get nodes
# or to see more information, such as IPs:
kubectl get nodes -o wide
```
2. Connect to the node:
```bash
kubectl debug node/constell-worker-xksa0-000000 -it --image=busybox
```
You will be presented with a prompt.
The nodes file system is mounted at `/host`.
3. Once finished, clean up the debug pod:
```bash
kubectl delete pod node-debugger-constell-worker-xksa0-000000-bjthj
```

54
docs/versioned_docs/version-2.16/workflows/trusted-launch.md Normal file
View file

@ -0,0 +1,54 @@
# Use Azure trusted launch VMs
Constellation also supports [trusted launch VMs](https://docs.microsoft.com/en-us/azure/virtual-machines/trusted-launch) on Microsoft Azure. Trusted launch VMs don't offer the same level of security as Confidential VMs, but are available in more regions and in larger quantities. The main difference between trusted launch VMs and normal VMs is that the former offer vTPM-based remote attestation. When used with trusted launch VMs, Constellation relies on vTPM-based remote attestation to verify nodes.
:::caution
Trusted launch VMs don't provide runtime encryption and don't keep the cloud service provider (CSP) out of your trusted computing base.
:::
Constellation supports trusted launch VMs with instance types `Standard_D*_v4` and `Standard_E*_v4`. Run `constellation config instance-types` for a list of all supported instance types.
## VM images
Azure currently doesn't support [community galleries for trusted launch VMs](https://docs.microsoft.com/en-us/azure/virtual-machines/share-gallery-community). Thus, you need to manually import the Constellation node image into your cloud subscription.
The latest image is available at <https://cdn.confidential.cloud/constellation/images/azure/trusted-launch/v2.2.0/constellation.img>. Simply adjust the version number to download a newer version.
After you've downloaded the image, create a resource group `constellation-images` in your Azure subscription and import the image.
You can use a script to do this:
```bash
wget https://raw.githubusercontent.com/edgelesssys/constellation/main/hack/importAzure.sh
chmod +x importAzure.sh
AZURE_IMAGE_VERSION=2.2.0 AZURE_RESOURCE_GROUP_NAME=constellation-images AZURE_IMAGE_FILE=./constellation.img ./importAzure.sh
```
The script creates the following resources:
1. A new image gallery with the default name `constellation-import`
2. A new image definition with the default name `constellation`
3. The actual image with the provided version. In this case `2.2.0`
Once the import is completed, use the `ID` of the image version in your `constellation-conf.yaml` for the `image` field. Set `confidentialVM` to `false`.
Fetch the image measurements:
```bash
IMAGE_VERSION=2.2.0
URL=https://public-edgeless-constellation.s3.us-east-2.amazonaws.com//communitygalleries/constellationcvm-b3782fa0-0df7-4f2f-963e-fc7fc42663df/images/constellation/versions/$IMAGE_VERSION/measurements.yaml
constellation config fetch-measurements -u$URL -s$URL.sig
```
:::info
The [`constellation apply`](create.md) command will issue a warning because manually imported images aren't recognized as production grade images:
```shell-session
Configured image doesn't look like a released production image. Double check image before deploying to production.
```
Please ignore this warning.
:::

110
docs/versioned_docs/version-2.16/workflows/upgrade.md Normal file
View file

@ -0,0 +1,110 @@
# Upgrade your cluster
Constellation provides an easy way to upgrade all components of your cluster, without disrupting it's availability.
Specifically, you can upgrade the Kubernetes version, the nodes' image, and the Constellation microservices.
You configure the desired versions in your local Constellation configuration and trigger upgrades with the `apply` command.
To learn about available versions you use the `upgrade check` command.
Which versions are available depends on the CLI version you are using.
## Update the CLI
Each CLI comes with a set of supported microservice and Kubernetes versions.
Most importantly, a given CLI version can only upgrade a cluster of the previous minor version, but not older ones.
This means that you have to upgrade your CLI and cluster one minor version at a time.
For example, if you are currently on CLI version v2.6 and the latest version is v2.8, you should
* upgrade the CLI to v2.7,
* upgrade the cluster to v2.7,
* and only then continue upgrading the CLI (and the cluster) to v2.8 after.
Also note that if your current Kubernetes version isn't supported by the next CLI version, use your current CLI to upgrade to a newer Kubernetes version first.
To learn which Kubernetes versions are supported by a particular CLI, run [constellation config kubernetes-versions](../reference/cli.md#constellation-config-kubernetes-versions).
## Migrate the configuration
The Constellation configuration file is located in the file `constellation-conf.yaml` in your workspace.
Refer to the [migration reference](../reference/migration.md) to check if you need to update fields in your configuration file.
Use [`constellation config migrate`](../reference/cli.md#constellation-config-migrate) to automatically update an old config file to a new format.
## Check for upgrades
To learn which versions the current CLI can upgrade to and what's installed in your cluster, run:
```bash
# Show possible upgrades
constellation upgrade check
# Show possible upgrades and write them to config file
constellation upgrade check --update-config
```
You can either enter the reported target versions into your config manually or run the above command with the `--update-config` flag.
When using this flag, the `kubernetesVersion`, `image`, `microserviceVersion`, and `attestation` fields are overwritten with the smallest available upgrade.
## Apply the upgrade
Once you updated your config with the desired versions, you can trigger the upgrade with this command:
```bash
constellation apply
```
Microservice upgrades will be finished within a few minutes, depending on the cluster size.
If you are interested, you can monitor pods restarting in the `kube-system` namespace with your tool of choice.
Image and Kubernetes upgrades take longer.
For each node in your cluster, a new node has to be created and joined.
The process usually takes up to ten minutes per node.
When applying an upgrade, the Helm charts for the upgrade as well as backup files of Constellation-managed Custom Resource Definitions, Custom Resources, and Terraform state are created.
You can use the Terraform state backup to restore previous resources in case an upgrade misconfigured or erroneously deleted a resource.
You can use the Custom Resource (Definition) backup files to restore Custom Resources and Definitions manually (e.g., via `kubectl apply`) if the automatic migration of those resources fails.
You can use the Helm charts to manually apply upgrades to the Kubernetes resources, should an upgrade fail.
:::note
For advanced users: the upgrade consists of several phases that can be individually skipped through the `--skip-phases` flag.
The phases are `infrastracture` for the cloud resource management through Terraform, `helm` for the chart management of the microservices, `image` for OS image upgrades, and `k8s` for Kubernetes version upgrades.
:::
## Check the status
Upgrades are asynchronous operations.
After you run `apply`, it will take a while until the upgrade has completed.
To understand if an upgrade is finished, you can run:
```bash
constellation status
```
This command displays the following information:
* The installed services and their versions
* The image and Kubernetes version the cluster is expecting on each node
* How many nodes are up to date
Here's an example output:
```shell-session
Target versions:
Image: v2.6.0
Kubernetes: v1.25.8
Service versions:
Cilium: v1.12.1
cert-manager: v1.10.0
constellation-operators: v2.6.0
constellation-services: v2.6.0
Cluster status: Some node versions are out of date
Image: 23/25
Kubernetes: 25/25
```
This output indicates that the cluster is running Kubernetes version `1.25.8`, and all nodes have the appropriate binaries installed.
23 out of 25 nodes have already upgraded to the targeted image version of `2.6.0`, while two are still in progress.
## Apply further upgrades
After the upgrade is finished, you can run `constellation upgrade check` again to see if there are more upgrades available. If so, repeat the process.

125
docs/versioned_docs/version-2.16/workflows/verify-cli.md Normal file
View file

@ -0,0 +1,125 @@
# Verify the CLI
:::info
This recording presents the essence of this page. It's recommended to read it in full for the motivation and all details.
:::
<asciinemaWidget src="/constellation/assets/verify-cli.cast" rows="20" cols="112" idleTimeLimit="3" preload="true" theme="edgeless" />
---
Edgeless Systems uses [sigstore](https://www.sigstore.dev/) and [SLSA](https://slsa.dev) to ensure supply-chain security for the Constellation CLI and node images ("artifacts"). sigstore consists of three components: [Cosign](https://docs.sigstore.dev/signing/quickstart), [Rekor](https://docs.sigstore.dev/logging/overview), and Fulcio. Edgeless Systems uses Cosign to sign artifacts. All signatures are uploaded to the public Rekor transparency log, which resides at <https://rekor.sigstore.dev/>.
:::note
The public key for Edgeless Systems' long-term code-signing key is:
```
-----BEGIN PUBLIC KEY-----
MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEf8F1hpmwE+YCFXzjGtaQcrL6XZVT
JmEe5iSLvG1SyQSAew7WdMKF6o9t8e2TFuCkzlOhhlws2OHWbiFZnFWCFw==
-----END PUBLIC KEY-----
```
The public key is also available for download at <https://edgeless.systems/es.pub> and in the Twitter profile [@EdgelessSystems](https://twitter.com/EdgelessSystems).
:::
The Rekor transparency log is a public append-only ledger that verifies and records signatures and associated metadata. The Rekor transparency log enables everyone to observe the sequence of (software) signatures issued by Edgeless Systems and many other parties. The transparency log allows for the public identification of dubious or malicious signatures.
You should always ensure that (1) your CLI executable was signed with the private key corresponding to the above public key and that (2) there is a corresponding entry in the Rekor transparency log. Both can be done as described in the following.
:::info
You don't need to verify the Constellation node images. This is done automatically by your CLI and the rest of Constellation.
:::
## Verify the signature
First, [install the Cosign CLI](https://docs.sigstore.dev/system_config/installation). Next, [download](https://github.com/edgelesssys/constellation/releases) and verify the signature that accompanies your CLI executable, for example:
```shell-session
$ cosign verify-blob --key https://edgeless.systems/es.pub --signature constellation-linux-amd64.sig constellation-linux-amd64
Verified OK
```
The above performs an offline verification of the provided public key, signature, and executable. To also verify that a corresponding entry exists in the public Rekor transparency log, add the variable `COSIGN_EXPERIMENTAL=1`:
```shell-session
$ COSIGN_EXPERIMENTAL=1 cosign verify-blob --key https://edgeless.systems/es.pub --signature constellation-linux-amd64.sig constellation-linux-amd64
tlog entry verified with uuid: afaba7f6635b3e058888692841848e5514357315be9528474b23f5dcccb82b13 index: 3477047
Verified OK
```
🏁 You now know that your CLI executable was officially released and signed by Edgeless Systems.
### Optional: Manually inspect the transparency log
To further inspect the public Rekor transparency log, [install the Rekor CLI](https://docs.sigstore.dev/logging/installation). A search for the CLI executable should give a single UUID. (Note that this UUID contains the UUID from the previous `cosign` command.)
```shell-session
$ rekor-cli search --artifact constellation-linux-amd64
Found matching entries (listed by UUID):
362f8ecba72f4326afaba7f6635b3e058888692841848e5514357315be9528474b23f5dcccb82b13
```
With this UUID you can get the full entry from the transparency log:
```shell-session
$ rekor-cli get --uuid=362f8ecba72f4326afaba7f6635b3e058888692841848e5514357315be9528474b23f5dcccb82b13
LogID: c0d23d6ad406973f9559f3ba2d1ca01f84147d8ffc5b8445c224f98b9591801d
Index: 3477047
IntegratedTime: 2022-09-12T22:28:16Z
UUID: afaba7f6635b3e058888692841848e5514357315be9528474b23f5dcccb82b13
Body: {
"HashedRekordObj": {
"data": {
"hash": {
"algorithm": "sha256",
"value": "40e137b9b9b8204d672642fd1e181c6d5ccb50cfc5cc7fcbb06a8c2c78f44aff"
}
},
"signature": {
"content": "MEUCIQCSER3mGj+j5Pr2kOXTlCIHQC3gT30I7qkLr9Awt6eUUQIgcLUKRIlY50UN8JGwVeNgkBZyYD8HMxwC/LFRWoMn180=",
"publicKey": {
"content": "LS0tLS1CRUdJTiBQVUJMSUMgS0VZLS0tLS0KTUZrd0V3WUhLb1pJemowQ0FRWUlLb1pJemowREFRY0RRZ0FFZjhGMWhwbXdFK1lDRlh6akd0YVFjckw2WFpWVApKbUVlNWlTTHZHMVN5UVNBZXc3V2RNS0Y2bzl0OGUyVEZ1Q2t6bE9oaGx3czJPSFdiaUZabkZXQ0Z3PT0KLS0tLS1FTkQgUFVCTElDIEtFWS0tLS0tCg=="
}
}
}
}
```
The field `publicKey` should contain Edgeless Systems' public key in Base64 encoding.
You can get an exhaustive list of artifact signatures issued by Edgeless Systems via the following command:
```bash
rekor-cli search --public-key https://edgeless.systems/es.pub --pki-format x509
```
Edgeless Systems monitors this list to detect potential unauthorized use of its private key.
## Verify the provenance
Provenance attests that a software artifact was produced by a specific repository and build system invocation. For more information on provenance visit [slsa.dev](https://slsa.dev/provenance/v0.2) and learn about the [adoption of SLSA for Constellation](../reference/slsa.md).
Just as checking its signature proves that the CLI hasn't been manipulated, checking the provenance proves that the artifact was produced by the expected build process and hasn't been tampered with.
To verify the provenance, first install the [slsa-verifier](https://github.com/slsa-framework/slsa-verifier). Then make sure you have the provenance file (`constellation.intoto.jsonl`) and Constellation CLI downloaded. Both are available on the [GitHub release page](https://github.com/edgelesssys/constellation/releases).
:::info
The same provenance file is valid for all Constellation CLI executables of a given version independent of the target platform.
:::
Use the verifier to perform the check:
```shell-session
$ slsa-verifier verify-artifact constellation-linux-amd64 \
--provenance-path constellation.intoto.jsonl \
--source-uri github.com/edgelesssys/constellation
Verified signature against tlog entry index 7771317 at URL: https://rekor.sigstore.dev/api/v1/log/entries/24296fb24b8ad77af2c04c8b4ae0d5bc5...
Verified build using builder https://github.com/slsa-framework/slsa-github-generator/.github/workflows/generator_generic_slsa3.yml@refs/tags/v1.2.2 at commit 18e9924b416323c37b9cdfd6cc728de8a947424a
PASSED: Verified SLSA provenance
```

96
docs/versioned_docs/version-2.16/workflows/verify-cluster.md Normal file
View file

@ -0,0 +1,96 @@
# Verify your cluster
Constellation's [attestation feature](../architecture/attestation.md) allows you, or a third party, to verify the integrity and confidentiality of your Constellation cluster.
## Fetch measurements
To verify the integrity of Constellation you need trusted measurements to verify against. For each node image released by Edgeless Systems, there are signed measurements, which you can download using the CLI:
```bash
constellation config fetch-measurements
```
This command performs the following steps:
1. Download the signed measurements for the configured image. By default, this will use Edgeless Systems' public measurement registry.
2. Verify the signature of the measurements. This will use Edgeless Systems' [public key](https://edgeless.systems/es.pub).
3. Write measurements into configuration file.
The configuration file then contains a list of `measurements` similar to the following:
```yaml
# ...
measurements:
0:
expected: "0f35c214608d93c7a6e68ae7359b4a8be5a0e99eea9107ece427c4dea4e439cf"
warnOnly: false
4:
expected: "02c7a67c01ec70ffaf23d73a12f749ab150a8ac6dc529bda2fe1096a98bf42ea"
warnOnly: false
5:
expected: "e6949026b72e5045706cd1318889b3874480f7a3f7c5c590912391a2d15e6975"
warnOnly: true
8:
expected: "0000000000000000000000000000000000000000000000000000000000000000"
warnOnly: false
9:
expected: "f0a6e8601b00e2fdc57195686cd4ef45eb43a556ac1209b8e25d993213d68384"
warnOnly: false
11:
expected: "0000000000000000000000000000000000000000000000000000000000000000"
warnOnly: false
12:
expected: "da99eb6cf7c7fbb692067c87fd5ca0b7117dc293578e4fea41f95d3d3d6af5e2"
warnOnly: false
13:
expected: "0000000000000000000000000000000000000000000000000000000000000000"
warnOnly: false
14:
expected: "d7c4cc7ff7933022f013e03bdee875b91720b5b86cf1753cad830f95e791926f"
warnOnly: true
15:
expected: "0000000000000000000000000000000000000000000000000000000000000000"
warnOnly: false
# ...
```
Each entry specifies the expected value of the Constellation node, and whether the measurement should be enforced (`warnOnly: false`), or only a warning should be logged (`warnOnly: true`).
By default, the subset of the [available measurements](../architecture/attestation.md#runtime-measurements) that can be locally reproduced and verified is enforced.
During attestation, the validating side (CLI or [join service](../architecture/microservices.md#joinservice)) compares each measurement reported by the issuing side (first node or joining node) individually.
For mismatching measurements that have set `warnOnly` to `true` only a warning is emitted.
For mismatching measurements that have set `warnOnly` to `false` an error is emitted and attestation fails.
If attestation fails for a new node, it isn't permitted to join the cluster.
## The *verify* command
:::note
The steps below are purely optional. They're automatically executed by `constellation apply` when you initialize your cluster. The `constellation verify` command mostly has an illustrative purpose.
:::
The `verify` command obtains and verifies an attestation statement from a running Constellation cluster.
```bash
constellation verify [--cluster-id ...]
```
From the attestation statement, the command verifies the following properties:
* The cluster is using the correct Confidential VM (CVM) type.
* Inside the CVMs, the correct node images are running. The node images are identified through the measurements obtained in the previous step.
* The unique ID of the cluster matches the one from your `constellation-state.yaml` file or passed in via `--cluster-id`.
Once the above properties are verified, you know that you are talking to the right Constellation cluster and it's in a good and trustworthy shape.
### Custom arguments
The `verify` command also allows you to verify any Constellation deployment that you have network access to. For this you need the following:
* The IP address of a running Constellation cluster's [VerificationService](../architecture/microservices.md#verificationservice). The `VerificationService` is exposed via a `NodePort` service using the external IP address of your cluster. Run `kubectl get nodes -o wide` and look for `EXTERNAL-IP`.
* The cluster's *clusterID*. See [cluster identity](../architecture/keys.md#cluster-identity) for more details.
For example:
```shell-session
constellation verify -e 192.0.2.1 --cluster-id Q29uc3RlbGxhdGlvbkRvY3VtZW50YXRpb25TZWNyZXQ=
```