revision iteration

This commit is contained in:
david-crypto 2024-11-28 16:28:46 +01:00
parent 1a2ed9eff2
commit 9fdba69ca2
7 changed files with 31 additions and 63 deletions

View File

@ -2,7 +2,7 @@
The command-line interface (CLI) is one of the key components of Constellation and is used for **orchestrating constellation clusters**. It is run by the cloud administrator on a local machine and is used to **create, manage, and update confidential clusters** directly from the command line.
You can use the CLI to create a cluster on the [supported cloud platforms](../../overview/product.md). It provisions the necessary resources in your cloud environment and initiates the cluster setup.
You can use the CLI to create a cluster on the [supported cloud platforms](../../overview/product.md).
The CLI uses a set of parameters and an optional configuration file to manage your cluster installation. You can also use the CLI to update your cluster.
@ -20,34 +20,9 @@ but you shouldn't move or delete them while the cluster is still being used.
The Constellation CLI takes care of managing the workspace.
Only when a cluster was terminated, and you are sure the files aren't needed anymore, should you remove a workspace.
## Cluster creation process
## Cluster creation
To allow for fine-grained configuration of your cluster and cloud environment, Constellation supports an extensive configuration file with strong defaults. [Generating the configuration file](../../workflows/config.md) is typically the first thing you do in the workspace.
Altogether, the following files are generated during the creation of a Constellation cluster and stored in the current workspace:
- a configuration file
- a state file
- a Base64-encoded master secret
- [Terraform artifacts](../../reference/terraform.md), stored in subdirectories
- a Kubernetes `kubeconfig` file.
After the initialization of your cluster, the CLI will provide you with a Kubernetes `kubeconfig` file.
This file grants you access to your Kubernetes cluster and configures the [kubectl](https://kubernetes.io/docs/concepts/configuration/organize-cluster-access-kubeconfig/) tool.
In addition, the cluster's [identifier](../components/cli.md#post-installation-configuration) is returned and stored in the state file.
### Creation flow
1. The CLI `apply` command first creates the confidential VM (CVM) resources in your cloud environment and configures the network
2. Each CVM boots the Constellation node image and measures every component in the boot chain
3. The first microservice launched in each node is the [_Bootstrapper_](../components/microservices.md#bootstrapper)
4. The _Bootstrapper_ waits until it either receives an initialization request or discovers an initialized cluster
5. The CLI then connects to the _Bootstrapper_ of a selected node — which we also refer to as _first node_, sends the configuration, and initiates the initialization of the cluster
6. The _Bootstrapper_ of the first node [initializes the Kubernetes cluster](../components/microservices.md#bootstrapper) and deploys the other Constellation [microservices](microservices.md) including the [_JoinService_](microservices.md#joinservice)
7. Subsequently, the _Bootstrappers_ of the other nodes discover the initialized cluster and send join requests to the _JoinService_
8. As part of the join request each node includes an attestation statement of its boot measurements as authentication
9. The _JoinService_ verifies the attestation statements and joins the nodes to the Kubernetes cluster
10. This process is repeated for every node joining the cluster later (e.g., through autoscaling)
The `apply` command creates the necessary resources in your cloud environment, bootstraps the Constellation cluster, and securely installs all Kubernetes binaries to ensure the cluster's integrity. For a detailed description, please visit our [Protocol overview](../overview.md#cluster-creation).
## Post-installation configuration
@ -67,18 +42,8 @@ Instead of managing this secret manually, you can [use your key management solut
The _clusterID_ uniquely identifies a cluster and can be used to [verify your cluster](../../workflows/verify-cluster.md).
## Constellation Upgrades
## Constellation upgrades
Constellation images and microservices may need to be upgraded to new versions during the lifetime of a cluster.
Constellation implements a rolling update mechanism ensuring no downtime of the control or data plane.
You can upgrade a Constellation cluster with a single operation by using the CLI.
For step-by-step instructions on how to do this, refer to [Upgrade your cluster](../../workflows/upgrade.md).
### Attestation of upgrades
With every new image, corresponding measurements are released.
During an update procedure, the CLI provides new measurements to the [JoinService](../components/microservices.md#joinservice) securely.
New measurements for an updated image are automatically pulled and verified by the CLI following the [supply chain security concept](../security/attestation.md#chain-of-trust) of Constellation.
The [attestation section](../security/attestation.md#cluster-facing-attestation) describes in detail how these measurements are then used by the JoinService for the attestation of nodes.
Constellation images and microservices may need to be upgraded to new versions during the lifetime of a cluster. This can be done by running the `upgrade` command. An explanation of the update protocol can be found in our [Protocol overview](../overview.md#cluster-upgrade).
<!-- soon: As the [builds of the Constellation images are reproducible](attestation.md#chain-of-trust), the updated measurements are auditable by the customer. -->

View File

@ -1,7 +1,7 @@
# Core services
Constellation takes care of bootstrapping and initializing a confidential Kubernetes cluster.
During the lifetime of the cluster, it handles day 2 operations such as **key management, remote attestation, and updates**.
During the lifetime of the cluster, it handles day-2 operations such as **key management, remote attestation, and updates**.
These features are provided by several microservices:
- The [Bootstrapper](microservices.md#bootstrapper) initializes a Constellation node and bootstraps the cluster
@ -32,6 +32,8 @@ flowchart LR
C -- deploys --> F
```
An overview of how these services interact with each other when setting up a cluster is provided in the [Protocol overview](../overview.md#cluster-growth). Details about the role of each service are provided below.
## Bootstrapper
The _Bootstrapper_ is the first microservice launched after booting a Constellation node image.
@ -43,9 +45,7 @@ Otherwise, it waits for an initialization request to create a new Kubernetes clu
## JoinService
The _JoinService_ runs as [DaemonSet](https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/) on each control-plane node.
New nodes (at cluster start, or later through autoscaling) send a request to the service over [attested TLS (aTLS)](../security/attestation.md#attested-tls-atls).
The _JoinService_ verifies the new node's certificate and attestation statement.
If attestation is successful, the new node is supplied with an encryption key from the [_KeyService_](microservices.md#keyservice) for its state disk, and a [Kubernetes bootstrap token](https://kubernetes.io/docs/reference/access-authn-authz/bootstrap-tokens/).
New nodes (either at cluster start or later through autoscaling) send a request to the service over [attested TLS (aTLS)](../security/attestation.md#attested-tls-atls). This connection is established by verifying the attestation statement of the new node, which includes the public key used for the connection. Afterward, the new node communicates with the JoinService as descriped in [Protocol overview](../overview.md#cluster-growth).
```mermaid
sequenceDiagram
@ -53,7 +53,7 @@ sequenceDiagram
participant JoinService
New node->>JoinService: aTLS handshake (server side verification)
JoinService-->>New node: #
New node->>+JoinService: IssueJoinTicket(DiskUUID, NodeName, IsControlPlane)
New node->>+JoinService: IssueJoinTicket(DiskUUID, CertificateRequest, IsControlPlane)
JoinService->>+KeyService: GetDataKey(DiskUUID)
KeyService-->>-JoinService: DiskEncryptionKey
JoinService-->>-New node: DiskEncryptionKey, KubernetesJoinToken, ...

View File

@ -7,8 +7,8 @@ This section offers a comprehensive overview of Constellation's inner workings.
- [**Key components**](./components/cli.md): This chapter outlines Constellation's main components, their roles, and how users interact with them:
- [The CLI](./components/cli.md): A command-line tool to efficiently create and manage your cluster.
- [Constellation's core services](./components/microservices.md): These services run on the control planes, enabling secure protocols for cluster scaling and performing integrity checks.
- [Operating system images](./components/node-images.md): Constellation offers optimized OS images for Kubernetes control-plane and worker nodes, tailored for containerized workloads and ready for confidential VMs.
- [Constellation's core services](./components/microservices.md): These services run on the control planes, enabling secure protocols for cluster scaling and performing integrity checks.
- [**Security concepts**](./security/attestation.md): A detailed exploration of the concepts that provide strong protection for your Kubernetes clusters, including:

View File

@ -62,7 +62,7 @@ In addition to the attestation config, the CLI contains the following data in ha
## Cluster creation
When a cluster is [created](https://docs.edgeless.systems/constellation/workflows/create), the CLI interacts with the API of the respective infrastructure provider, for example Azure, and launches CVMs with the applicable node image.
When a cluster is [created](https://docs.edgeless.systems/constellation/workflows/create) running the `apply` command, the CLI interacts with the API of the respective infrastructure provider, for example Azure, and launches CVMs with the applicable node image.
These CVMs are called _nodes_.
On each node, the Bootstrapper is launched.
@ -99,7 +99,7 @@ R = Sig-CPU(<launch digest>, <auxiliary data>, <payload>)
- `payload`: This is controlled by the software running inside the CVM. In the case of a Constellation node, the payload is always the public key of the respective Bootstrapper running inside the CVM. Thus, `R` can be seen as a certificate for that public key issued by the CPU. Based on this, nodes establish attested TLS (aTLS) connections.
aTLS is used during [cluster creation](#cluster-creation) and when [growing a cluster](#cluster-growth).
- `launch digest`: Abstactly, this is the hash of the initial memory contents of a CVM, provided by Intel TDX or AMD SEV-SNP. The initial setup of the CVM includes only third-party code provided by the cloud provider.
- `launch digest`: Abstractly, this refers to the hash of the initial memory contents of a CVM, as measured by Intel TDX or AMD SEV-SNP. The initial setup of the CVM consists solely of third-party code supplied by the cloud provider.
- `auxiliary data`: Includes the digest of the subsequent booting process within the CVM, such as booting the Constellation node image and loading the Bootstrapper. This digest is stored in several of the 16 virtual registers provided by a virtual Trusted Platform Module (vTPM). This mechanism ensures the integrity and identity of a node, as explained in the following sections.
### Measurements

View File

@ -1,6 +1,6 @@
# Attestation
Constellation's attestation methods are the cornerstone of establishing trust in a cluster deployment. They are the key to ensuring that clusters always operate in a **protected and trustworthy** state.
Constellation's attestation methods are the cornerstone of establishing trust in cluster deployments. They are the key to ensuring that clusters always operate in a **protected and trustworthy** state.
## Key components and concepts
@ -61,16 +61,16 @@ Instead of relying on a certificate authority, aTLS uses this attestation statem
The protocol can be used by clients to verify a server certificate, by a server to verify a client certificate, or for mutual verification (mutual aTLS).
## Overview
## Attestation in Constellation: from nodes to the entire cluster
The challenge for Constellation is to lift a CVM's attestation statement to the Kubernetes software layer and make it end-to-end verifiable.
From there, Constellation needs to expand the attestation from a single CVM to the entire cluster.
One of Constellation's core challenges is extending a CVM's attestation statement to the Kubernetes software layer and ensuring it is verifiable end-to-end. Beyond this, Constellation must scale the attestation from a single CVM to encompass the entire cluster.
The [_JoinService_](../components/microservices.md#joinservice) and [_VerificationService_](../components/microservices.md#verificationservice) are where all runs together.
Internally, the _JoinService_ uses remote attestation to securely join CVM nodes to the cluster.
Externally, the _VerificationService_ provides an attestation statement for the cluster's CVMs and configuration.
The process revolves around two key components: the [_JoinService_](../components/microservices.md#joinservice) and the [_VerificationService_](../components/microservices.md#verificationservice).
The following explains the details of both steps.
- **JoinService**: Internally, the _JoinService_ uses remote attestation to securely add CVM nodes to the cluster.
- **VerificationService**: Externally, the _VerificationService_ generates an attestation statement covering both the cluster's CVMs and its configuration.
The following sections provide a detailed explanation of these two steps.
## Node attestation
@ -115,11 +115,14 @@ The runtime measurements consist of two types of values:
- **Measurements produced by the Constellation bootloader and boot chain**:
The Constellation bootloader takes over from the CVM firmware and [measures the rest of the boot chain](../components/node-images.md).
The Constellation [Bootstrapper](../components/microservices.md#bootstrapper) is the first user mode component that runs in a Constellation image.
It extends PCR registers with the [IDs](keys.md#cluster-identity) of the cluster marking a node as initialized.
Note that the Bootstrapper is included in the root filesystem, with a hash of this filesystem embedded in the kernel command line. By using a minimal filesystem (initramfs) and [dm-verity](https://docs.kernel.org/admin-guide/device-mapper/verity.html), the whole root filesystem's — including the Bootstrapper's — integrity is securely enforced. For more details, see the [Node image](../components/node-images.md#initramfs) section. The kernel, kernel command line, and initramfs are tracked in the PCRs, enabling the Bootstrapper's integrity to be indirectly attested. This is possible because the logic for loading the Bootstrapper and verifying it against a specific hash is itself attested.
Finally, the Bootstrapper extends the PCR registers with the [cluster IDs](keys.md#cluster-identity), marking a node as initialized.
Constellation allows to specify in the config which measurements should be enforced during the attestation process.
Enforcing non-reproducible measurements controlled by the cloud provider means that changes in these values require manual updates to the cluster's config.
By default, Constellation only enforces measurements that are stable values produced by the infrastructure or by Constellation directly.
By default, Constellation exludes these from its attestation and only enforces measurements that are stable values produced by the infrastructure or by Constellation directly.
<Tabs groupId="csp">
<TabItem value="aws" label="AWS">
@ -346,7 +349,7 @@ The _JoinService_ is provided with the runtime measurements of the whitelisted C
During the initialization and the cluster bootstrapping, each node connects to the _JoinService_ using [aTLS](#attested-tls-atls).
During the handshake, the node transmits an attestation statement including its runtime measurements.
The _JoinService_ verifies that statement and compares the measurements against the ground truth.
For details of the initialization process check the [microservice descriptions](../components/microservices.md).
For details of the initialization process, check the [Protocol overview](../overview.md#cluster-growth).
After the initialization, every node updates its runtime measurements with the _clusterID_ value, marking it irreversibly as initialized.
When an initialized node tries to join another cluster, its measurements inevitably mismatch the measurements of an uninitialized node and it will be declined.

View File

@ -7,7 +7,7 @@ Currently, Constellation only supports WireGuard as the encryption engine.
You can read more about the cryptographic soundness of WireGuard [in their white paper](https://www.wireguard.com/papers/wireguard.pdf).
Cilium is actively working on implementing a feature called [`host-to-host`](https://github.com/cilium/cilium/pull/19401) encryption mode for WireGuard.
With `host-to-host`, all traffic between nodes will be tunneled via WireGuard (host-to-host, host-to-pod, pod-to-host, pod-to-pod).
With `host-to-host`, all traffic between nodes will be tunneled via WireGuard (node-to-node, node-to-pod, pod-to-node, pod-to-pod).
Until the `host-to-host` feature is released, Constellation enables `pod-to-pod` encryption.
This mode encrypts all traffic between Kubernetes pods using WireGuard tunnels.

View File

@ -9,7 +9,7 @@ As described in [Use persistent storage](../../workflows/storage.md), cloud serv
These CSI storage solutions often support some sort of encryption.
For example, Google Cloud [encrypts data at rest by default](https://cloud.google.com/security/encryption/default-encryption), without any action required by the customer.
## Cloud provider-managed encryption
## Cloud provider-managed encryption
CSP-managed storage solutions encrypt the data in the cloud backend before writing it physically to disk.
In the context of confidential computing and Constellation, the CSP and its managed services aren't trusted.
@ -20,7 +20,7 @@ Even with "bring your own key" or similar concepts, the CSP performs the encrypt
In the security model of Constellation, securing persistent storage and thereby data at rest requires that all cryptographic operations are performed inside a trusted execution environment.
Consequently, using CSP-managed encryption of persistent storage usually isn't an option.
## Constellation-managed encryption
## Constellation-managed encryption
Constellation provides CSI drivers for storage solutions in all major clouds with built-in encryption support.
Block storage provisioned by the CSP is [mapped](https://guix.gnu.org/manual/en/html_node/Mapped-Devices.html) using the [dm-crypt](https://www.kernel.org/doc/html/latest/admin-guide/device-mapper/dm-crypt.html), and optionally the [dm-integrity](https://www.kernel.org/doc/html/latest/admin-guide/device-mapper/dm-integrity.html), kernel modules, before it's formatted and accessed by the Kubernetes workloads.
@ -28,7 +28,7 @@ All cryptographic operations happen inside the trusted environment of the confid
Note that for integrity-protected disks, [volume expansion](https://kubernetes.io/blog/2018/07/12/resizing-persistent-volumes-using-kubernetes/) isn't supported.
By default the driver uses data encryption keys (DEKs) issued by the Constellation [_KeyService_](../components/microservices.md#keyservice).
By default, the driver uses data encryption keys (DEKs) issued by the Constellation [_KeyService_](../components/microservices.md#keyservice).
The DEKs are in turn derived from the Constellation's key encryption key (KEK), which is directly derived from the [master secret](keys.md#master-secret).
This is the recommended mode of operation, and also requires the least amount of setup by the cluster administrator.