Git-Mirrors/constellation
1
0
Fork 0
mirror of https://github.com/edgelesssys/constellation.git synced 2025-05-02 14:26:23 -04:00
Code Issues Projects Releases Packages Wiki Activity

Release docs for v2.1 (#222)

Browse source
This commit is contained in:
Paul Meyer 2022-10-07 12:14:55 +02:00 • committed by GitHub
parent 7f96b82288
commit fd63ca1251
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
53 changed files with 6173 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

237
docs/versioned_docs/version-2.1/architecture/attestation.md Normal file
View file

@ -0,0 +1,237 @@
# Attestation
This page explains Constellation's attestation process and highlights the cornerstones of its trust model.
## Terms
The following lists a few terms and concepts that helps to understand the attestation concept of Constellation.
### Trusted Platform Module (TPM)
A TPM chip is a dedicated tamper-resistant crypto-processor.
It can securely store artifacts such as passwords, certificates, encryption keys, or *runtime measurements* (more on this below).
When a TPM is implemented in software, it's typically called a *virtual* TPM (vTPM).
### Runtime measurement
A runtime measurement is a cryptographic hash of the memory pages of a so called *runtime component*. Runtime components of interest typically include a system's bootloader or OS kernel.
### Platform Configuration Register (PCR)
A Platform Configuration Register (PCR) is a memory location in the TPM that has some unique properties.
To store a new value in a PCR, the existing value is extended with a new value as follows:
```
PCR[N] = HASHalg( PCR[N] || ArgumentOfExtend )
```
The PCRs are typically used to store runtime measurements.
The new value of a PCR is always an extension of the existing value.
Thus, storing the measurements of multiple components into the same PCR irreversibly links them together.
### Measured boot
Measured boot builds on the concept of chained runtime measurements.
Each component in the boot-chain loads and measures the next component into the PCR before executing it.
By comparing the resulting PCR values against trusted reference values, the integrity of the entire boot chain and thereby, the running system can be ensured.
### Remote attestation (RA)
Remote attestation is the process of verifying certain properties of an application or platform, such as integrity and confidentiality, from a remote location.
In the case of a measured boot, the goal is to obtain a signed attestation statement on the PCR values of the boot measurements.
The statement can then be verified and compared to a set of trusted reference values.
This way, the integrity of the platform can be ensured before sharing secrets with it.
### Confidential virtual machine (CVM)
Confidential computing (CC) is the protection of data in-use with hardware-based trusted execution environments (TEEs).
With CVMs, TEEs encapsulate entire virtual machines and isolate them against the hypervisor, other VMs, and direct memory access.
After loading the initial VM image into encrypted memory, the hypervisor calls for a secure processor to measure these initial memory pages.
The secure processor locks these pages and generates an attestation report on the initial page measurements.
CVM memory pages are encrypted with a key that resides inside the secure processor, which makes sure only the guest VM can access them.
The attestation report is signed by the secure processor and can be verified using remote attestation via the certificate authority of the hardware vendor.
Such an attestation statement guarantees the confidentiality and integrity of a CVM.
### Attested TLS (aTLS)
In a CC environment, attested TLS (aTLS) can be used to establish secure connections between two parties utilizing the remote attestation features of the CC components.
aTLS modifies the TLS handshake by embedding an attestation statement into the TLS certificate.
Instead of relying on a certificate authority, aTLS uses this attestation statement to establish trust in the certificate.
The protocol can be used by clients to verify a server certificate, by a server to verify a client certificate, or for mutual verification (mutual aTLS).
## Overview
The challenge for Constellation is to lift a CVM's attestation statement to the Kubernetes software layer and make it end-to-end verifiable.
From there, Constellation needs to expand the attestation from a single CVM to the entire cluster.
The [*JoinService*](components.md#joinservice) and [*VerificationService*](components.md#verificationservice) are where all runs together.
Internally, the *JoinService* uses remote attestation to securely join CVM nodes to the cluster.
Externally, the *VerificationService* provides an attestation statement for the cluster's CVMs and configuration.
The following explains the details of both steps.
## Node attestation
The idea is that Constellation nodes should have verifiable integrity from the CVM hardware measurement up to the Kubernetes software layer.
The solution is a verifiable boot chain and an integrity-protected runtime environment.
Constellation uses measured boot within CVMs, measuring each component in the boot process before executing it.
Outside of CC, it's usually implemented via TPMs.
CVM technologies differ in how they implement runtime measurements, but the general concepts are similar to those of a TPM.
For simplicity, we use TPM terminology like *PCR* in the following.
When a Constellation node image boots inside a CVM, it uses measured boot for all stages and components of the boot chain.
This process goes up to the root filesystem.
The root filesystem is mounted read-only with integrity protection, guaranteeing forward integrity.
For the details on the image and boot stages see the [image architecture](../architecture/images.md) documentation.
Any changes to the image will inevitably also change the measured boot's PCR values.
To create a node attestation statement, the Constellation image obtains a CVM attestation statement from the hardware.
This includes the runtime measurements and thereby binds the measured boot results to the CVM hardware measurement.
In addition to the image measurements, Constellation extends a PCR during the [initialization phase](../workflows/create.md#the-init-step) that irrevocably marks the node as initialized.
The measurement is created using the [*clusterID*](../architecture/keys.md#cluster-identity), tying all future attestation statements to this ID.
Thereby, an attestation statement is unique for every cluster and a node can be identified unambiguously as being initialized.
To verify an attestation, the hardware's signature and a statement are verified first to establish trust in the contained runtime measurements.
If successful, the measurements are verified against the trusted values of the particular Constellation release version.
Finally, the measurement of the *clusterID* can be compared by calculating it with the [master secret](keys.md#master-secret).
### Runtime measurements
Constellation utilizes runtime measurement to implement the measured boot approach.
As stated above, the underlying hardware technology and guest firmware differ in their implementations of runtime measurements.
The following gives a detailed description of the available measurements in the different cloud environments.
The runtime measurements contain two types of values.
First, are the ones produced by the cloud infrastructure and firmware of the CVM.
Depending on the cloud environment they can contain non-reproducible values.
Those correspond to measurements of closed-source firmware components and other values controlled by the cloud provider.
While not being directly verifiable, they can be compared against previously observed values.
As part of the [signed image measurements](#chain-of-trust), Constellation provides measurements that are known, previously observed values.
Thereby, Constellation enables users to identify changes and deviations and allows them to act accordingly.
See how to [fetch](../workflows/verify-cluster.md#fetch-measurements) the latest measurements and verify a cluster.
Second, are the measurements produced by the Constellation bootloader and boot chain itself.
The Constellation Bootloader is the first part of the Constellation stack that takes over from the CVM firmware and measures the rest of the boot chain.
Refer to [images](images.md) for more details on the Constellation boot chain.
The Constellation [Bootstrapper](components.md#bootstrapper) is the first user mode component that runs in a Constellation image.
It extends PCR registers with the [IDs](keys.md#cluster-identity) of the cluster marking a node as initialized.
Constellation allows to specify in the config which measurements should be enforced during the attestation process
Enforcing non-reproducible measurements controlled by the cloud provider means that changes in these values require manual updates to the cluster's config.
By default, Constellation only enforces measurements that are stable values produced by the infrastructure or by Constellation directly.
<tabs groupId="csp">
<tabItem value="azure" label="Azure">
Constellation leverages the [vTPM](https://docs.microsoft.com/en-us/azure/virtual-machines/trusted-launch#vtpm) feature of Azure CVMs for runtime measurements.
The vTPM on Azure CVMs adheres to the [TPM 2.0](https://trustedcomputinggroup.org/resource/tpm-library-specification/) specification of the [Trusted Computing Group](https://trustedcomputinggroup.org/resource/trusted-platform-module-tpm-summary/).
It provides a [measured boot](https://docs.microsoft.com/en-us/azure/security/fundamentals/measured-boot-host-attestation#measured-boot) verification that's based on the trusted launch feature of [Trusted Launch VMs](https://docs.microsoft.com/en-us/azure/virtual-machines/trusted-launch).
The following table lists all PCR values of the vTPM and the measured components.
It also lists what components of the boot chain did the measurements and if the value is reproducible and verifiable.
The latter means that value can be generated offline and compared to the one in the vTPM
| PCR | Components | Measured by | Reproducible and Verifiable |
|---------------|-------------------------------------|---------------------------------|-----------------------------|
| 0 | Firmware | Azure | No |
| 1 | Firmware | Azure | No |
| 2 | Firmware | Azure | No |
| 3 | Firmware | Azure | No |
| 4 | Constellation Bootloader, GRUB | Azure, Constellation Bootloader | Yes |
| 5 | Reserved | Azure | No |
| 6 | VM Unique ID | Azure | No |
| 7 | Secure Boot State | Azure, Constellation Bootloader | No |
| 8 | Kernel command line, GRUB config | Constellation Bootloader | Yes |
| 9 | Kernel, initramfs | Constellation Bootloader | Yes |
| 10 | Reserved | - | No |
| 11 | Reserved | Constellation Bootstrapper | Yes |
| 12 | ClusterID | Constellation Bootstrapper | Yes |
| 13&ndash;23 | Unused | - | - |
</tabItem>
<tabItem value="gcp" label="GCP">
Constellation leverages the [vTPM](https://cloud.google.com/compute/confidential-vm/docs/about-cvm) feature of CVMs on GCP for runtime measurements.
Note that the vTPM in GCP doesn't run inside the hardware-protected CVM context, but is emulated on the hypervisor level.
The vTPM on GCP CVMs adheres to the [TPM 2.0](https://trustedcomputinggroup.org/resource/tpm-library-specification/) specification of the [Trusted Computing Group](https://trustedcomputinggroup.org/resource/trusted-platform-module-tpm-summary/).
It provides a [launch attestation report](https://cloud.google.com/compute/confidential-vm/docs/monitoring#about_launch_attestation_report_events) that's based on the Measured Boot feature of [Shielded VMs](https://cloud.google.com/compute/shielded-vm/docs/shielded-vm#measured-boot).
The following table lists all PCR values of the vTPM and the measured components.
It also lists what components of the boot chain did the measurements and if the value is reproducible and verifiable.
The latter means that value can be generated offline and compared to the one in the vTPM.
| PCR | Components | Measured by | Reproducible and Verifiable |
|---------------|----------------------------------|-------------------------------|-----------------------------|
| 0 | CVM constant string | GCP | No |
| 1 | Reserved | GCP | No |
| 2 | Reserved | GCP | No |
| 3 | Reserved | GCP | No |
| 4 | Constellation Bootloader, GRUB | GCP, Constellation Bootloader | Yes |
| 5 | Disk GUID partition table | GCP | No |
| 6 | Disk GUID partition table | GCP | No |
| 7 | GCP Secure Boot Policy | GCP, Constellation Bootloader | No |
| 8 | Kernel command line, GRUB config | Constellation Bootloader | Yes |
| 9 | Kernel, initramfs | Constellation Bootloader | Yes |
| 10 | Reserved | - | No |
| 11 | Reserved | Constellation Bootstrapper | Yes |
| 12 | ClusterID | Constellation Bootstrapper | Yes |
| 13&ndash;23 | Unused |- | - |
</tabItem>
</tabs>
## Cluster attestation
Cluster-facing, Constellation's [*JoinService*](components.md#joinservice) verifies each node joining the cluster given the configured ground truth runtime measurements.
User-facing, the [*VerificationService*](components.md#verificationservice) provides an interface to verify a node using remote attestation.
By verifying the first node during the [initialization](components.md#bootstrapper) and configuring the ground truth measurements that are subsequently enforced by the *JoinService*, the whole cluster is verified in a transitive way.
### Cluster-facing attestation
The *JoinService* is provided with the runtime measurements of the whitelisted Constellation image version as the ground truth.
During the initialization and the cluster bootstrapping, each node connects to the *JoinService* using [aTLS](#attested-tls-atls).
During the handshake, the node transmits an attestation statement including its runtime measurements.
The *JoinService* verifies that statement and compares the measurements against the ground truth.
For details of the initialization process check the [component descriptions](components.md).
After the initialization, every node updates its runtime measurements with the *clusterID* value, marking it irreversibly as initialized.
When an initialized node tries to join another cluster, its measurements inevitably mismatch the measurements of an uninitialized node and it will be declined.
### User-facing attestation
The [*VerificationService*](components.md#verificationservice) provides an endpoint for obtaining its hardware-based remote attestation statement, which includes the runtime measurements.
A user can [verify](../workflows/verify-cluster.md) this statement and compare the measurements against the configured ground truth and, thus, verify the identity and integrity of all Constellation components and the cluster configuration. Subsequently, the user knows that the entire cluster is in the expected state and is trustworthy.
## Chain of trust
So far, this page described how an entire Constellation cluster can be verified using hardware attestation capabilities and runtime measurements.
The last missing link is how the ground truth in the form of runtime measurements can be securely distributed to the verifying party.
When building Constellation images the process entails creating the ground truth runtime measurements.
The builds of Constellation images are reproducible and the measurements of an image can be recalculated and verified by everyone.
With every release, Edgeless Systems publishes signed runtime measurements.
When installing the Constellation CLI, the release binary is also signed by Edgeless Systems.
The [installation guide](../architecture/orchestration.md#verify-your-cli-installation) explains how this signature can be verified.
The CLI contains the public key required to verify signed runtime measurements from Edgeless Systems.
When a new cluster is [created](../workflows/create.md) or updated, the CLI automatically verifies the measurements for the selected image.
Alternatively, Constellation has the option to use and verify custom build images.
For this, the CLI can be provided with a custom public key for verification.
Thus, we've a chain of trust based on cryptographic signatures, which goes from CLI to runtime measurements to images. This is illustrated in the following diagram.
```mermaid
flowchart LR
A[Edgeless]-- "signs (cosign)" -->B[CLI]
C[User]-- "verifies (cosign)" -->B[CLI]
B[CLI]-- "contains" -->D["Public Key"]
A[Edgeless]-- "signs" -->E["Runtime measurements"]
D["Public Key"]-- "verifies" -->E["Runtime measurements"]
E["Runtime measurements"]-- "verify" -->F["Constellation cluster"]
```

81
docs/versioned_docs/version-2.1/architecture/components.md Normal file
View file

@ -0,0 +1,81 @@
# Components
Constellation takes care of bootstrapping and initializing a Confidential Kubernetes cluster.
During the lifetime of the cluster, it handles day 2 operations such as key management, remote attestation, and updates.
These features are provided by several components:
* The [Bootstrapper](components.md#bootstrapper) initializes a Constellation node and bootstraps the cluster
* The [JoinService](components.md#joinservice) joins new nodes to an existing cluster
* The [VerificationService](components.md#verificationservice) provides remote attestation functionality
* The [Key Management Service (KMS)](components.md#kms) manages Constellation-internal keys
* The [AccessManager](components.md#accessmanager) manages node SSH access
The relations between components are shown in the following diagram:
```mermaid
flowchart LR
subgraph admin [Admin's machine]
A[Constellation CLI]
end
subgraph img [CoreOS image]
B[CoreOS]
C[Bootstrapper]
end
subgraph Kubernetes
D[AccessManager]
E[JoinService]
F[KMS]
G[VerificationService]
end
A -- deploys -->
B -- starts --> C
C -- deploys --> D
C -- deploys --> E
C -- deploys --> F
C -- deploys --> G
```
## Bootstrapper
The *Bootstrapper* is the first component launched after booting a Constellation node image.
It sets up that machine as a Kubernetes node and integrates that node into the Kubernetes cluster.
To this end, the *Bootstrapper* first downloads and [verifies](https://blog.sigstore.dev/kubernetes-signals-massive-adoption-of-sigstore-for-protecting-open-source-ecosystem-73a6757da73) the [Kubernetes components](https://kubernetes.io/docs/concepts/overview/components/) at the configured versions.
The *Bootstrapper* tries to find an existing cluster and if successful, communicates with the [JoinService](components.md#joinservice) to join the node.
Otherwise, it waits for an initialization request to create a new Kubernetes cluster.
## JoinService
The *JoinService* runs as [DaemonSet](https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/) on each control-plane node.
New nodes (at cluster start, or later through autoscaling) send a request to the service over [attested TLS (aTLS)](attestation.md#attested-tls-atls).
The *JoinService* verifies the new node's certificate and attestation statement.
If attestation is successful, the new node is supplied with an encryption key from the [*KMS*](components.md#kms) for its state disk, and a Kubernetes bootstrap token.
```mermaid
sequenceDiagram
participant New node
participant JoinService
New node->>JoinService: aTLS handshake (server side verification)
JoinService-->>New node: #
New node->>+JoinService: IssueJoinTicket(DiskUUID, NodeName, IsControlPlane)
JoinService->>+KMS: GetDataKey(DiskUUID)
KMS-->>-JoinService: DiskEncryptionKey
JoinService-->>-New node: DiskEncryptionKey, KubernetesJoinToken, ...
```
## VerificationService
The *VerificationService* runs as DaemonSet on each node.
It provides user-facing functionality for remote attestation during the cluster's lifetime via an endpoint for [verifying the cluster](attestation.md#cluster-attestation).
Read more about the hardware-based [attestation feature](attestation.md) of Constellation and how to [verify](../workflows/verify-cluster.md) a cluster on the client side.
## KMS
The *KMS* runs as DaemonSet on each control-plane node.
It implements the key management for the [storage encryption keys](keys.md#storage-encryption) in Constellation. These keys are used for the [state disk](images.md#state-disk) of each node and the [transparently encrypted storage](encrypted-storage.md) for Kubernetes.
Depending on wether the [constellation-managed](keys.md#constellation-managed-key-management) or [user-managed](keys.md#user-managed-key-management) mode is used, the *KMS* holds the key encryption key (KEK) directly or calls an external service for key derivation respectively.
## AccessManager
The *AccessManager* runs as DaemonSet on each node.
It manages the user's SSH access to nodes as specified in the config.

57
docs/versioned_docs/version-2.1/architecture/encrypted-storage.md Normal file
View file

@ -0,0 +1,57 @@
# Encrypted persistent storage
Confidential VMs provide runtime memory encryption to protect data in use.
In the context of Kubernetes, this is sufficient for the confidentiality and integrity of stateless services.
Consider a front-end web server, for example, that keeps all connection information cached in main memory.
No sensitive data is ever written to an insecure medium.
However, many real-world applications need some form of state or data-lake service that's connected to a persistent storage device and requires encryption at rest.
As described in [Use persistent storage](../workflows/storage.md), cloud service providers (CSPs) use the container storage interface (CSI) to make their storage solutions available to Kubernetes workloads.
These CSI storage solutions often support some sort of encryption.
For example, Google Cloud [encrypts data at rest by default](https://cloud.google.com/security/encryption/default-encryption), without any action required by the customer.
## Cloud provider-managed encryption
CSP-managed storage solutions encrypt the data in the cloud backend before writing it physically to disk.
In the context of confidential computing and Constellation, the CSP and its managed services aren't trusted.
Hence, cloud provider-managed encryption protects your data from offline hardware access to physical storage devices.
It doesn't protect it from anyone with infrastructure-level access to the storage backend or a malicious insider in the cloud platform.
Even with "bring your own key" or similar concepts, the CSP performs the encryption process with access to the keys and plaintext data.
In the security model of Constellation, securing persistent storage and thereby data at rest requires that all cryptographic operations are performed inside a trusted execution environment.
Consequently, using CSP-managed encryption of persistent storage usually isn't an option.
## Constellation-managed encryption
Constellation provides CSI drivers for storage solutions in all major clouds with built-in encryption support.
Block storage provisioned by the CSP is [mapped](https://guix.gnu.org/manual/en/html_node/Mapped-Devices.html) using the [dm-crypt](https://www.kernel.org/doc/html/latest/admin-guide/device-mapper/dm-crypt.html), and optionally the [dm-integrity](https://www.kernel.org/doc/html/latest/admin-guide/device-mapper/dm-integrity.html), kernel modules, before it's formatted and accessed by the Kubernetes workloads.
All cryptographic operations happen inside the trusted environment of the confidential Constellation node.
Please note that for integrity-protected disks, [volume expansion](https://kubernetes.io/blog/2018/07/12/resizing-persistent-volumes-using-kubernetes/) isn't supported.
By default the driver uses data encryption keys (DEKs) issued by the Constellation [*KMS*](components.md#kms).
The DEKs are in turn derived from the Constellation's key encryption key (KEK), which is directly derived from the [master secret](keys.md#master-secret).
This is the recommended mode of operation, and also requires the least amount of setup by the cluster administrator.
Alternatively, the driver can be configured to use a key management system to store and access KEKs and DEKs.
Please refer to [keys and cryptography](keys.md) for more details on key management in Constellation.
Once deployed and configured, the CSI driver ensures transparent encryption and integrity of all persistent volumes provisioned via its storage class.
Data at rest is secured without any additional actions required by the developer.
## Cryptographic algorithms
This section gives an overview of the libraries, cryptographic algorithms, and their configurations, used in Constellation's CSI drivers.
### dm-crypt
To interact with the dm-crypt kernel module, Constellation uses [libcryptsetup](https://gitlab.com/cryptsetup/cryptsetup/).
New devices are formatted as [LUKS2](https://gitlab.com/cryptsetup/LUKS2-docs/-/tree/master) partitions with a sector size of 4096 bytes.
The used key derivation function is [Argon2id](https://datatracker.ietf.org/doc/html/rfc9106) with the [recommended parameters for memory-constrained environments](https://datatracker.ietf.org/doc/html/rfc9106#section-7.4) of 3 iterations and 64 MiB of memory, utilizing 4 parallel threads.
For encryption Constellation uses AES in XTS-Plain64. The key size is 512 bit.
### dm-integrity
To interact with the dm-integrity kernel module, Constellation uses [libcryptsetup](https://gitlab.com/cryptsetup/cryptsetup/).
When enabled, the used data integrity algorithm is [HMAC](https://datatracker.ietf.org/doc/html/rfc2104) with SHA256 as the hash function.
The tag size is 32 Bytes.

45
docs/versioned_docs/version-2.1/architecture/images.md Normal file
View file

@ -0,0 +1,45 @@
# Constellation images
Constellation uses [Fedora CoreOS](https://docs.fedoraproject.org/en-US/fedora-coreos/) as the operating system running inside confidential VMs. This Linux distribution is optimized for containers and is designed to have an immutable filesystem.
The Constellation images extend on that concept by leveraging measured boot and verification of the root filesystem.
## Measured boot
```mermaid
flowchart LR
Firmware --> Bootloader
Bootloader --> kernel
Bootloader --> initramfs
initramfs --> rootfs[root filesystem]
```
Measured boot uses a Trusted Platform Module (TPM) to measure every part of the boot process. This allows for verification of the integrity of a running system at any point in time. To ensure correct measurements of every stage, each stage is responsible to measure the next stage before transitioning.
### Firmware
With confidential VMs, the firmware is the root of trust and is measured automatically at boot. After initialization, the firmware will load and measure the bootloader before executing it.
### Bootloader
The bootloader is the first modifiable part of the boot chain. The bootloader is tasked with loading the kernel, initramfs and setting the kernel command line. The Constellation bootloader measures these components before starting the kernel.
### initramfs
The initramfs is a small filesystem loaded to prepare the actual root filesystem. The Constellation initramfs maps the block device containing the root filesystem with [dm-verity](https://www.kernel.org/doc/html/latest/admin-guide/device-mapper/verity.html). The initramfs then mounts the root filesystem from the mapped block device.
dm-verity provides integrity checking using a cryptographic hash tree. When a block is read, its integrity is checked by verifying the tree against a trusted root hash. The initramfs reads this root hash from the previously measured kernel command line. Thus, if any block of the root filesystem's device is modified on disk, trying to read the modified block will result in a kernel panic at runtime.
After mounting the root filesystem, the initramfs will switch over and start the `init` process of the integrity-protected root filesystem.
## State disk
In addition to the read-only root filesystem, each Constellation node has a disk for storing state data.
This disk is mounted readable and writable by the initramfs and contains data that should persist across reboots.
Such data can contain sensitive information and, therefore, must be stored securely.
To that end, the state disk is protected by authenticated encryption.
See the section on [keys and encryption](keys.md#storage-encryption) for more information on the cryptographic primitives in use.
## Kubernetes components
During initialization, the [*Bootstrapper*](components.md#bootstrapper) downloads and [verifies](https://blog.sigstore.dev/kubernetes-signals-massive-adoption-of-sigstore-for-protecting-open-source-ecosystem-73a6757da73) the [Kubernetes components](https://kubernetes.io/docs/concepts/overview/components/) as configured by the user.
They're stored on the state partition and can be updated once new releases need to be installed.

127
docs/versioned_docs/version-2.1/architecture/keys.md Normal file
View file

@ -0,0 +1,127 @@
# Key management and cryptographic primitives
Constellation protects and isolates your cluster and workloads.
To that end, cryptography is the foundation that ensures the confidentiality and integrity of all components.
Evaluating the security and compliance of Constellation requires a precise understanding of the cryptographic primitives and keys used.
The following gives an overview of the architecture and explains the technical details.
## Confidential VMs
Confidential VM (CVM) technology comes with hardware and software components for memory encryption, isolation, and remote attestation.
For details on the implementations and cryptographic soundness please refer to the hardware vendors' documentation and advisories.
## Master secret
The master secret is the cryptographic material used for deriving the [*clusterID*](#cluster-identity) and the *key encryption key (KEK)* for [storage encryption](#storage-encryption).
It's generated during the bootstrapping of a Constellation cluster.
It can either be managed by [Constellation](#constellation-managed-key-management) or an [external key management system](#user-managed-key-management)
In case of [recovery](#recovery-and-migration), the master secret allows to decrypt the state and recover a Constellation cluster.
## Cluster identity
The identity of a Constellation cluster consists of two parts:
* *baseID:* The identity of a valid and measured, uninitialized Constellation node
* *clusterID:* The identity unique to a single initialized Constellation cluster
Using the CVM's attestation mechanism and [measured boot up to the read-only root filesystem](images.md) guarantees *baseID*.
The *clusterID* is derived from the master secret and a cryptographically random salt. It's unique for every Constellation cluster.
The remote attestation statement of a Constellation cluster combines *baseID* and *clusterID* for a verifiable, unspoofable, unique identity.
## Network encryption
Constellation encrypts all cluster network communication using the [container network interface (CNI)](https://github.com/containernetworking/cni).
See [network encryption](networking.md) for more details.
The Cilium agent running on each cluster node will establish a secure WireGuard tunnel between it and all other known nodes in the cluster.
Each node automatically creates its own [Curve25519](http://cr.yp.to/ecdh.html) encryption key-pair and distributes its public key via Kubernetes.
Each node’s public key is then used by other nodes to decrypt and encrypt traffic from and to Cilium-managed endpoints running on that node.
Connections are always encrypted peer-to-peer using [ChaCha20](http://cr.yp.to/chacha.html) with [Poly1305](http://cr.yp.to/mac.html).
WireGuard implements [forward secrecy with key rotation every 2 minutes](https://lists.zx2c4.com/pipermail/wireguard/2017-December/002141.html).
Cilium supports [key rotation](https://docs.cilium.io/en/stable/gettingstarted/encryption-ipsec/#key-rotation) for the long-term node keys via Kubernetes secrets.
## Storage encryption
Constellation supports transparent encryption of persistent storage.
The Linux kernel's device mapper-based encryption features are used to encrypt the data on the block storage level.
Currently, the following primitives are used for block storage encryption:
* [dm-crypt](https://www.kernel.org/doc/html/latest/admin-guide/device-mapper/dm-crypt.html)
* [dm-integrity](https://www.kernel.org/doc/html/latest/admin-guide/device-mapper/dm-integrity.html)
Adding primitives for integrity protection in the CVM attacker model are under active development and will be available in a future version of Constellation.
See [encrypted storage](encrypted-storage.md) for more details.
As a cluster administrator, when creating a cluster, you can use the Constellation [installation program](orchestration.md) to select one of the following methods for key management:
* Constellation-managed key management
* User-managed key management
### Constellation-managed key management
#### Key material and key derivation
During the creation of a Constellation, the cluster's master secret is used to derive a KEK.
This means creating two clusters with the same master secret will yield the same KEK.
Any data encryption key (DEK) is derived from the KEK via HKDF.
Note that the master secret is recommended to be unique for every Constellation cluster and shouldn't be reused (except in case of [recovering](../workflows/recovery.md) a cluster).
#### State and storage
The KEK is derived from the master secret during the initialization.
Subsequently, all other key material is derived from the KEK.
Given the same KEK, any DEK can be derived deterministically from a given identifier.
Hence, there is no need to store DEKs. They can be derived on demand.
After the KEK was derived, it's stored in memory only and never leaves the CVM context.
#### Availability
Constellation-managed key management has the same availability as the underlying Kubernetes cluster.
Therefore, the KEK is stored in the [distributed Kubernetes etcd storage](https://kubernetes.io/docs/tasks/administer-cluster/configure-upgrade-etcd/) to allow for unexpected but non-fatal (control-plane) node failure.
The etcd storage is backed by the encrypted and integrity protected [state disk](images.md#state-disk) of the nodes.
#### Recovery
Constellation clusters can be recovered in the event of a disaster, even when all node machines have been stopped and need to be rebooted.
For details on the process see the [recovery workflow](../workflows/recovery.md).
### User-managed key management
User-managed key management is under active development and will be available soon.
In scenarios where constellation-managed key management isn't an option, this mode allows you to keep full control of your keys.
For example, compliance requirements may force you to keep your KEKs in an on-prem key management system (KMS).
During the creation of a Constellation, you specify a KEK present in a remote KMS.
This follows the common scheme of "bring your own key" (BYOK).
Constellation will support several KMSs for managing the storage and access of your KEK.
Initially, it will support the following KMSs:
* [AWS KMS](https://aws.amazon.com/kms/)
* [GCP KMS](https://cloud.google.com/security-key-management)
* [Azure Key Vault](https://azure.microsoft.com/en-us/services/key-vault/#product-overview)
* [KMIP-compatible KMS](https://www.oasis-open.org/committees/tc_home.php?wg_abbrev=kmip)
Storing the keys in Cloud KMS of AWS, GCP, or Azure binds the key usage to the particular cloud identity access management (IAM).
In the future, Constellation will support remote attestation-based access policies for Cloud KMS once available.
Note that using a Cloud KMS limits the isolation and protection to the guarantees of the particular offering.
KMIP support allows you to use your KMIP-compatible on-prem KMS and keep full control over your keys.
This follows the common scheme of "hold your own key" (HYOK).
The KEK is used to encrypt per-data "data encryption keys" (DEKs).
DEKs are generated to encrypt your data before storing it on persistent storage.
After being encrypted by the KEK, the DEKs are stored on dedicated cloud storage for persistence.
Currently, Constellation supports the following cloud storage options:
* [AWS S3](https://aws.amazon.com/s3/)
* [GCP Cloud Storage](https://cloud.google.com/storage)
* [Azure Blob Storage](https://azure.microsoft.com/en-us/services/storage/blobs/#overview)
The DEKs are only present in plaintext form in the encrypted main memory of the CVMs.
Similarly, the cryptographic operations for encrypting data before writing it to persistent storage are performed in the context of the CVMs.
#### Recovery and migration
In the case of a disaster, the KEK can be used to decrypt the DEKs locally and subsequently use them to decrypt and retrieve the data.
In case of migration, configuring the same KEK will provide seamless migration of data.
Thus, only the DEK storage needs to be transferred to the new cluster alongside the encrypted data for seamless migration.

23
docs/versioned_docs/version-2.1/architecture/networking.md Normal file
View file

@ -0,0 +1,23 @@
# Network encryption
Constellation encrypts all pod communication using the [container network interface (CNI)](https://github.com/containernetworking/cni).
To that end, Constellation deploys, configures, and operates the [Cilium](https://cilium.io/) CNI plugin.
Cilium provides [transparent encryption](https://docs.cilium.io/en/stable/gettingstarted/encryption) for all cluster traffic using either IPSec or [WireGuard](https://www.wireguard.com/).
Currently, Constellation only supports WireGuard as the encryption engine.
You can read more about the cryptographic soundness of WireGuard [in their white paper](https://www.wireguard.com/papers/wireguard.pdf).
Cilium is actively working on implementing a feature called [`host-to-host`](https://github.com/cilium/cilium/pull/19401) encryption mode for WireGuard.
With `host-to-host`, all traffic between nodes will be tunneled via WireGuard (host-to-host, host-to-pod, pod-to-host, pod-to-pod).
Until the `host-to-host` feature is released, Constellation enables `pod-to-pod` encryption.
This mode encrypts all traffic between Kubernetes pods using WireGuard tunnels.
Constellation uses an extended version of `pod-to-pod` called *strict* mode.
When using Cilium in the default setup but with encryption enabled, there is a [known issue](https://docs.cilium.io/en/v1.12/gettingstarted/encryption/#egress-traffic-to-not-yet-discovered-remote-endpoints-may-be-unencrypted)
that can cause pod-to-pod traffic to be unencrypted.
Constellation uses strict mode to mitigates this issue.
We change the default behavior of traffic that's destined for an unknown endpoint to not be send out in plaintext to instead being dropped.
The strict mode can distinguish between traffic that's send to an pod from traffic that's destined for an cluster-external endpoint, since it knows the pod's CIDR range.
The last remaining traffic that's not encrypted is traffic originating from hosts.
This mainly includes health checks from Kubernetes API server.
Also, traffic proxied over the API server via e.g. `kubectl port-forward` isn't encrypted.

88
docs/versioned_docs/version-2.1/architecture/orchestration.md Normal file
View file

@ -0,0 +1,88 @@
# Orchestrating Constellation clusters
You can use the CLI to create a cluster on the supported cloud platforms.
The CLI provisions the resources in your cloud environment and initiates the initialization of your cluster.
It uses a set of parameters and an optional configuration file to manage your cluster installation.
The CLI is also used for updating your cluster.
## Workspaces
Each Constellation cluster has an associated *workspace*.
The workspace is where the persistent data such as the Constellation state, config, and ID files are stored.
Each workspace is associated with a single cluster and configuration.
Currently, the CLI stores state in the local filesystem making the current directory the active workspace.
Multiple clusters require multiple workspaces, hence, multiple directories.
Note that every operation on a cluster always has to be performed from the directory associated with its workspace.
## Cluster creation process
Releases of Constellation are [published on GitHub](https://github.com/edgelesssys/constellation/releases).
To allow for fine-grained configuration of your cluster and cloud environment, Constellation supports an extensive configuration file with strong defaults.
The CLI provides you with a good default configuration which can be generated with `constellation config generate`. Some cloud account-specific information is always required and to be set by the user.
The following files are generated during the creation of a Constellation cluster and stored in the current workspace:
* a configuration file
* a state file
* an ID file
* a base64 encoded master secret
* a Kubernetes `kubeconfig` file.
Constellation must store the state of its created infrastructure and configuration.
This state is used by Constellation to map real-world resources to your configuration and keep track of metadata.
This state is stored by default in a local file named `constellation-state.json`.
After the successful creation of your cluster, the CLI will provide you with a Kubernetes `kubeconfig` file.
This file provides you with access to your Kubernetes cluster and configures the [kubectl](https://kubernetes.io/docs/concepts/configuration/organize-cluster-access-kubeconfig/) tool.
In addition, the cluster's [identifier](orchestration.md#post-installation-configuration) is returned and stored in a file called `constellation-id.json`
### Creation process details
1. The CLI `create` command creates the confidential VMs (CVMs) resources in your cloud environment and configures the network
2. Each CVM boots the Constellation node image and measures every component in the boot chain
3. The first component launched in each node is the [*Bootstrapper*](components.md#bootstrapper)
4. The *Bootstrapper* waits until it either receives an initialization request or discovers an initialized cluster
5. The CLI `init` command connects to the *Bootstrapper* of a selected node, sends the configuration, and initiates the initialization of the cluster
6. The *Bootstrapper* of **that** node [initializes the Kubernetes cluster](components.md#bootstrapper) and deploys the other Constellation [components](components.md) including the [*JoinService*](components.md#joinservice)
7. Subsequently, the *Bootstrappers* of the other nodes discover the initialized cluster and send join requests to the *JoinService*
8. As part of the join request each node includes an attestation statement of its boot measurements as a form of authentication
9. The *JoinService* verifies the attestation statements and joins the nodes to the Kubernetes cluster
10. This process is repeated for every node joining the cluster later (e.g., through autoscaling)
## Post-installation configuration
Post-installation the CLI provides a configuration for [accessing the cluster using the Kubernetes API](https://kubernetes.io/docs/tasks/administer-cluster/access-cluster-api/).
The `kubeconfig` file provides the credentials and configuration for connecting and authenticating to the API server.
Once configured, orchestrate the Kubernetes cluster via `kubectl`.
Keep the state files in the workspace directory such as the `constellation-state.json` for the CLI to be able to manage your cluster.
Without it, you won't be able to modify or terminate your cluster.
After the initialization, the CLI will present you with a couple of tokens:
* The [*master secret*](keys.md#master-secret) (stored in the `constellation-mastersecret.json` file by default)
* The [*clusterID*](keys.md#cluster-identity) of your cluster in base64 format
You can read more about these values and their meaning in the guide on [cluster identity](keys.md#cluster-identity).
The *master secret* must be kept secret and can be used to [recover your cluster](../workflows/recovery.md).
Instead of managing this secret manually, you can [use your key management solution of choice](keys.md#user-managed-key-management) with Constellation.
The *clusterID* uniquely identifies a cluster and can be used to [verify your cluster](../workflows/verify-cluster.md).
## Upgrades
Constellation images and components might need to be upgraded to new versions during the lifetime of a cluster.
Constellation implements a rolling update mechanism ensuring no downtime of the control or data plane.
You can upgrade a Constellation cluster with a single operation by using the CLI.
For step-by-step instructions on how to do this, refer to [Upgrade Constellation](../workflows/upgrade.md).
### Attestation of upgrades
The new verification hashes (measurements) are released together with every image.
During an update procedure, the CLI provides the new measurements to the [JoinService](components.md#joinservice) securely.
New measurements for an updated image are automatically pulled and verified by the CLI following the [supply chain security concept](attestation.md#chain-of-trust) of Constellation.
The [attestation section](attestation.md#cluster-facing-attestation) describes in detail how these measurements are then used by the JoinService for the attestation of nodes.
The updated measurements are reproducible based on the updated node images, hence, auditable by the customer.

23
docs/versioned_docs/version-2.1/architecture/overview.md Normal file
View file

@ -0,0 +1,23 @@
# Overview
Constellation is a cloud-based confidential orchestration platform.
The foundation of Constellation is Kubernetes and therefore shares the same technology stack and architecture principles.
To learn more about Constellation and Kubernetes, see [product overview](../overview/product.md).
## About installation and updates
As a cluster administrator, you can use the [Constellation CLI](orchestration.md) to install and deploy a cluster.
## About the components and attestation
Constellation manages the nodes and network in your cluster. All nodes are bootstrapped by the [*Bootstrapper*](components.md#bootstrapper). They're verified and authenticated by the [*JoinService*](components.md#joinservice) before being added to the cluster and the network. Finally, the entire cluster can be verified via the [*VerificationService*](components.md#verification-service) using [remote attestation](attestation.md).
## About node images and verified boot
Constellation comes with operating system images for Kubernetes control-plane and worker nodes.
They're highly optimized for running containerized workloads and specifically prepared for running inside confidential VMs.
You can learn more about [the images](images.md) and how verified boot ensures their integrity during boot and beyond.
## About key management and cryptographic primitives
Encryption of data at-rest, in-transit, and in-use is the fundamental building block for confidential computing and Constellation. Learn more about the [keys and cryptographic primitives](keys.md) used in Constellation and about [encrypted persistent storage](encrypted-storage.md).

17
docs/versioned_docs/version-2.1/architecture/versions.md Normal file
View file

@ -0,0 +1,17 @@
# Versions and support policy
This page details which guarantees Constellation makes regarding versions, compatibility, and life cycle.
All released components of Constellation use a three-digit version number, of the form `v<MAJOR>.<MINOR>.<PATCH>`.
## Release cadence
All [components](components.md) of Constellation are released in lock step on the first Tuesday of every month. This release primarily introduces new features, but may also include security or performance improvements. The `MINOR` version will be incremented as part of this release.
Additional `PATCH` releases may be created on demand, to fix security issues or bugs before the next `MINOR` release window.
New releases are published on [GitHub](https://github.com/edgelesssys/constellation/releases).
### Kubernetes support policy
Constellation is aligned to the [version support policy of Kubernetes](https://kubernetes.io/releases/version-skew-policy/#supported-versions), and therefore supports the most recent three minor versions.