Git-Mirrors/constellation
1
0
Fork 0
mirror of https://github.com/edgelesssys/constellation.git synced 2024-09-19 15:56:03 +00:00
Code Issues Packages Projects Releases Wiki Activity

rfc: update documentation of new versions API (#788)

Browse Source 
Signed-off-by: Paul Meyer <49727155+katexochen@users.noreply.github.com>
Co-authored-by: Malte Poll <mp@edgeless.systems>
This commit is contained in:
Paul Meyer 2023-01-03 15:49:58 +01:00 committed by GitHub
parent 7c017e2b67
commit baa1b37681
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
5 changed files with 190 additions and 53 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

132
rfc/apis.md Normal file
View File

@ -0,0 +1,132 @@
# Constellation APIs (v1)
## Base
The API base starts with `constellation` followed by the API version.
At this moment, the only valid API version is `v1`:
```
/constellation/v1/
```
## API groups
The API version is followed by the API group. Possible values are:
- [`versions`: version information for Constellation components](version-api.md)
- [`image`: metadata for individual Constellation OS images](image-api.md)
- `info`: image reference lookup for each cloud provider and additional metadata
- `measurements`: TPM measurements for Constellation OS images
- `raw`: raw OS images
- `sbom`: SBOMs for Constellation OS images
There may be more API groups in the future (e.g. `cli`)
## API paths overview
- [`/constellation/v1/ref/<ref>/stream/<stream>/versions/latest/<kind>.json`](version-api.md#latest)
- [`/constellation/v1/ref/<ref>/stream/<stream>/versions/major/<base>/<kind>.json`](version-api.md#major-to-minor-version-list)
- [`/constellation/v1/ref/<ref>/stream/<stream>/versions/minor/<base>/<kind>.json`](version-api.md#minor-to-patch-version-list)
- [`/constellation/v1/ref/<ref>/stream/<stream>/image/<version>/info.json`](image-api.md#image-lookup-table)
- [`/constellation/v1/ref/<ref>/stream/<stream>/image/<version>/sbom.<format>.json`](image-api.md)
- [`/constellation/v1/ref/<ref>/stream/<stream>/image/<version>/csp/<csp>/measurements.json`](image-api.md)
- [`/constellation/v1/ref/<ref>/stream/<stream>/image/<version>/csp/<csp>/measurements.json.sig`](image-api.md)
- [`/constellation/v1/ref/<ref>/stream/<stream>/image/<version>/csp/<csp>/measurements.image.json`](image-api.md)
- [`/constellation/v1/ref/<ref>/stream/<stream>/image/<version>/csp/<csp>/image.raw`](image-api.md)
## API path identifiers `ref`, `stream` and `version`
### Meaning of `ref`, `stream` and `version` for resource names
Components in this API are identified by `ref`, `stream` and `version`.
- a `ref` that is either a normalized git branch name or the special value `-`. Defaults to `-`.
- a `stream` that determines the specific type of the resource. Defaults to `stable`.
- a `version` that is a semantic version.
The special `ref` value `-` (dash) is reserved for releases. Every other value is a branch name where special characters are replaced by dashes.
Streams are used to distinguish different types of resources. Depending on the kind of resource and `ref`, only a subset of values might be allowed.
The `version` is always a valid semantic version. For the special stream `stable` it is guaranteed to be a semantic version `v<major>.<minor>.<patch>`,
for other stream values, it is a semantic version that may contain a suffix after the patch: `v<major>.<minor>.<patch>-<suffix>`.
The use of semantic versions and pseudo versions tries to match the versioning used in Go. If in doubt,
consult the module reference of Go, especially the sections about [versions](https://go.dev/ref/mod#versions)
and [pseudo-versions](https://go.dev/ref/mod#pseudo-versions).
The [pesudo-version tool](../hack/pseudo-version) can generate a valid pseudo-version for your current head.
### Consisten API path prefix of `ref` and `stream`
For API calls, paths will always start with `ref` and `stream`:
```
<base>/ref/<ref>/stream/<stream>/
```
### Meaning of `ref` and `stream` for images
The special value `-` (dash) for images is used for releases. They are always built from the corresponding release branch (and tag).
Every other value refers to a normalized branch name.
For the `ref` `-` (dash), the `stream` value can be one of the following constants:
- `stable`: Built with default settings (non-debug), for end users and with all security features enabled
- `console`: Built with default settings (non-debug), allows access to the serial console
- `debug`: Image containing the debugd, allows access to the serial console
For other `ref` values, the `stream` value can be one of the following constants:
- `nightly`: Built with default settings (non-debug), with all security features enabled
- `console`: Built with default settings (non-debug), allows access to the serial console
- `debug`: Image containing the debugd, allows access to the serial console
## Short paths for resource identification
Short paths allow an easier version handling, e.g., in end user configuration files or CLI tools.
When using the default values, short paths omit either the `ref` (if set to `-`) or the `ref` and `stream`
parts (if set to `-` and `stable`):
- `ref/-/stream/stable/<version>` is equivalent to the short path `<version>`
- `ref/-/stream/<stream>/<version>` is equivalent to the short path `stream/<stream>/<version>`
Resource group can be identified by `ref/<ref>/stream/<stream>/<version>` and allow for short form encoding as explained above.
Resources using this encoding include:
- `version`: version information for Constellation components
- `image`: metadata for individual Constellation OS images
## Examples
Release v2.3.0 would use the following image name for the default, end-user image:
```
short form: v2.3.0
long form: ref/-/stream/stable/v2.3.0
```
A debug image created from the same release:
```
short form: stream/debug/v2.3.0
long form: ref/-/stream/debug/v2.3.0
```
A debug image on the branch `feat/xyz`:
```
ref/feat-xyz/stream/debug/v2.4.0-pre.0.20230922011244-0744d001aa84
```
A nightly image on `main`:
```
ref/main/stream/nightly/v2.4.0-pre.0.20230922011244-0744d001aa84
```
A debug image on `main`:
```
ref/main/stream/debug/v2.4.0-pre.0.20230922011244-0744d001aa84
```

67
rfc/image-discoverability.md → rfc/image-api.md
View File

@ -4,7 +4,7 @@ The Constellation OS image build pipeline generates a set of images using a chos
```mermaid
graph LR
version["input: version (<code>v2.2.0</code>)"] --> imageid["image version uid (<code>v2.2.0</code>)"]
version["input: version (<code>v2.2.0</code>)"] --> imageid["image name (<code>v2.2.0</code>)"]
commit["input: Constellation repo commit hash (<code>cc0de5c</code>)"] --> pipeline
imageid --> pipeline{build images}
pipeline --> awsimage["raw AWS image"]
@ -42,28 +42,25 @@ The build pipeline takes as inputs:
- a version number that is one of
- a release version number (e.g. `v2.2.0`) for release images
- a pseudo-version number (e.g. `debug-v2.3.0-pre.0.`) for development images
- a pseudo-version number (e.g. `branch-name-v2.3.0-pre.0.`) for branch images
- a pseudo-version number (e.g. `v2.3.0-pre.0.`) for other images
- a branch name of the Constellation monorepo that is used to build the images
- a commit hash of the Constellation monorepo that is used to build the images (e.g. `cc0de5c68d41f31dd0b284d574f137e0b0ad106b`)
- a commit timestamp of the Constellation monorepo that is used to build the images (e.g. `20221115082220`)
To identify images belonging to one invocation of the build pipeline, the pipeline uses a unique identifier for the set of images, referred to as `image version uid`.
This is either the release version number (e.g. `v2.2.0`) or a pseudo version that combines the version number, commit timestamp and the commit hash (e.g. `debug-v2.3.0-pre.0.20221115082220-cc0de5c68d41f31dd0b284d574f137e0b0ad106b`).
The build pipeline produces as outputs:
- a raw OS image for every CSP
- a set of measurements for each raw OS image
- one or more images uploaded to each CSP (e.g. AWS AMIs, Azure images, GCP images)
- a lookup table that maps the `image version uid` to the CSP-specific image references
- a lookup table that maps the image name to the CSP-specific image references
The lookup table is uploaded to S3 and is used to identify the images that belong to a given `image version uid`.
Measurements are uploaded to S3 and can be looked up for each cloud service provider and `image version uid`.
The lookup table is uploaded to S3 and is used to identify the CSP specific references that belong to a given image.
Measurements are uploaded to S3 and can be looked up for each cloud service provider and image name.
## Image API
## Image API Endpoints
The build pipeline produces artifacts that are uploaded to S3 and can be accessed via HTTP.
The artifacts are organized in a directory structure that allows to look up the artifacts for a given `image version uid`.
The artifacts are organized in a directory structure that allows to look up the artifacts for a given image name.
Where applicable, the API uses the following CSP names:
@ -74,26 +71,30 @@ Where applicable, the API uses the following CSP names:
The following HTTP endpoints are available:
- `GET /constellation/v1/images/<image version uid>.json` returns the lookup table for the given `image version uid`.
- `GET /constellation/v1/measurements/<image version uid>/<csp>/` contains files with measurements and signatures for the given `image version uid` and CSP.
- `measurements.json` contains the final measurements for the given `image version uid` and CSP.
- `GET /constellation/v1/ref/<REF>/stream/<STREAM>/image/<VERSION>/`
- [`info.json` returns the lookup table for the given image version.](#image-lookup-table)
- `sbom.<format>.json` contains SBOM files for the given image version. The exact formats and file names are TBD.
- `GET /constellation/v1/ref/<REF>/stream/<STREAM>/<VERSION>/csp/<csp>/` contains files with measurements and signatures for the given image version and CSP.
- `measurements.json` contains the final measurements for the given image version and CSP.
- `measurements.json.sig` returns the signature of the measurements file.
- `measurements.image.json` returns the measurements generated statically from the image.
- `GET /constellation/v1/raw/<image version uid>/<csp>/image.raw` returns the raw image for the given `image version uid` and CSP.
- `GET /constellation/v1/sbom/<image version uid>/` contains SBOM files for the given `image version uid`. The exact formats and file names are TBD.
- `image.raw` returns the raw image for the given image version and CSP.
## Image lookup table
The image lookup table is a JSON file that maps the `image version uid` to the CSP-specific image references. It uses the `image version uid` as file name.
The image lookup table is a JSON file that maps the image name consisting of `ref`, `stream` and `version` to the CSP-specific image references:
```
s3://<BUCKET-NAME>/constellation/v1/images/<IMAGE-VERSION-UID>.json
/constellation/v1/ref/<REF>/stream/<STREAM>/image/<VERSION>/info.json
```
```json
{
"version": "<IMAGE-VERSION-UID>",
"debug": true,
"version": "<VERSION>",
"ref": "<REF>",
"stream": "<STREAM>",
"aws": {
"us-east-1": "ami-123",
"us-west-2": "ami-456",
@ -107,7 +108,7 @@ s3://<BUCKET-NAME>/constellation/v1/images/<IMAGE-VERSION-UID>.json
"sev-es": "gcp-image-123"
},
"qemu": {
"default": "https://cdn.confidential.cloud/constellation/v1/raw/v2.2.0/qemu/image.raw"
"default": "https://cdn.confidential.cloud/constellation/v1/ref/<REF>/stream/<STREAM>/image/<VERSION>/csp/qemu/image.raw"
}
}
```
@ -120,32 +121,19 @@ s3://<BUCKET-NAME>/constellation/v1/images/<IMAGE-VERSION-UID>.json
This document is not signed and can be extended in the future to include more image references (e.g. if an image is replicated to a new AWS region).
The same document can be used to identify old images that are no longer used and can be deleted for cost optimization.
## Image measurements
This RFC is not about the image measurements themselves, but about how they are stored and looked up.
The format of the image measurements is described in the [secure software distribution RFC](secure-software-distribution.md).
The image measurements are stored in a folder structure in S3 that is organized by CSP and `image version uid`.
```
s3://<BUCKET-NAME>/constellation/v1/measurements/<IMAGE-VERSION-UID>/<CSP>/measurements.json
s3://<BUCKET-NAME>/constellation/v1/measurements/<IMAGE-VERSION-UID>/<CSP>/measurements.json.sig
s3://<BUCKET-NAME>/constellation/v1/measurements/<IMAGE-VERSION-UID>/<CSP>/measurements.image.json
```
## CLI image discovery
The CLI needs to be able to discover the image references for a given `image version uid`.
By default, the CLI will prefill the `image` field of the `constellation-conf.yaml` when `constellation config generate <CSP>` is run with a hardcoded `image version uid` (e.g. `v2.2.0`).
The CLI needs to be able to discover the image references for a given image name.
By default, the CLI will prefill the `image` field of the `constellation-conf.yaml` when `constellation config generate <CSP>` is run with a hardcoded name (e.g. `v2.2.0`).
The `image` field is independent of the CSP and is a used to discover the CSP-specific image reference as needed for the following operations:
- `constellation create`
- `constellation upgrade apply`
The CLI can find a CSP- and region specific image reference by looking up the `image version uid` in the following order:
The CLI can find a CSP- and region specific image reference by looking up the image name in the following order:
- if a local file `<IMAGE-VERSION-UID>.json` exists, use the lookup table in that file
- otherwise, load the image lookup table from a well known URL (e.g. `https://cdn.confidential.cloud/constellation/v1/images/<IMAGE-VERSION-UID>.json`) and use the lookup table in that file
- if a local file `<IMAGE NAME>.json` exists, use the lookup table in that file
- otherwise, load the image lookup table from a well known URL (e.g. `https://cdn.confidential.cloud/constellation/v1/ref/<REF>/stream/<STREAM>/image/<VERSION>/info.json`) and use the lookup table in that file
- choose the CSP-specific image reference for the current region and security type:
- On AWS, use the AMI ID for the current region (e.g. `.aws.us-east-1`)
- On Azure, use the image ID for the security type (CVM or Trusted Launch) (e.g. `.azure.cvm`)
@ -160,6 +148,5 @@ This is a list of possible future extensions that are not part of this RFC.
Their implementation is not guaranteed.
They are listed here to ensure that the design of this RFC is flexible enough to support them.
- A lookup table for available image versions might be added in the future.
- The lookup table can be signed using a signing key that is only used for that purpose.
- User managed repositories can be added in the future. This would allow users to reupload Constellation OS images to their cloud subscription and host their own lookup tables that resolve the same image versions to image references pointing to self managed images. An optional `repository` field could be added to the configuration file to allow users to specify the repository to use for image discovery.

2
rfc/secure-software-distribution.md
View File

@ -104,7 +104,7 @@ measurements and keep them in a file (`measurements.yaml`) compatible with our c
Comments should be omitted in final file. They show why certain values might be missing.
Those measurements are signed and uploaded to AWS S3. Stored at a path matching the configured image (see [image discoverability](image-discoverability.md)).
Those measurements are signed and uploaded to AWS S3. Stored at a path matching the configured image (see [image API](image-api.md)).
```yaml
csp: azure

4
rfc/updates.md
View File

@ -111,7 +111,7 @@ First, the node operator consumes the JoiningNode CRD. It watches on changes in
Second, we need to extend the node operator to also handle Kubernetes updates. The operator already receives information about the Kubernetes version of each node.
The CLI hands users the same mechanism to deliver the Kubernetes version to the operator as we [currently use for the image reference](https://github.com/edgelesssys/constellation/blob/main/operators/constellation-node-operator/api/v1alpha1/nodeimage_types.go#L14):
The CLI hands users the same mechanism to deliver the Kubernetes version to the operator as we currently use for the image reference:
```patch
// NodeImageSpec defines the desired state of NodeImage.
@ -252,7 +252,7 @@ Since the service versions bundled inside a `microserviceVersion` are hidden, th
```bash
$ constellation upgrade apply
Upgrading Kubernetes: 1.24.2 --> 1.24.3 ...
Upgrading VM image: /communityGalleries/ConstellationCVM-b3782fa0-0df7-4f2f-963e-fc7fc42663df/images/constellation/versions/2.3.0 --> /communityGalleries/ConstellationCVM-b3782fa0-0df7-4f2f-963e-fc7fc42663df/images/constellation/versions/2.3.0 (not updated)
Upgrading VM image: 2.3.0 --> 2.3.0 (not updated)
Upgrading Kubernetes services version to 1.24.5:
Autoscaler: 1.24.3 --> 1.24.3 (not updated)

38
rfc/version-api.md
View File

@ -14,35 +14,41 @@ Design goals:
The following HTTP endpoints are available:
- `GET /constellation/v1/versions/stream/<stream>/latest/` contains files showing the latest version available
- [`GET /constellation/v1/ref/<ref>/stream/<stream>/versions/latest/` contains files showing the latest versions available](#latest)
- `image.json` contains the latest image version
- `microservice.json` contains the latest microservice version
- `cli.json` contains the latest cli version
- `kubernetes.json` contains the latest supported version of Kubernetes
- `GET /constellation/v1/versions/stream/<stream>/major/<major-version>/` contains files with version information for this major version
- [`GET /constellation/v1/ref/<ref>/stream/<stream>/versions/major/<major-version>/` contains files with version information for this major version](#major-to-minor-version-list)
- `image.json` contains a list of all minor image versions that belong to a major version
- `microservice.json` contains a list of all minor microservice versions that belong to a major version
- `cli.json` contains a list of all minor cli versions that belong to a major version
- `kubernetes.json` contains a list of all supported minor version of Kubernetes that belong to a major version
- `GET /constellation/v1/versions/stream/<stream>/minor/<minor-version>/` contains files with version information for this minor version
- [`GET /constellation/v1/ref/<ref>/stream/<stream>/versions/minor/<major-minor-version>/` contains files with version information for this minor version](#minor-to-patch-version-list)
- `image.json` contains a list of all patch image versions that belong to a minor version
- `microservice.json` contains a list of all patch microservice versions that belong to a minor version
- `cli.json` contains a list of all patch cli versions that belong to a minor version
- `kubernetes.json` contains a list of all supported patch version of Kubernetes that belong to a minor version
- [`GET /constellation/v1/trash` contains resources of deleted versions that couldn't be cleaned up](#thrash)
`stream` is used to distinguish between different release streams. For example, `stable` and `beta` could be two different streams.
Currently, only `stable` is supported. The parameter exists to allow for future expansion.
`ref` is used to distinguish between different Constellation branches (e.g. `main` and `feat-xyz`) and releases (`-`).
`stream` is used to distinguish between different release streams. For example, `stable` and `debug` could be two different streams.
## Examples
The following examples show how the version API could be used to retrieve version information.
### Latest
This shows the version information for the latest OS image. The file could be extended to include more metadata.
```
https://cdn.confidential.cloud/constellation/v1/versions/stream/stable/latest/image.json
https://cdn.confidential.cloud/constellation/v1/ref/-/stream/stable/versions/latest/image.json
```
```json
{
"ref": "-",
"stream": "stable",
"kind": "image",
"version": "v2.3.0"
@ -52,25 +58,29 @@ https://cdn.confidential.cloud/constellation/v1/versions/stream/stable/latest/im
This shows the version information for the latest Kubernetes release. The file could be extended to include more metadata.
```
https://cdn.confidential.cloud/constellation/v1/versions/stream/stable/latest/kubernetes.json
https://cdn.confidential.cloud/constellation/v1/ref/-/stream/stable/versions/latest/kubernetes.json
```
```json
{
"ref": "-",
"stream": "stable",
"kind": "kubernetes",
"version": "v1.25.4"
}
```
### Major to minor version list
This shows a list of all minor releases of the microservices for the major version `v2`. The file could be extended to include more metadata.
```
https://cdn.confidential.cloud/constellation/v1/versions/stream/stable/major/v2/microservice.json
https://cdn.confidential.cloud/constellation/v1/ref/-/stream/stable/versions/major/v2/microservice.json
```
```json
{
"ref": "-",
"stream": "stable",
"granularity": "major",
"base": "v2",
@ -81,12 +91,15 @@ https://cdn.confidential.cloud/constellation/v1/versions/stream/stable/major/v2/
This shows a list of all patch releases of the CLI for the minor version `v2.3`. The file could be extended to include more metadata.
### Minor to patch version list
```
https://cdn.confidential.cloud/constellation/v1/versions/stream/stable/minor/v2.3/cli.json
https://cdn.confidential.cloud/constellation/v1/ref/-/stream/stable/versions/minor/v2.3/cli.json
```
```json
{
"ref": "-",
"stream": "stable",
"granularity": "minor",
"base": "v2.3",
@ -108,10 +121,15 @@ The CLI can query the `latest` endpoint to inform users about new releases of th
The CLI can query the `minor` endpoint to retrieve a list of all patch releases of a Constellation OS image. This can be used to only select compatible versions for image updates if combined with a downgrade protection (only allow updating to image versions that are newer than the one currently in use).
### Trash
The trash endpoint collects files that were deleted as versions but contain references to resources that couldn't be cleaned up.
It allows admins to manually clean up resources.
## Possible future extensions
- Explicit compatbility information between different resource kinds
- Explicit compatibility information between different resource kinds
- Example: OS image `v2.3.4` requires microservice version > `v2.3.1`
- Allows fine grained control in case of unexpected incompatibility
- Version metadata