constellation/docs/secure-software-distribution.md

# Secure Distribution of CLI & Measurements

## Goal / Motivation

We need a mechanism to provide trusted measurements, for a specific version,
to users of `constellation` CLI, when verifying a Constellation cluster.

Since `constellation` CLI is responsible for carrying out the verification,
we also enable users to verify the CLI they are using.

## Limitations

+ We only support a single measurement per image.
  Updating measurements (e.g. when cloud provider replaced firmware) will be solved later.
+ First implementation only supports GitHub as "KMS" for our cosign key.
  Later on, we can support a proper KMS with key rotation, derivation, revocation & TEE.
+ In future, we might support multi-party-signatures for measurements.
  This would allow customers to explicitly trust a measurement.
+ This implementation will not protect against downgrade attacks. For a full
  implementation we should check how we can integrate with [TUF](https://theupdateframework.io/)
  & [in-toto](https://in-toto.io/).

## Alternative Solutions

+ We could enable users to reproduce measurements.
  This way users do not need to trust our signed measurements,
  but are able to generate this data themselves.
+ We could also enable users to build CLI themselves from source (open source).

We might support some or all alternative solutions in future.

## High Level Solution

We will create a single `cosign` key `Signer` in GitHub:

1. `Signer` signs `constellation` CLI
    + `Signer`'s public key is part of each CLI release
    + Customer verify CLI **before** first use
2. `constellation` CLI contains public key of `Signer`
    + By verifying integrity of CLI, user can trust this public key and do not need to supply it
    + `constellation config fetch-measurements` downloads measurements for current image
    version from a known (or configurable) place
      + CLI uses `Signer`'s public key to verify downloaded measurements
      + CLI stores verified measurements in local `constellation-config.yaml`
3. `constellation verify` reads measurements from config & compares to cluster's measurements
    + No changes necessary here.

## Detailed Solution

### Key Generation

#### Generate locally

```sh
COSIGN_PASSWORD=$(openssl rand --hex 32)
cosign generate-key-pair
```

Generates a password protected keypair locally.
This keypair can be configured manually to GitHub.

This keypair could also be backed-up onto USB stick / SD card.

### Sign & Verify CLI

```sh
# Set these beforehand!
# COSIGN_PASSWORD=
# COSIGN_PRIVATE_KEY=
# COSIGN_PUBLIC_KEY=
go build constellation
COSIGN_EXPERIMENTAL=1 cosign sign-blob --key env://COSIGN_PRIVATE_KEY constellation > constellation.sig
# We provide: cosign.pub, constellation.sig, constellation
echo "$COSIGN_PUBLIC_KEY" > cosign.pub
cosign verify-blob --key cosign.pub --signature constellation.sig constellation
# We provide: cosign.pub, constellation (requires access to rekor.dev, but no signature to be distributed)
uuid=$(rekor-cli search --artifact constellation | tail -n 1)
sig=$(rekor-cli get --uuid=$uuid --format=json | jq -r .Body.HashedRekordObj.signature.content)
cosign verify-blob --key cosign.pub --signature <(echo $sig) constellation
```

```mermaid
flowchart LR
    cli[Constellation CLI]
    clisig[Constellation CLI signature]
    signer[Cosign Keypair]

    signer -- signs --> cli
    signer -- produces --> clisig
    clisig -- uploaded to --> rekor
    clisig -- uploaded to --> gh
    cli -- uploaded to --> gh

    subgraph artifacts [Artifact Storage]
        rekor[Rekor]
        gh[GitHub Releases]
    end
```

### Sign Measurements

We regularly spin up a Constellation cluster in our pipeline, store the observed
measurements and keep them in a file (`measurements.yaml`) compatible with our config file.

Comments should be omitted in final file. They show why certain values might be missing.

Those measurements are signed and uploaded to AWS S3. Stored at a path matching the configured image.

```yaml
azure:
    image: /subscriptions/0d202bbb-4fa7-4af8-8125-58c269a05435/resourceGroups/constellation-images/providers/Microsoft.Compute/galleries/Constellation/images/constellation-coreos/versions/0.0.1655304334
    measurements:
        # 0: q27iAZeXGAiCPdu1bqRA2gAoyMO2KrXWY4YkTCQowc4=   # Unstable: UEFI on Azure
        1: 0GqVBBcu78dlLW03pON6OJbjQTMsKZmN+SV88kWHPss=
        2: PUWM/lXMA+ofRD8VYr7sjfUcdeFKn8+acjShPxmOeWk=
        3: PUWM/lXMA+ofRD8VYr7sjfUcdeFKn8+acjShPxmOeWk=
        4: Mnl6y16fHWpwSGWZsSFiLc4NYXRhQ39UqkClHcDbJ2s=
        5: qJ2QqWHIFfV9UILu76d3fGXdZz/RpZ/TcFyw7kPHzj4=
        # 6: glvMCHop3keeyU2xBHJTpYmEuqKqXJqCRQuQi8C3n4w=   # Unstable: VM Name Encoded on Azure
        7: DnCqTk4YKN60heuvyzPoPH9uJ3yn3SgjaK1w59xmvvg=
        8: hcLg6uP27lj28A+TExXlsv34EOmOh1jzdCofrBZS5gU=
        9: hwxFDmhNROlS7oBh4dG3jzB4OeQAGhcZD9f6bwBtK/k=
        # 10: PPiSI1eZHs+S/BgVWqewZAXDhkVgtIW8/PPgpR9Sr2o=  # Unstable: Linux IMA records on Azure
        # 11: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=  # Set by us
        # 12: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=  # Set by us
        # 13..23 are all unused
```

```sh
# Set these beforehand!
# COSIGN_PASSWORD=
# COSIGN_PRIVATE_KEY=
COSIGN_EXPERIMENTAL=1 cosign sign-blob --key cosign.key measurements.yaml > measurements.yaml.sig
```

```mermaid
flowchart LR
    measurementssig[measurements.yaml signature]
    measurements[measurements.yaml]
    signer[Cosign Keypair]
    cloud[Cloud Infrastructure<br />GCP/Azure]
    pcrreader[pcr-reader]
    cloud -- uses --> aig
    cloud -- uses --> gmi
    pcrreader -- writes --> measurements
    pcrreader -- reads --> cloud
    signer -- produces --> measurementssig
    signer -- signs --> measurements
    measurementssig -- uploaded to --> rekor
    measurementssig -- uploaded to --> s3
    measurements -- uploaded to --> s3

    subgraph artifacts [Artifact Storage]
        rekor[Rekor]
        boots[Bootstrapper]
        s3[S3]
        aig[Azure Image Gallery]
        gmi[Google Machine Images]
        gmi -- contains --> boots
        aig -- contains --> boots
    end
```

### Verify Measurements

`constellation config fetch-measurements`
1. Read `.provider.[azure|gcp].image` from `constellation-conf.yaml`
2. Fetch measurements for this image from S3
   + Alternatively: Use rekor to fetch signature for this artifact
3. Use embedded public key to verify signature.
4. Write measurements to local config

No changes required for `constellation verify`.

## Next Increments

### Key Management

#### Trust Public Key

Upload our public key to public places controlled by us, e.g.,
+ Twitter
+ Website

Users can check that public keys are the same in all those places, and therefore have not been tampered with.

#### Local Build Agent

Instead of using a GitHub managed runner when building and signing our release, we should use the one we have in the office. We also re-generate the key pair and store it locally on the build runner, as well as a back up on a USB stick.

The goal is to remove GitHub from the TCB.

#### Confidential Build Agent

We should investigate if an Enclave could be used as a confidential build agent for building and signing the CLI.

Private signing key can be stored encrypted anywhere and is only ever decrypted in TEE.

## Potential Problems

### How do we handle, if one of our private signing keys is leaked?

We will investigate how [TUF](https://theupdateframework.io/) can be used to improve key life cycle management. A [TUF / cosing integration](https://github.com/sigstore/cosign/issues/86) is still being worked on.

## Roads Not Taken

### Fulcio / OIDC Based Identities

Sigstore supports creating ephemeral keys and trusting in OIDC identities.

We could use our @edgeless.systems Office365 accounts, but in order to trust
these identities we would need to prove our Office365 subscription is secure and
trustworthy.

Additionally our TCB would increase and include Microsoft & Sigstore Root CA.

Therefore we decided to manage our own signing keys.