From 8e706d6de3d3dbcc39d6c355b5693518024301b6 Mon Sep 17 00:00:00 2001 From: Malte Poll Date: Mon, 18 Sep 2023 12:18:52 +0200 Subject: [PATCH] image: update README --- dev-docs/workflows/build-develop-deploy.md | 13 +- image/README.md | 188 +++------------------ 2 files changed, 32 insertions(+), 169 deletions(-) diff --git a/dev-docs/workflows/build-develop-deploy.md b/dev-docs/workflows/build-develop-deploy.md index 9a2e55ae6..913c42fd2 100644 --- a/dev-docs/workflows/build-develop-deploy.md +++ b/dev-docs/workflows/build-develop-deploy.md @@ -6,13 +6,24 @@ Prerequisites: * 20GB (minimum), better 40 GB disk space (required if you want to cross compile for all platforms) * [Latest version of Go](https://go.dev/doc/install). -* [Bazelisk installed as `bazel` in your path](https://github.com/bazelbuild/bazelisk/releases). +* Unless you use Nix / NixOS: [Bazelisk installed as `bazel` in your path](https://github.com/bazelbuild/bazelisk/releases). +* We recommend Nix installed via [determinate systems installer](https://github.com/DeterminateSystems/nix-installer) (or NixOS as host system). * [Docker](https://docs.docker.com/engine/install/). Can be installed with these commands on Ubuntu 22.04: `sudo apt update && sudo apt install docker.io`. As the build spawns docker containers your user account either needs to be in the `docker` group (Add with `sudo usermod -a -G docker $USER`) or you have to run builds with `sudo`. When using `sudo` remember that your root user might (depending on your distro and local config) not have the go binary in it's PATH. The current PATH can be forwarded to the root env with `sudo env PATH=$PATH `. ## Prequisites ### Linux +* Packages on NixOS or with Nix installed (use flake.nix in this repo): + + ```sh + # add "common --config=nix" to your .bazeloverwriterc if you want to get nix compatible toolchains + echo "common --config=nix" >> .bazeloverwriterc + # better would be: nix develop -i + # but this doesn't play nice with bashrc, colored output and non-hermetic tools + nix develop + ``` + * Packages on Ubuntu: ```sh diff --git a/image/README.md b/image/README.md index 4c031f543..63c11e24e 100644 --- a/image/README.md +++ b/image/README.md @@ -1,179 +1,31 @@ ## Setup -- Install mkosi (from git): - - ```sh - cd /tmp/ - git clone https://github.com/systemd/mkosi - cd mkosi - git checkout d8b32fbf3077b612db0024276e73cec3c2c87577 - tools/generate-zipapp.sh - cp builddir/mkosi /usr/local/bin/ - ``` - -- Build systemd tooling (from git): - - Ubuntu and Fedora ship outdated versions of systemd tools, so you need to build them from source: - - ```sh - # Ubuntu - echo "deb-src http://archive.ubuntu.com/ubuntu/ $(lsb_release -cs) main restricted universe multiverse" | sudo tee -a /etc/apt/sources.list - sudo apt-get update - sudo apt-get build-dep systemd - sudo apt-get install libfdisk-dev - # Fedora - sudo dnf builddep systemd - - git clone https://github.com/systemd/systemd --depth=1 - meson systemd/build systemd -Drepart=true -Defi=true -Dbootloader=true - BINARIES=( - bootctl - systemctl - systemd-analyze - systemd-dissect - systemd-nspawn - systemd-repart - ukify - ) - ninja -C systemd/build ${BINARIES[@]} - SYSTEMD_BIN=$(realpath systemd/build) - echo installed systemd tools to "${SYSTEMD_BIN}" - ``` - -- Install tools: - -
- Ubuntu / Debian - - ```sh - sudo apt-get update - sudo apt-get install --assume-yes --no-install-recommends \ - bubblewrap \ - coreutils \ - curl \ - dnf \ - e2fsprogs \ - efitools \ - jq \ - mtools \ - ovmf \ - python3-pefile \ - python3-pyelftools \ - python3-setuptools \ - qemu-system-x86 \ - qemu-utils \ - rpm \ - sbsigntool \ - squashfs-tools \ - systemd-container \ - util-linux \ - virt-manager - ``` - -
- -
- Fedora - - ```sh - sudo dnf install -y \ - bubblewrap \ - edk2-ovmf \ - systemd-container \ - qemu \ - e2fsprogs \ - squashfs-tools \ - efitools \ - sbsigntools \ - coreutils \ - curl \ - jq \ - util-linux \ - virt-manager - ``` - -
- -- Prepare secure boot PKI (see `secure-boot/genkeys.sh`) +Ensure you have Nix installed. This is recommended in general but a requirement for the following steps. +Consult the [developer docs](/dev-docs/workflows/build-develop-deploy.md) for more info. +At the very least, `nix` should be in your PATH and either `common --config=nix` +has to be set in the `.bazelrc` or you need to append `--config=nix` to each Bazel command. ## Build -When building your first image, prepare the secure boot PKI (see `secure-boot/genkeys.sh`) for self-signed, locally built images. - -After that, you can build the image with: +You can build any image using Bazel. +Start by querying the available images: ```sh -# export SYSTEMD_BIN= -# OPTIONAL: to create a debug image, export the following line -# export DEBUG=true -# OPTIONAL: to enable the serial console, export the following line -# export AUTOLOGIN=true -# OPTIONAL: symlink custom path to secure boot PKI to ./pki -# ln -s /path/to/pki/folder ./pki -sudo make EXTRA_SEARCH_PATHS="${SYSTEMD_BIN}" -j $(nproc) +bazel query //image/system/... ``` -Raw images will be placed in `mkosi.output./fedora~38/image.raw`. - -## Prepare Secure Boot - -The generated images are partially signed by Microsoft ([shim loader](https://github.com/rhboot/shim)), and partially signed by Edgeless Systems (systemd-boot and unified kernel images consisting of the linux kernel, initramfs and kernel commandline). - -For QEMU and Azure, you can pre-generate the NVRAM variables for secure boot. This is not necessary for GCP, as you can specify secure boot parameters via the GCP API on image creation. - -
-libvirt / QEMU / KVM +You can either build a group of images (all images for a cloud provider, a stream, ...) or a single image by selecting a target. ```sh -secure-boot/generate_nvram_vars.sh mkosi.output.qemu/fedora~38/image.raw +bazel build //image/system:openstack_qemu-vtpm_debug ``` -
- -
-Azure - -These steps only have to performed once for a fresh set of secure boot certificates. -VMGS blobs for testing and release images already exist. - -First, create a disk without embedded MOK EFI variables. +The location of the destination folder can be queried like this: ```sh -# set these variables -export AZURE_SECURITY_TYPE=ConfidentialVM # or TrustedLaunch -export AZURE_RESOURCE_GROUP_NAME= # e.g. "constellation-images" - -export AZURE_REGION=northeurope -export AZURE_REPLICATION_REGIONS= -export AZURE_DISK_NAME=constellation-$(date +%s) -export AZURE_SNAPSHOT_NAME=${AZURE_DISK_NAME} -export AZURE_RAW_IMAGE_PATH=${PWD}/mkosi.output.azure/fedora~38/image.raw -export AZURE_IMAGE_PATH=${PWD}/mkosi.output.azure/fedora~38/image.vhd -export AZURE_VMGS_FILENAME=${AZURE_SECURITY_TYPE}.vmgs -export AZURE_JSON_OUTPUT=${PWD}/mkosi.output.azure/fedora~38/image-upload.json -export BLOBS_DIR=${PWD}/blobs -upload/pack.sh azure "${AZURE_RAW_IMAGE_PATH}" "${AZURE_IMAGE_PATH}" -upload/upload_azure.sh --disk-name "${AZURE_DISK_NAME}-setup-secure-boot" "" -secure-boot/azure/launch.sh -n "${AZURE_DISK_NAME}-setup-secure-boot" -d --secure-boot true --disk-name "${AZURE_DISK_NAME}-setup-secure-boot" +bazel cquery --output=files //image/system:openstack_qemu-vtpm_debug ``` -Ignore the running launch script and connect to the serial console once available. -The console shows the message "Verification failed: (0x1A) Security Violation". You can import the MOK certificate via the UEFI shell: - -Press OK, then ENTER, then "Enroll key from disk". -Select the following key: `/EFI/loader/keys/auto/db.cer`. -Press Continue, then choose "Yes" to the question "Enroll the key(s)?". -Choose reboot. - -Extract the VMGS from the running VM (this includes the MOK EFI variables) and delete the VM: - -```sh -secure-boot/azure/extract_vmgs.sh --name "${AZURE_DISK_NAME}-setup-secure-boot" -secure-boot/azure/delete.sh --name "${AZURE_DISK_NAME}-setup-secure-boot" -``` - -
- ## Upload to CSP Warning! Never set `--version` to a value that is already used for a release image. @@ -191,7 +43,7 @@ Warning! Never set `--version` to a value that is already used for a release ima ```sh # Warning! Never set `--version` to a value that is already used for a release image. # Instead, use a `ref` that corresponds to your branch name. -bazel run //image/upload -- aws --verbose --raw-image mkosi.output.aws/fedora~38/image.raw --variant "" --version ref/foo/stream/nightly/v2.7.0-pre-asdf +bazel run //image/upload -- image aws --verbose --raw-image path/to/constellation.raw --attestation-variant "" --version ref/foo/stream/nightly/v2.7.0-pre-asdf ``` @@ -207,12 +59,12 @@ bazel run //image/upload -- aws --verbose --raw-image mkosi.output.aws/fedora~38 - `pki_prod` is used for release images ```sh -export GCP_RAW_IMAGE_PATH=${PWD}/mkosi.output.gcp/fedora~38/image.raw -export GCP_IMAGE_PATH=${PWD}/mkosi.output.gcp/fedora~38/image.tar.gz +export GCP_RAW_IMAGE_PATH=$(realpath path/to/constellation.raw) +export GCP_IMAGE_PATH=path/to/image.tar.gz upload/pack.sh gcp ${GCP_RAW_IMAGE_PATH} ${GCP_IMAGE_PATH} # Warning! Never set `--version` to a value that is already used for a release image. # Instead, use a `ref` that corresponds to your branch name. -bazel run //image/upload -- gcp --verbose --raw-image "${GCP_IMAGE_PATH}" --variant "sev-es" --version ref/foo/stream/nightly/v2.7.0-pre-asdf +bazel run //image/upload -- image gcp --verbose --raw-image "${GCP_IMAGE_PATH}" --attestation-variant "sev-es" --version ref/foo/stream/nightly/v2.7.0-pre-asdf ``` @@ -230,12 +82,12 @@ Note: - Optional (if Secure Boot should be enabled) [Prepare virtual machine guest state (VMGS) with customized NVRAM or use existing VMGS blob](#azure-secure-boot) ```sh -export AZURE_RAW_IMAGE_PATH=${PWD}/mkosi.output.azure/fedora~38/image.raw -export AZURE_IMAGE_PATH=${PWD}/mkosi.output.azure/fedora~38/image.vhd +export AZURE_RAW_IMAGE_PATH=path/to/constellation.raw +export AZURE_IMAGE_PATH=path/to/image.vhd upload/pack.sh azure "${AZURE_RAW_IMAGE_PATH}" "${AZURE_IMAGE_PATH}" # Warning! Never set `--version` to a value that is already used for a release image. # Instead, use a `ref` that corresponds to your branch name. -bazel run //image/upload -- azure --verbose --raw-image "${AZURE_IMAGE_PATH}" --variant "cvm" --version ref/foo/stream/nightly/v2.7.0-pre-asdf +bazel run //image/upload -- image azure --verbose --raw-image "${AZURE_IMAGE_PATH}" --attestation-variant "cvm" --version ref/foo/stream/nightly/v2.7.0-pre-asdf ``` @@ -254,7 +106,7 @@ Note: ```sh # Warning! Never set `--version` to a value that is already used for a release image. # Instead, use a `ref` that corresponds to your branch name. -bazel run //image/upload -- openstack --verbose --raw-image mkosi.output.openstack/fedora~38/image.raw --variant "sev" --version ref/foo/stream/nightly/v2.7.0-pre-asdf +bazel run //image/upload -- image openstack --verbose --raw-image path/to/constellation.raw --attestation-variant "sev" --version ref/foo/stream/nightly/v2.7.0-pre-asdf ``` @@ -268,7 +120,7 @@ bazel run //image/upload -- openstack --verbose --raw-image mkosi.output.opensta ```sh # Warning! Never set `--version` to a value that is already used for a release image. # Instead, use a `ref` that corresponds to your branch name. -bazel run //image/upload -- qemu --verbose --raw-image mkosi.output.qemu/fedora~38/image.raw --variant "default" --version ref/foo/stream/nightly/v2.7.0-pre-asdf +bazel run //image/upload -- image qemu --verbose --raw-image path/to/constellation.raw --attestation-variant "default" --version ref/foo/stream/nightly/v2.7.0-pre-asdf ```