Fedora CoreOS Assembler

We use the Fedora CoreOS Assembler to build the base image for Constellation nodes.

Setup

Prerequisites: podman and qemu-kvm are installed, nested virtualization is enabled. Make sure your user is allowed read and write access on /dev/kvm. If the device is not mounted in the container try the following command, and restart the container:

sudo chmod 666 /dev/kvm

Pull the assembler container image

podman pull quay.io/coreos-assembler/coreos-assembler

Create a working directory on your host system
```
mkdir fcos && cd fcos
```

Set up a bash alias

Add the following to your .bashrc to easily start the image assembler using cosa:

cosa() {
    env | grep COREOS_ASSEMBLER
    local -r COREOS_ASSEMBLER_CONTAINER_LATEST="quay.io/coreos-assembler/coreos-assembler:latest"
    if [[ -z ${COREOS_ASSEMBLER_CONTAINER} ]] && $(podman image exists ${COREOS_ASSEMBLER_CONTAINER_LATEST}); then
        local -r cosa_build_date_str="$(podman inspect -f "{{.Created}}" ${COREOS_ASSEMBLER_CONTAINER_LATEST} | awk '{print $1}')"
        local -r cosa_build_date="$(date -d ${cosa_build_date_str} +%s)"
        if [[ $(date +%s) -ge $((cosa_build_date + 60*60*24*7)) ]] ; then
            echo -e "\e[0;33m----" >&2
            echo "The COSA container image is more that a week old and likely outdated." >&2
            echo "You should pull the latest version with:" >&2
            echo "podman pull ${COREOS_ASSEMBLER_CONTAINER_LATEST}" >&2
            echo -e "----\e[0m" >&2
            sleep 10
        fi
    fi
    set -x
    podman run --rm -ti --security-opt label=disable --privileged                                    \
               --uidmap=1000:0:1 --uidmap=0:1:1000 --uidmap 1001:1001:64536                          \
               -v ${PWD}:/srv/ --device /dev/kvm --device /dev/fuse                                  \
               --tmpfs /tmp -v /var/tmp:/var/tmp --name cosa                                         \
               ${COREOS_ASSEMBLER_CONFIG_GIT:+-v $COREOS_ASSEMBLER_CONFIG_GIT:/srv/src/config/:ro}   \
               ${COREOS_ASSEMBLER_GIT:+-v $COREOS_ASSEMBLER_GIT/src/:/usr/lib/coreos-assembler/:ro}  \
               ${COREOS_ASSEMBLER_CONTAINER_RUNTIME_ARGS}                                            \
               ${COREOS_ASSEMBLER_CONTAINER:-$COREOS_ASSEMBLER_CONTAINER_LATEST} "$@"
    rc=$?; set +x; return $rc
}

Run the builder
```
cosa shell
```

Initialize the build

cosa init https://github.com/coreos/fedora-coreos-config

Fetch metadata and packages
```
cosa fetch
```
Build a qemu VM image
```
cosa build
```
Each build will create a new directory in $PWD/builds/, containing the generated OSTree commit and the qemu VM image.
Run the image
```
cosa run
```

Customization

The CoreOS Assembler offers three main customization options:

manifest.yaml

An rpm-ostree "manifest" or "treefile", primarily, a list of RPMs and their associated repositories. See the rpm-ostree documentation for the treefile format reference
overlay.d/

A generic way to embed architecture-independent configuration and scripts by creating subdirectories in overlay.d/. Each subdirectory is added to the OSTree commit in lexicographic order.
image.yaml

Configuration for the output disk images

Additionally, one may use overrides to embed local RPMs from the build environment, that should not be pulled from a remote repository:

Package the binary as an RPM
Add any dependencies of the RPM to manifest.yaml
Run cosa fetch to prepare dependencies
Place the RPM in overrides/rpm
Add the name of your RPM to manifest.yaml
Run cosa build. Your RPM will be added to the final image.

Example: We want to build FCOS with our own kernel

Follow Kernel Building to build the kernel

You should end up with at least three RPMs: kernel, kernel-core, kernel-modules. kernel depends on core and modules, modules on core, and core on common FCOS packages (bash, systemd, etc.). These dependencies should already be in the manifest.
Run cosa fetch
Place the kernel RPMs in overrides/rpm

kernel, kernel-core, kernel-modules should already be in the manifest (src/config/manifests/bootable-rpm-ostree.yaml)
Run cosa build to create the image
Test the image with cosa run
Run cosa buildextend-gcp and cosa buildextend-azure to additionaly create a VM image for GCP and Azure

RPM packaging

If we want to make the most use of CoreOS assembler we should package our applications as RPM packages. See creating rpm packages.

Brief overview of the required steps:

Create a directory with your source code or binary file
Add a .spec file

Run the following command to create a spec file template that you can update with information about your package
```
rpmdev-newspec <package>
```
Create the RPM
```
fedpkg --release f35 local
```

Kernel Building

See the building a custom kernel from the Fedora Project documentation.

The following assumes you are running on a current release of Fedora. We have a Fedora 35 image available on GCP, make sure you have enough space available and the VM is capable to build the kernel in a reasonable time (e2-standard-8 takes ~2h to finish the build).

Install dependencies and clone the kernel

sudo dnf install fedpkg fedora-packager rpmdevtools ncurses-devel pesign grubby qt3-devel libXi-devel gcc-c++
fedpkg clone -a kernel && cd kernel
sudo dnf builddep kernel.spec

Optionally install ccache to speed up rebuilds

sudo dnf install ccache

Check out the kernel branch you want to base your build on

Each release has its own branch. E.g. to customize the kernel for Fedora 35, check out origin/f35. rawhide tracks the latest iteration, following closely behind the mainline kernel.
```
git checkout origin/f35
git checkout -b custom-kernel
```
Customize buildid by chaning # define buildid .local to %define buildid .<your_custom_id_here> in kernel.spec
Apply your changes and patches to the kernel
Build the RPMs

This will take a while
```
fedpkg local
```
The built kernel RPMs will be in ./x86_64/

You can now use and install the kernel packages

sudo dnf install --nogpgcheck ./x86_64/kernel-$version.rpm

7.2 KiB Raw Blame History

Fedora CoreOS Assembler

Setup

Customization

RPM packaging

Kernel Building

7.2 KiB

Raw Blame History