privsec.dev/content/posts/linux/Root ZFS Encryption, Mirroring, and Remote Unlocking with Ubuntu.md

title	tags	date	author
Root ZFS Encryption, Mirroring, and Remote Unlocking on Ubuntu	Operating Systems Linux Security	2023-07-26	Tommy

While Unbutu supports ZFS on root filesystems with an easy snapshot and rollback mechanism called ZSYS, ZSYS is soon going to be deprecated and the installer does not offer an easy way to setup mirroring. In this guide, I will walk you through how to set up Ubuntu with root on ZFS, mirroring with both the root and EFI partitions, and remote unlocking + boot into a snapshot with ZFSBootMenu.

Enter the Shell

Enter the shell on your Ubuntu Installer:

Partitioning the Disk

Esentially, we need a 512MB ESP partition for ZFSBootMenu and a / partition for the rest of the system. If you are using a single disk, just make those 2 partitions on your disk. If you are planning to do mirroring, set up both on of these partitions on each disk.

There are a variety of tools you can use, but an easy to use one would be cfdisk.

cfdisk /dev/nvme0n1
cfdisk /dev/nvme0n2

Mirroring the ESP partition

Skip this if you are not doing mirroring

While the EFI specs do not support mdadm, we can setup mdadm with metadata v1.0, which will be put at the end of the parition and allows it to boot.

mdadm --create /dev/md0 --level 1 --raid-disks --metadata 1.0 /dev/nvme0n1p1 /dev/nvme0n1p2
mkfs.fat -F 32 /dev/md0

Setup the ZFS partition

This part is mostly based on the official ZFSBootMenu guide with some changes to work around some not-so-great instructions there.

Creating the zpool

Getting the Disk ID.

First, we must get the disk IDs from /dev/disk/by-id. The official guide uses the dynamically assigned drive identifier (/dev/sda, /dev/nvme0n1, etc), which is not what we want to do with zpools, since it will cause problems later on.

Installing ZFS-Utils

sudo apt install zfsutils-linux -y

Create the encryption key

echo 'SomeKeyphrase' > /etc/zfs/zroot.key
chmod 000 /etc/zfs/zroot.key

For Non-Mirrored Setups

sudo zpool create -o ashift=12 -O compression=zstd -O acltype=posixacl -O xattr=sa -O atime=off -O encryption=on -O keylocation=file:///etc/zfs/zroot.key -O keyformat=passphrase -o autotrim=on -m none zroot /dev/disk/by-id/nvme-SAMSUNG_MZQL21T9HCJR-00A07_XXXXXXX-part2 

For Mirrored Setups

zpool create  -o ashift=12  -O compression=zstd  -O acltype=posixacl  -O xattr=sa  -O atime=off  -O encryption=on  -O keylocation=file:///etc/zfs/zroot.key  -O keyformat=passphrase  -o autotrim=on  -m none zroot mirror /dev/disk/by-id/nvme-SAMSUNG_MZQL21T9HCJR-00A07_XXXXXXX-part2 /dev/disk/by-id/nvme-SAMSUNG_MZQL21T9HCJR-00A07_YYYYYYY-part2

Notes

We use slightly different options than the official guide. Most notably, atime is disabled as it has detrimental effect on performance and unnecessarily increases write operations. compression is changed from lz4 to zstd as it has much better compression ratio than lz4 while still maintaining good performance. We did not specify the encryption type here as aes-256-gcm is already the default with openZFS >= 0.8.4.

Creating the filesystems

zfs create -o mountpoint=none zroot/ROOT
zfs create -o mountpoint=/ -o canmount=noauto zroot/ROOT/ubuntu
zfs create -o mountpoint=/home zroot/home
zfs create -o mountpoint=/var/log zroot/ROOT/ubuntu/log
zfs create -o mountpoint=/var/spool zroot/ROOT/ubuntu/spool
zfs create -o mountpoint=/var/cache zroot/ROOT/ubuntu/cache

zpool set bootfs=zroot/ROOT/ubuntu zroot

Here, we deviate from the official guide by splitting out /var/log, /var/spool, /var/cache out into their own datasets. These are directories which are parts of Ubuntu that we do not want to be rolled back along with the system in case we need to boot into a prior snapshot.

If you plan to dual boot with a different system and have shared directory dataset, then you need to make sure that dataset is not under zroot/ROOT. zroot/home is an example of this.