Git-Mirrors/qubes-doc
1
0
Fork 0
mirror of https://github.com/QubesOS/qubes-doc.git synced 2025-08-08 06:32:31 -04:00
Code Issues Projects Releases Packages Wiki Activity

Revise in accordance with documentation style guide

Browse source
This commit is contained in:
Andrew David Wong 2021-08-04 22:22:12 -07:00
parent f4177b4554
commit 59aab04f08
No known key found for this signature in database
GPG key ID: 8CE137352A019A17
1 changed files with 137 additions and 102 deletions
Download patch file Download diff file Expand all files Collapse all files

239
developer/system/template-manager.md
View file

@ -1,133 +1,152 @@
--- ---
layout: doc layout: doc
title: Template Manager title: Template manager
permalink: /doc/template-manager/ permalink: /doc/template-manager/
--- ---
# Template Manager This document discusses the designs and technical details of `qvm-template`, a
template manager application. The goal of the project is to design a new
This document discusses the designs and technical details of `qvm-template`, a soon-to-be template manager application currently in development for Qubes OS. mechanism for template distribution and a unified tool for template management.
The goals of the project is to design a new mechanism for TemplateVM distribution and a unified tool for template management.
## Motivation ## Motivation
This project was originally proposed in the 2020 Google Summer of Code program. This project was originally proposed in the 2020 Google Summer of Code program.
Previously, TemplateVMs were distributed by RPM packages and managed by `yum`/`dnf`. Previously, templates were distributed by RPM packages and managed by
However, tracking inherently dynamic VM images with a package manager suited for static files creates some challenges. `yum`/`dnf`. However, tracking inherently dynamic VM images with a package
For example, users may accidentally update the images, overriding local changes ([#996], [#1647]). manager suited for static files creates some challenges. For example, users may
(Or in the case of [#2061], want to specifically override the changes.) accidentally update the images, overriding local changes
Other operations that work well on normal VMs are also somewhat inconsistent on RPM-managed templates. ([#996](https://github.com/QubesOS/qubes-issues/issues/996),
This includes actions such as renaming ([#839]), removal ([#5509]) and backup/restore ([#1385], [#1453], [discussion thread 1], [discussion thread 2]). [#1647](https://github.com/QubesOS/qubes-issues/issues/1647)). (Or in the case
In turn, this creates inconveniences and confusion for users ([#1403], [#4518]). of [#2061](https://github.com/QubesOS/qubes-issues/issues/2061), want to
specifically override the changes.) Other operations that work well on normal
VMs are also somewhat inconsistent on RPM-managed templates. This includes
actions such as renaming
([#839](https://github.com/QubesOS/qubes-issues/issues/839)), removal
([#5509](https://github.com/QubesOS/qubes-issues/issues/5509)) and
backup/restore ([#1385](https://github.com/QubesOS/qubes-issues/issues/1385),
[#1453](https://github.com/QubesOS/qubes-issues/issues/1453), [discussion
thread
1](https://groups.google.com/forum/#!topic/qubes-devel/rwc2_miCNNE/discussion),
[discussion thread
2](https://groups.google.com/forum/#!topic/qubes-users/uQEUpv4THsY/discussion)).
In turn, this creates inconveniences and confusion for users
([#1403](https://github.com/QubesOS/qubes-issues/issues/1403),
[#4518](https://github.com/QubesOS/qubes-issues/issues/4518)).
Also, the usage of RPM packages meant that installing a template results in arbitrary code execution, which is not ideal. Also, the usage of RPM packages meant that installing a template results in
arbitrary code execution, which is not ideal.
Besides distribution, users may also wish to have an integrated template management application ([#2062], [#2064], [#2534], [#3040]), as opposed to the situation where multiple programs are required for different purposes, e.g., `qubes-dom0-update`, `dnf`, `qvm-remove`, `qubes-manager`. Besides distribution, users may also wish to have an integrated template
management application
([#2062](https://github.com/QubesOS/qubes-issues/issues/2062),
[#2064](https://github.com/QubesOS/qubes-issues/issues/2064),
[#2534](https://github.com/QubesOS/qubes-issues/issues/2534),
[#3040](https://github.com/QubesOS/qubes-issues/issues/3040)), as opposed to
the situation where multiple programs are required for different purposes,
e.g., `qubes-dom0-update`, `dnf`, `qvm-remove`, `qubes-manager`.
To tackle these issues, `qvm-template` is created. To tackle these issues, `qvm-template` was created. It strives to provide not
It strives to provide not only a better mechanism for handling template installation but also a consistent user-facing interface to deal with template management. only a better mechanism for handling template installation but also a
consistent user-facing interface to deal with template management.
[#996]: https://github.com/QubesOS/qubes-issues/issues/996
[#1647]: https://github.com/QubesOS/qubes-issues/issues/1647
[#2061]: https://github.com/QubesOS/qubes-issues/issues/2061
[#839]: https://github.com/QubesOS/qubes-issues/issues/839
[#5509]: https://github.com/QubesOS/qubes-issues/issues/5509
[#1385]: https://github.com/QubesOS/qubes-issues/issues/1385
[#1453]: https://github.com/QubesOS/qubes-issues/issues/1453
[#1403]: https://github.com/QubesOS/qubes-issues/issues/1403
[#4518]: https://github.com/QubesOS/qubes-issues/issues/4518
[#2062]: https://github.com/QubesOS/qubes-issues/issues/2062
[#2064]: https://github.com/QubesOS/qubes-issues/issues/2064
[#2534]: https://github.com/QubesOS/qubes-issues/issues/2534
[#3040]: https://github.com/QubesOS/qubes-issues/issues/3040
[discussion thread 1]: https://groups.google.com/forum/#!topic/qubes-devel/rwc2_miCNNE/discussion
[discussion thread 2]: https://groups.google.com/forum/#!topic/qubes-users/uQEUpv4THsY/discussion
## Features ## Features
- Install/reinstall/downgrade/upgrade templates, either from local packages or remote repositories - Install/reinstall/downgrade/upgrade templates, either from local packages or
remote repositories
- Ability to install templates in alternative pools - Ability to install templates in alternative pools
- Possibility for the template package to specify options such as kernel or `virt_mode` used by the resulting template VM - Possibility for the template package to specify options such as the kernel
or `virt_mode` used by the resulting template
- List and show information about local and available templates - List and show information about local and available templates
- Machine-readable output for easy extension - Machine-readable output for easy extension
- Search for templates - Search for templates
- Remove templates - Remove templates
- Optionally, VMs based on the template to be removed can be either removed or "disassociated" -- namely, have their templates changed to a "dummy" one. - Optionally, VMs based on the template to be removed can be either removed
or "disassociated" -- namely, have their templates changed to a "dummy"
one.
- Show available repositories - Show available repositories
- Works in both dom0 and management VMs by utilizing the Admin API - Works in both dom0 and management VMs by utilizing the Admin API
- Works well with existing tools - Works well with existing tools
- Command-line interface with DNF-like usage - Command-line interface with DNF-like usage
- A graphical interface also available - A graphical interface also available
## Package Format ## Package format
The RPM package format is still used. The RPM package format is still used. However, the contents are manually
However, the contents are manually extracted instead of installing the whole package. extracted instead of installing the whole package. This allows us to take
This allows us to take advantage of existing tools for things like repository management. advantage of existing tools for things like repository management. We can also
We can also avoid the burden of dealing with verification, reducing the risk of issues like [QSB-28]. avoid the burden of dealing with verification, reducing the risk of issues like
[QSB-028](/news/2016/12/19/qsb-28/).
The package name should be in the form `qubes-template-${template_name}`. The package name should be in the form `qubes-template-<TEMPLATE_NAME>`.
The package metadata (summary, description, etc.) should not contain the `|` character to avoid possibly cryptic errors. The package metadata (summary, description, etc.) should not contain the `|`
This is because of its use as an internal separator. character to avoid possibly cryptic errors. This is because of its use as an
Note that as we already consider the repository metadata untrusted, this should not result in security issues. internal separator. Note that as we already consider the repository metadata
untrusted. This should not result in security issues.
The file structure should be quite similar to previous template RPMs. The file structure should be quite similar to previous template RPMs. Namely,
Namely, there should be the following files in the package: there should be the following files in the package:
- `var/lib/qubes/vm-templates/${template_name}/root.img.part.{00,01,...}` - `var/lib/qubes/vm-templates/<TEMPLATE_NAME>/root.img.part.[00,01,...]`
- Split tarball of template `root.img` - Split tarball of template `root.img`
- Note that the file is still split due to tools such as `rpm2cpio` not supporting large files. - Note that the file is still split due to tools such as `rpm2cpio` not
(Notably, the cpio format does not support files over 4GiB.) supporting large files. (Notably, the cpio format does not support files
- `var/lib/qubes/vm-templates/${template_name}/template.conf` over 4GiB.)
- `var/lib/qubes/vm-templates/<TEMPLATE_NAME>/template.conf`
- Stores custom package metadata (as RPM does not support custom attributes). - Stores custom package metadata (as RPM does not support custom attributes).
- Uses `KEY=VALUE` format. - Uses `KEY=VALUE` format.
- Fields (corresponding to [qvm-prefs]/[qvm-features] tags with the same names) - Fields (corresponding to
[qvm-prefs](https://dev.qubes-os.org/projects/core-admin-client/en/stable/manpages/qvm-prefs.html#common-properties)
and
[qvm-features](https://dev.qubes-os.org/projects/core-admin-client/en/stable/manpages/qvm-features.html#list-of-known-features)
tags with the same names)
- `virt_mode` - `virt_mode`
- Setting this to `pv` requires user confirmation. - Setting this to `pv` requires user confirmation.
- Permitted values: `pv`, `pvh`, `hvm`. - Permitted values: `pv`, `pvh`, `hvm`.
- `kernel` - `kernel`
- Only allowed to be set to "" (without quotes), i.e., "none", for PVGrub. - Only allowed to be set to "" (without quotes), i.e., "none", for
- Network-related flags: (Must be set to IPv4 addresses in the form of `x.x.x.x`.) PVGrub.
- Network-related flags: (Must be set to IPv4 addresses in the form of
`x.x.x.x`.)
- `net.fake-ip` - `net.fake-ip`
- `net.fake-gateway` - `net.fake-gateway`
- `net.fake-netmask` - `net.fake-netmask`
- Boolean flags: (Permitted values are "1" and "0", denoting "true" and "false" respectively.) - Boolean flags: (Permitted values are "1" and "0", denoting "true" and
"false" respectively.)
- `no-monitor-layout` - `no-monitor-layout`
- `pci-e820-host` - `pci-e820-host`
- `linux-stubdom` - `linux-stubdom`
- `gui` - `gui`
- `gui-emulated` - `gui-emulated`
- `qrexec` - `qrexec`
- `var/lib/qubes/vm-templates/${template_name}/whitelisted-appmenus.list` - `var/lib/qubes/vm-templates/<TEMPLATE_NAME>/whitelisted-appmenus.list`
- Contains default app menu entries of the template itself. - Contains default app menu entries of the template itself.
- `var/lib/qubes/vm-templates/${template_name}/vm-whitelisted-appmenus.list` - `var/lib/qubes/vm-templates/<TEMPLATE_NAME>/vm-whitelisted-appmenus.list`
- Contains default app menu entries of VMs based on the template. - Contains default app menu entries of VMs based on the template.
- `var/lib/qubes/vm-templates/${template_name}/netvm-whitelisted-appmenus.list` - `var/lib/qubes/vm-templates/<TEMPLATE_NAME>/netvm-whitelisted-appmenus.list`
- Contains default app menu entries of NetVMs based on the template. - Contains default app menu entries of NetVMs based on the template.
- These three files are the same as the current format. - These three files are the same as the current format.
- Note that the contents of these files are stored in `qvm-features` upon installation. - Note that the contents of these files are stored in `qvm-features` upon
See the section below for details. installation. See the section below for details.
[QSB-28]: https://github.com/QubesOS/qubes-secpack/blob/master/QSBs/qsb-028-2016.txt ## Metadata storage
[qvm-prefs]: https://dev.qubes-os.org/projects/core-admin-client/en/stable/manpages/qvm-prefs.html#common-properties
[qvm-features]: https://dev.qubes-os.org/projects/core-admin-client/en/stable/manpages/qvm-features.html#list-of-known-features
## Metadata Storage The template manager needs to keep metadata of installed templates such as
versions and origin. This data can be stored via `qvm-features` to keep things
consistent when, e.g., `qvm-remove` is used. Besides, backups are also more
easily handled this way.
The template manager needs to keep metadata of installed templates such as versions and origin. Also, the fields can serve as an indicator of whether a template is installed
This data can be stored via `qvm-features` to keep things consistent when, e.g., `qvm-remove` is used. by `qvm-template`.
Besides, backups are also more easily handled this way.
Also, the fields can serve as an indicator of whether a template is installed by `qvm-template`.
### Fields ### Fields
Most of the fields should be fairly self-explanatory. Most of the fields should be fairly self-explanatory.
- `template-name` - `template-name`
- Note that this field needs to be consistent with the template name to be considered valid. - Note that this field needs to be consistent with the template name to be
considered valid.
- `template-epoch` - `template-epoch`
- `template-version` - `template-version`
- `template-release` - `template-release`
@ -139,31 +158,38 @@ Most of the fields should be fairly self-explanatory.
- `template-url` - `template-url`
- `template-summary` - `template-summary`
- `template-description` - `template-description`
- Note that the newlines in this field are converted to `|` to work better with existing tools like `qvm-features`. - Note that the newlines in this field are converted to `|` to work better
with existing tools like `qvm-features`.
- `menu-items` - `menu-items`
- `default-menu-items` - `default-menu-items`
- `netvm-menu-items` - `netvm-menu-items`
- The `*menu-items` entries store the contents of `var/lib/qubes/vm-templates/${template_name}/whitelisted-appmenus.list`, `var/lib/qubes/vm-templates/${template_name}/vm-whitelisted-appmenus.list`, `var/lib/qubes/vm-templates/${template_name}/netvm-whitelisted-appmenus.list` respectively. - The `*menu-items` entries store the contents of
- Note that newlines are converted to spaces, again for it to work better with existing tools. `var/lib/qubes/vm-templates/<TEMPLATE_NAME>/whitelisted-appmenus.list`,
This should not cause ambiguity as [the FreeDesktop specifications][freedesktop-spec] forbid spaces in .desktop file names. `var/lib/qubes/vm-templates/<TEMPLATE_NAME>/vm-whitelisted-appmenus.list`,
`var/lib/qubes/vm-templates/<TEMPLATE_NAME>/netvm-whitelisted-appmenus.list`
respectively.
- Note that newlines are converted to spaces, again for it to work better
with existing tools. This should not cause ambiguity as [the FreeDesktop
specifications](https://specifications.freedesktop.org/desktop-entry-spec/desktop-entry-spec-latest.html)
forbid spaces in .desktop file names.
[freedesktop-spec]: https://specifications.freedesktop.org/desktop-entry-spec/desktop-entry-spec-latest.html ## Repository management
## Repository Management For UpdateVMs to access the repository configuration, the package
[qubes-repo-templates](https://github.com/WillyPillow/qubes-repo-templates) is
For UpdateVMs to access repository configuration, the package [qubes-repo-templates] is created with the following contents: created with the following contents:
- `/etc/qubes/repo-templates/*.repo`: repository configuration - `/etc/qubes/repo-templates/*.repo`: repository configuration
- `/etc/qubes/repo-templates/keys`: PGP keys - `/etc/qubes/repo-templates/keys`: PGP keys
As template keys may be less trusted, they are *not* added to the system RPM keychain but instead managed separately. As template keys may be less trusted, they are *not* added to the system RPM
keychain but instead managed separately.
[qubes-repo-templates]: https://github.com/WillyPillow/qubes-repo-templates ## Qrexec protocol
## Qrexec Protocol Dom0 and management VMs without network access also need to interact with
template repositories. The following qrexec calls that list and download
Dom0 and management VMs without network access also need to interact with template repositories. templates are thus proposed.
The following qrexec calls that list and download templates are thus proposed.
- `qubes.TemplateSearch`: wraps `dnf repoquery` - `qubes.TemplateSearch`: wraps `dnf repoquery`
- `qubes.TemplateDownload`: wraps `dnf download` - `qubes.TemplateDownload`: wraps `dnf download`
@ -183,8 +209,8 @@ repo config
``` ```
In other words, the input consists of two parts separated by the line `---`. In other words, the input consists of two parts separated by the line `---`.
The first part contains some arguments and `package-file-spec` that indicates the pattern to be queried or downloaded. The first part contains some arguments and `package-file-spec` that indicates
The following arguments are allowed: the pattern to be queried or downloaded. The following arguments are allowed:
- `--enablerepo=<repoid>` - `--enablerepo=<repoid>`
- `--disablerepo=<repoid>` - `--disablerepo=<repoid>`
@ -194,39 +220,48 @@ The following arguments are allowed:
where the usage is identical to that of DNF. where the usage is identical to that of DNF.
For the exact definition of `package-file-spec`, refer to the DNF documentation. For the exact definition of `package-file-spec`, refer to the DNF
documentation.
The second part contains the repository configurations in `yum.repos.d` format. The second part contains the repository configurations in `yum.repos.d` format.
### Output ### Output
`qubes.TemplateSearch` prints each package in `%{name}|%{epoch}|%{version}|%{release}|%{reponame}|%{downloadsize}|%{buildtime}|%{license}|%{url}|%{summary}|%{description}|` format to standard output, separated by newlines. `qubes.TemplateSearch` prints each package in
Note that there is a `|` at the end of the line. `%{name}|%{epoch}|%{version}|%{release}|%{reponame}|%{downloadsize}|%{buildtime}|%{license}|%{url}|%{summary}|%{description}|`
This is because `%{description}` may contain newlines, and doing so allows us to split the entries by `|\n`. format to standard output, separated by newlines. Note that there is a `|` at
(As we are using `dnf repoquery --qf`, we are unable to escape the newlines in advance.) the end of the line. This is because `%{description}` may contain newlines, and
doing so allows us to split the entries by `|\n`. (As we are using `dnf
repoquery --qf`, we are unable to escape the newlines in advance.)
`qubes.TemplateDownload`, on the other hand, directly outputs the downloaded content to standard output. `qubes.TemplateDownload`, on the other hand, directly outputs the downloaded
content to standard output.
## Machine-readable Output ## Machine-readable output
The commands `qvm-template list` and `qvm-template info` provide machine-readable output in both pipe(`|`)-separated and JSON format. The commands `qvm-template list` and `qvm-template info` provide
See the `qvm-template` man page for details. machine-readable output in both pipe(`|`)-separated and JSON format. See the
`qvm-template` man page for details.
## Interactions with Existing Tools ## Interactions with existing tools
### `qvm-remove` ### `qvm-remove`
The existing `qvm-remove` tool should behave identically to `qvm-template remove` -- albeit without fancy features like disassociation. The existing `qvm-remove` tool should behave identically to `qvm-template
This is unlike the previous situation where `qvm-remove` cannot remove RPM-installed templates. remove` -- albeit without fancy features like disassociation. This is unlike
the previous situation where `qvm-remove` cannot remove RPM-installed
templates.
Notably, the metadata needs no special handling as it is stored in VM features and thus automatically consistent. Notably, the metadata needs no special handling as it is stored in VM features
and thus automatically consistent.
### Renaming / Cloning ### Renaming and cloning
A template is treated as non-manager-installed once renamed or cloned. A template is treated as non-manager-installed once renamed or cloned. However,
However, relevant metadata in the VM features is still retained for future extension and to serve as a hint for the user. relevant metadata in the VM features is still retained for future extension and
to serve as a hint for the user.
## Further Reading ## Further reading
Initial Google Summer of Code (2020) project proposal: Initial Google Summer of Code (2020) project proposal: