Git-Mirrors/qubes-doc
1
0
Fork 0
mirror of https://github.com/QubesOS/qubes-doc.git synced 2025-01-11 15:29:41 -05:00
Code Issues Packages Projects Releases Wiki Activity

Convert to RST

Browse Source
This commit is contained in:
Marek Marczykowski-Górecki 2024-05-21 20:59:46 +02:00
parent b8f24e762e
commit b93b3c571e
No known key found for this signature in database
GPG Key ID: F32894BE9684938A
412 changed files with 43831 additions and 28740 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

25
.readthedocs.yaml Normal file
View File

@ -0,0 +1,25 @@
version: 2
build:
os: "ubuntu-22.04"
tools:
python: "3.10"
submodules:
include: all
recursive: true
sphinx:
builder: html
configuration: conf.py
fail_on_warning: false
python:
install:
- requirements: requirements.txt
formats:
- pdf
- epub

23
_dev/index.rst
View File

@ -1,9 +1,26 @@
==============================================
Welcome to Qubes OS developer's documentation!
==============================================
This is documentation for the source code. Please choose specific repostitory:
* `core-admin </projects/core-admin>`_
* `core-admin-client </projects/core-admin-client>`_
.. toctree::
:maxdepth: 1
Or see `the main Qubes OS documentation <https://www.qubes-os.org/doc/>`_.
core-admin <>
.. _core-admin: /projects/core-admin
core-admin-client <>
.. _core-admin-client: /projects/core-admin-client
Or see the main Qubes OS documentation <https://www.qubes-os.org/doc/>
.. _the main qubes os documentation: https://www.qubes-os.org/doc/
.

131
conf.py Normal file
View File

@ -0,0 +1,131 @@
"""
ReST directive for embedding Youtube and Vimeo videos.
There are two directives added: ``youtube`` and ``vimeo``. The only
argument is the video id of the video to include.
Both directives have three optional arguments: ``height``, ``width``
and ``align``. Default height is 281 and default width is 500.
Example::
.. youtube:: anwy2MPT5RE
:height: 315
:width: 560
:align: left
:copyright: (c) 2012 by Danilo Bargen.
:license: BSD 3-clause
"""
from __future__ import absolute_import
from docutils import nodes
from docutils.parsers.rst import Directive, directives
# -- Project information -----------------------------------------------------
project = 'Qubes OS'
copyright = '2024, Qubes OS Project'
author = 'Qubes OS Project'
title = "Qubes Docs"
html_title = "Qubes Docs"
# The full version, including alpha/beta/rc tags
release = '4.2'
# -- General configuration ---------------------------------------------------
html_static_path = ['attachment/doc']
extensions = [
'sphinx.ext.autosectionlabel',
'sphinxnotes.strike',
'sphinx_reredirects'
]
redirects = {
"user/hardware/hcl": "https://www.qubes-os.org/hcl/",
"user/downloading-installing-upgrading/downloads:mirrors":"https://www.qubes-os.org/downloads/mirrors/",
"developer/general/visual-style-guide": "https://www.qubes-os.org/doc/visual-style-guide/",
"developer/general/website-style-guide":"https://www.qubes-os.org/doc/website-style-guide/",
"user/downloading-installing-upgrading/downloads":"https://www.qubes-os.org/downloads/",
"developer/general/how-to-edit-the-documentation":"https://www.qubes-os.org/doc/how-to-edit-the-documentation/"
}
autosectionlabel_prefix_document = True
source_suffix = {
'.rst': 'restructuredtext',
}
root_doc = "index"
exclude_patterns = [
'_dev/*',
'attachment/*',
'**/*.txt'
]
html_theme = 'sphinx_rtd_theme'
# html_theme = 'default'
# html_theme = 'classic'
html_theme_options = {
'externalrefs': True,
'bgcolor': 'white',
'linkcolor': '#99bfff',
'textcolor': '#000000',
'visitedlinkcolor': '#7b7b7b',
'bodyfont': '"Open Sans", Arial, sans-serif',
'codebgcolor': 'grey',
'body_min_width': '50%',
'body_max_width': '90%',
'collapse_navigation': True,
}
gettext_uuid = True
gettext_compact = False
# epub_show_urls = 'footnote'
# latex_show_urls ='footnote'
locale_dirs = ['_translated']
# from https://gist.github.com/ehles/bed012d78aad5d3cd6c35a49bef32f9f
def align(argument):
"""Conversion function for the "align" option."""
return directives.choice(argument, ('left', 'center', 'right'))
class IframeVideo(Directive):
has_content = False
required_arguments = 1
optional_arguments = 0
final_argument_whitespace = False
option_spec = {
'height': directives.nonnegative_int,
'width': directives.nonnegative_int,
'align': align,
}
default_width = 500
default_height = 281
def run(self):
self.options['video_id'] = directives.uri(self.arguments[0])
if not self.options.get('width'):
self.options['width'] = self.default_width
if not self.options.get('height'):
self.options['height'] = self.default_height
if not self.options.get('align'):
self.options['align'] = 'left'
return [nodes.raw('', self.html % self.options, format='html')]
class GeneralVid(IframeVideo):
html = '<iframe src="%(video_id)s" width="%(width)u" height="%(height)u" frameborder="0" webkitAllowFullScreen mozallowfullscreen allowfullscreen class="responsive" referrerpolicy="no-referrer" scrolling="no"></iframe>'
class Youtube(IframeVideo):
html = '<iframe src="https://www.youtube-nocookie.com/embed/%(video_id)s" \
width="%(width)u" height="%(height)u" frameborder="0" \
webkitAllowFullScreen mozallowfullscreen allowfullscreen \
class="responsive" referrerpolicy="no-referrer" scrolling="no"></iframe>'
def setup(builder):
directives.register_directive('youtube', Youtube)
directives.register_directive('generalvid', GeneralVid)

538
developer/building/development-workflow.md
View File

@ -1,538 +0,0 @@
---
lang: en
layout: doc
permalink: /doc/development-workflow/
redirect_from:
- /en/doc/development-workflow/
- /doc/DevelopmentWorkflow/
- /wiki/DevelopmentWorkflow/
ref: 66
title: Development workflow
---
A workflow for developing Qubes OS+
First things first, setup [QubesBuilder](/doc/qubes-builder/). This guide
assumes you're using qubes-builder to build Qubes.
## Repositories and committing Code
Qubes is split into a bunch of git repos. These are all contained in the
`qubes-src` directory under qubes-builder. Subdirectories there are separate
components, stored in separate git repositories.
The best way to write and contribute code is to create a git repo somewhere
(e.g., github) for the repo you are interested in editing (e.g.,
`qubes-manager`, `core-agent-linux`, etc). To integrate your repo with the rest
of Qubes, cd to the repo directory and add your repository as a remote in git
**Example:**
~~~
$ cd qubes-builder/qubes-src/qubes-manager
$ git remote add abel git@github.com:abeluck/qubes-manager.git
~~~
You can then proceed to easily develop in your own branches, pull in new
commits from the dev branches, merge them, and eventually push to your own repo
on github.
When you are ready to submit your changes to Qubes to be merged, push your
changes, then create a signed git tag (using `git tag -s`). Finally, send a
letter to the Qubes listserv describing the changes and including the link to
your repository. You can also create pull request on github. Don't forget to
include your public PGP key you use to sign your tags.
### Kernel-specific notes
#### Prepare fresh version of kernel sources, with Qubes-specific patches applied
In qubes-builder/qubes-src/linux-kernel:
~~~
make prep
~~~
The resulting tree will be in kernel-\<VERSION\>/linux-\<VERSION\>:
~~~
ls -ltrd kernel*/linux*
~~~
~~~
drwxr-xr-x 23 user user 4096 Nov 5 09:50 kernel-3.4.18/linux-3.4.18
drwxr-xr-x 6 user user 4096 Nov 21 20:48 kernel-3.4.18/linux-obj
~~~
#### Go to the kernel tree and update the version
In qubes-builder/qubes-src/linux-kernel:
~~~
cd kernel-3.4.18/linux-3.4.18
~~~
#### Changing the config
In kernel-3.4.18/linux-3.4.18:
~~~
cp ../../config .config
make oldconfig
~~~
Now change the configuration. For example, in kernel-3.4.18/linux-3.4.18:
~~~
make menuconfig
~~~
Copy the modified config back into the kernel tree:
~~~
cp .config ../../../config
~~~
#### Patching the code
TODO: describe the workflow for patching the code, below are some random notes, not working well
~~~
ln -s ../../patches.xen
export QUILT_PATCHES=patches.xen
export QUILT_REFRESH_ARGS="-p ab --no-timestamps --no-index"
export QUILT_SERIES=../../series-pvops.conf
quilt new patches.xen/pvops-3.4-0101-usb-xen-pvusb-driver-bugfix.patch
quilt add drivers/usb/host/Kconfig drivers/usb/host/Makefile \
drivers/usb/host/xen-usbback/* drivers/usb/host/xen-usbfront.c \
include/xen/interface/io/usbif.h
*edit something*
quilt refresh
cd ../..
vi series.conf
~~~
#### Building RPMs
TODO: Is this step generic for all subsystems?
Now it is a good moment to make sure you have changed kernel release name in
rel file. For example, if you change it to '1debug201211116c' the
resulting RPMs will be named
'kernel-3.4.18-1debug20121116c.pvops.qubes.x86\_64.rpm'. This will help
distinguish between different versions of the same package.
You might want to take a moment here to review (git diff, git status), commit
your changes locally.
To actually build RPMs, in qubes-builder:
~~~
make linux-kernel
~~~
RPMs will appear in qubes-src/linux-kernel/pkgs/fc20/x86\_64:
~~~
-rw-rw-r-- 1 user user 42996126 Nov 17 04:08 kernel-3.4.18-1debug20121116c.pvops.qubes.x86_64.rpm
-rw-rw-r-- 1 user user 43001450 Nov 17 05:36 kernel-3.4.18-1debug20121117a.pvops.qubes.x86_64.rpm
-rw-rw-r-- 1 user user 8940138 Nov 17 04:08 kernel-devel-3.4.18-1debug20121116c.pvops.qubes.x86_64.rpm
-rw-rw-r-- 1 user user 8937818 Nov 17 05:36 kernel-devel-3.4.18-1debug20121117a.pvops.qubes.x86_64.rpm
-rw-rw-r-- 1 user user 54490741 Nov 17 04:08 kernel-qubes-vm-3.4.18-1debug20121116c.pvops.qubes.x86_64.rpm
-rw-rw-r-- 1 user user 54502117 Nov 17 05:37 kernel-qubes-vm-3.4.18-1debug20121117a.pvops.qubes.x86_64.rpm
~~~
### Useful [QubesBuilder](/doc/qubes-builder/) commands
1. `make check` - will check if all the code was committed into repository and
if all repository are tagged with signed tag.
2. `make show-vtags` - show version of each component (based on git tags) -
mostly useful just before building ISO. **Note:** this will not show version
for components containing changes since last version tag.
3. `make push` - push change from **all** repositories to git server. You must
set proper remotes (see above) for all repositories first.
4. `make prepare-merge` - fetch changes from remote repositories (can be
specified on commandline via GIT\_SUBDIR or GIT\_REMOTE vars), (optionally)
verify tags and show the changes. This do not merge the changes - there are
left for review as FETCH\_HEAD ref. You can merge them using `git merge
FETCH_HEAD` (in each repo directory). Or `make do-merge` to merge all of them.
## Copying Code to dom0
When developing it is convenient to be able to rapidly test changes. Assuming
you're developing Qubes on Qubes, you should be working in a special VM for
Qubes and occasionally you will want to transfer code or RPMs back to dom0 for
testing.
Here are some handy scripts Marek has shared to facilitate this.
You may also like to run your [test environment on separate
machine](/doc/test-bench/).
### Syncing dom0 files
TODO: edit this script to be more generic
~~~
#!/bin/sh
set -x
set -e
QUBES_PY_DIR=/usr/lib64/python2.6/site-packages/qubes
QUBES_PY=$QUBES_PY_DIR/qubes.py
QUBESUTILS_PY=$QUBES_PY_DIR/qubesutils.py
qvm-run -p qubes-devel 'cd qubes-builder/qubes-src/core/dom0; tar c qmemman/qmemman*.py qvm-core/*.py qvm-tools/* misc/vm-template-hvm.conf misc/qubes-start.desktop ../misc/block-snapshot aux-tools ../qrexec' |tar xv
cp $QUBES_PY qubes.py.bak$$
cp $QUBESUTILS_PY qubesutils.py.bak$$
cp /etc/xen/scripts/block-snapshot block-snapshot.bak$$
sudo cp qvm-core/qubes.py $QUBES_PY
sudo cp qvm-core/qubesutils.py $QUBESUTILS_PY
sudo cp qvm-core/guihelpers.py $QUBES_PY_DIR/
sudo cp qmemman/qmemman*.py $QUBES_PY_DIR/
sudo cp misc/vm-template-hvm.conf /usr/share/qubes/
sudo cp misc/qubes-start.desktop /usr/share/qubes/
sudo cp misc/block-snapshot /etc/xen/scripts/
sudo cp aux-tools/qubes-dom0-updates.cron /etc/cron.daily/
# FIXME(Abel Luck): I hope to
~~~
### Apply qvm-tools
TODO: make it more generic
~~~
#!/bin/sh
BAK=qvm-tools.bak$$
mkdir -p $BAK
cp -a /usr/bin/qvm-* /usr/bin/qubes-* $BAK/
sudo cp qvm-tools/qvm-* qvm-tools/qubes-* /usr/bin/
~~~
### Copy from dom0 to an appvm
~~~
#/bin/sh
#
# usage ./cp-domain <vm_name> <file_to_copy>
#
domain=$1
file=$2
fname=`basename $file`
qvm-run $domain 'mkdir /home/user/incoming/dom0 -p'
cat $file| qvm-run --pass-io $domain "cat > /home/user/incoming/dom0/$fname"
~~~
## Git connection between VMs
Sometimes it's useful to transfer git commits between VMs. You can use `git
format-patch` for that and simply copy the files. But you can also setup
custom qrexec service for it.
Below example assumes that you use `builder-RX` directory in target VM to
store sources in qubes-builder layout (where `X` is some number). Make sure that
all the scripts are executable.
Service file (save in `/usr/local/etc/qubes-rpc/local.Git` in target VM):
~~~
#!/bin/sh
exec 2>/tmp/log2
read service rel repo
echo "Params: $service $rel $repo" >&2
# Adjust regexps if needed
echo "$repo" | grep -q '^[A-Za-z0-9-]\+$' || exit 1
echo "$rel" | grep -q '^[0-9.]\+$' || exit 1
path="/home/user/builder-R$rel/qubes-src/$repo"
if [ "$repo" = "builder" ]; then
path="/home/user/builder-R$rel"
fi
case $service in
git-receive-pack|git-upload-pack)
echo "starting $service $path" >&2
exec $service $path
;;
*)
echo "Unsupported service: $service" >&2
;;
esac
~~~
Client script (save in `~/bin/git-qrexec` in source VM):
~~~
#!/bin/sh
VMNAME=$1
(echo $GIT_EXT_SERVICE $2 $3; exec cat) | qrexec-client-vm $VMNAME local.Git
~~~
You will also need to setup qrexec policy in dom0 (`/etc/qubes-rpc/policy/local.Git`).
Usage:
~~~
[user@source core-agent-linux]$ git remote add testbuilder "ext::git-qrexec testbuilder 3 core-agent-linux"
[user@source core-agent-linux]$ git push testbuilder master
~~~
You can create `~/bin/add-remote` script to ease adding remotes:
~~~
#!/bin/sh
[ -n "$1" ] || exit 1
if [ "$1" = "tb" ]; then
git remote add $1 "ext::git-qrexec testbuilder 3 `basename $PWD`"
exit $?
fi
git remote add $1 git@github.com:$1/qubes-`basename $PWD`
~~~
It should be executed from component top level directory. This script takes one
argument - remote name. If it is `tb`, then it creates qrexec-based git remote
to `testbuilder` VM. Otherwise it creates remote pointing at github account of
the same name. In any case it points at repository matching current directory
name.
## Sending packages to different VM
Other useful script(s) can be used to setup local package repository hosted in
some VM. This way you can keep your development VM behind firewall, while
having an option to expose some yum/apt repository to the local network (to
have them installed on test machine).
To achieve this goal, a dummy repository can be created, which instead of
populating metadata locally, will upload the packages to some other VM and
trigger repository update there (using qrexec). You can use `unstable`
repository flavor, because there is no release managing rules bundled (unlike
current and current-testing).
### RPM packages - yum repo
In source VM, grab [linux-yum](https://github.com/QubesOS/qubes-linux-yum) repository (below is assumed you've made it in
`~/repo-yum-upload` directory) and replace `update_repo.sh` script with:
~~~
#!/bin/sh
VMNAME=repo-vm
set -e
qvm-copy-to-vm $VMNAME $1
# remove only files, leave directory structure
find -type f -name '*.rpm' -delete
# trigger repo update
qrexec-client-vm $VMNAME local.UpdateYum
~~~
In target VM, setup actual yum repository (also based on [linux-yum](https://github.com/QubesOS/qubes-linux-yum), this time
without modifications). You will also need to setup some gpg key for signing
packages (it is possible to force yum to install unsigned packages, but it
isn't possible for `qubes-dom0-update` tool). Fill `~/.rpmmacros` with
key description:
~~~
%_gpg_name Test packages signing key
~~~
Then setup `local.UpdateYum` qrexec service (`/usr/local/etc/qubes-rpc/local.UpdateYum`):
~~~
#!/bin/sh
if [ -z "$QREXEC_REMOTE_DOMAIN" ]; then
exit 1
fi
real_repository=/home/user/linux-yum
incoming=/home/user/QubesIncoming/$QREXEC_REMOTE_DOMAIN
find $incoming -name '*.rpm' |xargs rpm -K |grep -iv pgp |cut -f1 -d: |xargs -r setsid -w rpm --addsign 2>&1
rsync -lr --remove-source-files $incoming/ $real_repository
cd $real_repository
export SKIP_REPO_CHECK=1
if [ -d $incoming/r3.1 ]; then
./update_repo-unstable.sh r3.1
fi
if [ -d $incoming/r3.0 ]; then
./update_repo-unstable.sh r3.0
fi
if [ -d $incoming/r2 ]; then
./update_repo-unstable.sh r2
fi
find $incoming -type d -empty -delete
exit 0
~~~
Of course you will also need to setup qrexec policy in dom0
`/etc/qubes-rpc/policy/local.UpdateYum`.
If you want to access the repository from network, you need to setup HTTP
server serving it, and configure the system to let other machines actually
reach this HTTP server. You can use for example using [port
forwarding](/doc/firewall/#port-forwarding-to-a-qube-from-the-outside-world) or setting up Tor hidden service. Configuration
details of those services are outside of the scope of this page.
Usage: setup `builder.conf` in source VM to use your dummy-uploader repository:
~~~
LINUX_REPO_BASEDIR = ../../repo-yum-upload/r3.1
~~~
Then use `make update-repo-unstable` to upload the packages. You can also
specify selected components on command line, then build them and upload to the
repository:
~~~
make COMPONENTS="core-agent-linux gui-agent-linux linux-utils" qubes update-repo-unstable
~~~
On the test machine, add yum repository (`/etc/yum.repos.d`) pointing at just
configured HTTP server. For example:
~~~
[local-test]
name=Test
baseurl=http://local-test.lan/linux-yum/r$releasever/unstable/dom0/fc20
~~~
Remember to also import gpg public key using `rpm --import`.
### Deb packages - Apt repo
Steps are mostly the same as in the case of yum repo. The only details that differ:
- use [linux-deb](https://github.com/QubesOS/qubes-linux-deb) instead of [linux-yum](https://github.com/QubesOS/qubes-linux-yum) as a base - both in source and target VM
- use different `update_repo.sh` script in source VM (below)
- use `local.UpdateApt` qrexec service in target VM (code below)
- in target VM additionally place `update-local-repo.sh` script in repository dir (code below)
`update_repo.sh` script:
~~~
#!/bin/sh
set -e
current_release=$1
VMNAME=repo-vm
qvm-copy-to-vm $VMNAME $1
find $current_release -type f -name '*.deb' -delete
rm -f $current_release/vm/db/*
qrexec-client-vm $VMNAME local.UpdateApt
~~~
`local.UpdateApt` service code (`/usr/local/etc/qubes-rpc/local.UpdateApt` in repo-serving VM):
~~~
#!/bin/sh
if [ -z "$QREXEC_REMOTE_DOMAIN" ]; then
exit 1
fi
incoming=/home/user/QubesIncoming/$QREXEC_REMOTE_DOMAIN
rsync -lr --remove-source-files $incoming/ /home/user/linux-deb/
cd /home/user/linux-deb
export SKIP_REPO_CHECK=1
if [ -d $incoming/r3.1 ]; then
for dist in `ls r3.1/vm/dists`; do
./update-local-repo.sh r3.1/vm $dist
done
fi
if [ -d $incoming/r3.0 ]; then
for dist in `ls r3.0/vm/dists`; do
./update-local-repo.sh r3.0/vm $dist
done
fi
if [ -d $incoming/r2 ]; then
for dist in `ls r2/vm/dists`; do
./update-local-repo.sh r2/vm $dist
done
fi
find $incoming -type d -empty -delete
exit 0
~~~
`update-local-repo.sh`:
~~~
#!/bin/sh
set -e
# Set this to your local repository signing key
SIGN_KEY=01ABCDEF
[ -z "$1" ] && { echo "Usage: $0 <repo> <dist>"; exit 1; }
REPO_DIR=$1
DIST=$2
if [ "$DIST" = "wheezy-unstable" ]; then
DIST_TAG=deb7
elif [ "$DIST" = "jessie-unstable" ]; then
DIST_TAG=deb8
elif [ "$DIST" = "stretch-unstable" ]; then
DIST_TAG=deb9
fi
pushd $REPO_DIR
mkdir -p dists/$DIST/main/binary-amd64
dpkg-scanpackages --multiversion --arch "*$DIST_TAG*" . > dists/$DIST/main/binary-amd64/Packages
gzip -9c dists/$DIST/main/binary-amd64/Packages > dists/$DIST/main/binary-amd64/Packages.gz
cat > dists/$DIST/Release <<EOF
Label: Test repo
Suite: $DIST
Codename: $DIST
Date: `date -R`
Architectures: amd64
Components: main
SHA1:
EOF
function calc_sha1() {
f=dists/$DIST/$1
echo -n " "
echo -n `sha1sum $f|cut -d' ' -f 1` ""
echo -n `stat -c %s $f` ""
echo $1
}
calc_sha1 main/binary-amd64/Packages >> dists/$DIST/Release
rm -f $DIST/Release.gpg
rm -f $DIST/InRelease
gpg -abs -u "$SIGN_KEY" \
< dists/$DIST/Release > dists/$DIST/Release.gpg
gpg -a -s --clearsign -u "$SIGN_KEY" \
< dists/$DIST/Release > dists/$DIST/InRelease
popd
if [ `id -u` -eq 0 ]; then
chown -R --reference=$REPO_DIR $REPO_DIR
fi
~~~
Usage: add this line to `/etc/apt/sources.list` on test machine (adjust host and path):
~~~
deb http://local-test.lan/linux-deb/r3.1 jessie-unstable main
~~~

647
developer/building/development-workflow.rst Normal file
View File

@ -0,0 +1,647 @@
====================
Development workflow
====================
A workflow for developing Qubes OS+
First things first, setup :doc:`QubesBuilder </developer/building/qubes-builder>`. This
guide assumes you’re using qubes-builder to build Qubes.
Repositories and committing Code
--------------------------------
Qubes is split into a bunch of git repos. These are all contained in the
``qubes-src`` directory under qubes-builder. Subdirectories there are
separate components, stored in separate git repositories.
The best way to write and contribute code is to create a git repo
somewhere (e.g., github) for the repo you are interested in editing
(e.g., ``qubes-manager``, ``core-agent-linux``, etc). To integrate your
repo with the rest of Qubes, cd to the repo directory and add your
repository as a remote in git
**Example:**
.. code:: bash
$ cd qubes-builder/qubes-src/qubes-manager
$ git remote add abel git@github.com:abeluck/qubes-manager.git
You can then proceed to easily develop in your own branches, pull in new
commits from the dev branches, merge them, and eventually push to your
own repo on github.
When you are ready to submit your changes to Qubes to be merged, push
your changes, then create a signed git tag (using ``git tag -s``).
Finally, send a letter to the Qubes listserv describing the changes and
including the link to your repository. You can also create pull request
on github. Don’t forget to include your public PGP key you use to sign
your tags.
Kernel-specific notes
^^^^^^^^^^^^^^^^^^^^^
Prepare fresh version of kernel sources, with Qubes-specific patches applied
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
In qubes-builder/qubes-src/linux-kernel:
.. code:: bash
make prep
The resulting tree will be in kernel-<VERSION>/linux-<VERSION>:
.. code:: bash
ls -ltrd kernel*/linux*
.. code:: bash
drwxr-xr-x 23 user user 4096 Nov 5 09:50 kernel-3.4.18/linux-3.4.18
drwxr-xr-x 6 user user 4096 Nov 21 20:48 kernel-3.4.18/linux-obj
Go to the kernel tree and update the version
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
In qubes-builder/qubes-src/linux-kernel:
.. code:: bash
cd kernel-3.4.18/linux-3.4.18
Changing the config
^^^^^^^^^^^^^^^^^^^
In kernel-3.4.18/linux-3.4.18:
.. code:: bash
cp ../../config .config
make oldconfig
Now change the configuration. For example, in
kernel-3.4.18/linux-3.4.18:
.. code:: bash
make menuconfig
Copy the modified config back into the kernel tree:
.. code:: bash
cp .config ../../../config
Patching the code
^^^^^^^^^^^^^^^^^
TODO: describe the workflow for patching the code, below are some random
notes, not working well
.. code:: bash
ln -s ../../patches.xen
export QUILT_PATCHES=patches.xen
export QUILT_REFRESH_ARGS="-p ab --no-timestamps --no-index"
export QUILT_SERIES=../../series-pvops.conf
quilt new patches.xen/pvops-3.4-0101-usb-xen-pvusb-driver-bugfix.patch
quilt add drivers/usb/host/Kconfig drivers/usb/host/Makefile \
drivers/usb/host/xen-usbback/* drivers/usb/host/xen-usbfront.c \
include/xen/interface/io/usbif.h
*edit something*
quilt refresh
cd ../..
vi series.conf
Building RPMs
^^^^^^^^^^^^^
TODO: Is this step generic for all subsystems?
Now it is a good moment to make sure you have changed kernel release
name in rel file. For example, if you change it to ‘1debug201211116c’
the resulting RPMs will be named
‘kernel-3.4.18-1debug20121116c.pvops.qubes.x86_64.rpm’. This will help
distinguish between different versions of the same package.
You might want to take a moment here to review (git diff, git status),
commit your changes locally.
To actually build RPMs, in qubes-builder:
.. code:: bash
make linux-kernel
RPMs will appear in qubes-src/linux-kernel/pkgs/fc20/x86_64:
.. code:: bash
-rw-rw-r-- 1 user user 42996126 Nov 17 04:08 kernel-3.4.18-1debug20121116c.pvops.qubes.x86_64.rpm
-rw-rw-r-- 1 user user 43001450 Nov 17 05:36 kernel-3.4.18-1debug20121117a.pvops.qubes.x86_64.rpm
-rw-rw-r-- 1 user user 8940138 Nov 17 04:08 kernel-devel-3.4.18-1debug20121116c.pvops.qubes.x86_64.rpm
-rw-rw-r-- 1 user user 8937818 Nov 17 05:36 kernel-devel-3.4.18-1debug20121117a.pvops.qubes.x86_64.rpm
-rw-rw-r-- 1 user user 54490741 Nov 17 04:08 kernel-qubes-vm-3.4.18-1debug20121116c.pvops.qubes.x86_64.rpm
-rw-rw-r-- 1 user user 54502117 Nov 17 05:37 kernel-qubes-vm-3.4.18-1debug20121117a.pvops.qubes.x86_64.rpm
Useful :doc:`QubesBuilder </developer/building/qubes-builder>` commands
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1. ``make check`` - will check if all the code was committed into
repository and if all repository are tagged with signed tag.
2. ``make show-vtags`` - show version of each component (based on git
tags) - mostly useful just before building ISO. **Note:** this will
not show version for components containing changes since last version
tag.
3. ``make push`` - push change from **all** repositories to git server.
You must set proper remotes (see above) for all repositories first.
4. ``make prepare-merge`` - fetch changes from remote repositories (can
be specified on commandline via GIT_SUBDIR or GIT_REMOTE vars),
(optionally) verify tags and show the changes. This do not merge the
changes - there are left for review as FETCH_HEAD ref. You can merge
them using ``git merge FETCH_HEAD`` (in each repo directory). Or
``make do-merge`` to merge all of them.
Copying Code to dom0
--------------------
When developing it is convenient to be able to rapidly test changes.
Assuming you’re developing Qubes on Qubes, you should be working in a
special VM for Qubes and occasionally you will want to transfer code or
RPMs back to dom0 for testing.
Here are some handy scripts Marek has shared to facilitate this.
You may also like to run your :doc:`test environment on separate machine </developer/debugging/test-bench>`.
Syncing dom0 files
^^^^^^^^^^^^^^^^^^
TODO: edit this script to be more generic
.. code:: bash
#!/bin/sh
set -x
set -e
QUBES_PY_DIR=/usr/lib64/python2.6/site-packages/qubes
QUBES_PY=$QUBES_PY_DIR/qubes.py
QUBESUTILS_PY=$QUBES_PY_DIR/qubesutils.py
qvm-run -p qubes-devel 'cd qubes-builder/qubes-src/core/dom0; tar c qmemman/qmemman*.py qvm-core/*.py qvm-tools/* misc/vm-template-hvm.conf misc/qubes-start.desktop ../misc/block-snapshot aux-tools ../qrexec' |tar xv
cp $QUBES_PY qubes.py.bak$$
cp $QUBESUTILS_PY qubesutils.py.bak$$
cp /etc/xen/scripts/block-snapshot block-snapshot.bak$$
sudo cp qvm-core/qubes.py $QUBES_PY
sudo cp qvm-core/qubesutils.py $QUBESUTILS_PY
sudo cp qvm-core/guihelpers.py $QUBES_PY_DIR/
sudo cp qmemman/qmemman*.py $QUBES_PY_DIR/
sudo cp misc/vm-template-hvm.conf /usr/share/qubes/
sudo cp misc/qubes-start.desktop /usr/share/qubes/
sudo cp misc/block-snapshot /etc/xen/scripts/
sudo cp aux-tools/qubes-dom0-updates.cron /etc/cron.daily/
# FIXME(Abel Luck): I hope to
Apply qvm-tools
^^^^^^^^^^^^^^^
TODO: make it more generic
.. code:: bash
#!/bin/sh
BAK=qvm-tools.bak$$
mkdir -p $BAK
cp -a /usr/bin/qvm-* /usr/bin/qubes-* $BAK/
sudo cp qvm-tools/qvm-* qvm-tools/qubes-* /usr/bin/
Copy from dom0 to an appvm
^^^^^^^^^^^^^^^^^^^^^^^^^^
.. code:: bash
#/bin/sh
#
# usage ./cp-domain <vm_name> <file_to_copy>
#
domain=$1
file=$2
fname=`basename $file`
qvm-run $domain 'mkdir /home/user/incoming/dom0 -p'
cat $file| qvm-run --pass-io $domain "cat > /home/user/incoming/dom0/$fname"
Git connection between VMs
--------------------------
Sometimes it’s useful to transfer git commits between VMs. You can use
``git format-patch`` for that and simply copy the files. But you can
also setup custom qrexec service for it.
Below example assumes that you use ``builder-RX`` directory in target VM
to store sources in qubes-builder layout (where ``X`` is some number).
Make sure that all the scripts are executable.
Service file (save in ``/usr/local/etc/qubes-rpc/local.Git`` in target
VM):
.. code:: bash
#!/bin/sh
exec 2>/tmp/log2
read service rel repo
echo "Params: $service $rel $repo" >&2
# Adjust regexps if needed
echo "$repo" | grep -q '^[A-Za-z0-9-]\+$' || exit 1
echo "$rel" | grep -q '^[0-9.]\+$' || exit 1
path="/home/user/builder-R$rel/qubes-src/$repo"
if [ "$repo" = "builder" ]; then
path="/home/user/builder-R$rel"
fi
case $service in
git-receive-pack|git-upload-pack)
echo "starting $service $path" >&2
exec $service $path
;;
*)
echo "Unsupported service: $service" >&2
;;
esac
Client script (save in ``~/bin/git-qrexec`` in source VM):
.. code:: bash
#!/bin/sh
VMNAME=$1
(echo $GIT_EXT_SERVICE $2 $3; exec cat) | qrexec-client-vm $VMNAME local.Git
You will also need to setup qrexec policy in dom0
(``/etc/qubes-rpc/policy/local.Git``).
Usage:
.. code:: bash
[user@source core-agent-linux]$ git remote add testbuilder "ext::git-qrexec testbuilder 3 core-agent-linux"
[user@source core-agent-linux]$ git push testbuilder master
You can create ``~/bin/add-remote`` script to ease adding remotes:
.. code:: bash
#!/bin/sh
[ -n "$1" ] || exit 1
if [ "$1" = "tb" ]; then
git remote add $1 "ext::git-qrexec testbuilder 3 `basename $PWD`"
exit $?
fi
git remote add $1 git@github.com:$1/qubes-`basename $PWD`
It should be executed from component top level directory. This script
takes one argument - remote name. If it is ``tb``, then it creates
qrexec-based git remote to ``testbuilder`` VM. Otherwise it creates
remote pointing at github account of the same name. In any case it
points at repository matching current directory name.
Sending packages to different VM
--------------------------------
Other useful script(s) can be used to setup local package repository
hosted in some VM. This way you can keep your development VM behind
firewall, while having an option to expose some yum/apt repository to
the local network (to have them installed on test machine).
To achieve this goal, a dummy repository can be created, which instead
of populating metadata locally, will upload the packages to some other
VM and trigger repository update there (using qrexec). You can use
``unstable`` repository flavor, because there is no release managing
rules bundled (unlike current and current-testing).
RPM packages - yum repo
^^^^^^^^^^^^^^^^^^^^^^^
In source VM, grab
`linux-yum <https://github.com/QubesOS/qubes-linux-yum>`__ repository
(below is assumed you’ve made it in ``~/repo-yum-upload`` directory) and
replace ``update_repo.sh`` script with:
.. code:: bash
#!/bin/sh
VMNAME=repo-vm
set -e
qvm-copy-to-vm $VMNAME $1
# remove only files, leave directory structure
find -type f -name '*.rpm' -delete
# trigger repo update
qrexec-client-vm $VMNAME local.UpdateYum
In target VM, setup actual yum repository (also based on
`linux-yum <https://github.com/QubesOS/qubes-linux-yum>`__, this time
without modifications). You will also need to setup some gpg key for
signing packages (it is possible to force yum to install unsigned
packages, but it isn’t possible for ``qubes-dom0-update`` tool). Fill
``~/.rpmmacros`` with key description:
.. code:: bash
%_gpg_name Test packages signing key
Then setup ``local.UpdateYum`` qrexec service
(``/usr/local/etc/qubes-rpc/local.UpdateYum``):
.. code:: bash
#!/bin/sh
if [ -z "$QREXEC_REMOTE_DOMAIN" ]; then
exit 1
fi
real_repository=/home/user/linux-yum
incoming=/home/user/QubesIncoming/$QREXEC_REMOTE_DOMAIN
find $incoming -name '*.rpm' |xargs rpm -K |grep -iv pgp |cut -f1 -d: |xargs -r setsid -w rpm --addsign 2>&1
rsync -lr --remove-source-files $incoming/ $real_repository
cd $real_repository
export SKIP_REPO_CHECK=1
if [ -d $incoming/r3.1 ]; then
./update_repo-unstable.sh r3.1
fi
if [ -d $incoming/r3.0 ]; then
./update_repo-unstable.sh r3.0
fi
if [ -d $incoming/r2 ]; then
./update_repo-unstable.sh r2
fi
find $incoming -type d -empty -delete
exit 0
Of course you will also need to setup qrexec policy in dom0
``/etc/qubes-rpc/policy/local.UpdateYum``.
If you want to access the repository from network, you need to setup
HTTP server serving it, and configure the system to let other machines
actually reach this HTTP server. You can use for example using :ref:`port forwarding <user/security-in-qubes/firewall:port forwarding to a qube from the outside world>`
or setting up Tor hidden service. Configuration details of those
services are outside of the scope of this page.
Usage: setup ``builder.conf`` in source VM to use your dummy-uploader
repository:
.. code:: bash
LINUX_REPO_BASEDIR = ../../repo-yum-upload/r3.1
Then use ``make update-repo-unstable`` to upload the packages. You can
also specify selected components on command line, then build them and
upload to the repository:
.. code:: bash
make COMPONENTS="core-agent-linux gui-agent-linux linux-utils" qubes update-repo-unstable
On the test machine, add yum repository (``/etc/yum.repos.d``) pointing
at just configured HTTP server. For example:
.. code:: bash
[local-test]
name=Test
baseurl=http://local-test.lan/linux-yum/r$releasever/unstable/dom0/fc20
Remember to also import gpg public key using ``rpm --import``.
Deb packages - Apt repo
^^^^^^^^^^^^^^^^^^^^^^^
Steps are mostly the same as in the case of yum repo. The only details
that differ:
- use `linux-deb <https://github.com/QubesOS/qubes-linux-deb>`__
instead of `linux-yum <https://github.com/QubesOS/qubes-linux-yum>`__
as a base - both in source and target VM
- use different ``update_repo.sh`` script in source VM (below)
- use ``local.UpdateApt`` qrexec service in target VM (code below)
- in target VM additionally place ``update-local-repo.sh`` script in
repository dir (code below)
``update_repo.sh`` script:
.. code:: bash
#!/bin/sh
set -e
current_release=$1
VMNAME=repo-vm
qvm-copy-to-vm $VMNAME $1
find $current_release -type f -name '*.deb' -delete
rm -f $current_release/vm/db/*
qrexec-client-vm $VMNAME local.UpdateApt
``local.UpdateApt`` service code
(``/usr/local/etc/qubes-rpc/local.UpdateApt`` in repo-serving VM):
.. code:: bash
#!/bin/sh
if [ -z "$QREXEC_REMOTE_DOMAIN" ]; then
exit 1
fi
incoming=/home/user/QubesIncoming/$QREXEC_REMOTE_DOMAIN
rsync -lr --remove-source-files $incoming/ /home/user/linux-deb/
cd /home/user/linux-deb
export SKIP_REPO_CHECK=1
if [ -d $incoming/r3.1 ]; then
for dist in `ls r3.1/vm/dists`; do
./update-local-repo.sh r3.1/vm $dist
done
fi
if [ -d $incoming/r3.0 ]; then
for dist in `ls r3.0/vm/dists`; do
./update-local-repo.sh r3.0/vm $dist
done
fi
if [ -d $incoming/r2 ]; then
for dist in `ls r2/vm/dists`; do
./update-local-repo.sh r2/vm $dist
done
fi
find $incoming -type d -empty -delete
exit 0
``update-local-repo.sh``:
.. code:: bash
#!/bin/sh
set -e
# Set this to your local repository signing key
SIGN_KEY=01ABCDEF
[ -z "$1" ] && { echo "Usage: $0 <repo> <dist>"; exit 1; }
REPO_DIR=$1
DIST=$2
if [ "$DIST" = "wheezy-unstable" ]; then
DIST_TAG=deb7
elif [ "$DIST" = "jessie-unstable" ]; then
DIST_TAG=deb8
elif [ "$DIST" = "stretch-unstable" ]; then
DIST_TAG=deb9
fi
pushd $REPO_DIR
mkdir -p dists/$DIST/main/binary-amd64
dpkg-scanpackages --multiversion --arch "*$DIST_TAG*" . > dists/$DIST/main/binary-amd64/Packages
gzip -9c dists/$DIST/main/binary-amd64/Packages > dists/$DIST/main/binary-amd64/Packages.gz
cat > dists/$DIST/Release <<EOF
Label: Test repo
Suite: $DIST
Codename: $DIST
Date: `date -R`
Architectures: amd64
Components: main
SHA1:
EOF
function calc_sha1() {
f=dists/$DIST/$1
echo -n " "
echo -n `sha1sum $f|cut -d' ' -f 1` ""
echo -n `stat -c %s $f` ""
echo $1
}
calc_sha1 main/binary-amd64/Packages >> dists/$DIST/Release
rm -f $DIST/Release.gpg
rm -f $DIST/InRelease
gpg -abs -u "$SIGN_KEY" \
< dists/$DIST/Release > dists/$DIST/Release.gpg
gpg -a -s --clearsign -u "$SIGN_KEY" \
< dists/$DIST/Release > dists/$DIST/InRelease
popd
if [ `id -u` -eq 0 ]; then
chown -R --reference=$REPO_DIR $REPO_DIR
fi
Usage: add this line to ``/etc/apt/sources.list`` on test machine
(adjust host and path):
.. code:: bash
deb http://local-test.lan/linux-deb/r3.1 jessie-unstable main

59
developer/building/qubes-builder-details.md
View File

@ -1,59 +0,0 @@
---
lang: en
layout: doc
permalink: /doc/qubes-builder-details/
redirect_from:
- /en/doc/qubes-builder-details/
- /doc/QubesBuilderDetails/
- /wiki/QubesBuilderDetails/
ref: 65
title: Qubes builder details
---
<div class="alert alert-warning" role="alert">
<i class="fa fa-exclamation-circle"></i>
<b>Note:</b> This information concerns the old Qubes builder (v1). It supports
only building Qubes 4.1 or earlier.<br>The build process has been completely rewritten in <a href="https://github.com/QubesOS/qubes-builderv2/">qubes-builder v2</a>. This can be used for building Qubes R4.1 and later, and all related components.</div>
Components Makefile.builder file
--------------------------------
[QubesBuilder](/doc/qubes-builder/) expects that each component have *Makefile.builder* file in its root directory. This file specifies what should be done to build the package. As name suggests, this is normal makefile, which is included by builder as its configuration. Its main purpose is to set some variables. Generally all available variables/settings are described as comments at the beginning of Makefile.\* in [QubesBuilder](/doc/qubes-builder/).
Variables for Linux build:
- `RPM_SPEC_FILES` List (space separated) of spec files for RPM package build. Path should be relative to component root directory. [QubesBuilder](/doc/qubes-builder/) will install all BuildRequires (in chroot environment) before the build. In most Qubes components all spec files are kept in *rpm\_spec* directory. This is mainly used for Fedora packages build.
- `ARCH_BUILD_DIRS` List (space separated) of directories with PKGBUILD files for Archlinux package build. Similar to RPM build, [QubesBuilder](/doc/qubes-builder/) will install all makedepends, then build the package.
Most components uses *archlinux* directory for this purpose, so its good to keep this style.
Variables for Windows build:
- `WIN_COMPILER` Choose which compiler should be used for this component, thus which build scripts. Currently two options available:
- `WDK` - Windows Driver Kit (default). Command used to build: *build -cZg*.
- `mingw` - MinGW (Windows gcc port). Command used to build: *make all*
- `WIN_SOURCE_SUBDIRS` List of directories in which above command should be run. In most cases it will be only one entry: current directory (*.*).
- `WIN_PREBUILD_CMD` Command to run before build, mostly useful for WDK build (in mingw case, you can use makefile for this purpose). Can be used to set some variables, preprocess some files etc.
- `WIN_SIGN_CMD` Command used to sign resulting binaries. Note that default value is *sign.bat*. If you don't want to sign binaries, specify some placeholder here (eg. *true*). Check existing components (eg. vmm-xen-windows-pvdrivers) for example scripts. This command will be run with certain environment variables:
- `CERT_FILENAME` Path to key file (pfx format)
- `CERT_PASSWORD` Key password
- `CERT_PUBLIC_FILENAME` Certificate path in the case of self-signed cert
- `CERT_CROSS_CERT_FILENAME` Certificate path in the case of correct autheticode cert
- `SIGNTOOL` Path to signtool
- `WIN_PACKAGE_CMD` Command used to produce installation package (msi or msm). Default value is *wix.bat*, similar to above - use *true* if you don't want this command.
- `WIN_OUTPUT_HEADERS` Directory (relative to `WIN_SOURCE_SUBDIRS` element) with public headers of the package - for use in other components.
- `WIN_OUTPUT_LIBS` Directory (relative to `WIN_SOURCE_SUBDIRS` element) with libraries (both DLL and implib) of the package - for use in other components. Note that [QubesBuilder](/doc/qubes-builder/) will copy files specified as *\$(WIN\_OUTPUT\_LIBS)/\*/\** to match WDK directory layout (*\<specified directory\>/\<arch directory\>/\<actual libraries\>*), so you in mingw build you need to place libraries in some additional subdirectory.
- `WIN_BUILD_DEPS` List of components required to build this one. [QubesBuilder](/doc/qubes-builder/) will copy files specified with `WIN_OUTPUT_HEADERS` and `WIN_OUTPUT_LIBS` of those components to some directory and provide its path with `QUBES_INCLUDES` and `QUBES_LIBS` variables. Use those variables in your build scripts (*sources* or *Makefile* - depending on selected compiler). You can assume that the variables are always set and directories always exists, even if empty.
builder.conf settings
---------------------
Most settings are documented in *builder.conf.default* file, which can be used as template the actual configuration.
**TODO**
Notes
-----
* For a list of custom TemplateVMs available in QubesBuilder look at [Supported Versions page](/doc/supported-releases/).

126
developer/building/qubes-builder-details.rst Normal file
View File

@ -0,0 +1,126 @@
=====================
Qubes builder details
=====================
.. warning::
Note: This information concerns the old Qubes builder (v1). It
supports only building Qubes 4.1 or earlier.The build process has
been completely rewritten in qubes-builder v2. This can be used for
building Qubes R4.1 and later, and all related components.
Components Makefile.builder file
--------------------------------
:doc:`QubesBuilder </developer/building/qubes-builder>` expects that each component have
*Makefile.builder* file in its root directory. This file specifies what
should be done to build the package. As name suggests, this is normal
makefile, which is included by builder as its configuration. Its main
purpose is to set some variables. Generally all available
variables/settings are described as comments at the beginning of
Makefile.* in :doc:`QubesBuilder </developer/building/qubes-builder>`.
Variables for Linux build:
- ``RPM_SPEC_FILES`` List (space separated) of spec files for RPM
package build. Path should be relative to component root directory.
:doc:`QubesBuilder </developer/building/qubes-builder>` will install all BuildRequires
(in chroot environment) before the build. In most Qubes components
all spec files are kept in *rpm_spec* directory. This is mainly used
for Fedora packages build.
- ``ARCH_BUILD_DIRS`` List (space separated) of directories with
PKGBUILD files for Archlinux package build. Similar to RPM build,
:doc:`QubesBuilder </developer/building/qubes-builder>` will install all makedepends,
then build the package.
Most components uses *archlinux* directory for this purpose, so its good
to keep this style.
Variables for Windows build:
- ``WIN_COMPILER`` Choose which compiler should be used for this
component, thus which build scripts. Currently two options available:
- ``WDK`` - Windows Driver Kit (default). Command used to build:
*build -cZg*.
- ``mingw`` - MinGW (Windows gcc port). Command used to build: *make all*
- ``WIN_SOURCE_SUBDIRS`` List of directories in which above command
should be run. In most cases it will be only one entry: current
directory (*.*).
- ``WIN_PREBUILD_CMD`` Command to run before build, mostly useful for
WDK build (in mingw case, you can use makefile for this purpose). Can
be used to set some variables, preprocess some files etc.
- ``WIN_SIGN_CMD`` Command used to sign resulting binaries. Note that
default value is *sign.bat*. If you don’t want to sign binaries,
specify some placeholder here (eg. *true*). Check existing components
(eg. vmm-xen-windows-pvdrivers) for example scripts. This command
will be run with certain environment variables:
- ``CERT_FILENAME`` Path to key file (pfx format)
- ``CERT_PASSWORD`` Key password
- ``CERT_PUBLIC_FILENAME`` Certificate path in the case of
self-signed cert
- ``CERT_CROSS_CERT_FILENAME`` Certificate path in the case of
correct autheticode cert
- ``SIGNTOOL`` Path to signtool
- ``WIN_PACKAGE_CMD`` Command used to produce installation package (msi
or msm). Default value is *wix.bat*, similar to above - use *true* if
you don’t want this command.
- ``WIN_OUTPUT_HEADERS`` Directory (relative to ``WIN_SOURCE_SUBDIRS``
element) with public headers of the package - for use in other
components.
- ``WIN_OUTPUT_LIBS`` Directory (relative to ``WIN_SOURCE_SUBDIRS``
element) with libraries (both DLL and implib) of the package - for
use in other components. Note that
:doc:`QubesBuilder </developer/building/qubes-builder>` will copy files specified as
*$(WIN_OUTPUT_LIBS)/*/** to match WDK directory layout (*<specified directory>/<arch directory>/<actual libraries>*), so you in mingw
build you need to place libraries in some additional subdirectory.
- ``WIN_BUILD_DEPS`` List of components required to build this one.
:doc:`QubesBuilder </developer/building/qubes-builder>` will copy files specified with
``WIN_OUTPUT_HEADERS`` and ``WIN_OUTPUT_LIBS`` of those components to
some directory and provide its path with ``QUBES_INCLUDES`` and
``QUBES_LIBS`` variables. Use those variables in your build scripts
(*sources* or *Makefile* - depending on selected compiler). You can
assume that the variables are always set and directories always
exists, even if empty.
builder.conf settings
---------------------
Most settings are documented in *builder.conf.default* file, which can
be used as template the actual configuration.
**TODO**
Notes
-----
- For a list of custom TemplateVMs available in QubesBuilder look at
:doc:`Supported Versions page </user/downloading-installing-upgrading/supported-releases>`.

218
developer/building/qubes-builder.md
View File

@ -1,218 +0,0 @@
---
lang: en
layout: doc
permalink: /doc/qubes-builder/
redirect_from:
- /en/doc/qubes-builder/
- /doc/QubesBuilder/
- /wiki/QubesBuilder/
ref: 64
title: Qubes builder
---
<div class="alert alert-warning" role="alert">
<i class="fa fa-exclamation-circle"></i>
<b>Note:</b> These instructions concern the older Qubes builder (v1). It supports
only building Qubes 4.1 or earlier.<br>The build process has been completely rewritten in <a href="https://github.com/QubesOS/qubes-builderv2/">qubes-builder v2</a>. This can be used for building Qubes R4.1 and later, and all related components.
</div>
**Note: See [ISO building instructions](/doc/qubes-iso-building/) for a streamlined overview on how to use the build system.**
We have a fully automated build system for Qubes, that downloads, builds and
packages all the Qubes components, and finally should spit out a ready-to-use
installation ISO, all in a [secure](/news/2016/05/30/build-security/) way.
In order to use it, you should use an rpm-based distro, like Fedora :), and should ensure the following packages are installed:
- sudo
- gnupg
- git
- createrepo
- rpm-build
- dnf-plugins-core
- make
- wget
- rpmdevtools
- python3-sh
- dialog
- rpm-sign
- dpkg-dev
- debootstrap
- python3-pyyaml
- devscripts
- perl-Digest-MD5
- perl-Digest-SHA
Usually you can install those packages by just issuing:
```shell
sudo dnf install gnupg git createrepo rpm-build make wget rpmdevtools python3-sh dialog rpm-sign dpkg-dev debootstrap python3-pyyaml devscripts perl-Digest-MD5 perl-Digest-SHA
```
The build system creates build environments in chroots and so no other packages are needed on the host.
All files created by the build system are contained within the qubes-builder directory.
The full build requires some 25GB of free space, so keep that in mind when deciding where to place this directory.
The build system is configured via builder.conf file.
You can use the setup.sh script to create and modify this file.
Alternatively, you can copy the provided default builder.conf, and modify it as needed, e.g.:
```shell
cp example-configs/qubes-os-master.conf builder.conf
# edit the builder.conf file and set the following variables:
NO_SIGN=1
```
One additional useful requirement is that 'sudo root' must work without any prompt, which is default on most distros (e.g. 'sudo bash' brings you the root shell without asking for any password).
This is important as the builder needs to switch to root and then back to user several times during the build process.
Additionally, if building with signing enabled (NO\_SIGN is not set), you must adjust `\~/.rpmmacros` file so that it points to the GPG key used for package signing, e.g.:
```bash
%_signature gpg
%_gpg_path /home/user/.gnupg
%_gpg_name AC1BF9B3 # <-- Key ID used for signing
```
It is also recommended that you use an empty passphrase for the private key used for signing.
Contrary to a popular belief, this doesn't affect your key or sources security -- if somebody compromised your system, then the game is over anyway, whether you have used an additional passphrase for the key or not.
So, to build Qubes you would do:
```shell
# Import the Qubes master key
gpg --recv-keys 0xDDFA1A3E36879494
# Verify its fingerprint, set as 'trusted'.
# This is described here:
# https://www.qubes-os.org/doc/VerifyingSignatures
wget https://keys.qubes-os.org/keys/qubes-developers-keys.asc
gpg --import qubes-developers-keys.asc
git clone https://github.com/QubesOS/qubes-builder.git qubes-builder
cd qubes-builder
# Verify its integrity:
git tag -v `git describe`
cp example-configs/qubes-os-master.conf builder.conf
# edit the builder.conf file and set the following variables:
# NO_SIGN="1"
# Download all components:
make get-sources
# And now to build all Qubes RPMs (this will take a few hours):
make qubes
# ... and then to build the ISO
make iso
```
And this should produce a shiny new ISO.
You can also build selected component separately. Eg. to compile only gui virtualization agent/daemon:
```shell
make gui-daemon
```
You can get a full list from make help.
## Making customized build
### Manual source modification
You can also modify sources somehow if you wish.
Here are some basic steps:
1. Download qubes-builder as described above (if you want to use marmarek's branches, you should also download qubes-builder from his repo - replace 'QubesOS' with 'marmarek' in above git clone command)
2. Edit builder.conf (still the same as above), some useful additions:
- You can also set GIT\_PREFIX="marmarek/qubes-" to use marmarek's repo instead of "mainstream" - it contains newer (but less tested) versions
3. Download unmodified sources
```shell
make get-sources
```
4. **Make your modifications here**
5. Build the Qubes
`make qubes` actually is just meta target which builds all required
components in correct order. The list of components is configured in
builder.conf. You can also check the current value at the end of `make
help`, or using `make build-info`.
6. `get-sources` is already done, so continue with the next one. You can skip `sign-all` if you've disabled signing
```shell
make vmm-xen core-admin linux-kernel gui-daemon template desktop-linux-kde installer-qubes-os manager linux-dom0-updates
```
7. build iso installation image
```shell
make iso
```
### Use pre-built Qubes packages
For building just a few selected packages, it's very useful to download pre-built qubes-specific dependencies from `{yum,deb}.qubes-os.org`.
This is especially true for `gcc`, which takes several hours to build.
Before creating the `chroot`, add this to your `builder.conf`:
```
USE_QUBES_REPO_VERSION = $(RELEASE)
```
It will add the 'current' Qubes repository to your `chroot` environment.
Next, specify which components (`gcc`, for example) you want to download instead of compiling:
```
COMPONENTS := $(filter-out gcc,$(COMPONENTS))
```
Alternatively, edit the actual COMPONENTS list which is defined in the included version-dependent config from example-configs (see series of include directives near the beginning of `builder.conf`).
This way, you can build only the packages in which you are interested.
If you also want to use the 'current-testing' repository, add this to your configuration:
```
USE_QUBES_REPO_TESTING = 1
```
In the case of an existing `chroot`, for mock-enabled builds, this works immediately because `chroot` is constructed each time separately.
For legacy builds, it will not add the necessary configuration into the build environment unless a specific builder change or configuration would force rebuilding chroot.
Also, once enabled, disabling this setting will not disable repositories in relevant chroots.
And even if it did, there could be some leftover packages installed from those repos (which may or may not be desirable).
**Note**
If you are building Ubuntu templates, you cannot use this option.
This is because Qubes does not provide official packages for Ubuntu templates.
## Code verification keys management
[QubesBuilder](/doc/qubes-builder/) by default verifies signed tags on every downloaded code.
Public keys used for that are stored in `keyrings/git`.
By default Qubes developers' keys are imported automatically, but if you need some additional keys (for example your own), you can add them using:
```shell
GNUPGHOME=$PWD/keyrings/git gpg --import /path/to/key.asc
GNUPGHOME=$PWD/keyrings/git gpg --edit-key ID_OF_JUST_IMPORTED_KEY
# here use "trust" command to set key fully or ultimately trusted - only those keys are accepted by QubesBuilder
```
All Qubes developers' keys are signed by the Qubes Master Signing Key (which is set as ultimately trusted key), so are trusted automatically.
If you are the owner of Master key and want to revoke such signature, use the `revsig` gpg key edit command and update the key in qubes-developers-keys.asc - now the key will be no longer trusted (unless manually set as such).
## Further information
For advanced [QubesBuilder](/doc/qubes-builder/) use, and preparing sources, take a look at [QubesBuilderDetails](/doc/qubes-builder-details/) page, or [QubesBuilder's doc directory](https://github.com/marmarek/qubes-builder/tree/master/doc).

303
developer/building/qubes-builder.rst Normal file
View File

@ -0,0 +1,303 @@
=============
Qubes builder
=============
.. warning::
Note: These instructions concern the older Qubes builder (v1). It
supports only building Qubes 4.1 or earlier.The build process has
been completely rewritten in qubes-builder v2. This can be used for
building Qubes R4.1 and later, and all related components.
**Note: See** :doc:`ISO building instructions </developer/building/qubes-iso-building>`
**for a streamlined overview on how to use the build system.**
We have a fully automated build system for Qubes, that downloads, builds
and packages all the Qubes components, and finally should spit out a
ready-to-use installation ISO, all in a
`secure <https://www.qubes-os.org/news/2016/05/30/build-security/>`__ way.
In order to use it, you should use an rpm-based distro, like Fedora :),
and should ensure the following packages are installed:
- sudo
- gnupg
- git
- createrepo
- rpm-build
- dnf-plugins-core
- make
- wget
- rpmdevtools
- python3-sh
- dialog
- rpm-sign
- dpkg-dev
- debootstrap
- python3-pyyaml
- devscripts
- perl-Digest-MD5
- perl-Digest-SHA
Usually you can install those packages by just issuing:
.. code:: bash
sudo dnf install gnupg git createrepo rpm-build make wget rpmdevtools python3-sh dialog rpm-sign dpkg-dev debootstrap python3-pyyaml devscripts perl-Digest-MD5 perl-Digest-SHA
The build system creates build environments in chroots and so no other
packages are needed on the host. All files created by the build system
are contained within the qubes-builder directory. The full build
requires some 25GB of free space, so keep that in mind when deciding
where to place this directory.
The build system is configured via builder.conf file. You can use the
setup.sh script to create and modify this file. Alternatively, you can
copy the provided default builder.conf, and modify it as needed, e.g.:
.. code:: bash
cp example-configs/qubes-os-master.conf builder.conf
# edit the builder.conf file and set the following variables:
NO_SIGN=1
One additional useful requirement is that ‘sudo root’ must work without
any prompt, which is default on most distros (e.g. ‘sudo bash’ brings
you the root shell without asking for any password). This is important
as the builder needs to switch to root and then back to user several
times during the build process.
Additionally, if building with signing enabled (NO_SIGN is not set), you
must adjust ``\~/.rpmmacros`` file so that it points to the GPG key used
for package signing, e.g.:
.. code:: bash
%_signature gpg
%_gpg_path /home/user/.gnupg
%_gpg_name AC1BF9B3 # <-- Key ID used for signing
It is also recommended that you use an empty passphrase for the private
key used for signing. Contrary to a popular belief, this doesn’t affect
your key or sources security – if somebody compromised your system, then
the game is over anyway, whether you have used an additional passphrase
for the key or not.
So, to build Qubes you would do:
.. code:: bash
# Import the Qubes master key
gpg --recv-keys 0xDDFA1A3E36879494
# Verify its fingerprint, set as 'trusted'.
# This is described here:
# https://www.qubes-os.org/doc/VerifyingSignatures
wget https://keys.qubes-os.org/keys/qubes-developers-keys.asc
gpg --import qubes-developers-keys.asc
git clone https://github.com/QubesOS/qubes-builder.git qubes-builder
cd qubes-builder
# Verify its integrity:
git tag -v `git describe`
cp example-configs/qubes-os-master.conf builder.conf
# edit the builder.conf file and set the following variables:
# NO_SIGN="1"
# Download all components:
make get-sources
# And now to build all Qubes RPMs (this will take a few hours):
make qubes
# ... and then to build the ISO
make iso
And this should produce a shiny new ISO.
You can also build selected component separately. Eg. to compile only
gui virtualization agent/daemon:
.. code:: bash
make gui-daemon
You can get a full list from make help.
Making customized build
-----------------------
Manual source modification
^^^^^^^^^^^^^^^^^^^^^^^^^^
You can also modify sources somehow if you wish. Here are some basic
steps:
1. Download qubes-builder as described above (if you want to use
marmarek’s branches, you should also download qubes-builder from his
repo - replace ‘QubesOS’ with ‘marmarek’ in above git clone command)
2. Edit builder.conf (still the same as above), some useful additions:
- You can also set GIT_PREFIX=“marmarek/qubes-” to use marmarek’s repo
instead of “mainstream” - it contains newer (but less tested)
versions
3. Download unmodified sources
.. code:: bash
make get-sources
4. **Make your modifications here**
5. Build the Qubes ``make qubes`` actually is just meta target which
builds all required components in correct order. The list of
components is configured in builder.conf. You can also check the
current value at the end of ``make help``, or using
``make build-info``.
6. ``get-sources`` is already done, so continue with the next one. You
can skip ``sign-all`` if you’ve disabled signing
.. code:: bash
make vmm-xen core-admin linux-kernel gui-daemon template desktop-linux-kde installer-qubes-os manager linux-dom0-updates
7. build iso installation image
.. code:: bash
make iso
Use pre-built Qubes packages
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
For building just a few selected packages, it’s very useful to download
pre-built qubes-specific dependencies from ``{yum,deb}.qubes-os.org``.
This is especially true for ``gcc``, which takes several hours to build.
Before creating the ``chroot``, add this to your ``builder.conf``:
.. code:: bash
USE_QUBES_REPO_VERSION = $(RELEASE)
It will add the ‘current’ Qubes repository to your ``chroot``
environment. Next, specify which components (``gcc``, for example) you
want to download instead of compiling:
.. code:: bash
COMPONENTS := $(filter-out gcc,$(COMPONENTS))
Alternatively, edit the actual COMPONENTS list which is defined in the
included version-dependent config from example-configs (see series of
include directives near the beginning of ``builder.conf``). This way,
you can build only the packages in which you are interested.
If you also want to use the ‘current-testing’ repository, add this to
your configuration:
.. code:: bash
USE_QUBES_REPO_TESTING = 1
In the case of an existing ``chroot``, for mock-enabled builds, this
works immediately because ``chroot`` is constructed each time
separately. For legacy builds, it will not add the necessary
configuration into the build environment unless a specific builder
change or configuration would force rebuilding chroot.
Also, once enabled, disabling this setting will not disable repositories
in relevant chroots. And even if it did, there could be some leftover
packages installed from those repos (which may or may not be desirable).
**Note** If you are building Ubuntu templates, you cannot use this
option. This is because Qubes does not provide official packages for
Ubuntu templates.
Code verification keys management
---------------------------------
:doc:`QubesBuilder </developer/building/qubes-builder>` by default verifies signed tags
on every downloaded code. Public keys used for that are stored in
``keyrings/git``. By default Qubes developers’ keys are imported
automatically, but if you need some additional keys (for example your
own), you can add them using:
.. code:: bash
GNUPGHOME=$PWD/keyrings/git gpg --import /path/to/key.asc
GNUPGHOME=$PWD/keyrings/git gpg --edit-key ID_OF_JUST_IMPORTED_KEY
# here use "trust" command to set key fully or ultimately trusted - only those keys are accepted by QubesBuilder
All Qubes developers’ keys are signed by the Qubes Master Signing Key
(which is set as ultimately trusted key), so are trusted automatically.
If you are the owner of Master key and want to revoke such signature,
use the ``revsig`` gpg key edit command and update the key in
qubes-developers-keys.asc - now the key will be no longer trusted
(unless manually set as such).
Further information
-------------------
For advanced :doc:`QubesBuilder </developer/building/qubes-builder>` use, and preparing
sources, take a look at
:doc:`QubesBuilderDetails </developer/building/qubes-builder-details>` page, or
`QubesBuilder’s doc directory <https://github.com/marmarek/qubes-builder/tree/master/doc>`__.

219
developer/building/qubes-iso-building.md
View File

@ -1,219 +0,0 @@
---
lang: en
layout: doc
permalink: /doc/qubes-iso-building/
redirect_from:
- /doc/qubes-r3-building/
- /en/doc/qubes-r3-building/
- /en/doc/qubes-iso-building/
- /doc/QubesR3Building/
- /wiki/QubesR3Building/
ref: 63
title: Qubes ISO building
---
<div class="alert alert-warning" role="alert">
<i class="fa fa-exclamation-circle"></i>
<b>Note:</b> These instructions concern the older Qubes builder (v1). It supports
only building Qubes 4.1 or earlier.<br>The build process has been completely rewritten in <a href="https://github.com/QubesOS/qubes-builderv2/">qubes-builder v2</a>. This can be used for building Qubes R4.1 and later, and all related components.
</div>
Build Environment
-----------------
Fedora 36 (and 37) has been successfully used to build Qubes R4.1 with the below steps.
Other rpm-based operating systems may also work.
Travis-CI uses Ubuntu 18.04 to perform test builds, except it can not test the `./setup` script.
**Notes:** On modern Fedora system (like Fedora 37) SeLinux is enforced by
default and is blocking the build system. You would get error like
"can't create transaction lock on /.../rpm/.rpm.lock (Permission denied)".
You can set SeLinux to permissive mode with
~~~bash
sudo setenforce 0
~~~
In `dom0`, install the Fedora 36 (or 37) template if you don't already have it.
~~~
sudo qubes-dom0-update qubes-template-fedora-36
~~~
Create a standalone AppVM from the Fedora template.
Set private storage to at least 60 GB if you will be building only the default templates; 100 GB or more if you plan on additional.
It's not required, but if you allocate additional CPU cores, the build process can utilize them at some steps such as the kernel build.
Likewise, more memory (up to 16 GB) can help.
Last, you may want to disable memory balancing, but keep in mind the impact on your other qubes.
Once you've built the development AppVM, open a Terminal window to it and install the necessary dependencies (see [QubesBuilder](/doc/qubes-builder/) for more info):
~~~
$ sudo dnf install git createrepo rpm-build rpm-sign make python3-sh rpmdevtools rpm-sign dialog perl-open python3-pyyaml perl-Digest-MD5 perl-Digest-SHA
~~~
Get the necessary keys to verify the sources (run these and other commands below as a regular user, not root):
~~~
wget https://keys.qubes-os.org/keys/qubes-master-signing-key.asc
gpg --import qubes-master-signing-key.asc
gpg --edit-key 36879494
fpr
# Verify fingerprint! See Note below!
# Once verified, set trust to *ultimate*
# (Typical sequence is trust, 5, Y, q)
wget https://keys.qubes-os.org/keys/qubes-developers-keys.asc
gpg --import qubes-developers-keys.asc
~~~
**Note** In the above process, we do *not* rely on the security of our server (keys.qubes-os.org) nor the connection (ssl, cert) -- we only rely on you getting the Qubes Master Signing Key fingerprint *somehow* and ensuring they match!
See [verifying signatures](/security/verifying-signatures/#how-to-import-and-authenticate-the-qubes-master-signing-key) for verification sources.
Now let's bootstrap the builder. Unfortunately, the builder cannot verify itself (the classic Chicken and Egg problem), so we need to verify the signature manually:
~~~
git clone https://github.com/QubesOS/qubes-builder.git
cd qubes-builder
git tag -v `git describe`
~~~
**Note** It's very important to check if the verification message contains "Good signature from ..." and does not contain "WARNING: This key is not certified with a trusted signature!".
Assuming the verification went fine, we're good to go with all the rest without ever thinking more about verifying digital signatures on all the rest of the components, apart from an additional step if doing a non-scripted build.
The builder will do that for us for each component, every time we build, even for all auxiliary files (e.g. Xen or Linux kernel sources).
Build using setup script
-----------------
Let's configure the builder first (see [procedure](/doc/qubes-iso-building/#build-using-manual-steps) at bottom if you would prefer to manually configure):
~~~
cd ~/qubes-builder
./setup
# Select Yes to add Qubes Master Signing Key
# Select Yes to add Qubes OS Signing Key
# Select 4.1 for version
# Stable
# Select Yes for fast Git cloning
# Select Current (if you want the option to use pre-built packages)
# Select No (we want a full build)
# Select fc36 and bullseye (for the currently shipping templates)
# Select builder-rpm, builder-debian, template-whonix, mgmt-salt
# Select Yes to add adrelanos's third party key
# Select Yes (to download)
~~~
Once it completes downloading, re-run `setup` to add the Whonix templates:
~~~
./setup
# Choose the same options as above, except at templates select:
# fc36, bullseye, whonix-gateway-16, whonix-workstation-16
~~~
Continue the build process with:
~~~
make install-deps
make get-sources
~~~
When building the Whonix templates, you will often need to add/update the `WHONIX_TBB_VERSION` variable in `builder.conf` at this stage to specify the currently shipping Tor Browser version.
See the related note under [Extra Whonix Build Options](/doc/building-whonix-template/).
You may also want to add `COMPONENTS := $(filter-out gcc,$(COMPONENTS))` to bypass a multiple hour compile step.
See [QubesBuilder](/doc/qubes-builder/#use-pre-built-qubes-packages) for more detail.
Finally, if you are making a test build, use:
~~~
make qubes
make iso
~~~
Or for a fully signed build (this requires setting `SIGN_KEY` in `builder.conf`):
~~~
make qubes
make sign-all
make iso
~~~
Enjoy your new ISO!
Build using manual steps
-----------------
Instead of using `./setup`, you can manually configure the build.
The script takes care of a lot of the keyring preparation for us, so we first need to set that up.
If you will be building Whonix templates:
~~~
cd ~
gpg --keyserver pgp.mit.edu --recv-keys 916B8D99C38EAF5E8ADC7A2A8D66066A2EEACCDA
gpg --fingerprint 916B8D99C38EAF5E8ADC7A2A8D66066A2EEACCDA
~~~
**Note:** It's very important to check the fingerprint displayed against multiple sources such as the [Whonix web site](https://www.whonix.org/wiki/Whonix_Signing_Key), etc.
It should look something like this:
~~~
pub rsa4096 2014-01-16 [SC] [expires: 2026-01-23]
Key fingerprint = 916B 8D99 C38E AF5E 8ADC 7A2A 8D66 066A 2EEA CCDA
uid [ unknown] Patrick Schleizer <adrelanos@kicksecure.com>
uid [ unknown] Patrick Schleizer <adrelanos@riseup.net>
uid [ unknown] Patrick Schleizer <adrelanos@whonix.org>
sub rsa4096 2014-01-16 [E] [expires: 2026-01-23]
sub rsa4096 2014-01-16 [A] [expires: 2026-01-23]
sub rsa4096 2014-01-16 [S] [expires: 2026-01-23]
~~~
Next, prepare the Git keyring directory and copy them in:
~~~
export GNUPGHOME=~/qubes-builder/keyrings/git
mkdir --parents "$GNUPGHOME"
cp ~/.gnupg/pubring.gpg "$GNUPGHOME"
cp ~/.gnupg/trustdb.gpg "$GNUPGHOME"
chmod --recursive 700 "$GNUPGHOME"
~~~
Copy one of the example configurations:
~~~
cd ~/qubes-builder
cp example-configs/qubes-os-master.conf builder.conf
~~~
Edit `builder.conf`, referring to `doc/Configuration.md` for a description of all available options.
Continue the build process with:
~~~
make install-deps
make get-sources
unset GNUPGHOME
~~~
When building the Whonix templates, you will often need to add/update the `WHONIX_TBB_VERSION` variable at this stage to specify the currently shipping Tor Browser version.
See the related note under [Extra Whonix Build Options](/doc/building-whonix-template/).
Finally, if you are making a test build, use:
~~~
make qubes
make iso
~~~
Or for a fully signed build (this requires setting `SIGN_KEY` in `builder.conf`):
~~~
make qubes
make sign-all
make iso
~~~
Enjoy your new ISO!

278
developer/building/qubes-iso-building.rst Normal file
View File

@ -0,0 +1,278 @@
==================
Qubes ISO building
==================
.. warning::
Note: These instructions concern the older Qubes builder (v1). It
supports only building Qubes 4.1 or earlier.The build process has
been completely rewritten in qubes-builder v2. This can be used for
building Qubes R4.1 and later, and all related components.
Build Environment
-----------------
Fedora 36 (and 37) has been successfully used to build Qubes R4.1 with
the below steps. Other rpm-based operating systems may also work.
Travis-CI uses Ubuntu 18.04 to perform test builds, except it can not
test the ``./setup`` script.
**Notes:** On modern Fedora system (like Fedora 37) SeLinux is enforced
by default and is blocking the build system. You would get error like
“can’t create transaction lock on /…/rpm/.rpm.lock (Permission denied)”.
You can set SeLinux to permissive mode with
.. code:: bash
sudo setenforce 0
In ``dom0``, install the Fedora 36 (or 37) template if you don’t already
have it.
.. code:: bash
sudo qubes-dom0-update qubes-template-fedora-36
Create a standalone AppVM from the Fedora template. Set private storage
to at least 60 GB if you will be building only the default templates;
100 GB or more if you plan on additional. It’s not required, but if you
allocate additional CPU cores, the build process can utilize them at
some steps such as the kernel build. Likewise, more memory (up to 16 GB)
can help. Last, you may want to disable memory balancing, but keep in
mind the impact on your other qubes.
Once you’ve built the development AppVM, open a Terminal window to it
and install the necessary dependencies (see
:doc:`QubesBuilder </developer/building/qubes-builder>` for more info):
.. code:: bash
$ sudo dnf install git createrepo rpm-build rpm-sign make python3-sh rpmdevtools rpm-sign dialog perl-open python3-pyyaml perl-Digest-MD5 perl-Digest-SHA
Get the necessary keys to verify the sources (run these and other
commands below as a regular user, not root):
.. code:: bash
wget https://keys.qubes-os.org/keys/qubes-master-signing-key.asc
gpg --import qubes-master-signing-key.asc
gpg --edit-key 36879494
fpr
# Verify fingerprint! See Note below!
# Once verified, set trust to *ultimate*
# (Typical sequence is trust, 5, Y, q)
wget https://keys.qubes-os.org/keys/qubes-developers-keys.asc
gpg --import qubes-developers-keys.asc
**Note** In the above process, we do *not* rely on the security of our
server (keys.qubes-os.org) nor the connection (ssl, cert) – we only rely
on you getting the Qubes Master Signing Key fingerprint *somehow* and
ensuring they match! See :ref:`verifying signatures <project-security/verifying-signatures:how to import and authenticate the qubes master signing key>`
for verification sources.
Now let’s bootstrap the builder. Unfortunately, the builder cannot
verify itself (the classic Chicken and Egg problem), so we need to
verify the signature manually:
.. code:: bash
git clone https://github.com/QubesOS/qubes-builder.git
cd qubes-builder
git tag -v `git describe`
**Note** It’s very important to check if the verification message
contains “Good signature from …” and does not contain “WARNING: This key
is not certified with a trusted signature!”.
Assuming the verification went fine, we’re good to go with all the rest
without ever thinking more about verifying digital signatures on all the
rest of the components, apart from an additional step if doing a
non-scripted build. The builder will do that for us for each component,
every time we build, even for all auxiliary files (e.g. Xen or Linux
kernel sources).
Build using setup script
------------------------
Let’s configure the builder first (see
:ref:`procedure <developer/building/qubes-iso-building:build using manual steps>` at
bottom if you would prefer to manually configure):
.. code:: bash
cd ~/qubes-builder
./setup
# Select Yes to add Qubes Master Signing Key
# Select Yes to add Qubes OS Signing Key
# Select 4.1 for version
# Stable
# Select Yes for fast Git cloning
# Select Current (if you want the option to use pre-built packages)
# Select No (we want a full build)
# Select fc36 and bullseye (for the currently shipping templates)
# Select builder-rpm, builder-debian, template-whonix, mgmt-salt
# Select Yes to add adrelanos's third party key
# Select Yes (to download)
Once it completes downloading, re-run ``setup`` to add the Whonix
templates:
.. code:: bash
./setup
# Choose the same options as above, except at templates select:
# fc36, bullseye, whonix-gateway-16, whonix-workstation-16
Continue the build process with:
.. code:: bash
make install-deps
make get-sources
When building the Whonix templates, you will often need to add/update
the ``WHONIX_TBB_VERSION`` variable in ``builder.conf`` at this stage to
specify the currently shipping Tor Browser version. See the related note
under `Extra Whonix Build Options <https://forum.qubes-os.org/t/18981>`__.
You may also want to add
``COMPONENTS := $(filter-out gcc,$(COMPONENTS))`` to bypass a multiple
hour compile step. See
:ref:`QubesBuilder <developer/building/qubes-builder:use pre-built qubes packages>` for
more detail.
Finally, if you are making a test build, use:
.. code:: bash
make qubes
make iso
Or for a fully signed build (this requires setting ``SIGN_KEY`` in
``builder.conf``):
.. code:: bash
make qubes
make sign-all
make iso
Enjoy your new ISO!
Build using manual steps
------------------------
Instead of using ``./setup``, you can manually configure the build. The
script takes care of a lot of the keyring preparation for us, so we
first need to set that up.
If you will be building Whonix templates:
.. code:: bash
cd ~
gpg --keyserver pgp.mit.edu --recv-keys 916B8D99C38EAF5E8ADC7A2A8D66066A2EEACCDA
gpg --fingerprint 916B8D99C38EAF5E8ADC7A2A8D66066A2EEACCDA
**Note:** It’s very important to check the fingerprint displayed against
multiple sources such as the `Whonix web site <https://www.whonix.org/wiki/Whonix_Signing_Key>`__, etc. It should
look something like this:
.. code:: bash
pub rsa4096 2014-01-16 [SC] [expires: 2026-01-23]
Key fingerprint = 916B 8D99 C38E AF5E 8ADC 7A2A 8D66 066A 2EEA CCDA
uid [ unknown] Patrick Schleizer <adrelanos@kicksecure.com>
uid [ unknown] Patrick Schleizer <adrelanos@riseup.net>
uid [ unknown] Patrick Schleizer <adrelanos@whonix.org>
sub rsa4096 2014-01-16 [E] [expires: 2026-01-23]
sub rsa4096 2014-01-16 [A] [expires: 2026-01-23]
sub rsa4096 2014-01-16 [S] [expires: 2026-01-23]
Next, prepare the Git keyring directory and copy them in:
.. code:: bash
export GNUPGHOME=~/qubes-builder/keyrings/git
mkdir --parents "$GNUPGHOME"
cp ~/.gnupg/pubring.gpg "$GNUPGHOME"
cp ~/.gnupg/trustdb.gpg "$GNUPGHOME"
chmod --recursive 700 "$GNUPGHOME"
Copy one of the example configurations:
.. code:: bash
cd ~/qubes-builder
cp example-configs/qubes-os-master.conf builder.conf
Edit ``builder.conf``, referring to ``doc/Configuration.md`` for a
description of all available options.
Continue the build process with:
.. code:: bash
make install-deps
make get-sources
unset GNUPGHOME
When building the Whonix templates, you will often need to add/update
the ``WHONIX_TBB_VERSION`` variable at this stage to specify the
currently shipping Tor Browser version. See the related note under
`Extra Whonix Build Options <https://forum.qubes-os.org/t/18981>`__.
Finally, if you are making a test build, use:
.. code:: bash
make qubes
make iso
Or for a fully signed build (this requires setting ``SIGN_KEY`` in
``builder.conf``):
.. code:: bash
make qubes
make sign-all
make iso
Enjoy your new ISO!

8
developer/building/qubes-template-configs.md
View File

@ -1,8 +0,0 @@
---
lang: en
layout: doc
permalink: /doc/qubes-template-configs/
redirect_to: https://github.com/QubesOS/qubes-template-configs
ref: 248
title: Qubes template configs
---

197
developer/code/code-signing.md
View File

@ -1,197 +0,0 @@
---
lang: en
layout: doc
permalink: /doc/code-signing/
ref: 51
title: Code signing
---
All contributions to the Qubes OS [source code](/doc/source-code/) must be cryptographically signed by the author's PGP key.
## Generating a Key
(Note: If you already have a PGP key, you may skip this step.)
Alex Cabal has written an excellent [guide](https://alexcabal.com/creating-the-perfect-gpg-keypair/) on creating a PGP keypair.
Below, we reproduce just the minimum steps in generating a keypair using GnuPG.
Please read Cabal's full guide for further important details.
~~~
$ gpg --gen-key
gpg (GnuPG) 2.2.27; Copyright (C) 2021 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
gpg: directory '/home/user/.gnupg' created
gpg: keybox '/home/user/.gnupg/pubring.kbx' created
Note: Use "gpg --full-generate-key" for a full featured key generation dialog.
GnuPG needs to construct a user ID to identify your key.
Real name: Bilbo Baggins
Email address: bilbo@shire.org
You selected this USER-ID:
"Bilbo Baggins <bilbo@shire.org>"
Change (N)ame, (E)mail, or (O)kay/(Q)uit? O
We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.
<type your passphrase>
We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.
gpg: /home/user/.gnupg/trustdb.gpg: trustdb created
gpg: key 6E2F4E7AF50A5827 marked as ultimately trusted
gpg: directory '/home/user/.gnupg/openpgp-revocs.d' created
gpg: revocation certificate stored as '/home/user/.gnupg/openpgp-revocs.d/87975838063F97A968D503266E2F4E7AF50A5827.rev'
public and secret key created and signed.
pub rsa3072 2021-12-30 [SC] [expires: 2023-12-30]
87975838063F97A968D503266E2F4E7AF50A5827
uid Bilbo Baggins <bilbo@shire.org>
sub rsa3072 2021-12-30 [E] [expires: 2023-12-30]
~~~
## Upload the Key
For others to find the public key, please upload it to a server.
Currently, [these](https://github.com/marmarek/signature-checker/blob/master/check-git-signature#L133-L135) are the recognized servers.
In the example below, we will use `keyserver.ubuntu.com`.
Replace 6E2F4E7AF50A5827 with your key ID, preferably the **long keyID**
which is the last 16 hex digits of the long number in the second line
of the output above:
```
pub rsa3072 2021-12-30 [SC] [expires: 2023-12-30]
87975838063F97A968D503266E2F4E7AF50A5827
```
```shell_session
$ gpg --send-keys --keyserver hkps://keyserver.ubuntu.com 6E2F4E7AF50A5827
gpg: sending key 6E2F4E7AF50A5827 to hkps://keyserver.ubuntu.com
```
## Using PGP with Git
If you're submitting a patch via GitHub (or a similar Git server), please sign
your Git commits.
1. Set up Git to use your key:
~~~
git config --global user.signingkey <KEYID>
~~~
2. Set up Git to sign your commits with your key:
~~~
git config --global commit.gpgsign true
~~~
Alternatively, manually specify when a commit is to be signed:
~~~
git commit -S
~~~
3. (Optional) Create signed tags.
Signed commits are totally sufficient to contribute to Qubes OS.
However, if you have commits which are not signed and you do not want to change them,
you can create a signed tag for the commit and push it before the check.
This is useful for example, if you have a commit back in the git history which
you like to sign now without rewriting the history.
~~~
git tag -s <tag_name> -m "<tag_message>"
~~~
You can also create an alias to make this easier.
Edit your `~/.gitconfig` file.
In the `[alias]` section, add `stag` to create signed tags and `spush` to create signed tags and push them.
~~~
[alias]
stag = "!bash -c 'id=\"`git rev-parse --verify HEAD`\"; tag_name="signed_tag_for_${id:0:8}"; git tag -s "$tag_name" -m \"Tag for commit $id\"; echo \"$tag_name\"'"
spush = "!bash -c 'git push origin `git stag`'"
~~~
You may also find it convenient to have an alias for verifying the tag on the
latest commit:
~~~
vtag = !git tag -v `git describe`
~~~
## GitHub Signature Verification (optional)
GitHub shows a green `Verified` label indicating that the GPG signature could be
verified using any of the contributor’s GPG keys uploaded to GitHub. You can
upload your public key on GitHub by adding your public GPG key on the [New GPG
key](https://github.com/settings/gpg/new) under the [SSH GPG keys page](https://github.com/settings/keys).
## Code Signature Checks
The [signature-checker](https://github.com/marmarek/signature-checker) checks if code contributions are signed.
Although GitHub adds a little green `Verified` button next to the commit, the [signature-checker](https://github.com/marmarek/signature-checker) uses this algorithm to check if a commit is correctly signed:
1. Is the commit signed?
If the commit is not signed, you can see the message
> policy/qubesos/code-signing — No signature found
2. If the commit is signed, the key is downloaded from a GPG key server.
If you can see the following error message, please check if you have uploaded the key to a key server.
> policy/qubesos/code-signing — Unable to verify (no valid key found)
### No Signature Found
> policy/qubesos/code-signing — No signature found
In this case, you have several options to sign the commit:
1. Amend the commit and replace it with a signed commit.
You can use this command to create a new signed commit:
```
git commit --amend -S
```
This also rewrites the commit so you need to push it forcefully:
```
git push -f
```
2. Create a signed tag for the unsigned commit.
If the commit is back in history and you do not want to change it,
you can create a signed tag for this commit and push the signature.
You can use the alias from above:
```
git checkout <commit>
git spush
```
Now, the signature checker needs to re-check the signature.
Please comment on the pull request that you would like to have the signatures checked again.
### Unable To Verify
> policy/qubesos/code-signing — Unable to verify (no valid key found)
This means that the [signature-checker](https://github.com/marmarek/signature-checker) has found a signature for the commit
but is not able to verify it using the any key available.
This might be that you forgot to upload the key to a key server.
Please upload it.
## Using PGP with Email
If you're submitting a patch by emailing the [developer mailing list](/support/#qubes-devel), simply sign your email with your PGP key.
One good way to do this is with a program like [Enigmail](https://www.enigmail.net/).
Enigmail is a security addon for the Mozilla Thunderbird email client that allows you to easily digitally encrypt and sign your emails.

248
developer/code/code-signing.rst Normal file
View File

@ -0,0 +1,248 @@
============
Code signing
============
All contributions to the Qubes OS :doc:`source code </developer/code/source-code>`
must be cryptographically signed by the author’s PGP key.
Generating a Key
----------------
(Note: If you already have a PGP key, you may skip this step.)
Alex Cabal has written an excellent
`guide <https://alexcabal.com/creating-the-perfect-gpg-keypair/>`__ on
creating a PGP keypair. Below, we reproduce just the minimum steps in
generating a keypair using GnuPG. Please read Cabal’s full guide for
further important details.
.. code:: bash
$ gpg --gen-key
gpg (GnuPG) 2.2.27; Copyright (C) 2021 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
gpg: directory '/home/user/.gnupg' created
gpg: keybox '/home/user/.gnupg/pubring.kbx' created
Note: Use "gpg --full-generate-key" for a full featured key generation dialog.
GnuPG needs to construct a user ID to identify your key.
Real name: Bilbo Baggins
Email address: bilbo@shire.org
You selected this USER-ID:
"Bilbo Baggins <bilbo@shire.org>"
Change (N)ame, (E)mail, or (O)kay/(Q)uit? O
We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.
<type your passphrase>
We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.
gpg: /home/user/.gnupg/trustdb.gpg: trustdb created
gpg: key 6E2F4E7AF50A5827 marked as ultimately trusted
gpg: directory '/home/user/.gnupg/openpgp-revocs.d' created
gpg: revocation certificate stored as '/home/user/.gnupg/openpgp-revocs.d/87975838063F97A968D503266E2F4E7AF50A5827.rev'
public and secret key created and signed.
pub rsa3072 2021-12-30 [SC] [expires: 2023-12-30]
87975838063F97A968D503266E2F4E7AF50A5827
uid Bilbo Baggins <bilbo@shire.org>
sub rsa3072 2021-12-30 [E] [expires: 2023-12-30]
Upload the Key
--------------
For others to find the public key, please upload it to a server.
Currently,
`these <https://github.com/marmarek/signature-checker/blob/master/check-git-signature#L133-L135>`__
are the recognized servers.
In the example below, we will use ``keyserver.ubuntu.com``.
Replace 6E2F4E7AF50A5827 with your key ID, preferably the **long keyID**
which is the last 16 hex digits of the long number in the second line of
the output above:
.. code:: bash
pub rsa3072 2021-12-30 [SC] [expires: 2023-12-30]
87975838063F97A968D503266E2F4E7AF50A5827
.. code:: bash
$ gpg --send-keys --keyserver hkps://keyserver.ubuntu.com 6E2F4E7AF50A5827
gpg: sending key 6E2F4E7AF50A5827 to hkps://keyserver.ubuntu.com
Using PGP with Git
------------------
If you’re submitting a patch via GitHub (or a similar Git server),
please sign your Git commits.
1. Set up Git to use your key:
.. code:: bash
git config --global user.signingkey <KEYID>
2. Set up Git to sign your commits with your key:
.. code:: bash
git config --global commit.gpgsign true
Alternatively, manually specify when a commit is to be signed:
.. code:: bash
git commit -S
3. (Optional) Create signed tags. Signed commits are totally sufficient
to contribute to Qubes OS. However, if you have commits which are not
signed and you do not want to change them, you can create a signed
tag for the commit and push it before the check.
This is useful for example, if you have a commit back in the git
history which you like to sign now without rewriting the history.
.. code:: bash
git tag -s <tag_name> -m "<tag_message>"
You can also create an alias to make this easier. Edit your
``~/.gitconfig`` file. In the ``[alias]`` section, add ``stag`` to
create signed tags and ``spush`` to create signed tags and push them.
.. code:: bash
[alias]
stag = "!bash -c 'id=\"`git rev-parse --verify HEAD`\"; tag_name="signed_tag_for_${id:0:8}"; git tag -s "$tag_name" -m \"Tag for commit $id\"; echo \"$tag_name\"'"
spush = "!bash -c 'git push origin `git stag`'"
You may also find it convenient to have an alias for verifying the
tag on the latest commit:
.. code:: bash
vtag = !git tag -v `git describe`
GitHub Signature Verification (optional)
----------------------------------------
GitHub shows a green ``Verified`` label indicating that the GPG
signature could be verified using any of the contributor’s GPG keys
uploaded to GitHub. You can upload your public key on GitHub by adding
your public GPG key on the `New GPG key <https://github.com/settings/gpg/new>`__ under the `SSH GPG keys page <https://github.com/settings/keys>`__.
Code Signature Checks
---------------------
The
`signature-checker <https://github.com/marmarek/signature-checker>`__
checks if code contributions are signed. Although GitHub adds a little
green ``Verified`` button next to the commit, the
`signature-checker <https://github.com/marmarek/signature-checker>`__
uses this algorithm to check if a commit is correctly signed:
1. Is the commit signed? If the commit is not signed, you can see the
message > policy/qubesos/code-signing — No signature found
2. If the commit is signed, the key is downloaded from a GPG key server.
If you can see the following error message, please check if you have
uploaded the key to a key server. > policy/qubesos/code-signing —
Unable to verify (no valid key found)
No Signature Found
^^^^^^^^^^^^^^^^^^
policy/qubesos/code-signing — No signature found
In this case, you have several options to sign the commit:
1. Amend the commit and replace it with a signed commit. You can use
this command to create a new signed commit:
.. code:: bash
git commit --amend -S
This also rewrites the commit so you need to push it forcefully:
.. code:: bash
git push -f
2. Create a signed tag for the unsigned commit. If the commit is back in
history and you do not want to change it, you can create a signed tag
for this commit and push the signature. You can use the alias from
above:
.. code:: bash
git checkout <commit>
git spush
Now, the signature checker needs to re-check the signature. Please
comment on the pull request that you would like to have the
signatures checked again.
Unable To Verify
^^^^^^^^^^^^^^^^
policy/qubesos/code-signing — Unable to verify (no valid key found)
This means that the
`signature-checker <https://github.com/marmarek/signature-checker>`__
has found a signature for the commit but is not able to verify it using
the any key available. This might be that you forgot to upload the key
to a key server. Please upload it.
Using PGP with Email
--------------------
If you’re submitting a patch by emailing the :ref:`developer mailing list <introduction/support:qubes-devel>`, simply sign your email with your PGP
key. One good way to do this is with a program like
`Enigmail <https://www.enigmail.net/>`__. Enigmail is a security addon
for the Mozilla Thunderbird email client that allows you to easily
digitally encrypt and sign your emails.

185
developer/code/coding-style.md
View File

@ -1,185 +0,0 @@
---
lang: en
layout: doc
permalink: /doc/coding-style/
redirect_from:
- /en/doc/coding-style/
- /doc/CodingStyle/
- /wiki/CodingStyle/
- /trac/wiki/CodingStyle/
ref: 53
title: Coding style
---
Rationale
---------
Maintaining proper coding style is very important for any large software project, such as Qubes. Here's why:
- It eases maintenance tasks, such as adding new functionality or generalizing code later,
- It allows others (as well as the future you!) to easily understand fragments of code and what they were supposed to do, and thus makes it easier to later extend them with newer functionality or bug fixes,
- It allows others to easily review the code and catch various bugs,
- It provides for an aesthetically pleasing experience when one reads the code...
Often, developers, usually smart ones, undersell the value of proper coding style, thinking that it's much more important how their code works. These developers expect that if their code solves some problem using a nice and neat trick, then that's all that is really required. Such thinking shows, however, immaturity and is a signal that the developer, no matter how bright and smart, might not be a good fit for larger projects. Writing a clever exploit for a Black Hat show is one thing - writing useful software supposed to be used and maintained for years is quite a different story. If you want to show off what a smart programmer you are, then you should become a researcher and write exploits. If, on the other hand, you want to be part of a team that makes real, useful software, you should ensure your coding style is impeccable. At Qubes project, we often took shortcuts and wrote nasty code, and this has always back fired at us, sometime months, sometime years later, the net result being we had to spend time fixing code, rather than implementing new functionality.
And here's a [link to the real case](https://groups.google.com/forum/#!msg/qubes-devel/XgTo6L8-5XA/JLOadvBqnqMJ) (one Qubes Security Bulletin) demonstrating how the lackadaisical coding style lead to a real security bug. Never assume you're smart enough to disregard clean and rigorous coding!
General typographic conventions
-------------------------------
- **Use space-expanded tabs that equal 4 spaces.** Yes, we know, there are many arguments for using "real" tabs instead of space-expanded tabs, but we need to pick one convention to make the project consistent. One argument for using space-expanded tabs is that this way the programmer is in control of how the code will look like, despite how other users have configured their editors to visualize the tabs (of course, we assume any sane person uses a fixed-width font for viewing the source code). If it makes you feel any better, assume this is just an arbitrary choice made to enforce a unified style.
- **Maintain max. line length of 80 characters**. Even though today's monitors often are very wide and it's often not a problem to have 120 characters displayed in an editor, maintaining shorter line lengths improves readability. It also allows others to have two parallel windows open, side by side, each with different parts of the source code.
- **Naming conventions for any OS *other than Windows***:
- `ClassName`
- `some_variable`, `some_function`, `some_argument`
- **Naming convention *for Windows OS*** -- exceptionally to preserve Windows conventions please use the following:
- `ClassName`, `FunctionName`
- `pszArgumentOne`, `hPipe` -- use Hungarian notation for argument and variables
- **Maintain a decent amount of horizontal spacing**, e.g. add a space after `if` or before `{` in C, and similar in other languages. Whether and where to also use spaces within expressions, such as (x\*2+5) vs. (x \* 2 + 5) is left to the developer's judgment. Do not put spaces immediately after or before the brackets in expressions, so avoid constructs like this: `if ( condition )` and use ones like this: `if (condition)` instead.
- **Use single new lines** ('\\n' aka LF) in any non-Windows source code. On Windows, exceptionally, use the CRLF line endings (--). This will allow the source code to be easily viewable in various Windows-based programs.
- **Use descriptive names for variables and functions**! Really, at a time when most editors have auto-completion features, there is no excuse for using short variable names.
- **Comments should be indented together with the code**, e.g. like this:
~~~
for (...) {
// The following code finds PGP private key matching the given public key in O(1)
while (key_found) {
(...)
}
}
~~~
File naming conventions
-----------------------
- All file names written with small letters, use dash to separate words, rather than underscores, e.g. `qubes-dom0-update`. Never use spaces!
**File naming in Linux/Unix-like systems:**
- User commands that operate on particular VMs (also those accessible in VMs): `/usr/bin/qvm-*`
- User commands that apply to the whole system (Dom0 only): `/usr/bin/qubes-*`
- Auxiliary executables and scripts in `/usr/libexec/qubes/` (Note: previously we used `/usr/lib/qubes` but decided to change that)
- Helper, non-executable files in `/usr/share/qubes/`
- Various config files in `/etc/qubes`
- Qubes RPC services in `/etc/qubes-rpc`. Qubes RPC Policy definitions (only in Dom0) in `/etc/qubes-rpc/policy/`
- All VM-related configs, images, and other files in `/var/lib/qubes/`
- System-wide temporary files which reflect the current state of system in `/var/run/qubes`
- Logs: either log to the system-wide messages, or to `/var/log/qubes/`
**File naming in Windows systems:**
- All base qubes-related files in `C:\Program Files\Invisible Things Lab\Qubes\` (Exceptionally spaces are allowed here to adhere to Windows naming conventions)
- Other, third-party files, not Qubes-specific, such as e.g. Xen PV drivers might be in different vendor subdirs, e.g. `C:\Program Files\Xen PV Drivers`
General programming style guidelines
------------------------------------
- Do not try to impress with your coding kung-fu, do not use tricks to save 2 lines of code, always prefer readability over trickiness!
- Make sure your code compiles and builds without warnings.
- Always think first about interfaces (e.g. function arguments, or class methods) and data structures before you start writing the actual code.
- Use comments to explain non-trivial code fragments, or expected behavior of more complex functions, if it is not clear from their name.
- Do **not** use comments for code fragments where it is immediately clear what the code does. E.g. avoid constructs like this:
~~~
// Return window id
int get_window_id (...) {
(...)
return id;
}
~~~
- Do **not** use comments to disable code fragments. In production code there should really be no commented or disabled code fragments. If you really, really have a good reason to retain some fragment of unused code, use \#if or \#ifdef to disable it, e.g.:
~~~
#if 0
(...) // Some unused code here
#endif
~~~
... and preferably use some descriptive macro instead of just `0`, e.g.:
~~~
#if USE_OLD_WINDOW_TRAVERSING
(...) // Some unused code here
#endif
~~~
Not sure how to do similar thing in Python... Anyone?
> But generally, there is little excuse to keep old, unused code fragments in the code. One should really use the functionality provided by the source code management system, such as git, instead. E.g. create a special branch for storing the old, unused code -- this way you will always be able to merge this code into upstream in the future.
- Do not hardcode values in the code! The only three numbers that are an exception here and for which it is acceptable to hardcode them are: `0`, `1` and `-1`, and frankly the last two are still controversial...
Source Code management (Git) guidelines
---------------------------------------
- Use git to maintain all code for Qubes project.
- Before you start using git, make sure you understand that git is a decentralized Source Code Management system, and that it doesn't behave like traditional, centralized source code management systems, such as SVN. Here's a good [introductory book on git](https://git-scm.com/book). Read it.
- Qubes code is divided into many git repositories. There are several reasons for that:
- This creates natural boundaries between different code blocks, enforcing proper interfaces, and easing independent development to be conducted on various code parts at the same time, without the fear of running into conflicts.
- By maintaining relatively small git repositories, it is easy for new developers to understand the code and contribute new patches, without the need to understand all the other code.
- Code repositories represent also licensing boundaries. So, e.g. because `core-agent-linux` and `core-agent-windows` are maintained in two different repositories, it is possible to have the latter under a proprietary, non-GPL license, while keeping the former fully open source.
- We have drastically changed the layout and naming of the code repositories shortly after Qubes OS R2 Beta 2 release. For details on the current code layout, please read [this article](https://blog.invisiblethings.org/2013/03/21/introducing-qubes-odyssey-framework.html).
Commit message guidelines
-------------------------
Please attempt to follow these conventions when writing your Git commit messages:
- Separate the subject line from the body with a blank line.
- Limit the subject line to approximately 50 characters.
- Capitalize the subject line.
- Do not end the subject line with a period.
- Use the imperative mood in the subject line.
- Wrap the body at 72 characters.
- Use the body to explain *what* and *why* rather than *how*.
For details, examples, and the rationale behind each of these conventions, please see [this blog post](https://chris.beams.io/posts/git-commit/), which is the source of this list.
Security coding guidelines
--------------------------
- As a general rule: **untrusted input** is our \#1 enemy!
- Any input that comes from untrusted, or less trusted, or just differently-trusted, entity should always be considered as malicious and should always be sanitized and verified. So, if your software runs in Dom0 and processes some input from any of the VMs, this input should be considered to be malicious. Even if your software runs in a VM, and processes input from some other VM, you should also assume that the input is malicious and verify it.
- Use `untrusted_` prefix for all variables that hold values read from untrusted party and which have not yet been verified to be decent, e.g.:
~~~
read_struct(untrusted_conf);
/* sanitize start */
if (untrusted_conf.width > MAX_WINDOW_WIDTH)
untrusted_conf.width = MAX_WINDOW_WIDTH;
if (untrusted_conf.height > MAX_WINDOW_HEIGHT)
untrusted_conf.height = MAX_WINDOW_HEIGHT;
width = untrusted_conf.width;
height = untrusted_conf.height;
~~~
- Use others variables, without the `untrusted_` prefix to hold the sanitized values, as shown above.
Python-specific guidelines
--------------------------
- Please follow the guidelines [here](https://peps.python.org/pep-0008/), unless they were in conflict with what is written on this page.
C and C++ specific guidelines
-----------------------------
- Do not place code in `*.h` files.
- Use `const` whenever possible, e.g. in function arguments passed via pointers.
- Do not mix procedural and objective code together -- if you write in C++, use classes and objects.
- Think about classes hierarchy, before starting to implement specific methods.
Bash-specific guidelines
------------------------
- Avoid writing scripts in bash whenever possible. Use python instead. Bash-scripts are Unix-specific and will not work under Windows VMs, or in Windows admin domain, or Windows gui domain.

378
developer/code/coding-style.rst Normal file
View File

@ -0,0 +1,378 @@
============
Coding style
============
Rationale
---------
Maintaining proper coding style is very important for any large software
project, such as Qubes. Here’s why:
- It eases maintenance tasks, such as adding new functionality or
generalizing code later,
- It allows others (as well as the future you!) to easily understand
fragments of code and what they were supposed to do, and thus makes
it easier to later extend them with newer functionality or bug fixes,
- It allows others to easily review the code and catch various bugs,
- It provides for an aesthetically pleasing experience when one reads
the code…
Often, developers, usually smart ones, undersell the value of proper
coding style, thinking that it’s much more important how their code
works. These developers expect that if their code solves some problem
using a nice and neat trick, then that’s all that is really required.
Such thinking shows, however, immaturity and is a signal that the
developer, no matter how bright and smart, might not be a good fit for
larger projects. Writing a clever exploit for a Black Hat show is one
thing - writing useful software supposed to be used and maintained for
years is quite a different story. If you want to show off what a smart
programmer you are, then you should become a researcher and write
exploits. If, on the other hand, you want to be part of a team that
makes real, useful software, you should ensure your coding style is
impeccable. At Qubes project, we often took shortcuts and wrote nasty
code, and this has always back fired at us, sometime months, sometime
years later, the net result being we had to spend time fixing code,
rather than implementing new functionality.
And here’s a `link to the real case <https://groups.google.com/forum/#!msg/qubes-devel/XgTo6L8-5XA/JLOadvBqnqMJ>`__
(one Qubes Security Bulletin) demonstrating how the lackadaisical coding
style lead to a real security bug. Never assume you’re smart enough to
disregard clean and rigorous coding!
General typographic conventions
-------------------------------
- **Use space-expanded tabs that equal 4 spaces.** Yes, we know, there
are many arguments for using “real” tabs instead of space-expanded
tabs, but we need to pick one convention to make the project
consistent. One argument for using space-expanded tabs is that this
way the programmer is in control of how the code will look like,
despite how other users have configured their editors to visualize
the tabs (of course, we assume any sane person uses a fixed-width
font for viewing the source code). If it makes you feel any better,
assume this is just an arbitrary choice made to enforce a unified
style.
- **Maintain max. line length of 80 characters**. Even though today’s
monitors often are very wide and it’s often not a problem to have 120
characters displayed in an editor, maintaining shorter line lengths
improves readability. It also allows others to have two parallel
windows open, side by side, each with different parts of the source
code.
- **Naming conventions for any OS other than Windows**:
- ``ClassName``
- ``some_variable``, ``some_function``, ``some_argument``
- **Naming convention for Windows OS** – exceptionally to preserve
Windows conventions please use the following:
- ``ClassName``, ``FunctionName``
- ``pszArgumentOne``, ``hPipe`` – use Hungarian notation for
argument and variables
- **Maintain a decent amount of horizontal spacing**, e.g. add a space
after ``if`` or before ``{`` in C, and similar in other languages.
Whether and where to also use spaces within expressions, such as
(x*2+5) vs. (x * 2 + 5) is left to the developer’s judgment. Do not
put spaces immediately after or before the brackets in expressions,
so avoid constructs like this: ``if ( condition )`` and use ones like
this: ``if (condition)`` instead.
- **Use single new lines** (‘\n’ aka LF) in any non-Windows source
code. On Windows, exceptionally, use the CRLF line endings (–). This
will allow the source code to be easily viewable in various
Windows-based programs.
- **Use descriptive names for variables and functions**! Really, at a
time when most editors have auto-completion features, there is no
excuse for using short variable names.
- **Comments should be indented together with the code**, e.g. like
this:
.. code:: c
for (...) {
// The following code finds PGP private key matching the given public key in O(1)
while (key_found) {
(...)
}
}
File naming conventions
-----------------------
- All file names written with small letters, use dash to separate
words, rather than underscores, e.g. ``qubes-dom0-update``. Never use
spaces!
**File naming in Linux/Unix-like systems:**
- User commands that operate on particular VMs (also those accessible
in VMs): ``/usr/bin/qvm-*``
- User commands that apply to the whole system (Dom0 only):
``/usr/bin/qubes-*``
- Auxiliary executables and scripts in ``/usr/libexec/qubes/`` (Note:
previously we used ``/usr/lib/qubes`` but decided to change that)
- Helper, non-executable files in ``/usr/share/qubes/``
- Various config files in ``/etc/qubes``
- Qubes RPC services in ``/etc/qubes-rpc``. Qubes RPC Policy
definitions (only in Dom0) in ``/etc/qubes-rpc/policy/``
- All VM-related configs, images, and other files in
``/var/lib/qubes/``
- System-wide temporary files which reflect the current state of system
in ``/var/run/qubes``
- Logs: either log to the system-wide messages, or to
``/var/log/qubes/``
**File naming in Windows systems:**
- All base qubes-related files in
``C:\Program Files\Invisible Things Lab\Qubes\`` (Exceptionally
spaces are allowed here to adhere to Windows naming conventions)
- Other, third-party files, not Qubes-specific, such as e.g. Xen PV
drivers might be in different vendor subdirs,
e.g. ``C:\Program Files\Xen PV Drivers``
General programming style guidelines
------------------------------------
- Do not try to impress with your coding kung-fu, do not use tricks to
save 2 lines of code, always prefer readability over trickiness!
- Make sure your code compiles and builds without warnings.
- Always think first about interfaces (e.g. function arguments, or
class methods) and data structures before you start writing the
actual code.
- Use comments to explain non-trivial code fragments, or expected
behavior of more complex functions, if it is not clear from their
name.
- Do **not** use comments for code fragments where it is immediately
clear what the code does. E.g. avoid constructs like this:
.. code:: c
// Return window id
int get_window_id (...) {
(...)
return id;
}
- Do **not** use comments to disable code fragments. In production code
there should really be no commented or disabled code fragments. If
you really, really have a good reason to retain some fragment of
unused code, use #if or #ifdef to disable it, e.g.:
.. code:: c
#if 0
(...) // Some unused code here
#endif
… and preferably use some descriptive macro instead of just ``0``,
e.g.:
.. code:: c
#if USE_OLD_WINDOW_TRAVERSING
(...) // Some unused code here
#endif
Not sure how to do similar thing in Python… Anyone?
But generally, there is little excuse to keep old, unused code
fragments in the code. One should really use the functionality
provided by the source code management system, such as git, instead.
E.g. create a special branch for storing the old, unused code – this
way you will always be able to merge this code into upstream in the
future.
- Do not hardcode values in the code! The only three numbers that are
an exception here and for which it is acceptable to hardcode them
are: ``0``, ``1`` and ``-1``, and frankly the last two are still
controversial…
Source Code management (Git) guidelines
---------------------------------------
- Use git to maintain all code for Qubes project.
- Before you start using git, make sure you understand that git is a
decentralized Source Code Management system, and that it doesn’t
behave like traditional, centralized source code management systems,
such as SVN. Here’s a good `introductory book on git <https://git-scm.com/book>`__. Read it.
- Qubes code is divided into many git repositories. There are several
reasons for that:
- This creates natural boundaries between different code blocks,
enforcing proper interfaces, and easing independent development to
be conducted on various code parts at the same time, without the
fear of running into conflicts.
- By maintaining relatively small git repositories, it is easy for
new developers to understand the code and contribute new patches,
without the need to understand all the other code.
- Code repositories represent also licensing boundaries. So,
e.g. because ``core-agent-linux`` and ``core-agent-windows`` are
maintained in two different repositories, it is possible to have
the latter under a proprietary, non-GPL license, while keeping the
former fully open source.
- We have drastically changed the layout and naming of the code
repositories shortly after Qubes OS R2 Beta 2 release. For details
on the current code layout, please read `this article <https://blog.invisiblethings.org/2013/03/21/introducing-qubes-odyssey-framework.html>`__.
Commit message guidelines
-------------------------
Please attempt to follow these conventions when writing your Git commit
messages:
- Separate the subject line from the body with a blank line.
- Limit the subject line to approximately 50 characters.
- Capitalize the subject line.
- Do not end the subject line with a period.
- Use the imperative mood in the subject line.
- Wrap the body at 72 characters.
- Use the body to explain *what* and *why* rather than *how*.
For details, examples, and the rationale behind each of these
conventions, please see `this blog post <https://chris.beams.io/posts/git-commit/>`__, which is the source
of this list.
Security coding guidelines
--------------------------
- As a general rule: **untrusted input** is our #1 enemy!
- Any input that comes from untrusted, or less trusted, or just
differently-trusted, entity should always be considered as malicious
and should always be sanitized and verified. So, if your software
runs in Dom0 and processes some input from any of the VMs, this input
should be considered to be malicious. Even if your software runs in a
VM, and processes input from some other VM, you should also assume
that the input is malicious and verify it.
- Use ``untrusted_`` prefix for all variables that hold values read
from untrusted party and which have not yet been verified to be
decent, e.g.:
.. code:: c
read_struct(untrusted_conf);
/* sanitize start */
if (untrusted_conf.width > MAX_WINDOW_WIDTH)
untrusted_conf.width = MAX_WINDOW_WIDTH;
if (untrusted_conf.height > MAX_WINDOW_HEIGHT)
untrusted_conf.height = MAX_WINDOW_HEIGHT;
width = untrusted_conf.width;
height = untrusted_conf.height;
- Use others variables, without the ``untrusted_`` prefix to hold the
sanitized values, as shown above.
Python-specific guidelines
--------------------------
- Please follow the guidelines
`here <https://peps.python.org/pep-0008/>`__, unless they were in
conflict with what is written on this page.
C and C++ specific guidelines
-----------------------------
- Do not place code in ``*.h`` files.
- Use ``const`` whenever possible, e.g. in function arguments passed
via pointers.
- Do not mix procedural and objective code together – if you write in
C++, use classes and objects.
- Think about classes hierarchy, before starting to implement specific
methods.
Bash-specific guidelines
------------------------
- Avoid writing scripts in bash whenever possible. Use python instead.
Bash-scripts are Unix-specific and will not work under Windows VMs,
or in Windows admin domain, or Windows gui domain.

24
developer/code/license.md
View File

@ -1,24 +0,0 @@
---
lang: en
layout: doc
permalink: /doc/license/
redirect_from:
- /en/doc/license/
- /doc/QubesLicensing/
- /wiki/QubesLicensing/
ref: 52
title: Software license
---
Qubes OS is a compilation of software packages, each under its own license. The
compilation is made available under the GNU General Public License version 2
(GPLv2). However, the license for this compilation does not supersede the
license of any package included in the compilation.
The source code of Qubes OS is contained in repositories under the
[@QubesOS](https://github.com/QubesOS) account on GitHub. This source code is
made available under GPLv2, unless there is a `LICENSE` file in the root of the
containing repository that specifies a different license.
The full text of the GPLv2 license can be found
[here](https://www.gnu.org/licenses/gpl-2.0.html).

19
developer/code/license.rst Normal file
View File

@ -0,0 +1,19 @@
================
Software license
================
Qubes OS is a compilation of software packages, each under its own
license. The compilation is made available under the GNU General Public
License version 2 (GPLv2). However, the license for this compilation
does not supersede the license of any package included in the
compilation.
The source code of Qubes OS is contained in repositories under the
`@QubesOS <https://github.com/QubesOS>`__ account on GitHub. This source
code is made available under GPLv2, unless there is a ``LICENSE`` file
in the root of the containing repository that specifies a different
license.
The full text of the GPLv2 license can be found
`here <https://www.gnu.org/licenses/gpl-2.0.html>`__.

82
developer/code/source-code.md
View File

@ -1,82 +0,0 @@
---
lang: en
layout: doc
permalink: /doc/source-code/
redirect_from:
- /en/doc/source-code/
- /doc/SourceCode/
- /wiki/SourceCode/
ref: 54
title: Source code
---
All the Qubes code is kept in Git repositories. We have divided the project into
several components, each of which has its own separate repository, for example:
* `core-admin.git` -- The core Qubes infrastructure, responsible for VM
management, VM templates, fs sharing, etc.
* `gui-daemon.git` -- GUI virtualization, Dom0 side.
* `gui-agent-linux.git` -- GUI virtualization, Linux VM side.
* `linux-template-builder.git` -- Scripts and other files used to create Qubes
template images.
All of our repositories are available under the [QubesOS GitHub account](https://github.com/QubesOS/).
To clone a repository:
~~~
git clone https://github.com/QubesOS/qubes-<repo_name>.git <repo_name>
~~~
e.g.:
~~~
git clone https://github.com/QubesOS/qubes-core-admin.git core-admin
~~~
To build Qubes you do not need to download all these repositories.
If you use [qubes builder](/doc/QubesBuilder/) you can specify *what* you want to build, and download only the repositories needed to build that target.
If you really do want to clone **all** of the repositories, you can use these commands:
~~~
curl "https://api.github.com/orgs/QubesOS/repos?page=1&per_page=100" | grep -e 'clone_url*' | cut -d \" -f 4 | xargs -L1 git clone
curl "https://api.github.com/orgs/QubesOS/repos?page=2&per_page=100" | grep -e 'clone_url*' | cut -d \" -f 4 | xargs -L1 git clone
~~~
To update (git fetch) **all** of these repositories :
~~~
find . -mindepth 1 -maxdepth 1 -type d -exec git -C {} fetch --tags --recurse-submodules=on-demand --all \;
~~~
(Alternatively, you can pull instead of just fetching.)
How to Send Patches
-------------------
If you want to [contribute code](/doc/contributing/#contributing-code) to the project, there are two ways. Whichever
method you choose, you must [sign your code](/doc/code-signing/) before it can be accepted.
* **Preferred**: Use GitHub's [fork & pull requests](https://guides.github.com/activities/forking/).
Opening a pull request on GitHub greatly eases the code review and tracking
process. In addition, especially for bigger changes, it's a good idea to send
a message to the [qubes-devel mailing list](/support/#qubes-devel) in order to notify people who
do not receive GitHub notifications.
* Send a patch to the [qubes-devel mailing list](/support/#qubes-devel) (`git format-patch`).
1. Make all the changes in your working directory, i.e. edit files, move them
around (you can use 'git mv' for this), etc.
2. Add the changes and commit them (`git add`, `git commit`). Never mix
different changes into one commit! Write a good description of the commit.
The first line should contain a short summary, and then, if you feel like
more explanations are needed, enter an empty new line, and then start the
long, detailed description (optional).
3. Test your changes NOW: check if RPMs build fine, etc.
4. Create the patch using `git format-patch`. This has an advantage over
`git diff`, because the former will also include your commit message, your
name and email, so that *your* name will be used as a commit's author.
5. Send your patch to `qubes-devel`. Start the message subject with
`[PATCH]`.

102
developer/code/source-code.rst Normal file
View File

@ -0,0 +1,102 @@
===========
Source code
===========
All the Qubes code is kept in Git repositories. We have divided the
project into several components, each of which has its own separate
repository, for example:
- ``core-admin.git`` – The core Qubes infrastructure, responsible for
VM management, VM templates, fs sharing, etc.
- ``gui-daemon.git`` – GUI virtualization, Dom0 side.
- ``gui-agent-linux.git`` – GUI virtualization, Linux VM side.
- ``linux-template-builder.git`` – Scripts and other files used to
create Qubes template images.
All of our repositories are available under the `QubesOS GitHub account <https://github.com/QubesOS/>`__.
To clone a repository:
.. code:: bash
git clone https://github.com/QubesOS/qubes-<repo_name>.git <repo_name>
e.g.:
.. code:: bash
git clone https://github.com/QubesOS/qubes-core-admin.git core-admin
To build Qubes you do not need to download all these repositories. If
you use :doc:`qubes builder </developer/building/qubes-builder>` you can specify *what*
you want to build, and download only the repositories needed to build
that target.
If you really do want to clone **all** of the repositories, you can use
these commands:
.. code:: bash
curl "https://api.github.com/orgs/QubesOS/repos?page=1&per_page=100" | grep -e 'clone_url*' | cut -d \" -f 4 | xargs -L1 git clone
curl "https://api.github.com/orgs/QubesOS/repos?page=2&per_page=100" | grep -e 'clone_url*' | cut -d \" -f 4 | xargs -L1 git clone
To update (git fetch) **all** of these repositories :
.. code:: bash
find . -mindepth 1 -maxdepth 1 -type d -exec git -C {} fetch --tags --recurse-submodules=on-demand --all \;
(Alternatively, you can pull instead of just fetching.)
How to Send Patches
-------------------
If you want to :ref:`contribute code <introduction/contributing:contributing code>` to the project, there are
two ways. Whichever method you choose, you must :doc:`sign your code </developer/code/code-signing>` before it can be accepted.
- **Preferred**: Use GitHub’s `fork & pull requests <https://guides.github.com/activities/forking/>`__.
Opening a pull request on GitHub greatly eases the code review and
tracking process. In addition, especially for bigger changes, it’s a
good idea to send a message to the :ref:`qubes-devel mailing list <introduction/support:qubes-devel>` in order to notify people who do not
receive GitHub notifications.
- Send a patch to the :ref:`qubes-devel mailing list <introduction/support:qubes-devel>` (``git format-patch``).
1. Make all the changes in your working directory, i.e. edit files,
move them around (you can use ‘git mv’ for this), etc.
2. Add the changes and commit them (``git add``, ``git commit``).
Never mix different changes into one commit! Write a good
description of the commit. The first line should contain a short
summary, and then, if you feel like more explanations are needed,
enter an empty new line, and then start the long, detailed
description (optional).
3. Test your changes NOW: check if RPMs build fine, etc.
4. Create the patch using ``git format-patch``. This has an advantage
over ``git diff``, because the former will also include your
commit message, your name and email, so that *your* name will be
used as a commit’s author.
5. Send your patch to ``qubes-devel``. Start the message subject with
``[PATCH]``.

279
developer/debugging/automated-tests.md
View File

@ -1,279 +0,0 @@
---
lang: en
layout: doc
permalink: /doc/automated-tests/
redirect_from:
- /en/doc/automated-tests/
- /doc/AutomatedTests/
ref: 45
title: Automated tests
---
## Unit and Integration Tests
Starting with Qubes R3 we use [python unittest](https://docs.python.org/3/library/unittest.html) to perform automatic tests of Qubes OS.
Despite the name, we use it for both [unit tests](https://en.wikipedia.org/wiki/Unit_tests) and [integration tests](https://en.wikipedia.org/wiki/Integration_tests).
The main purpose is, of course, to deliver much more stable releases.
The integration tests must be run in dom0, but some unit tests can run inside a VM as well.
### Integration & unit testing in dom0
Integration tests are written with the assumption that they will be executed on dedicated hardware and must be run in dom0. All other unit tests can also be run in dom0.
**Do not run the tests on installations with important data, because you might lose it.**
All the VMs with a name starting with `test-` on the installation are removed during the process, and all the tests are recklessly started from dom0, even when testing (& possibly breaking) VM components.
First you need to build all packages that you want to test. Please do not mix branches as this will inevitably lead to failures. Then setup Qubes OS with these packages installed.
For testing you'll have to stop the `qubesd` service as the tests will use its own custom variant of the service:
`sudo systemctl stop qubesd`
Don't forget to start it after testing again.
To start testing you can then use the standard python unittest runner:
`sudo -E python3 -m unittest -v qubes.tests`
Alternatively, use the custom Qubes OS test runner:
`sudo -E python3 -m qubes.tests.run -v`
Our test runner runs mostly the same as the standard one, but it has some nice additional features like colored output and not needing the "qubes.test" prefix.
You can use `python3 -m qubes.tests.run -h` to get usage information:
```
[user@dom0 ~]$ python3 -m qubes.tests.run -h
usage: run.py [-h] [--verbose] [--quiet] [--list] [--failfast] [--no-failfast]
[--do-not-clean] [--do-clean] [--loglevel LEVEL]
[--logfile FILE] [--syslog] [--no-syslog] [--kmsg] [--no-kmsg]
[TESTNAME [TESTNAME ...]]
positional arguments:
TESTNAME list of tests to run named like in description
(default: run all tests)
optional arguments:
-h, --help show this help message and exit
--verbose, -v increase console verbosity level
--quiet, -q decrease console verbosity level
--list, -l list all available tests and exit
--failfast, -f stop on the first fail, error or unexpected success
--no-failfast disable --failfast
--loglevel LEVEL, -L LEVEL
logging level for file and syslog forwarding (one of:
NOTSET, DEBUG, INFO, WARN, WARNING, ERROR, CRITICAL;
default: DEBUG)
--logfile FILE, -o FILE
if set, test run will be also logged to file
--syslog reenable logging to syslog
--no-syslog disable logging to syslog
--kmsg, --very-brave-or-very-stupid
log most important things to kernel ring-buffer
--no-kmsg, --i-am-smarter-than-kay-sievers
do not abuse kernel ring-buffer
--allow-running-along-qubesd
allow running in parallel with qubesd; this is
DANGEROUS and WILL RESULT IN INCONSISTENT SYSTEM STATE
--break-to-repl break to REPL after tests
When running only specific tests, write their names like in log, in format:
MODULE+"/"+CLASS+"/"+FUNCTION. MODULE should omit initial "qubes.tests.".
Example: basic/TC_00_Basic/test_000_create
```
For instance, to run only the tests for the fedora-21 template, you can use the `-l` option, then filter the list:
```
[user@dom0 ~]$ python3 -m qubes.tests.run -l | grep fedora-21
network/VmNetworking_fedora-21/test_000_simple_networking
network/VmNetworking_fedora-21/test_010_simple_proxyvm
network/VmNetworking_fedora-21/test_020_simple_proxyvm_nm
network/VmNetworking_fedora-21/test_030_firewallvm_firewall
network/VmNetworking_fedora-21/test_040_inter_vm
vm_qrexec_gui/TC_00_AppVM_fedora-21/test_000_start_shutdown
vm_qrexec_gui/TC_00_AppVM_fedora-21/test_010_run_gui_app
vm_qrexec_gui/TC_00_AppVM_fedora-21/test_050_qrexec_simple_eof
vm_qrexec_gui/TC_00_AppVM_fedora-21/test_051_qrexec_simple_eof_reverse
vm_qrexec_gui/TC_00_AppVM_fedora-21/test_052_qrexec_vm_service_eof
vm_qrexec_gui/TC_00_AppVM_fedora-21/test_053_qrexec_vm_service_eof_reverse
vm_qrexec_gui/TC_00_AppVM_fedora-21/test_060_qrexec_exit_code_dom0
vm_qrexec_gui/TC_00_AppVM_fedora-21/test_065_qrexec_exit_code_vm
vm_qrexec_gui/TC_00_AppVM_fedora-21/test_100_qrexec_filecopy
vm_qrexec_gui/TC_00_AppVM_fedora-21/test_110_qrexec_filecopy_deny
vm_qrexec_gui/TC_00_AppVM_fedora-21/test_120_qrexec_filecopy_self
vm_qrexec_gui/TC_20_DispVM_fedora-21/test_000_prepare_dvm
vm_qrexec_gui/TC_20_DispVM_fedora-21/test_010_simple_dvm_run
vm_qrexec_gui/TC_20_DispVM_fedora-21/test_020_gui_app
vm_qrexec_gui/TC_20_DispVM_fedora-21/test_030_edit_file
[user@dom0 ~]$ sudo -E python3 -m qubes.tests.run -v `python3 -m qubes.tests.run -l | grep fedora-21`
```
Example test run:
![snapshot-tests2.png](/attachment/doc/snapshot-tests2.png)
Tests are also compatible with nose2 test runner, so you can use this instead:
```bash
sudo systemctl stop qubesd; sudo -E nose2 -v --plugin nose2.plugins.loader.loadtests qubes.tests; sudo systemctl start qubesd
```
This may be especially useful together with various nose2 plugins to store tests results (for example `nose2.plugins.junitxml`), to ease presenting results. This is what we use on [OpenQA](https://open.qa/).
### Unit testing inside a VM
Many unit tests will also work inside a VM. However all of the tests requiring a dedicated VM to be run (mostly the integration tests) will be skipped.
Whereas integration tests are mostly stored in the [qubes-core-admin](https://github.com/QubesOS/qubes-core-admin) repository, unit tests can be found in each of the Qubes OS repositories.
To for example run the `qubes-core-admin` unit tests, you currently have to clone at least [qubes-core-admin](https://github.com/QubesOS/qubes-core-admin) and
its dependency [qubes-core-qrexec](https://github.com/QubesOS/qubes-core-qrexec) repository in the branches that you want to test.
The below example however will assume that you set up a build environment as described in the [Qubes Builder documentation](/doc/qubes-builder/).
Assuming you cloned the `qubes-builder` repository to your home directory inside a fedora VM, you can use the following commands to run the unit tests:
```{.bash}
cd ~
sudo dnf install python3-pip lvm2 python35 python3-virtualenv
virtualenv -p /usr/bin/python35 python35
source python35/bin/activate
python3 -V
cd ~/qubes-builder/qubes-src/core-admin
pip3 install -r ci/requirements.txt
export PYTHONPATH=../core-qrexec:test-packages
./run-tests
```
To run only the tests related to e.g. `lvm`, you may use:
`./run-tests -v $(python3 -m qubes.tests.run -l | grep lvm)`
You can later re-use the created virtual environment including all of the via `pip3` installed packages with `source ~/python35/bin/activate`.
We recommend to run the unit tests with the Python version that the code is meant to be run with in dom0 (3.5 was just an example above). For instance, the `release4.0` (Qubes 4.0) branch is intended
to be run with Python 3.5 whereas the Qubes 4.1 branch (`master` as of 2020-07) is intended to be run with Python 3.7 or higher. You can always check your dom0 installation for the Python version of
the current stable branch.
### Tests configuration
Test runs can be altered using environment variables:
- `DEFAULT_LVM_POOL` - LVM thin pool to use for tests, in `VolumeGroup/ThinPool` format
- `QUBES_TEST_PCIDEV` - PCI device to be used in PCI passthrough tests (for example sound card)
- `QUBES_TEST_TEMPLATES` - space separated list of templates to run tests on; if not set, all installed templates are tested
- `QUBES_TEST_LOAD_ALL` - load all tests (including tests for all templates) when relevant test modules are imported; this needs to be set for test runners not supporting [load_tests protocol](https://docs.python.org/3/library/unittest.html#load-tests-protocol)
### Adding a new test to core-admin
After adding a new unit test to [core-admin/qubes/tests](https://github.com/QubesOS/qubes-core-admin/tree/master/qubes/tests) you'll have to include it in [core-admin/qubes/tests/\_\_init\_\_.py](https://github.com/QubesOS/qubes-core-admin/tree/master/qubes/tests/__init__.py)
#### Editing `__init__.py`
You'll also need to add your test at the bottom of the `__init__.py` file, in the method `def load_tests`, in the for loop with `modname`.
Again, given the hypothetical `example.py` test:
~~~python
for modname in (
'qubes.tests.basic',
'qubes.tests.dom0_update',
'qubes.tests.network',
'qubes.tests.vm_qrexec_gui',
'qubes.tests.backup',
'qubes.tests.backupcompatibility',
'qubes.tests.regressions',
'qubes.tests.example', # This is our newly added test
):
~~~
### Testing PyQt applications
When testing (Py)QT applications, it's useful to create a separate QApplication object for each test.
But QT framework does not allow multiple QApplication objects in the same process at the same time.
This means it's critical to reliably cleanup the previous instance before creating a new one.
This turns out to be a non-trivial task, especially if _any_ test uses the event loop.
Failure to perform proper cleanup in many cases results in SEGV.
Below you can find steps for the proper cleanup:
~~~python
import asyncio
import quamash
import unittest
import gc
class SomeTestCase(unittest.TestCase):
def setUp(self):
[...]
# force "cleanlooks" style, the default one on Xfce (GtkStyle) use
# static variable internally and caches pointers to later destroyed
# objects (result: SEGV)
self.qtapp = QtGui.QApplication(["test", "-style", "cleanlooks"])
# construct event loop even if this particular test doesn't use it,
# otherwise events with qtapp references will be queued there anyway and the
# first test that actually use event loop will try to dereference (already
# destroyed) objects, resulting in SEGV
self.loop = quamash.QEventLoop(self.qtapp)
def tearDown(self):
[...]
# process any pending events before destroying the object
self.qtapp.processEvents()
# queue destroying the QApplication object, do that for any other QT
# related objects here too
self.qtapp.deleteLater()
# process any pending events (other than just queued destroy), just in case
self.qtapp.processEvents()
# execute main loop, which will process all events, _including just queued destroy_
self.loop.run_until_complete(asyncio.sleep(0))
# at this point it QT objects are destroyed, cleanup all remaining references;
# del other QT object here too
self.loop.close()
del self.qtapp
del self.loop
gc.collect()
~~~
## Automated tests with openQA
**URL:** <https://openqa.qubes-os.org/>
**Tests:** <https://github.com/marmarek/openqa-tests-qubesos>
Manually testing Qubes OS and its installation is a time-consuming process.
We use [OpenQA](https://open.qa/) to automate this process.
It works by installing Qubes in KVM and interacting with it as a user would, including simulating mouse clicks and keyboard presses.
Then, it checks the output to see whether various tests were passed, e.g. by comparing the virtual screen output to screenshots of a successful installation.
Using openQA to automatically test the Qubes installation process works as of Qubes 4.0-rc4 on 2018-01-26, provided that the versions of KVM and QEMU are new enough and the hardware has VT-x and EPT.
KVM also supports nested virtualization, so HVM should theoretically work.
In practice, however, either Xen or QEMU crashes when this is attempted.
Nonetheless, PV works well, which is sufficient for automated installation testing.
Thanks to present and past donors who have provided the infrastructure for Qubes' openQA system with hardware that meets these requirements.
### Looking for patterns in tests
In order to better visualize patterns in tests the [`openqa_investigator`](https://github.com/QubesOS/openqa-tests-qubesos/blob/master/utils/openqa_investigator.py) script can be used.
It feeds off of the openQA test data to make graph plots. Here is an example:
![openqa-investigator-splitgpg-example.png](/attachment/doc/openqa-investigator-splitgpg-example.png)
Some outputs:
- plot by tests
- plot by errors
- markdown
Some filters:
- filter by error
- filter by test name
Check out the script's help with `python3 openqa_investigator.py --help`
to see all available options.

355
developer/debugging/automated-tests.rst Normal file
View File

@ -0,0 +1,355 @@
===============
Automated tests
===============
Unit and Integration Tests
--------------------------
Starting with Qubes R3 we use `python unittest <https://docs.python.org/3/library/unittest.html>`__ to perform
automatic tests of Qubes OS. Despite the name, we use it for both `unit tests <https://en.wikipedia.org/wiki/Unit_tests>`__ and `integration tests <https://en.wikipedia.org/wiki/Integration_tests>`__. The main
purpose is, of course, to deliver much more stable releases.
The integration tests must be run in dom0, but some unit tests can run
inside a VM as well.
Integration & unit testing in dom0
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Integration tests are written with the assumption that they will be
executed on dedicated hardware and must be run in dom0. All other unit
tests can also be run in dom0.
**Do not run the tests on installations with important data, because you might lose it.**
All the VMs with a name starting with ``test-`` on the installation are
removed during the process, and all the tests are recklessly started
from dom0, even when testing (& possibly breaking) VM components.
First you need to build all packages that you want to test. Please do
not mix branches as this will inevitably lead to failures. Then setup
Qubes OS with these packages installed.
For testing you’ll have to stop the ``qubesd`` service as the tests will
use its own custom variant of the service:
``sudo systemctl stop qubesd``
Don’t forget to start it after testing again.
To start testing you can then use the standard python unittest runner:
``sudo -E python3 -m unittest -v qubes.tests``
Alternatively, use the custom Qubes OS test runner:
``sudo -E python3 -m qubes.tests.run -v``
Our test runner runs mostly the same as the standard one, but it has
some nice additional features like colored output and not needing the
“qubes.test” prefix.
You can use ``python3 -m qubes.tests.run -h`` to get usage information:
.. code:: bash
[user@dom0 ~]$ python3 -m qubes.tests.run -h
usage: run.py [-h] [--verbose] [--quiet] [--list] [--failfast] [--no-failfast]
[--do-not-clean] [--do-clean] [--loglevel LEVEL]
[--logfile FILE] [--syslog] [--no-syslog] [--kmsg] [--no-kmsg]
[TESTNAME [TESTNAME ...]]
positional arguments:
TESTNAME list of tests to run named like in description
(default: run all tests)
optional arguments:
-h, --help show this help message and exit
--verbose, -v increase console verbosity level
--quiet, -q decrease console verbosity level
--list, -l list all available tests and exit
--failfast, -f stop on the first fail, error or unexpected success
--no-failfast disable --failfast
--loglevel LEVEL, -L LEVEL
logging level for file and syslog forwarding (one of:
NOTSET, DEBUG, INFO, WARN, WARNING, ERROR, CRITICAL;
default: DEBUG)
--logfile FILE, -o FILE
if set, test run will be also logged to file
--syslog reenable logging to syslog
--no-syslog disable logging to syslog
--kmsg, --very-brave-or-very-stupid
log most important things to kernel ring-buffer
--no-kmsg, --i-am-smarter-than-kay-sievers
do not abuse kernel ring-buffer
--allow-running-along-qubesd
allow running in parallel with qubesd; this is
DANGEROUS and WILL RESULT IN INCONSISTENT SYSTEM STATE
--break-to-repl break to REPL after tests
When running only specific tests, write their names like in log, in format:
MODULE+"/"+CLASS+"/"+FUNCTION. MODULE should omit initial "qubes.tests.".
Example: basic/TC_00_Basic/test_000_create
For instance, to run only the tests for the fedora-21 template, you can
use the ``-l`` option, then filter the list:
.. code:: bash
[user@dom0 ~]$ python3 -m qubes.tests.run -l | grep fedora-21
network/VmNetworking_fedora-21/test_000_simple_networking
network/VmNetworking_fedora-21/test_010_simple_proxyvm
network/VmNetworking_fedora-21/test_020_simple_proxyvm_nm
network/VmNetworking_fedora-21/test_030_firewallvm_firewall
network/VmNetworking_fedora-21/test_040_inter_vm
vm_qrexec_gui/TC_00_AppVM_fedora-21/test_000_start_shutdown
vm_qrexec_gui/TC_00_AppVM_fedora-21/test_010_run_gui_app
vm_qrexec_gui/TC_00_AppVM_fedora-21/test_050_qrexec_simple_eof
vm_qrexec_gui/TC_00_AppVM_fedora-21/test_051_qrexec_simple_eof_reverse
vm_qrexec_gui/TC_00_AppVM_fedora-21/test_052_qrexec_vm_service_eof
vm_qrexec_gui/TC_00_AppVM_fedora-21/test_053_qrexec_vm_service_eof_reverse
vm_qrexec_gui/TC_00_AppVM_fedora-21/test_060_qrexec_exit_code_dom0
vm_qrexec_gui/TC_00_AppVM_fedora-21/test_065_qrexec_exit_code_vm
vm_qrexec_gui/TC_00_AppVM_fedora-21/test_100_qrexec_filecopy
vm_qrexec_gui/TC_00_AppVM_fedora-21/test_110_qrexec_filecopy_deny
vm_qrexec_gui/TC_00_AppVM_fedora-21/test_120_qrexec_filecopy_self
vm_qrexec_gui/TC_20_DispVM_fedora-21/test_000_prepare_dvm
vm_qrexec_gui/TC_20_DispVM_fedora-21/test_010_simple_dvm_run
vm_qrexec_gui/TC_20_DispVM_fedora-21/test_020_gui_app
vm_qrexec_gui/TC_20_DispVM_fedora-21/test_030_edit_file
[user@dom0 ~]$ sudo -E python3 -m qubes.tests.run -v `python3 -m qubes.tests.run -l | grep fedora-21`
Example test run:
.. figure:: /attachment/doc/snapshot-tests2.png
:alt: snapshot-tests2.png
snapshot-tests2.png
Tests are also compatible with nose2 test runner, so you can use this
instead:
.. code:: bash
sudo systemctl stop qubesd; sudo -E nose2 -v --plugin nose2.plugins.loader.loadtests qubes.tests; sudo systemctl start qubesd
This may be especially useful together with various nose2 plugins to
store tests results (for example ``nose2.plugins.junitxml``), to ease
presenting results. This is what we use on
`OpenQA <https://open.qa/>`__.
Unit testing inside a VM
^^^^^^^^^^^^^^^^^^^^^^^^
Many unit tests will also work inside a VM. However all of the tests
requiring a dedicated VM to be run (mostly the integration tests) will
be skipped.
Whereas integration tests are mostly stored in the
`qubes-core-admin <https://github.com/QubesOS/qubes-core-admin>`__
repository, unit tests can be found in each of the Qubes OS
repositories.
To for example run the ``qubes-core-admin`` unit tests, you currently
have to clone at least
`qubes-core-admin <https://github.com/QubesOS/qubes-core-admin>`__ and
its dependency
`qubes-core-qrexec <https://github.com/QubesOS/qubes-core-qrexec>`__
repository in the branches that you want to test.
The below example however will assume that you set up a build
environment as described in the :doc:`Qubes Builder documentation </developer/building/qubes-builder>`.
Assuming you cloned the ``qubes-builder`` repository to your home
directory inside a fedora VM, you can use the following commands to run
the unit tests:
.. code:: bash
cd ~
sudo dnf install python3-pip lvm2 python35 python3-virtualenv
virtualenv -p /usr/bin/python35 python35
source python35/bin/activate
python3 -V
cd ~/qubes-builder/qubes-src/core-admin
pip3 install -r ci/requirements.txt
export PYTHONPATH=../core-qrexec:test-packages
./run-tests
To run only the tests related to e.g. ``lvm``, you may use:
``./run-tests -v $(python3 -m qubes.tests.run -l | grep lvm)``
You can later re-use the created virtual environment including all of
the via ``pip3`` installed packages with
``source ~/python35/bin/activate``.
We recommend to run the unit tests with the Python version that the code
is meant to be run with in dom0 (3.5 was just an example above). For
instance, the ``release4.0`` (Qubes 4.0) branch is intended to be run
with Python 3.5 whereas the Qubes 4.1 branch (``master`` as of 2020-07)
is intended to be run with Python 3.7 or higher. You can always check
your dom0 installation for the Python version of the current stable
branch.
Tests configuration
^^^^^^^^^^^^^^^^^^^
Test runs can be altered using environment variables:
- ``DEFAULT_LVM_POOL`` - LVM thin pool to use for tests, in
``VolumeGroup/ThinPool`` format
- ``QUBES_TEST_PCIDEV`` - PCI device to be used in PCI passthrough
tests (for example sound card)
- ``QUBES_TEST_TEMPLATES`` - space separated list of templates to run
tests on; if not set, all installed templates are tested
- ``QUBES_TEST_LOAD_ALL`` - load all tests (including tests for all
templates) when relevant test modules are imported; this needs to be
set for test runners not supporting `load_tests protocol <https://docs.python.org/3/library/unittest.html#load-tests-protocol>`__
Adding a new test to core-admin
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
After adding a new unit test to
`core-admin/qubes/tests <https://github.com/QubesOS/qubes-core-admin/tree/master/qubes/tests>`__
you’ll have to include it in
`core-admin/qubes/tests/__init__.py <https://github.com/QubesOS/qubes-core-admin/tree/master/qubes/tests/__init__.py>`__
Editing ``__init__.py``
^^^^^^^^^^^^^^^^^^^^^^^
You’ll also need to add your test at the bottom of the ``__init__.py``
file, in the method ``def load_tests``, in the for loop with
``modname``. Again, given the hypothetical ``example.py`` test:
.. code:: python
for modname in (
'qubes.tests.basic',
'qubes.tests.dom0_update',
'qubes.tests.network',
'qubes.tests.vm_qrexec_gui',
'qubes.tests.backup',
'qubes.tests.backupcompatibility',
'qubes.tests.regressions',
'qubes.tests.example', # This is our newly added test
):
Testing PyQt applications
^^^^^^^^^^^^^^^^^^^^^^^^^
When testing (Py)QT applications, it’s useful to create a separate
QApplication object for each test. But QT framework does not allow
multiple QApplication objects in the same process at the same time. This
means it’s critical to reliably cleanup the previous instance before
creating a new one. This turns out to be a non-trivial task, especially
if *any* test uses the event loop. Failure to perform proper cleanup in
many cases results in SEGV. Below you can find steps for the proper
cleanup:
.. code:: python
import asyncio
import quamash
import unittest
import gc
class SomeTestCase(unittest.TestCase):
def setUp(self):
[...]
# force "cleanlooks" style, the default one on Xfce (GtkStyle) use
# static variable internally and caches pointers to later destroyed
# objects (result: SEGV)
self.qtapp = QtGui.QApplication(["test", "-style", "cleanlooks"])
# construct event loop even if this particular test doesn't use it,
# otherwise events with qtapp references will be queued there anyway and the
# first test that actually use event loop will try to dereference (already
# destroyed) objects, resulting in SEGV
self.loop = quamash.QEventLoop(self.qtapp)
def tearDown(self):
[...]
# process any pending events before destroying the object
self.qtapp.processEvents()
# queue destroying the QApplication object, do that for any other QT
# related objects here too
self.qtapp.deleteLater()
# process any pending events (other than just queued destroy), just in case
self.qtapp.processEvents()
# execute main loop, which will process all events, _including just queued destroy_
self.loop.run_until_complete(asyncio.sleep(0))
# at this point it QT objects are destroyed, cleanup all remaining references;
# del other QT object here too
self.loop.close()
del self.qtapp
del self.loop
gc.collect()
Automated tests with openQA
---------------------------
**URL:** https://openqa.qubes-os.org/ **Tests:**
https://github.com/marmarek/openqa-tests-qubesos
Manually testing Qubes OS and its installation is a time-consuming
process. We use `OpenQA <https://open.qa/>`__ to automate this process.
It works by installing Qubes in KVM and interacting with it as a user
would, including simulating mouse clicks and keyboard presses. Then, it
checks the output to see whether various tests were passed, e.g. by
comparing the virtual screen output to screenshots of a successful
installation.
Using openQA to automatically test the Qubes installation process works
as of Qubes 4.0-rc4 on 2018-01-26, provided that the versions of KVM and
QEMU are new enough and the hardware has VT-x and EPT. KVM also supports
nested virtualization, so HVM should theoretically work. In practice,
however, either Xen or QEMU crashes when this is attempted. Nonetheless,
PV works well, which is sufficient for automated installation testing.
Thanks to present and past donors who have provided the infrastructure
for Qubes’ openQA system with hardware that meets these requirements.
Looking for patterns in tests
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
In order to better visualize patterns in tests the
`openqa_investigator <https://github.com/QubesOS/openqa-tests-qubesos/blob/master/utils/openqa_investigator.py>`__
script can be used. It feeds off of the openQA test data to make graph
plots. Here is an example:
.. figure:: /attachment/doc/openqa-investigator-splitgpg-example.png
:alt: openqa-investigator-splitgpg-example.png
openqa-investigator-splitgpg-example.png
Some outputs: - plot by tests - plot by errors - markdown
Some filters: - filter by error - filter by test name
Check out the script’s help with
``python3 openqa_investigator.py --help`` to see all available options.

54
developer/debugging/mount-lvm-image.md
View File

@ -1,54 +0,0 @@
---
lang: en
layout: doc
permalink: /doc/mount-lvm-image/
ref: 46
title: How to mount LVM images
---
You want to read your LVM image (e.g., there is a problem where you can't start any VMs except dom0).
1: make the image available for qubesdb.
From dom0 terminal:
```bash
# Example: /dev/qubes_dom0/vm-debian-9-tmp-root
[user@dom0]$ dev=$(basename $(readlink /dev/YOUR_LVM_VG/YOUR_LVM_IMAGE))
[user@dom0]$ qubesdb-write /qubes-block-devices/$dev/desc "YOUR_LVM_IMAGE"
```
2: Create a new disposable VM
```bash
[user@dom0]$ qvm-run -v --dispvm=YOUR_DVM_TEMPLATE --service qubes.StartApp+xterm &
```
3: Attach the device to your newly created disp VM
From the GUI, or from the command line:
```bash
[user@dom0]$ qvm-block attach NEWLY_CREATED_DISPVM dom0:$dev
```
4: Mount the partition you want to, and do what you want with it
```bash
[user@dispXXXX]$ mount /dev/xvdiX /mnt/
```
5: Umount and kill the VM
```bash
[user@dispXXXX]$ umount /mnt/
```
6: Remove the image from qubesdb
```bash
[user@dom0]$ qubesdb-rm /qubes-block-devices/$dev/
```
# References
Please consult this issue's [comment](https://github.com/QubesOS/qubes-issues/issues/4687#issuecomment-451626625).

60
developer/debugging/mount-lvm-image.rst Normal file
View File

@ -0,0 +1,60 @@
=======================
How to mount LVM images
=======================
You want to read your LVM image (e.g., there is a problem where you
can’t start any VMs except dom0).
1: make the image available for qubesdb. From dom0 terminal:
.. code:: bash
# Example: /dev/qubes_dom0/vm-debian-9-tmp-root
[user@dom0]$ dev=$(basename $(readlink /dev/YOUR_LVM_VG/YOUR_LVM_IMAGE))
[user@dom0]$ qubesdb-write /qubes-block-devices/$dev/desc "YOUR_LVM_IMAGE"
2: Create a new disposable VM
.. code:: bash
[user@dom0]$ qvm-run -v --dispvm=YOUR_DVM_TEMPLATE --service qubes.StartApp+xterm &
3: Attach the device to your newly created disp VM
From the GUI, or from the command line:
.. code:: bash
[user@dom0]$ qvm-block attach NEWLY_CREATED_DISPVM dom0:$dev
4: Mount the partition you want to, and do what you want with it
.. code:: bash
[user@dispXXXX]$ mount /dev/xvdiX /mnt/
5: Umount and kill the VM
.. code:: bash
[user@dispXXXX]$ umount /mnt/
6: Remove the image from qubesdb
.. code:: bash
[user@dom0]$ qubesdb-rm /qubes-block-devices/$dev/
References
----------
Please consult this issue’s
`comment <https://github.com/QubesOS/qubes-issues/issues/4687#issuecomment-451626625>`__.

76
developer/debugging/safe-remote-ttys.md
View File

@ -1,76 +0,0 @@
---
lang: en
layout: doc
permalink: /doc/safe-remote-ttys/
redirect_from:
- /en/doc/safe-remote-ttys/
ref: 49
title: Safe remote dom0 terminals
---
If you do not have working graphics in Dom0, then using a terminal can be quite annoying!
This was the case for the author while trying to debug PCI-passthrough of a machine's primary (only) GPU.
Your first thought might be to just allow network access to Dom0, enable ssh, and connect in remotely.
But, this gravely violates the Qubes security model.
Instead, a better solution is to split the input and output paths of using a terminal.
Use your normal keyboard for input, but have the output go to a remote machine in a unidirectional manner.
To do this, we make use of script(1), qvm-run, and optionally your network transport of choice.
To a different VM
-----------------
As an example of forwarding terminal output to another VM on the same machine:
~~~
$ mkfifo /tmp/foo
$ qvm-run -p some-vm 'xterm -e "cat 0<&5" 5<&0' </tmp/foo >/dev/null 2>&1 &
$ script -f /tmp/foo
~~~
To a different machine
----------------------
In this case over SSH (from a network-connected VM):
~~~
$ mkfifo /tmp/foo
$ qvm-run -p some-vm \
'ssh user@host sh -c "DISPLAY=:0 xterm -e \"cat 0<&5\" 5<&0"' \
</tmp/foo >/dev/null 2>&1 &
$ script -f /tmp/foo
~~~
Note that no data received over SSH is ever treated as terminal input in Dom0.
The input path remains only from your trusted local keyboard.
Multiple terminals
------------------
For multiple terminals, you may find it easier to just use tmux than to try to blindly switch to the correct window.
Terminal size
-------------
It is up to you to ensure the sizes of the local and remote terminal are the same, otherwise things may display incorrectly (especially in interactive programs).
Depending on your shell, the size of your local (blind) terminal is likely stored in the `$LINES` and `$COLUMNS` variables.
~~~
$ echo $COLUMNS $LINES
80 24
~~~
A note on serial consoles
-------------------------
If your machine has a serial console, you may with to use that, but note that a similar split-I/O model should be used to ensure Dom0 integrity.
If you use the serial console as normal (via e.g. getty on ttyX, and logging in as normal), then the machine at the end of the serial cable could compromise your machine!
Ideally, you would take input from your trusted keyboard, and only send the output over the serial cable via e.g. disabling getty and using:
~~~
script -f /dev/ttyS0
~~~
You don't even need to connect the TX pin.

97
developer/debugging/safe-remote-ttys.rst Normal file
View File

@ -0,0 +1,97 @@
==========================
Safe remote dom0 terminals
==========================
If you do not have working graphics in Dom0, then using a terminal can
be quite annoying! This was the case for the author while trying to
debug PCI-passthrough of a machine’s primary (only) GPU.
Your first thought might be to just allow network access to Dom0, enable
ssh, and connect in remotely. But, this gravely violates the Qubes
security model.
Instead, a better solution is to split the input and output paths of
using a terminal. Use your normal keyboard for input, but have the
output go to a remote machine in a unidirectional manner.
To do this, we make use of script(1), qvm-run, and optionally your
network transport of choice.
To a different VM
-----------------
As an example of forwarding terminal output to another VM on the same
machine:
.. code:: bash
$ mkfifo /tmp/foo
$ qvm-run -p some-vm 'xterm -e "cat 0<&5" 5<&0' </tmp/foo >/dev/null 2>&1 &
$ script -f /tmp/foo
To a different machine
----------------------
In this case over SSH (from a network-connected VM):
.. code:: bash
$ mkfifo /tmp/foo
$ qvm-run -p some-vm \
'ssh user@host sh -c "DISPLAY=:0 xterm -e \"cat 0<&5\" 5<&0"' \
</tmp/foo >/dev/null 2>&1 &
$ script -f /tmp/foo
Note that no data received over SSH is ever treated as terminal input in
Dom0. The input path remains only from your trusted local keyboard.
Multiple terminals
------------------
For multiple terminals, you may find it easier to just use tmux than to
try to blindly switch to the correct window.
Terminal size
-------------
It is up to you to ensure the sizes of the local and remote terminal are
the same, otherwise things may display incorrectly (especially in
interactive programs). Depending on your shell, the size of your local
(blind) terminal is likely stored in the ``$LINES`` and ``$COLUMNS``
variables.
.. code:: bash
$ echo $COLUMNS $LINES
80 24
A note on serial consoles
-------------------------
If your machine has a serial console, you may with to use that, but note
that a similar split-I/O model should be used to ensure Dom0 integrity.
If you use the serial console as normal (via e.g. getty on ttyX, and
logging in as normal), then the machine at the end of the serial cable
could compromise your machine! Ideally, you would take input from your
trusted keyboard, and only send the output over the serial cable via
e.g. disabling getty and using:
.. code:: bash
script -f /dev/ttyS0
You don’t even need to connect the TX pin.

235
developer/debugging/test-bench.md
View File

@ -1,235 +0,0 @@
---
lang: en
layout: doc
permalink: /doc/test-bench/
redirect_from:
- /en/doc/test-bench/
- /doc/TestBench/
- /wiki/TestBench/
ref: 44
title: How to set up a test bench
---
This guide shows how to set up simple test bench that automatically test your code you're about to push. It is written especially for `core3` branch of `core-admin.git` repo, but some ideas are universal.
We will set up a spare machine (bare metal, not a virtual) that will be hosting our experimental Dom0. We will communicate with it via Ethernet and SSH. This tutorial assumes you are familiar with [QubesBuilder](/doc/qubes-builder/) and you have it set up and running flawlessly.
> **Notice:**
> This setup intentionally weakens some security properties in the testing system. So make sure you understand the risks and use exclusively for testing.
## Setting up the Machine
### Install ISO
First, do a clean install from the `.iso` [you built](/doc/qubes-iso-building/) or grabbed elsewhere (for example [here](https://forum.qubes-os.org/t/qubesos-4-1-alpha-signed-weekly-builds/3601)).
### Enabling Network Access in Dom0
Internet access is intentionally disabled by default in dom0. But to ease the deployment process we will give it access. The following steps should be done in `dom0`.
> **Note:** the following assume you have only one network card. If you have two, pick one and leave the other attached to `sys-net`.
1. Remove the network card (PCI device) from `sys-net`
2. Restart your computer (for the removal to take effect)
3. Install `dhcp-client` and `openssh-server` on your testbench's dom0.
4. Save the following script in `/home/user/bin/dom0_network.sh` and make it executable. It should enable your network card in dom0. *Be sure to adjust the script's variables to suit your needs.*
```bash
#!/bin/sh
# adjust this for your NIC (run lspci)
BDF=0000:02:00.0
# adjust this for your network driver
DRIVER=e1000e
prog=$(basename $0)
pciunbind() {
local path
path=/sys/bus/pci/devices/${1}/driver/unbind
if ! [ -w ${path} ]; then
echo "${prog}: Device ${1} not bound"
return 1
fi
echo -n ${1} >${path}
}
pcibind() {
local path
path=/sys/bus/pci/drivers/${2}/bind
if ! [ -w ${path} ]; then
echo "${prog}: Driver ${2} not found"
return 1
fi
echo ${1} >${path}
}
pciunbind ${BDF}
pcibind ${BDF} ${DRIVER}
sleep 1
dhclient
```
5. Configure your DHCP server so your testbench gets static IP and connect your machine to your local network. You should ensure that your testbench can reach the Internet.
6. You'll need to run the above script on every startup. To automate this save the following systemd service `/etc/systemd/system/dom0-network-direct.service`
```
[Unit]
Description=Connect network to dom0
[Service]
Type=oneshot
ExecStart=/home/user/bin/dom0_network.sh
[Install]
WantedBy=multi-user.target
```
7. Then, enable and start the SSH Server and the script on boot:
```bash
sudo systemctl enable sshd
sudo systemctl start sshd
sudo systemctl enable dom0-network-direct
sudo systemctl start dom0-network-direct
```
> **Note:** If you want to install additional software in dom0 and your only network card was assigned to dom0, then _instead_ of the usual `sudo qubes-dom0-update <PACKAGE>` now you run `sudo dnf --setopt=reposdir=/etc/yum.repos.d install <PACKAGE>`.
### Install Tests and Their Dependencies
A regular Qubes installation isn't ready to run the full suite of tests. For example, in order to run the [Split GPG tests](https://github.com/QubesOS/qubes-app-linux-split-gpg/blob/4bc201bb70c011119eed19df25dc5b46120d04ed/tests/splitgpg/tests.py) you need to have the `qubes-gpg-split-tests` package installed in your app qubes.
Because of the above reason, some additional configurations need to be done to your testing environment. This can be done in an automated manner with the help of the [Salt](/doc/salt) configuration that provisions the [automated testing environment](/doc/automated-tests/).
The following commands should work for you, but do keep in mind that the provisioning scripts are designed for the [openQA environment](https://openqa.qubes-os.org/) and not your specific local testing system. Run the following in `dom0`:
```bash
# For future reference the following commands are an adaptation of
# https://github.com/marmarek/openqa-tests-qubesos/blob/master/tests/update.pm
# Install git
sudo qubes-dom0-update git || sudo dnf --setopt=reposdir=/etc/yum.repos.d install git
# Download the openQA automated testing environment Salt configuration
git clone https://github.com/marmarek/openqa-tests-qubesos/
cd openqa-tests-qubesos/extra-files
sudo cp -a system-tests/ /srv/salt/
sudo qubesctl top.enable system-tests
# Install the same configuration as the one in openQA
QUBES_VERSION=4.1
PILLAR_DIR=/srv/pillar/base/update
sudo mkdir -p $PILLAR_DIR
printf 'update:\n qubes_ver: '$QUBES_VERSION'\n' | sudo tee $PILLAR_DIR/init.sls
printf "base:\n '*':\n - update\n" | sudo tee $PILLAR_DIR/init.top
sudo qubesctl top.enable update pillar=True
# Apply states to dom0 and VMs
# NOTE: These commands can take several minutes (if not more) without showing output
sudo qubesctl --show-output state.highstate
sudo qubesctl --max-concurrency=2 --skip-dom0 --templates --show-output state.highstate
```
## Development VM
### SSH
Arrange firewall so you can reach the testbench from your `qubes-dev` VM. Generate SSH key in `qubes-dev`:
~~~
ssh-keygen -t ecdsa -b 521
~~~
Add the following section in `.ssh/config` in `qubes-dev`:
~~~
Host testbench
# substitute username in testbench
User user
# substitute address of your testbench
HostName 192.168.123.45
~~~
#### Passwordless SSH Login
To log to your testbench without entering password every time, copy your newly generated public key (`id_ecdsa.pub`) to `~/.ssh/authorized_keys` on your testbench. You can do this easily by running this command on `qubes-dev`: `ssh-copy-id -i ~/.ssh/id_ecdsa.pub user@192.168.123.45` (substituting with the actual username address of your testbench).
### Scripting
This step is optional, but very helpful. Put these scripts somewhere in your `${PATH}`, like `/usr/local/bin`.
`qtb-runtests`:
```bash
#!/bin/sh
ssh testbench python -m qubes.tests.run
```
`qtb-install`:
```bash
#!/bin/sh
TMPDIR=/tmp/qtb-rpms
if [ $# -eq 0 ]; then
echo "usage: $(basename $0) <rpmfile> ..."
exit 2
fi
set -e
ssh testbench mkdir -p "${TMPDIR}"
scp "${@}" testbench:"${TMPDIR}" || echo "check if you have 'scp' installed on your testbench"
while [ $# -gt 0 ]; do
ssh testbench sudo rpm -i --replacepkgs --replacefiles "${TMPDIR}/$(basename ${1})"
shift
done
```
`qtb-iterate`:
```bash
#!/bin/sh
set -e
# substitute path to your builder installation
pushd ${HOME}/builder >/dev/null
# the following are needed only if you have sources outside builder
#rm -rf qubes-src/core-admin
#make COMPONENTS=core-admin get-sources
make core-admin
qtb-install qubes-src/core-admin/rpm/x86_64/qubes-core-dom0-*.rpm
qtb-runtests
```
### Hooking git
I (woju) have those two git hooks. They ensure tests are passing (or are marked as expected failure) when committing and pushing. For committing it is only possible to run tests that may be executed from git repo (even if the rest were available, I probably wouldn't want to do that). For pushing, I also install RPM and run tests on testbench.
`core-admin/.git/hooks/pre-commit`: (you may retain also the default hook, here omitted for readability)
```bash
#!/bin/sh
set -e
python -c "import sys, qubes.tests.run; sys.exit(not qubes.tests.run.main())"
```
`core-admin/.git/hooks/pre-push`:
```bash
#!/bin/sh
exec qtb-iterate
```

306
developer/debugging/test-bench.rst Normal file
View File

@ -0,0 +1,306 @@
==========================
How to set up a test bench
==========================
This guide shows how to set up simple test bench that automatically test
your code you’re about to push. It is written especially for ``core3``
branch of ``core-admin.git`` repo, but some ideas are universal.
We will set up a spare machine (bare metal, not a virtual) that will be
hosting our experimental Dom0. We will communicate with it via Ethernet
and SSH. This tutorial assumes you are familiar with
:doc:`QubesBuilder </developer/building/qubes-builder>` and you have it set up and
running flawlessly.
**Notice:** This setup intentionally weakens some security properties
in the testing system. So make sure you understand the risks and use
exclusively for testing.
Setting up the Machine
----------------------
Install ISO
^^^^^^^^^^^
First, do a clean install from the ``.iso`` :doc:`you built </developer/building/qubes-iso-building>` or grabbed elsewhere (for example
`here <https://forum.qubes-os.org/t/qubesos-4-1-alpha-signed-weekly-builds/3601>`__).
Enabling Network Access in Dom0
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Internet access is intentionally disabled by default in dom0. But to
ease the deployment process we will give it access. The following steps
should be done in ``dom0``.
**Note:** the following assume you have only one network card. If you
have two, pick one and leave the other attached to ``sys-net``.
1. Remove the network card (PCI device) from ``sys-net``
2. Restart your computer (for the removal to take effect)
3. Install ``dhcp-client`` and ``openssh-server`` on your testbench’s
dom0.
4. Save the following script in ``/home/user/bin/dom0_network.sh`` and
make it executable. It should enable your network card in dom0. *Be sure to adjust the script’s variables to suit your needs.*
.. code:: bash
#!/bin/sh
# adjust this for your NIC (run lspci)
BDF=0000:02:00.0
# adjust this for your network driver
DRIVER=e1000e
prog=$(basename $0)
pciunbind() {
local path
path=/sys/bus/pci/devices/${1}/driver/unbind
if ! [ -w ${path} ]; then
echo "${prog}: Device ${1} not bound"
return 1
fi
echo -n ${1} >${path}
}
pcibind() {
local path
path=/sys/bus/pci/drivers/${2}/bind
if ! [ -w ${path} ]; then
echo "${prog}: Driver ${2} not found"
return 1
fi
echo ${1} >${path}
}
pciunbind ${BDF}
pcibind ${BDF} ${DRIVER}
sleep 1
dhclient
5. Configure your DHCP server so your testbench gets static IP and
connect your machine to your local network. You should ensure that
your testbench can reach the Internet.
6. You’ll need to run the above script on every startup. To automate
this save the following systemd service
``/etc/systemd/system/dom0-network-direct.service``
.. code:: bash
[Unit]
Description=Connect network to dom0
[Service]
Type=oneshot
ExecStart=/home/user/bin/dom0_network.sh
[Install]
WantedBy=multi-user.target
7. Then, enable and start the SSH Server and the script on boot:
.. code:: bash
sudo systemctl enable sshd
sudo systemctl start sshd
sudo systemctl enable dom0-network-direct
sudo systemctl start dom0-network-direct
**Note:** If you want to install additional software in dom0 and your
only network card was assigned to dom0, then *instead* of the usual
``sudo qubes-dom0-update <PACKAGE>`` now you run
``sudo dnf --setopt=reposdir=/etc/yum.repos.d install <PACKAGE>``.
Install Tests and Their Dependencies
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
A regular Qubes installation isn’t ready to run the full suite of tests.
For example, in order to run the `Split GPG tests <https://github.com/QubesOS/qubes-app-linux-split-gpg/blob/4bc201bb70c011119eed19df25dc5b46120d04ed/tests/splitgpg/tests.py>`__
you need to have the ``qubes-gpg-split-tests`` package installed in your
app qubes.
Because of the above reason, some additional configurations need to be
done to your testing environment. This can be done in an automated
manner with the help of the :doc:`Salt </user/advanced-topics/salt>` configuration that
provisions the :doc:`automated testing environment </developer/debugging/automated-tests>`.
The following commands should work for you, but do keep in mind that the
provisioning scripts are designed for the `openQA environment <https://openqa.qubes-os.org/>`__ and not your specific
local testing system. Run the following in ``dom0``:
.. code:: bash
# For future reference the following commands are an adaptation of
# https://github.com/marmarek/openqa-tests-qubesos/blob/master/tests/update.pm
# Install git
sudo qubes-dom0-update git || sudo dnf --setopt=reposdir=/etc/yum.repos.d install git
# Download the openQA automated testing environment Salt configuration
git clone https://github.com/marmarek/openqa-tests-qubesos/
cd openqa-tests-qubesos/extra-files
sudo cp -a system-tests/ /srv/salt/
sudo qubesctl top.enable system-tests
# Install the same configuration as the one in openQA
QUBES_VERSION=4.1
PILLAR_DIR=/srv/pillar/base/update
sudo mkdir -p $PILLAR_DIR
printf 'update:\n qubes_ver: '$QUBES_VERSION'\n' | sudo tee $PILLAR_DIR/init.sls
printf "base:\n '*':\n - update\n" | sudo tee $PILLAR_DIR/init.top
sudo qubesctl top.enable update pillar=True
# Apply states to dom0 and VMs
# NOTE: These commands can take several minutes (if not more) without showing output
sudo qubesctl --show-output state.highstate
sudo qubesctl --max-concurrency=2 --skip-dom0 --templates --show-output state.highstate
Development VM
--------------
SSH
^^^
Arrange firewall so you can reach the testbench from your ``qubes-dev``
VM. Generate SSH key in ``qubes-dev``:
.. code:: bash
ssh-keygen -t ecdsa -b 521
Add the following section in ``.ssh/config`` in ``qubes-dev``:
.. code:: bash
Host testbench
# substitute username in testbench
User user
# substitute address of your testbench
HostName 192.168.123.45
Passwordless SSH Login
^^^^^^^^^^^^^^^^^^^^^^
To log to your testbench without entering password every time, copy your
newly generated public key (``id_ecdsa.pub``) to
``~/.ssh/authorized_keys`` on your testbench. You can do this easily by
running this command on ``qubes-dev``:
``ssh-copy-id -i ~/.ssh/id_ecdsa.pub user@192.168.123.45`` (substituting
with the actual username address of your testbench).
Scripting
^^^^^^^^^
This step is optional, but very helpful. Put these scripts somewhere in
your ``${PATH}``, like ``/usr/local/bin``.
``qtb-runtests``:
.. code:: bash
#!/bin/sh
ssh testbench python -m qubes.tests.run
``qtb-install``:
.. code:: bash
#!/bin/sh
TMPDIR=/tmp/qtb-rpms
if [ $# -eq 0 ]; then
echo "usage: $(basename $0) <rpmfile> ..."
exit 2
fi
set -e
ssh testbench mkdir -p "${TMPDIR}"
scp "${@}" testbench:"${TMPDIR}" || echo "check if you have 'scp' installed on your testbench"
while [ $# -gt 0 ]; do
ssh testbench sudo rpm -i --replacepkgs --replacefiles "${TMPDIR}/$(basename ${1})"
shift
done
``qtb-iterate``:
.. code:: bash
#!/bin/sh
set -e
# substitute path to your builder installation
pushd ${HOME}/builder >/dev/null
# the following are needed only if you have sources outside builder
#rm -rf qubes-src/core-admin
#make COMPONENTS=core-admin get-sources
make core-admin
qtb-install qubes-src/core-admin/rpm/x86_64/qubes-core-dom0-*.rpm
qtb-runtests
Hooking git
^^^^^^^^^^^
I (woju) have those two git hooks. They ensure tests are passing (or are
marked as expected failure) when committing and pushing. For committing
it is only possible to run tests that may be executed from git repo
(even if the rest were available, I probably wouldn’t want to do that).
For pushing, I also install RPM and run tests on testbench.
``core-admin/.git/hooks/pre-commit``: (you may retain also the default
hook, here omitted for readability)
.. code:: bash
#!/bin/sh
set -e
python -c "import sys, qubes.tests.run; sys.exit(not qubes.tests.run.main())"
``core-admin/.git/hooks/pre-push``:
.. code:: bash
#!/bin/sh
exec qtb-iterate

225
developer/debugging/vm-interface.md
View File

@ -1,225 +0,0 @@
---
lang: en
layout: doc
permalink: /doc/vm-interface/
redirect_from:
- /en/doc/vm-interface/
- /doc/VMInterface/
- /doc/SystemDoc/VMInterface/
- /wiki/SystemDoc/VMInterface/
ref: 47
title: Qube configuration interface
---
Qubes VM have some settings set by dom0 based on VM settings. There are multiple configuration channels, which includes:
- QubesDB
- XenStore (in Qubes 2, data the same as in QubesDB, keys without leading `/`)
- Qubes RPC (called at VM startup, or when configuration changed)
- GUI protocol
## QubesDB
### Keys exposed by dom0 to VM
- `/qubes-vm-type` - VM type, the same as `type` field in `qvm-prefs`. One of `AppVM`, `ProxyVM`, `NetVM`, `TemplateVM`, `HVM`, `TemplateHVM`
- `/qubes-vm-updatable` - flag whether VM is updatable (whether changes in root.img will survive VM restart). One of `True`, `False`
- `/qubes-vm-persistence` - what data do persist between VM restarts:
- `full` - all disks
- `rw-only` - only `/rw` disk
- `none` - none
- `/qubes-timezone` - name of timezone based on dom0 timezone. For example `Europe/Warsaw`
- `/qubes-keyboard` (deprecated in R4.1) - keyboard layout based on dom0 layout. Its syntax is suitable for `xkbcomp` command (after expanding escape sequences like `\n` or `\t`). This is meant only as some default value, VM can ignore this option and choose its own keyboard layout (this is what keyboard setting from Qubes Manager does). This entry is created as part of gui-daemon initialization (so not available when gui-daemon disabled, or not started yet).
- `/keyboard-layout` - keyboard layout based on GuiVM layout. Its syntax can be `layout+variant+options`, `layout+variant`, `layout++options` or simply `layout`. For example, `fr+oss`, `pl++compose:caps` or `fr`. This is meant only as some default value, VM can ignore this option and choose its own keyboard layout (this is what keyboard setting from Qubes Manager does).
- `/qubes-debug-mode` - flag whether VM has debug mode enabled (qvm-prefs setting). One of `1`, `0`
- `/qubes-service/SERVICE_NAME` - subtree for VM services controlled from dom0 (using the `qvm-service` command or Qubes Manager). One of `1`, `0`. Note that not every service will be listed here, if entry is missing, it means "use VM default". A list of currently supported services is in the `qvm-service` man page.
- `/qubes-netmask` - network mask (only when VM has netvm set); currently hardcoded "255.255.255.0"
- `/qubes-ip` - IP address for this VM (only when VM has netvm set)
- `/qubes-gateway` - default gateway IP (only when VM has netvm set); VM should add host route to this address directly via eth0 (or whatever default interface name is)
- `/qubes-primary-dns` - primary DNS address (only when VM has netvm set)
- `/qubes-secondary-dns` - secondary DNS address (only when VM has netvm set)
- `/qubes-netvm-gateway` - same as `qubes-gateway` in connected VMs (only when VM serves as network backend - ProxyVM and NetVM)
- `/qubes-netvm-netmask` - same as `qubes-netmask` in connected VMs (only when VM serves as network backend - ProxyVM and NetVM)
- `/qubes-netvm-network` - network address (only when VM serves as network backend - ProxyVM and NetVM); can be also calculated from qubes-netvm-gateway and qubes-netvm-netmask
- `/qubes-netvm-primary-dns` - same as `qubes-primary-dns` in connected VMs (only when VM serves as network backend - ProxyVM and NetVM); traffic sent to this IP on port 53 should be redirected to primary DNS server
- `/qubes-netvm-secondary-dns` - same as `qubes-secondary-dns` in connected VMs (only when VM serves as network backend - ProxyVM and NetVM); traffic sent to this IP on port 53 should be redirected to secondary DNS server
- `/guivm-windows-prefix` - title prefix for any window not originating from another qube. This means windows of applications running in GuiVM itself
#### Firewall rules in 3.x
QubesDB is also used to configure firewall in ProxyVMs. Rules are stored in
separate key for each target VM. Entries:
- `/qubes-iptables` - control entry - dom0 writing `reload` here signals `qubes-firewall` service to reload rules
- `/qubes-iptables-header` - rules not related to any particular VM, should be applied before domains rules
- `/qubes-iptables-domainrules/NNN` - rules for domain `NNN` (arbitrary number)
in `iptables-save` format. Rules are self-contained - fill `FORWARD` iptables
chain and contains all required matches (source IP address etc), as well as
final default action (`DROP`/`ACCEPT`)
VM after applying rules may signal some error, writing a message to
`/qubes-iptables-error` key. This does not exclude any other way of
communicating problems - like a popup.
#### Firewall rules in 4.x
QubesDB is also used to configure firewall in ProxyVMs. Each rule is stored as
a separate entry, grouped on target VM:
- `/qubes-firewall/SOURCE_IP` - base tree under which rules are placed. All
rules there should be applied to filter traffic coming from `SOURCE_IP`. This
can be either IPv4 or IPv6 address.
Dom0 will do an empty write to this top level entry after finishing rules
update, so VM can setup a watch here to trigger rules reload.
- `/qubes-firewall/SOURCE_IP/policy` - default action if no rule matches:
`drop` or `accept`.
- `/qubes-firewall/SOURCE_IP/NNNN` - rule number `NNNN` - decimal number,
padded with zeros. Se below for rule format. All the rules should be
applied in order of rules implied by those numbers. Note that QubesDB
itself does not impose any ordering (you need to sort the rules after
retrieving them). The first rule has number `0000`.
Each rule is a single QubesDB entry, consisting of pairs `key=value` separated
by space. QubesDB enforces limit on a single entry length - 3072 bytes.
Possible options for a single rule:
- `action`, values: `accept`, `drop`; this is present in every rule
- `dst4`, value: destination IPv4 address with a mask; for example: `192.168.0.0/24`
- `dst6`, value: destination IPv6 address with a mask; for example: `2000::/3`
- `dsthost`, value: DNS hostname of destination host
- `proto`, values: `tcp`, `udp`, `icmp`
- `specialtarget`, value: One of predefined target, currently defined values:
- `dns` - such option should match DNS traffic to default DNS server (but
not any DNS server), on both TCP and UDP
- `dstports`, value: destination ports range separated with `-`, valid only
together with `proto=tcp` or `proto=udp`; for example `1-1024`, `80-80`
- `icmptype`, value: numeric (decimal) icmp message type, for example `8` for
echo request, valid only together with `proto=icmp`
- `dpi`, value: Deep Packet Inspection protocol (like: HTTP, SSL, SMB, SSH, SMTP) or the default 'NO' as no DPI, only packet filtering
Options must appear in the rule in the order listed above. Duplicated options
are forbidden.
A rule matches only when all predicates match. Only one of `dst4`, `dst6` or
`dsthost` can be used in a single rule.
If tool applying firewall encounters any parse error (unknown option, invalid
value, duplicated option, etc), it should drop all the traffic coming from that `SOURCE_IP`,
regardless of properly parsed rules.
Example valid rules:
- `action=accept dst4=8.8.8.8 proto=udp dstports=53-53`
- `action=drop dst6=2a00:1450:4000::/37 proto=tcp`
- `action=accept specialtarget=dns`
- `action=drop proto=tcp specialtarget=dns` - drop DNS queries sent using TCP
- `action=drop`
### Keys set by VM for passing info to dom0
- `memory/meminfo` (**xenstore**) - used memory (updated by qubes-meminfo-writer), input information for qmemman;
- Qubes 3.x format: 6 lines (EOL encoded as `\n`), each in format "FIELD: VALUE kB"; fields: `MemTotal`, `MemFree`, `Buffers`, `Cached`, `SwapTotal`, `SwapFree`; meaning the same as in `/proc/meminfo` in Linux.
- Qubes 4.0+ format: used memory size in the VM, in kbytes
- `/qubes-block-devices` - list of block devices exposed by this VM, each device (subdirectory) should be named in a way that VM can attach the device based on it. Each should contain these entries:
- `desc` - device description (ASCII text)
- `size` - device size in bytes
- `mode` - default connection mode; `r` for read-only, `w` for read-write
- `/qubes-usb-devices` - list of USB devices exposed by this VM, each device (subdirectory) should contain:
- `desc` - device description (ASCII text)
- `usb-ver` - USB version (1, 2 or 3)
## Qubes RPC
Services called by dom0 to provide some VM configuration:
- `qubes.SetMonitorLayout` - provide list of monitors, one per line. Each line contains four numbers: `width height X Y width_mm height_mm` (physical dimensions - `width_mm` and `height_mm` - are optional)
- `qubes.WaitForSession` - called to wait for full VM startup
- `qubes.GetAppmenus` - receive appmenus from given VM (template); TODO: describe format here
- `qubes.GetImageRGBA` - receive image/application icon. Protocol:
1. Caller sends name of requested icon. This can be one of:
* `xdgicon:NAME` - search for NAME in standard icons theme
* `-` - get icon data from stdin (the caller), can be prefixed with format name, for example `png:-`
* file name
2. The service responds with image dimensions: width and height as
decimal numbers, separated with space and with EOL marker at the and; then
image data in RGBA format (32 bits per pixel)
- `qubes.SetDateTime` - set VM time, called periodically by dom0 (can be
triggered manually from dom0 by calling `qvm-sync-clock`). The service
receives one line at stdin - time in format of `date -u -Iseconds`, for
example `2015-07-31T16:10:43+0000`.
- `qubes.SetGuiMode` - called in HVM to switch between fullscreen and seamless
GUI mode. The service receives a single word on stdin - either `FULLSCREEN`
or `SEAMLESS`
- `qubes.ResizeDisk` - called to inform that underlying disk was resized.
Name of disk image is passed on standard input (`root`, `private`, `volatile`,
or other). This is used starting with Qubes 4.0.
Other Qrexec services installed by default:
- `qubes.Backup` - store Qubes backup. The service receives location chosen by
the user (one line, terminated by `\n`), the backup archive ([description of
backup format](/doc/BackupEmergencyRestoreV2/))
- `qubes.DetachPciDevice` - service called in reaction to `qvm-pci -d` call on
running VM. The service receives one word - BDF of device to detach. When the
service call ends, the device will be detached
- `qubes.Filecopy` - receive some files from other VM. Files sent in [qfile format](/doc/qfilecopy/)
- `qubes.OpenInVM` - open a file in called VM. Service receives a single file on stdin (in
[qfile format](/doc/qfilecopy/). After a file viewer/editor is terminated, if
the file was modified, can be sent back (just raw content, without any
headers); otherwise service should just terminate without sending anything.
This service is used by both `qvm-open-in-vm` and `qvm-open-in-dvm` tools. When
called in DispVM, service termination will trigger DispVM cleanup.
- `qubes.Restore` - retrieve Qubes backup. The service receives backup location
entered by the user (one line, terminated by `\n`), then should output backup
archive in [qfile format](/doc/qfilecopy/) (core-agent-linux component contains
`tar2qfile` utility to do the conversion)
- `qubes.SelectDirectory`, `qubes.SelectFile` - services which should show
file/directory selection dialog and return (to stdout) a single line
containing selected path, or nothing in the case of cancellation
- `qubes.SuspendPre` - service called in every VM with PCI device attached just
before system suspend
- `qubes.SuspendPost` - service called in every VM with PCI device attached just
after system resume
- `qubes.SyncNtpClock` - service called to trigger network time synchronization.
Service should synchronize local VM time and terminate when done.
- `qubes.WindowIconUpdater` - service called by VM to send icons of individual
windows. The protocol there is simple one direction stream: VM sends window ID
followed by icon in `qubes.GetImageRGBA` format, then next window ID etc. VM
can send icon for the same window multiple times to replace previous one (for
example for animated icons)
- `qubes.VMShell` - call any command in the VM; the command(s) is passed one per line
- `qubes.VMShell+WaitForSession` waits for full VM startup first
- `qubes.VMExec` - call any command in the VM, without using shell, the command
needs to be passed as argument and encoded as follows:
- the executable name and arguments are separated by `+`
- everything except alphanumeric characters, `.` and `_` needs to be
escaped
- bytes are escaped as `-HH` (where `HH` is hex code, capital letters only)
- `-` itself can be escaped as `--`
- example: to run `ls -a /home/user`, use
`qubes.VMExec+ls+--a+-2Fhome-2Fuser`
- `qubes.VMExecGUI` - a variant of `qubes.VMExec` that waits for full VM
startup first
Services called in GuiVM:
- `policy.Ask`, `policy.Notify` - confirmation prompt and notifications for
Qubes RPC calls, see [qrexec-policy implementation](/doc/qrexec-internals/#qrexec-policy-implementation)
for a detailed description.
Currently Qubes still calls few tools in VM directly, not using service
abstraction. This will change in the future. Those tools are:
- `/usr/lib/qubes/qubes-download-dom0-updates.sh` - script to download updates (or new packages to be installed) for dom0 (`qubes-dom0-update` tool)
- `date -u -Iseconds` - called directly to retrieve time after calling `qubes.SyncNtpClock` service (`qvm-sync-clock` tool)
- `nm-online -x` - called before `qubes.SyncNtpClock` service call by `qvm-sync-clock` tool
- `resize2fs` - called to resize filesystem on /rw partition by `qvm-grow-private` tool
- `gpk-update-viewer` - called by Qubes Manager to display available updates in a TemplateVM
- `systemctl start qubes-update-check.timer` (and similarly stop) - called when enabling/disabling updates checking in given VM (`qubes-update-check` [qvm-service](/doc/qubes-service/))
Additionally, automatic tests extensively run various commands directly in VMs. We do not plan to change that.
## GUI protocol
GUI initialization includes passing the whole screen dimensions from dom0 to VM. This will most likely be overwritten by qubes.SetMonitorLayout Qubes RPC call.

441
developer/debugging/vm-interface.rst Normal file
View File

@ -0,0 +1,441 @@
============================
Qube configuration interface
============================
Qubes VM have some settings set by dom0 based on VM settings. There are
multiple configuration channels, which includes:
- QubesDB
- XenStore (in Qubes 2, data the same as in QubesDB, keys without
leading ``/``)
- Qubes RPC (called at VM startup, or when configuration changed)
- GUI protocol
QubesDB
-------
Keys exposed by dom0 to VM
^^^^^^^^^^^^^^^^^^^^^^^^^^
- ``/qubes-vm-type`` - VM type, the same as ``type`` field in
``qvm-prefs``. One of ``AppVM``, ``ProxyVM``, ``NetVM``,
``TemplateVM``, ``HVM``, ``TemplateHVM``
- ``/qubes-vm-updatable`` - flag whether VM is updatable (whether
changes in root.img will survive VM restart). One of ``True``,
``False``
- ``/qubes-vm-persistence`` - what data do persist between VM restarts:
- ``full`` - all disks
- ``rw-only`` - only ``/rw`` disk
- ``none`` - none
- ``/qubes-timezone`` - name of timezone based on dom0 timezone. For
example ``Europe/Warsaw``
- ``/qubes-keyboard`` (deprecated in R4.1) - keyboard layout based on
dom0 layout. Its syntax is suitable for ``xkbcomp`` command (after
expanding escape sequences like ``\n`` or ``\t``). This is meant only
as some default value, VM can ignore this option and choose its own
keyboard layout (this is what keyboard setting from Qubes Manager
does). This entry is created as part of gui-daemon initialization (so
not available when gui-daemon disabled, or not started yet).
- ``/keyboard-layout`` - keyboard layout based on GuiVM layout. Its
syntax can be ``layout+variant+options``, ``layout+variant``,
``layout++options`` or simply ``layout``. For example, ``fr+oss``,
``pl++compose:caps`` or ``fr``. This is meant only as some default
value, VM can ignore this option and choose its own keyboard layout
(this is what keyboard setting from Qubes Manager does).
- ``/qubes-debug-mode`` - flag whether VM has debug mode enabled
(qvm-prefs setting). One of ``1``, ``0``
- ``/qubes-service/SERVICE_NAME`` - subtree for VM services controlled
from dom0 (using the ``qvm-service`` command or Qubes Manager). One
of ``1``, ``0``. Note that not every service will be listed here, if
entry is missing, it means “use VM default”. A list of currently
supported services is in the ``qvm-service`` man page.
- ``/qubes-netmask`` - network mask (only when VM has netvm set);
currently hardcoded “255.255.255.0”
- ``/qubes-ip`` - IP address for this VM (only when VM has netvm set)
- ``/qubes-gateway`` - default gateway IP (only when VM has netvm set);
VM should add host route to this address directly via eth0 (or
whatever default interface name is)
- ``/qubes-primary-dns`` - primary DNS address (only when VM has netvm
set)
- ``/qubes-secondary-dns`` - secondary DNS address (only when VM has
netvm set)
- ``/qubes-netvm-gateway`` - same as ``qubes-gateway`` in connected VMs
(only when VM serves as network backend - ProxyVM and NetVM)
- ``/qubes-netvm-netmask`` - same as ``qubes-netmask`` in connected VMs
(only when VM serves as network backend - ProxyVM and NetVM)
- ``/qubes-netvm-network`` - network address (only when VM serves as
network backend - ProxyVM and NetVM); can be also calculated from
qubes-netvm-gateway and qubes-netvm-netmask
- ``/qubes-netvm-primary-dns`` - same as ``qubes-primary-dns`` in
connected VMs (only when VM serves as network backend - ProxyVM and
NetVM); traffic sent to this IP on port 53 should be redirected to
primary DNS server
- ``/qubes-netvm-secondary-dns`` - same as ``qubes-secondary-dns`` in
connected VMs (only when VM serves as network backend - ProxyVM and
NetVM); traffic sent to this IP on port 53 should be redirected to
secondary DNS server
- ``/guivm-windows-prefix`` - title prefix for any window not
originating from another qube. This means windows of applications
running in GuiVM itself
Firewall rules in 3.x
^^^^^^^^^^^^^^^^^^^^^
QubesDB is also used to configure firewall in ProxyVMs. Rules are stored
in separate key for each target VM. Entries:
- ``/qubes-iptables`` - control entry - dom0 writing ``reload`` here
signals ``qubes-firewall`` service to reload rules
- ``/qubes-iptables-header`` - rules not related to any particular VM,
should be applied before domains rules
- ``/qubes-iptables-domainrules/NNN`` - rules for domain ``NNN``
(arbitrary number) in ``iptables-save`` format. Rules are
self-contained - fill ``FORWARD`` iptables chain and contains all
required matches (source IP address etc), as well as final default
action (``DROP``/``ACCEPT``)
VM after applying rules may signal some error, writing a message to
``/qubes-iptables-error`` key. This does not exclude any other way of
communicating problems - like a popup.
Firewall rules in 4.x
^^^^^^^^^^^^^^^^^^^^^
QubesDB is also used to configure firewall in ProxyVMs. Each rule is
stored as a separate entry, grouped on target VM:
- ``/qubes-firewall/SOURCE_IP`` - base tree under which rules are
placed. All rules there should be applied to filter traffic coming
from ``SOURCE_IP``. This can be either IPv4 or IPv6 address. Dom0
will do an empty write to this top level entry after finishing rules
update, so VM can setup a watch here to trigger rules reload.
- ``/qubes-firewall/SOURCE_IP/policy`` - default action if no rule
matches: ``drop`` or ``accept``.
- ``/qubes-firewall/SOURCE_IP/NNNN`` - rule number ``NNNN`` - decimal
number, padded with zeros. Se below for rule format. All the rules
should be applied in order of rules implied by those numbers. Note
that QubesDB itself does not impose any ordering (you need to sort
the rules after retrieving them). The first rule has number ``0000``.
Each rule is a single QubesDB entry, consisting of pairs ``key=value``
separated by space. QubesDB enforces limit on a single entry length -
3072 bytes. Possible options for a single rule:
- ``action``, values: ``accept``, ``drop``; this is present in every
rule
- ``dst4``, value: destination IPv4 address with a mask; for example:
``192.168.0.0/24``
- ``dst6``, value: destination IPv6 address with a mask; for example:
``2000::/3``
- ``dsthost``, value: DNS hostname of destination host
- ``proto``, values: ``tcp``, ``udp``, ``icmp``
- ``specialtarget``, value: One of predefined target, currently defined
values:
- ``dns`` - such option should match DNS traffic to default DNS
server (but not any DNS server), on both TCP and UDP
- ``dstports``, value: destination ports range separated with ``-``,
valid only together with ``proto=tcp`` or ``proto=udp``; for example
``1-1024``, ``80-80``
- ``icmptype``, value: numeric (decimal) icmp message type, for example
``8`` for echo request, valid only together with ``proto=icmp``
- ``dpi``, value: Deep Packet Inspection protocol (like: HTTP, SSL,
SMB, SSH, SMTP) or the default ‘NO’ as no DPI, only packet filtering
Options must appear in the rule in the order listed above. Duplicated
options are forbidden.
A rule matches only when all predicates match. Only one of ``dst4``,
``dst6`` or ``dsthost`` can be used in a single rule.
If tool applying firewall encounters any parse error (unknown option,
invalid value, duplicated option, etc), it should drop all the traffic
coming from that ``SOURCE_IP``, regardless of properly parsed rules.
Example valid rules:
- ``action=accept dst4=8.8.8.8 proto=udp dstports=53-53``
- ``action=drop dst6=2a00:1450:4000::/37 proto=tcp``
- ``action=accept specialtarget=dns``
- ``action=drop proto=tcp specialtarget=dns`` - drop DNS queries sent
using TCP
- ``action=drop``
Keys set by VM for passing info to dom0
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
- ``memory/meminfo`` (**xenstore**) - used memory (updated by
qubes-meminfo-writer), input information for qmemman;
- Qubes 3.x format: 6 lines (EOL encoded as ``\n``), each in format
“FIELD: VALUE kB”; fields: ``MemTotal``, ``MemFree``, ``Buffers``,
``Cached``, ``SwapTotal``, ``SwapFree``; meaning the same as in
``/proc/meminfo`` in Linux.
- Qubes 4.0+ format: used memory size in the VM, in kbytes
- ``/qubes-block-devices`` - list of block devices exposed by this VM,
each device (subdirectory) should be named in a way that VM can
attach the device based on it. Each should contain these entries:
- ``desc`` - device description (ASCII text)
- ``size`` - device size in bytes
- ``mode`` - default connection mode; ``r`` for read-only, ``w`` for
read-write
- ``/qubes-usb-devices`` - list of USB devices exposed by this VM, each
device (subdirectory) should contain:
- ``desc`` - device description (ASCII text)
- ``usb-ver`` - USB version (1, 2 or 3)
Qubes RPC
---------
Services called by dom0 to provide some VM configuration:
- ``qubes.SetMonitorLayout`` - provide list of monitors, one per line.
Each line contains four numbers:
``width height X Y width_mm height_mm`` (physical dimensions -
``width_mm`` and ``height_mm`` - are optional)
- ``qubes.WaitForSession`` - called to wait for full VM startup
- ``qubes.GetAppmenus`` - receive appmenus from given VM (template);
TODO: describe format here
- ``qubes.GetImageRGBA`` - receive image/application icon. Protocol:
1. Caller sends name of requested icon. This can be one of:
- ``xdgicon:NAME`` - search for NAME in standard icons theme
- ``-`` - get icon data from stdin (the caller), can be prefixed
with format name, for example ``png:-``
- file name
2. The service responds with image dimensions: width and height as
decimal numbers, separated with space and with EOL marker at the
and; then image data in RGBA format (32 bits per pixel)
- ``qubes.SetDateTime`` - set VM time, called periodically by dom0 (can
be triggered manually from dom0 by calling ``qvm-sync-clock``). The
service receives one line at stdin - time in format of
``date -u -Iseconds``, for example ``2015-07-31T16:10:43+0000``.
- ``qubes.SetGuiMode`` - called in HVM to switch between fullscreen and
seamless GUI mode. The service receives a single word on stdin -
either ``FULLSCREEN`` or ``SEAMLESS``
- ``qubes.ResizeDisk`` - called to inform that underlying disk was
resized. Name of disk image is passed on standard input (``root``,
``private``, ``volatile``, or other). This is used starting with
Qubes 4.0.
Other Qrexec services installed by default:
- ``qubes.Backup`` - store Qubes backup. The service receives location
chosen by the user (one line, terminated by ``\n``), the backup
archive (:doc:`description of backup format </user/how-to-guides/backup-emergency-restore-v2>`)
- ``qubes.DetachPciDevice`` - service called in reaction to
``qvm-pci -d`` call on running VM. The service receives one word -
BDF of device to detach. When the service call ends, the device will
be detached
- ``qubes.Filecopy`` - receive some files from other VM. Files sent in
:doc:`qfile format </developer/services/qfilecopy>`
- ``qubes.OpenInVM`` - open a file in called VM. Service receives a
single file on stdin (in :doc:`qfile format </developer/services/qfilecopy>`. After a
file viewer/editor is terminated, if the file was modified, can be
sent back (just raw content, without any headers); otherwise service
should just terminate without sending anything. This service is used
by both ``qvm-open-in-vm`` and ``qvm-open-in-dvm`` tools. When called
in DispVM, service termination will trigger DispVM cleanup.
- ``qubes.Restore`` - retrieve Qubes backup. The service receives
backup location entered by the user (one line, terminated by ``\n``),
then should output backup archive in :doc:`qfile format </developer/services/qfilecopy>` (core-agent-linux component contains
``tar2qfile`` utility to do the conversion)
- ``qubes.SelectDirectory``, ``qubes.SelectFile`` - services which
should show file/directory selection dialog and return (to stdout) a
single line containing selected path, or nothing in the case of
cancellation
- ``qubes.SuspendPre`` - service called in every VM with PCI device
attached just before system suspend
- ``qubes.SuspendPost`` - service called in every VM with PCI device
attached just after system resume
- ``qubes.SyncNtpClock`` - service called to trigger network time
synchronization. Service should synchronize local VM time and
terminate when done.
- ``qubes.WindowIconUpdater`` - service called by VM to send icons of
individual windows. The protocol there is simple one direction
stream: VM sends window ID followed by icon in ``qubes.GetImageRGBA``
format, then next window ID etc. VM can send icon for the same window
multiple times to replace previous one (for example for animated
icons)
- ``qubes.VMShell`` - call any command in the VM; the command(s) is
passed one per line
- ``qubes.VMShell+WaitForSession`` waits for full VM startup first
- ``qubes.VMExec`` - call any command in the VM, without using shell,
the command needs to be passed as argument and encoded as follows:
- the executable name and arguments are separated by ``+``
- everything except alphanumeric characters, ``.`` and ``_`` needs
to be escaped
- bytes are escaped as ``-HH`` (where ``HH`` is hex code, capital
letters only)
- ``-`` itself can be escaped as ``--``
- example: to run ``ls -a /home/user``, use
``qubes.VMExec+ls+--a+-2Fhome-2Fuser``
- ``qubes.VMExecGUI`` - a variant of ``qubes.VMExec`` that waits for
full VM startup first
Services called in GuiVM:
- ``policy.Ask``, ``policy.Notify`` - confirmation prompt and
notifications for Qubes RPC calls, see :ref:`qrexec-policy implementation <developer/services/qrexec-internals:\`\`qrexec-policy\`\` implementation>`
for a detailed description.
Currently Qubes still calls few tools in VM directly, not using service
abstraction. This will change in the future. Those tools are:
- ``/usr/lib/qubes/qubes-download-dom0-updates.sh`` - script to
download updates (or new packages to be installed) for dom0
(``qubes-dom0-update`` tool)
- ``date -u -Iseconds`` - called directly to retrieve time after
calling ``qubes.SyncNtpClock`` service (``qvm-sync-clock`` tool)
- ``nm-online -x`` - called before ``qubes.SyncNtpClock`` service call
by ``qvm-sync-clock`` tool
- ``resize2fs`` - called to resize filesystem on /rw partition by
``qvm-grow-private`` tool
- ``gpk-update-viewer`` - called by Qubes Manager to display available
updates in a TemplateVM
- ``systemctl start qubes-update-check.timer`` (and similarly stop) -
called when enabling/disabling updates checking in given VM
(``qubes-update-check`` :doc:`qvm-service </user/advanced-topics/qubes-service>`)
Additionally, automatic tests extensively run various commands directly
in VMs. We do not plan to change that.
GUI protocol
------------
GUI initialization includes passing the whole screen dimensions from
dom0 to VM. This will most likely be overwritten by
qubes.SetMonitorLayout Qubes RPC call.

89
developer/debugging/windows-debugging.md
View File

@ -1,89 +0,0 @@
---
lang: en
layout: doc
permalink: /doc/windows-debugging/
redirect_from:
- /en/doc/windows-debugging/
- /doc/WindowsDebugging/
- /wiki/WindowsDebugging/
ref: 50
title: Windows debugging
---
Debugging Windows code can be tricky in a virtualized environment. The guide below assumes Qubes 4.2 and Windows 7 or later VMs.
User-mode debugging is usually straightforward if it can be done on one machine. Just duplicate your normal debugging environment in the VM.
Things get complicated if you need to perform kernel debugging or troubleshoot problems that only manifest on system boot, user logoff or similar. For that you need two Windows VMs: the *host* and the *target*. The *host* will contain the debugger, your source code and private symbols. The *target* will run the code being debugged. We will use kernel debugging over network which is supported from Windows 7 onwards. The main caveat is that Windows kernel supports only specific network adapters for this, and the default one in Qubes won't work.
## Important note
- Do not install Xen network PV drivers in the target VM. Network kernel debugging needs a specific type of NIC or it won't work, the network PV drivers interfere with that.
- If you have kernel debugging active when the Xen PV drivers are being installed, make sure to disable it before rebooting (`bcdedit /set debug off`). You can re-enable debugging after the reboot. The OS won't boot otherwise. I'm not sure what's the exact cause. I know that busparams for the debugging NIC change when PV drivers are installed (see later), but even changing that accordingly in the debug settings doesn't help -- so it's best to disable debug for this one reboot.
## Modifying the NIC of the target VM
You will need to create a custom libvirt config for the target VM. See [the documentation](https://dev.qubes-os.org/projects/core-admin/en/latest/libvirt.html) for overview of how libvirt templates work in Qubes. The following assumes the target VM is named `target-vm`.
- Edit `/usr/share/qubes/templates/libvirt/xen.xml` to prepare our custom config to override just the NIC part of the global template:
- add `{% raw %}{% block network %}{% endraw %}` before `{% raw %}{% if vm.netvm %}{% endraw %}`
- add `{% raw %}{% endblock %}{% endraw %}` after the matching `{% raw %}{% endif %}{% endraw %}`
- Copy `/usr/share/qubes/templates/libvirt/devices/net.xml` to `/etc/qubes/templates/libvirt/xen/by-name/target-vm.xml`.
- Add `<model type='e1000'/>` to the `<interface>` section.
- Enclose everything within `{% raw %}{% block network %}{% endraw %}` + `{% raw %}{% endblock %}{% endraw %}`.
- Add `{% raw %}{% extends 'libvirt/xen.xml' %}{% endraw %}` at the start.
- The final `target-vm.xml` should look something like this:
~~~
{% raw %}
{% extends 'libvirt/xen.xml' %}
{% block network %}
<interface type='ethernet'>
<mac address="{{ vm.mac }}" />
<ip address="{{ vm.ip }}" />
<backenddomain name="{{ vm.netvm.name }}" />
<script path='vif-route-qubes' />
<model type='e1000' />
</interface>
{% endblock %}
{% endraw %}
~~~
- Start `target-vm` and verify in the device manager that a "Intel PRO/1000 MT" adapter is present.
## Host and target preparation
- On `host-vm` you will need WinDbg, which is a part of the [Windows SDK](https://developer.microsoft.com/en-us/windows/downloads/windows-sdk/).
- Copy the `Debuggers` directory from Windows SDK to `target-vm`.
- In both `host-vm` and `target-vm` switch the windows network to private (it tends to be public by default).
- Either turn off the windows firewall or enable all ICMP-in rules in both VMs.
- In `firewall-vm` edit `/rw/config/qubes-firewall-user-script` to connect both Windows VMs, add:
- `iptables -I FORWARD 2 -s <target-vm-ip> -d <host-vm-ip> -j ACCEPT`
- `iptables -I FORWARD 2 -s <host-vm-ip> -d <target-vm-ip> -j ACCEPT`
- run `/rw/config/qubes-firewall-user-script` so the changes take effect immediately
- Make sure both VMs can ping each other.
- In `target-vm`:
- start elevated `cmd` session
- `cd sdk\Debuggers\x64`
- `kdnet` should show that the NIC is supported, note the busparams:
~~~
Network debugging is supported on the following NICs:
busparams=0.6.0, Intel(R) PRO/1000 MT Network Connection, KDNET is running on this NIC.
~~~
- `bcdedit /debug on`
- `bcdedit /dbgsettings net hostip:<host-vm-ip> port:50000 key:1.1.1.1` (you can customize the key)
- `bcdedit /set "{dbgsettings}" busparams x.y.z` (use the busparams `kdnet` has shown earlier)
- In `host-vm` start WinDbg: `windbg -k net:port=50000,key=1.1.1.1`. It will listen for target's connection.
- Reboot `target-vm`, debugging should start:
~~~
Waiting to reconnect...
Connected to target 10.137.0.19 on port 50000 on local IP 10.137.0.20.
You can get the target MAC address by running .kdtargetmac command.
Connected to Windows 10 19041 x64 target at (Thu Aug 3 14:05:48.069 2023 (UTC + 2:00)), ptr64 TRUE
~~~
Happy debugging!

169
developer/debugging/windows-debugging.rst Normal file
View File

@ -0,0 +1,169 @@
=================
Windows debugging
=================
Debugging Windows code can be tricky in a virtualized environment. The
guide below assumes Qubes 4.2 and Windows 7 or later VMs.
User-mode debugging is usually straightforward if it can be done on one
machine. Just duplicate your normal debugging environment in the VM.
Things get complicated if you need to perform kernel debugging or
troubleshoot problems that only manifest on system boot, user logoff or
similar. For that you need two Windows VMs: the *host* and the *target*.
The *host* will contain the debugger, your source code and private
symbols. The *target* will run the code being debugged. We will use
kernel debugging over network which is supported from Windows 7 onwards.
The main caveat is that Windows kernel supports only specific network
adapters for this, and the default one in Qubes won’t work.
Important note
--------------
- Do not install Xen network PV drivers in the target VM. Network
kernel debugging needs a specific type of NIC or it won’t work, the
network PV drivers interfere with that.
- If you have kernel debugging active when the Xen PV drivers are being
installed, make sure to disable it before rebooting
(``bcdedit /set debug off``). You can re-enable debugging after the
reboot. The OS won’t boot otherwise. I’m not sure what’s the exact
cause. I know that busparams for the debugging NIC change when PV
drivers are installed (see later), but even changing that accordingly
in the debug settings doesn’t help – so it’s best to disable debug
for this one reboot.
Modifying the NIC of the target VM
----------------------------------
You will need to create a custom libvirt config for the target VM. See
`the documentation <https://dev.qubes-os.org/projects/core-admin/en/latest/libvirt.html>`__
for overview of how libvirt templates work in Qubes. The following
assumes the target VM is named ``target-vm``.
- Edit ``/usr/share/qubes/templates/libvirt/xen.xml`` to prepare our
custom config to override just the NIC part of the global template:
- add ``{% raw %}{% block network %}{% endraw %}`` before
``{% raw %}{% if vm.netvm %}{% endraw %}``
- add ``{% raw %}{% endblock %}{% endraw %}`` after the matching
``{% raw %}{% endif %}{% endraw %}``
- Copy ``/usr/share/qubes/templates/libvirt/devices/net.xml`` to
``/etc/qubes/templates/libvirt/xen/by-name/target-vm.xml``.
- Add ``<model type='e1000'/>`` to the ``<interface>`` section.
- Enclose everything within
``{% raw %}{% block network %}{% endraw %}`` +
``{% raw %}{% endblock %}{% endraw %}``.
- Add ``{% raw %}{% extends 'libvirt/xen.xml' %}{% endraw %}`` at the
start.
- The final ``target-vm.xml`` should look something like this:
.. code:: bash
{% raw %}
{% extends 'libvirt/xen.xml' %}
{% block network %}
<interface type='ethernet'>
<mac address="{{ vm.mac }}" />
<ip address="{{ vm.ip }}" />
<backenddomain name="{{ vm.netvm.name }}" />
<script path='vif-route-qubes' />
<model type='e1000' />
</interface>
{% endblock %}
{% endraw %}
- Start ``target-vm`` and verify in the device manager that a “Intel
PRO/1000 MT” adapter is present.
Host and target preparation
---------------------------
- On ``host-vm`` you will need WinDbg, which is a part of the `Windows SDK <https://developer.microsoft.com/en-us/windows/downloads/windows-sdk/>`__.
- Copy the ``Debuggers`` directory from Windows SDK to ``target-vm``.
- In both ``host-vm`` and ``target-vm`` switch the windows network to
private (it tends to be public by default).
- Either turn off the windows firewall or enable all ICMP-in rules in
both VMs.
- In ``firewall-vm`` edit ``/rw/config/qubes-firewall-user-script`` to
connect both Windows VMs, add:
- ``iptables -I FORWARD 2 -s <target-vm-ip> -d <host-vm-ip> -j ACCEPT``
- ``iptables -I FORWARD 2 -s <host-vm-ip> -d <target-vm-ip> -j ACCEPT``
- run ``/rw/config/qubes-firewall-user-script`` so the changes take
effect immediately
- Make sure both VMs can ping each other.
- In ``target-vm``:
- start elevated ``cmd`` session
- ``cd sdk\Debuggers\x64``
- ``kdnet`` should show that the NIC is supported, note the
busparams:
.. code:: bash
Network debugging is supported on the following NICs:
busparams=0.6.0, Intel(R) PRO/1000 MT Network Connection, KDNET is running on this NIC.
- ``bcdedit /debug on``
- ``bcdedit /dbgsettings net hostip:<host-vm-ip> port:50000 key:1.1.1.1``
(you can customize the key)
- ``bcdedit /set "{dbgsettings}" busparams x.y.z`` (use the
busparams ``kdnet`` has shown earlier)
- In ``host-vm`` start WinDbg:
``windbg -k net:port=50000,key=1.1.1.1``. It will listen for target’s
connection.
- Reboot ``target-vm``, debugging should start:
.. code:: bash
Waiting to reconnect...
Connected to target 10.137.0.19 on port 50000 on local IP 10.137.0.20.
You can get the target MAC address by running .kdtargetmac command.
Connected to Windows 10 19041 x64 target at (Thu Aug 3 14:05:48.069 2023 (UTC + 2:00)), ptr64 TRUE
Happy debugging!

39
developer/general/continuous-integration.md
View File

@ -1,39 +0,0 @@
---
lang: en
layout: doc
permalink: /doc/continuous-integration/
title: Continuous integration (CI)
---
This page explains the [continuous integration
(CI)](https://en.wikipedia.org/wiki/Continuous_integration) infrastructure that
the Qubes OS Project uses.
## Website and documentation
The following commands may be useful as a way to interact with our CI
infrastructure on website
([qubesos.github.io](https://github.com/QubesOS/qubesos.github.io)) and
documentation ([qubes-doc](https://github.com/QubesOS/qubes-doc)) pull requests
(PRs). Note that special permissions may be required to use some of these
commands. These commands are generally issued by adding a comment to a PR
containing only the command.
- `PipelineRetry`: Attempts to run the entire build pipeline over again. This
can be useful if CI incorrectly uses a stale branch instead of testing the PR
as if it were merged into `master`.
- `PipelineRetryFailed`: Retry just failed CI jobs. Do not update the branch.
- `PipelineRefresh`: Like `PipelineRetry`, except it only fetches the job status
from GitLab. It doesn't schedule a new build.
- `TestDeploy`: Deploys a test website, which is a live version of the Qubes
website as if this PR had been merged. This can be useful for previewing a PR
on a live public website. **Note:** You must wait for the site to finish
building before issuing this command, or else it will deploy an empty
website. To find the URL of the test website, look for text similar to "This
branch was successfully deployed" and a button named something like "View
deployment." Note that there are two different testing sites: `wwwtest` is
manually updated, whereas `wwwpreview` is managed by the `TestDeploy`
command.

41
developer/general/continuous-integration.rst Normal file
View File

@ -0,0 +1,41 @@
===========================
Continuous integration (CI)
===========================
This page explains the `continuous integration (CI) <https://en.wikipedia.org/wiki/Continuous_integration>`__
infrastructure that the Qubes OS Project uses.
Website and documentation
-------------------------
The following commands may be useful as a way to interact with our CI
infrastructure on website
(`qubesos.github.io <https://github.com/QubesOS/qubesos.github.io>`__)
and documentation (`qubes-doc <https://github.com/QubesOS/qubes-doc>`__)
pull requests (PRs). Note that special permissions may be required to
use some of these commands. These commands are generally issued by
adding a comment to a PR containing only the command.
- ``PipelineRetry``: Attempts to run the entire build pipeline over
again. This can be useful if CI incorrectly uses a stale branch
instead of testing the PR as if it were merged into ``master``.
- ``PipelineRetryFailed``: Retry just failed CI jobs. Do not update the
branch.
- ``PipelineRefresh``: Like ``PipelineRetry``, except it only fetches
the job status from GitLab. It doesn’t schedule a new build.
- ``TestDeploy``: Deploys a test website, which is a live version of
the Qubes website as if this PR had been merged. This can be useful
for previewing a PR on a live public website. **Note:** You must wait
for the site to finish building before issuing this command, or else
it will deploy an empty website. To find the URL of the test website,
look for text similar to “This branch was successfully deployed” and
a button named something like “View deployment.” Note that there are
two different testing sites: ``wwwtest`` is manually updated, whereas
``wwwpreview`` is managed by the ``TestDeploy`` command.

31
developer/general/devel-books.md
View File

@ -1,31 +0,0 @@
---
lang: en
layout: doc
permalink: /doc/devel-books/
redirect_from:
- /en/doc/devel-books/
- /doc/DevelBooks/
- /wiki/DevelBooks/
ref: 32
title: Developer books
---
Below is a list of various books that might be useful in learning some basics needed for Qubes development.
- A must-read about Xen internals:
- _[The Definitive Guide to the Xen Hypervisor](https://www.amazon.com/Definitive-Guide-Xen-Hypervisor/dp/013234971X)_, by David Chisnall
- Some good books about the Linux kernel:
- _[Linux Kernel Development](https://www.amazon.com/Linux-Kernel-Development-Robert-Love/dp/0672329468)_, by Robert Love
- _[Linux Device Drivers](https://www.amazon.com/Linux-Device-Drivers-Jonathan-Corbet/dp/0596005903)_, by Jonathan Corbet
- Solid intro into Trusted Computing:
- _[Dynamics of a Trusted Platform](https://www.amazon.com/Dynamics-Trusted-Platform-Buildin-Grawrock/dp/1934053082)_, by David Grawrock (original Intel architect for TXT)
- Good book about GIT:
- _[Pro Git](https://git-scm.com/book/en/v2)_, by Scott Chacon and Ben Straub (complete book available free online)
- Useful books about Python:
- _[Programming in Python 3](https://www.qtrac.eu/py3book.html)_, by Mark Summerfield
- _[Rapid GUI Programming with Python and Qt](https://www.qtrac.eu/pyqtbook.html)_, by Mark Summerfield
(Although note that [Qt is being replaced by GTK](/doc/usability-ux/#gnome-kde-and-xfce) in Qubes code.)

50
developer/general/devel-books.rst Normal file
View File

@ -0,0 +1,50 @@
===============
Developer books
===============
Below is a list of various books that might be useful in learning some
basics needed for Qubes development.
- A must-read about Xen internals:
- `The Definitive Guide to the Xen Hypervisor <https://www.amazon.com/Definitive-Guide-Xen-Hypervisor/dp/013234971X>`__,
by David Chisnall
- Some good books about the Linux kernel:
- `Linux Kernel Development <https://www.amazon.com/Linux-Kernel-Development-Robert-Love/dp/0672329468>`__,
by Robert Love
- `Linux Device Drivers <https://www.amazon.com/Linux-Device-Drivers-Jonathan-Corbet/dp/0596005903>`__,
by Jonathan Corbet
- Solid intro into Trusted Computing:
- `Dynamics of a Trusted Platform <https://www.amazon.com/Dynamics-Trusted-Platform-Buildin-Grawrock/dp/1934053082>`__,
by David Grawrock (original Intel architect for TXT)
- Good book about GIT:
- `Pro Git <https://git-scm.com/book/en/v2>`__, by Scott Chacon and
Ben Straub (complete book available free online)
- Useful books about Python:
- `Programming in Python 3 <https://www.qtrac.eu/py3book.html>`__,
by Mark Summerfield
- `Rapid GUI Programming with Python and Qt <https://www.qtrac.eu/pyqtbook.html>`__, by Mark Summerfield
(Although note that :ref:`Qt is being replaced by GTK <developer/general/usability-ux:gnome, kde, and xfce>` in Qubes code.)

102
developer/general/developing-gui-applications.md
View File

@ -1,102 +0,0 @@
---
lang: en
layout: doc
permalink: /doc/developing-gui-applications/
ref: 333
title: Developing Qubes OS GUI tools
---
In order to avoid installing Qubes OS frontend tools you are working on in your own `dom0` or just to test them with less problems, you can use the mock Qubes object from the `qubesadmin` package.
## Running programs using mock Qubes object
Where you would normally provide the Qubes object, use the `qubesadmin.tests.mock_app` package and one of the mock Qubes objects from it.
For example, the following code can be used to run the `qui-domains` tool using the mock Qubes object (this code would replace the initial part of the main function):
```python
def main():
''' main function '''
# qapp = qubesadmin.Qubes()
# dispatcher = qubesadmin.events.EventsDispatcher(qapp)
# stats_dispatcher = qubesadmin.events.EventsDispatcher(qapp, api_method='admin.vm.Stats')
import qubesadmin.tests.mock_app as mock_app
qapp = mock_app.MockQubesComplete()
dispatcher = mock_app.MockDispatcher(qapp)
stats_dispatcher = mock_app.MockDispatcher(
qapp, api_method='admin.vm.Stats')
# continue as normal
```
To run a mocked program without installing it in a qube, remember to extend PYTHONPATH appropriately, for example:
```bash
~/qubes-sources/manager $ PYTHONPATH=../core-admin-client:. python3 qui/tray/domains.py
```
The mock object does not provide events (yet).
Note: in order to see all qubes-relevant icons (like VM icons), install the `qubes-artwork` package.
## How does it actually work
The mock Qubes object has a collection of expected Qubes RPC calls and the responses that a real system would provide. Writing these calls manually is a bit tedious, given that most frontend tools query a lot of qube properties. For example, on a medium-sized system, initializing Qube Manager involves about 300 separate RPC calls.
If you need more calls, you can add them to the mock object using the following syntax (the following example adds listing available vm kernels):
```python
mock_app.expected_calls[('dom0', 'admin.pool.volume.List', 'linux-kernel', None)] = \
b'0\x006.1.57-1.fc37\n6.1.43-1.fc37\ncustom_kernel\n'
```
If error should be thrown, you need to provide the error code and name, for example:
```python
mock_app.expected_calls[("vmname", "admin.vm.property.Get", "property_name", None)] = \
b'2\x00QubesNoSuchPropertyError\x00\x00No such property\x00'
```
For details of particular calls, you can use [Extending the mock Qubes object].
## Available mocks
Three mocks are available in the `mock_app` file:
* MockQubes, an extremely bare-bones Qubes testing instance, with just dom0, sys-net, and one template (fedora-36).
* MockQubesComplete, a more complex setup [![Qubes Manager running MockQubesComplete](/attachment/doc/doc-mock-app-ex1.png)](/attachment/doc/doc-mock-app-ex1.png)
* MockQubesWhonix, the setup above extended with several Whonix-related qubes
## Extending the mock Qubes object
To collect information to modify this script, you can use the wrapper function to wrap and output all qubesd calls used by a program running on a live qubes instance.
```python
qapp = qubesadmin.Qubes()
import qubesadmin.tests.mock_app as mock_app
qapp.qubesd_call = mock_app.wrapper(qapp.qubesd_call)
qapp._parse_qubesd_response = mock_app.wrapper(qapp._parse_qubesd_response)
```
## Writing tests
The same mock Qubes can also be used to write tests. You can use the wrappers above to check which calls are made when certain actions are performed, and add them to the mock objects in the following way:
```python
# this is an excerpt from tests for Qubes Global Config tool
clockvm_combo.set_active_id('test-blue')
mock_qapp.expected_calls[('dom0', 'admin.property.Set',
'clockvm', b'test-blue')] = b'0\x00'
basics_handler.save()
```
If the call is made correctly, the test will continue successfully; if an unexpected call is made, the test will fail.
Caution: the mock Qubes object does not react to changes like a normal Qubes object does. Further queries to the test object will continue to return initial values.

142
developer/general/developing-gui-applications.rst Normal file
View File

@ -0,0 +1,142 @@
=============================
Developing Qubes OS GUI tools
=============================
In order to avoid installing Qubes OS frontend tools you are working on
in your own ``dom0`` or just to test them with less problems, you can
use the mock Qubes object from the ``qubesadmin`` package.
Running programs using mock Qubes object
----------------------------------------
Where you would normally provide the Qubes object, use the
``qubesadmin.tests.mock_app`` package and one of the mock Qubes objects
from it.
For example, the following code can be used to run the ``qui-domains``
tool using the mock Qubes object (this code would replace the initial
part of the main function):
.. code:: python
def main():
''' main function '''
# qapp = qubesadmin.Qubes()
# dispatcher = qubesadmin.events.EventsDispatcher(qapp)
# stats_dispatcher = qubesadmin.events.EventsDispatcher(qapp, api_method='admin.vm.Stats')
import qubesadmin.tests.mock_app as mock_app
qapp = mock_app.MockQubesComplete()
dispatcher = mock_app.MockDispatcher(qapp)
stats_dispatcher = mock_app.MockDispatcher(
qapp, api_method='admin.vm.Stats')
# continue as normal
To run a mocked program without installing it in a qube, remember to
extend PYTHONPATH appropriately, for example:
.. code:: bash
~/qubes-sources/manager $ PYTHONPATH=../core-admin-client:. python3 qui/tray/domains.py
The mock object does not provide events (yet).
Note: in order to see all qubes-relevant icons (like VM icons), install
the ``qubes-artwork`` package.
How does it actually work
-------------------------
The mock Qubes object has a collection of expected Qubes RPC calls and
the responses that a real system would provide. Writing these calls
manually is a bit tedious, given that most frontend tools query a lot of
qube properties. For example, on a medium-sized system, initializing
Qube Manager involves about 300 separate RPC calls.
If you need more calls, you can add them to the mock object using the
following syntax (the following example adds listing available vm
kernels):
.. code:: python
mock_app.expected_calls[('dom0', 'admin.pool.volume.List', 'linux-kernel', None)] = \
b'0\x006.1.57-1.fc37\n6.1.43-1.fc37\ncustom_kernel\n'
If error should be thrown, you need to provide the error code and name,
for example:
.. code:: python
mock_app.expected_calls[("vmname", "admin.vm.property.Get", "property_name", None)] = \
b'2\x00QubesNoSuchPropertyError\x00\x00No such property\x00'
For details of particular calls, you can use `Extending the mock Qubes object <#extending-the-mock-qubes-object>`__.
Available mocks
---------------
Three mocks are available in the ``mock_app`` file:
- MockQubes, an extremely bare-bones Qubes testing instance, with just
dom0, sys-net, and one template (fedora-36).
- MockQubesComplete, a more complex setup |Qubes Manager running
MockQubesComplete|
- MockQubesWhonix, the setup above extended with several Whonix-related
qubes
Extending the mock Qubes object
-------------------------------
To collect information to modify this script, you can use the wrapper
function to wrap and output all qubesd calls used by a program running
on a live qubes instance.
.. code:: python
qapp = qubesadmin.Qubes()
import qubesadmin.tests.mock_app as mock_app
qapp.qubesd_call = mock_app.wrapper(qapp.qubesd_call)
qapp._parse_qubesd_response = mock_app.wrapper(qapp._parse_qubesd_response)
Writing tests
-------------
The same mock Qubes can also be used to write tests. You can use the
wrappers above to check which calls are made when certain actions are
performed, and add them to the mock objects in the following way:
.. code:: python
# this is an excerpt from tests for Qubes Global Config tool
clockvm_combo.set_active_id('test-blue')
mock_qapp.expected_calls[('dom0', 'admin.property.Set',
'clockvm', b'test-blue')] = b'0\x00'
basics_handler.save()
If the call is made correctly, the test will continue successfully; if
an unexpected call is made, the test will fail.
Caution: the mock Qubes object does not react to changes like a normal
Qubes object does. Further queries to the test object will continue to
return initial values.
.. |Qubes Manager running MockQubesComplete| image:: /attachment/doc/doc-mock-app-ex1.png

355
developer/general/documentation-style-guide.md
View File

@ -1,355 +0,0 @@
---
lang: en
layout: doc
permalink: /doc/documentation-style-guide/
redirect_from:
- /doc/doc-guidelines/
- /en/doc/doc-guidelines/
- /wiki/DocStyle/
- /doc/DocStyle/
ref: 30
title: Documentation style guide
---
_Also see [how to edit the documentation](/doc/how-to-edit-the-documentation/)._
Qubes OS documentation pages are stored as plain text Markdown files in the [qubes-doc](https://github.com/QubesOS/qubes-doc) repository. By cloning and regularly pulling from this repo, users can maintain their own up-to-date offline copy of all Qubes documentation rather than relying solely on the web.
The documentation is a volunteer community effort. People like you are constantly working to make it better. If you notice something that can be fixed or improved, please [edit the documentation](/doc/how-to-edit-the-documentation/)!
This page explains the standards we follow for writing, formatting, and organizing the documentation. Please follow these guidelines and conventions when editing the documentation. For the standards governing the website as a whole, please see the [website style guide](/doc/website-style-guide).
## Markdown conventions
All the documentation is written in Markdown for maximum accessibility. When making contributions, please observe the following style conventions. If you're not familiar with Markdown syntax, [this](https://daringfireball.net/projects/markdown/) is a great resource.
### Hyperlink syntax
Use non-reference-style links like `[website](https://example.com/)`. Do *not* use reference-style links like `[website][example]`, `[website][]` or `[website]`. This facilitates the localization process.
### Relative vs. absolute links
Always use relative rather than absolute paths for internal website links. For example, use `/doc/documentation-style-guide/` instead of `https://www.qubes-os.org/doc/documentation-style-guide/`.
You may use absolute URLs in the following cases:
- External links
- URLs that appear inside code blocks (e.g., in comments and document templates, and the plain text reproductions of [QSBs](/security/qsb/) and [Canaries](/security/canary/)), since they're not hyperlinks
- Git repo files like `README.md` and `CONTRIBUTING.md`, since they're not part of the website itself but rather of the auxiliary infrastructure supporting the website
This rule is important because using absolute URLs for internal website links breaks:
- Serving the website offline
- Website localization
- Generating offline documentation
- Automatically redirecting Tor Browser visitors to the correct page on the onion service mirror
### Image linking
See [how to add images](/doc/how-to-edit-the-documentation/#how-to-add-images) for the required syntax. This will make the image a hyperlink to the image file, allowing the reader to click on the image in order to view the full image by itself. This is important. Following best practices, our website has a responsive design, which allows the website to render appropriately across all screen sizes. When viewing this page on a smaller screen, such as on a mobile device, the image will automatically shrink down to fit the screen. If visitors cannot click on the image to view it in full size, then, depending on their device, they may have no way see the details in the image clearly.
In addition, make sure to link only to images in the [qubes-attachment](https://github.com/QubesOS/qubes-attachment) repository. Do not attempt to link to images hosted on other websites.
### HTML and CSS
Do not write HTML inside Markdown documents (except in rare, unavoidable cases, such as [alerts](#alerts)). In particular, never include HTML or CSS for styling, formatting, or white space control. That belongs in the (S)CSS files instead.
### Headings
Do not use `h1` headings (single `#` or `======` underline). These are automatically generated from the `title:` line in the YAML front matter.
Use Atx-style syntax for headings: `##h2`, `### h3`, etc. Do not use underlining syntax (`-----`).
### Indentation
Use spaces instead of tabs. Use hanging indentations where appropriate.
### Lists
If appropriate, make numerals in numbered lists match between Markdown source and HTML output. Some users read the Markdown source directly, and this makes numbered lists easier to follow.
### Code blocks
When writing code blocks, use [syntax highlighting](https://github.github.com/gfm/#info-string) where possible (see [here](https://github.com/jneen/rouge/wiki/List-of-supported-languages-and-lexers) for a list of supported languages). Use `[...]` for anything omitted.
### Line wrapping
Do not hard wrap text, except where necessary (e.g., inside code blocks).
### Do not use Markdown syntax for styling
For example, there is a common temptation to use block quotations (created by beginning lines with the `>` character) in order to stylistically distinguish some portion of text from the rest of the document, e.g.:
```
> Note: This is an important note!
```
This renders as:
> Note: This is an important note!
There are two problems with this:
1. It is a violation of the [separation of content and presentation](https://en.wikipedia.org/wiki/Separation_of_content_and_presentation), since it abuses markup syntax in order to achieve unintended stylistic results. The Markdown (and HTML, if any) should embody the *content* of the documentation, while the *presentation* is handled by (S)CSS.
2. It is an abuse of quotation syntax for text that is not actually a quotation. (You are not quoting anyone here. You're just telling the reader to note something and trying to draw their attention to your note visually.)
Instead, an example of an appropriate way to stylistically distinguish a portion of text is by using [alerts](#alerts). Consider also that extra styling and visual distinction may not even be necessary. In most cases, traditional writing methods are perfectly sufficient, e.g.,:
```
**Note:** This is an important note.
```
This renders as:
**Note:** This is an important note.
### Alerts
Alerts are sections of HTML used to draw the reader's attention to important information, such as warnings, and for stylistic purposes. They are typically styled as colored text boxes, usually accompanied by icons. Alerts should generally be used somewhat sparingly, so as not to cause [alert fatigue](https://en.wikipedia.org/wiki/Alarm_fatigue) and since they must be written in HTML instead of Markdown, which makes the source less readable and more difficult to work with for localization and automation purposes. Here are examples of several types of alerts and their recommended icons:
```
<div class="alert alert-success" role="alert">
<i class="fa fa-check-circle"></i>
<b>Did you know?</b> The Qubes OS installer is completely offline. It doesn't
even load any networking drivers, so there is no possibility of
internet-based data leaks or attacks during the installation process.
</div>
<div class="alert alert-info" role="alert">
<i class="fa fa-info-circle"></i>
<b>Note:</b> Using Rufus to create the installation medium means that you <a
href="https://github.com/QubesOS/qubes-issues/issues/2051">won't be able</a>
to choose the "Test this media and install Qubes OS" option mentioned in the
example below. Instead, choose the "Install Qubes OS" option.
</div>
<div class="alert alert-warning" role="alert">
<i class="fa fa-exclamation-circle"></i>
<b>Note:</b> Qubes OS is not meant to be installed inside a virtual machine
as a guest hypervisor. In other words, <b>nested virtualization</b> is not
supported. In order for a strict compartmentalization to be enforced, Qubes
OS needs to be able to manage the hardware directly.
</div>
<div class="alert alert-danger" role="alert">
<i class="fa fa-exclamation-triangle"></i>
<b>Warning:</b> Qubes has no control over what happens on your computer
before you install it. No software can provide security if it is installed on
compromised hardware. Do not install Qubes on a computer you don't trust. See
<a href="/doc/install-security/">installation security</a> for more
information.
</div>
```
These render as:
<div class="alert alert-success" role="alert">
<i class="fa fa-check-circle"></i>
<b>Did you know?</b> The Qubes OS installer is completely offline. It doesn't
even load any networking drivers, so there is no possibility of
internet-based data leaks or attacks during the installation process.
</div>
<div class="alert alert-info" role="alert">
<i class="fa fa-info-circle"></i>
<b>Note:</b> Using Rufus to create the installation medium means that you <a
href="https://github.com/QubesOS/qubes-issues/issues/2051">won't be able</a>
to choose the "Test this media and install Qubes OS" option mentioned in the
example below. Instead, choose the "Install Qubes OS" option.
</div>
<div class="alert alert-warning" role="alert">
<i class="fa fa-exclamation-circle"></i>
<b>Note:</b> Qubes OS is not meant to be installed inside a virtual machine
as a guest hypervisor. In other words, <b>nested virtualization</b> is not
supported. In order for a strict compartmentalization to be enforced, Qubes
OS needs to be able to manage the hardware directly.
</div>
<div class="alert alert-danger" role="alert">
<i class="fa fa-exclamation-triangle"></i>
<b>Warning:</b> Qubes has no control over what happens on your computer
before you install it. No software can provide security if it is installed on
compromised hardware. Do not install Qubes on a computer you don't trust. See
<a href="/doc/install-security/">installation security</a> for more
information.
</div>
## Writing guidelines
### Correct use of terminology
Familiarize yourself with the terms defined in the [glossary](/doc/glossary/). Use these terms consistently and accurately throughout your writing.
### Sentence case in headings
Use sentence case (rather than title case) in headings for the reasons explained [here](https://www.sallybagshaw.com.au/articles/sentence-case-v-title-case/). In particular, since the authorship of the Qubes documentation is decentralized and widely distributed among users from around the world, many contributors come from regions with different conventions for implementing title case, not to mention that there are often differing style guide recommendations even within a single region. It is much easier for all of us to implement sentence case consistently across our growing body of pages, which is very important for managing the ongoing maintenance burden and sustainability of the documentation.
### Writing command-line examples
When providing command-line examples:
- Tell the reader where to open a terminal (dom0 or a specific domU), and show the command along with its output (if any) in a code block, e.g.:
~~~markdown
Open a terminal in dom0 and run:
```shell_session
$ cd test
$ echo Hello
Hello
```
~~~
- Precede each command with the appropriate command prompt: At a minimum, the prompt should contain a trailing `#` (for the user `root`) or `$` (for other users) on Linux systems and `>` on Windows systems, respectively.
- Don't try to add comments inside the code block. For example, *don't* do this:
~~~markdown
Open a terminal in dom0 and run:
```shell_session
# Navigate to the new directory
$ cd test
# Generate a greeting
$ echo Hello
Hello
```
~~~
The `#` symbol preceding each comment is ambiguous with a root command prompt. Instead, put your comments *outside* of the code block in normal prose.
### Variable names in commands
Syntactically distinguish variables in commands. For example, this is ambiguous:
$ qvm-run --dispvm=disposable-template --service qubes.StartApp+xterm
It should instead be:
$ qvm-run --dispvm=<DISPOSABLE_TEMPLATE> --service qubes.StartApp+xterm
Note that we syntactically distinguish variables in three ways:
1. Surrounding them in angled brackets (`< >`)
2. Using underscores (`_`) instead of spaces between words
3. Using all capital letters
We have observed that many novices make the mistake of typing the surrounding angled brackets (`< >`) on the command line, even after substituting the desired real value between them. Therefore, in documentation aimed at novices, we also recommend clarifying that the angled brackets should not be typed. This can be accomplished in one of several ways:
- Explicitly say something like "without the angled brackets."
- Provide an example command using real values that excludes the angled brackets.
- If you know that almost all users will want to use (or should use) a specific command containing all real values and no variables, you might consider providing exactly that command and forgoing the version with variables. Novices may not realize which parts of the command they can substitute with different values, but if you've correctly judged that they should use the command you've provided as is, then this shouldn't matter.
### Capitalization of "qube"
We introduced the term ["qube"](/doc/glossary/#qube) as a user-friendly alternative to the term ["virtual machine" ("VM")](/doc/glossary/#vm) in the context of Qubes OS. Nonetheless, "qube" is a common noun like the words "compartment" and "container." Therefore, in English, "qube" follows the standard capitalization rules for common nouns. For example, "I have three qubes" is correct, while "I have three Qubes" is incorrect. Like other common nouns, "qube" should still be capitalized at the beginnings of sentences, the beginnings of sentence-case headings, and in title-case headings. Note, however, that starting a sentence with the plural of "qube" (e.g., "Qubes can be shut down...") can be ambiguous, since it may not be clear whether the referent is a plurality of qubes, [Qubes OS](/doc/glossary/#qubes-os), or even the Qubes OS Project itself. Hence, it is generally a good idea to rephrase such sentences in order to avoid this ambiguity.
Many people feel a strong temptation to capitalize the word "qube" all the time, like a proper noun, perhaps because it's a new and unfamiliar term that's closely associated with a particular piece of software (namely, Qubes OS). However, these factors are not relevant to the capitalization rules of English. In fact, it's not unusual for new common nouns to be introduced into English, especially in the context of technology. For example, "blockchain" is a relatively recent technical term that's a common noun. Why is it a common noun rather than a proper noun? Because proper nouns refer to *particular* people, places, things, and ideas. There are many different blockchains. However, even when there was just one, the word still denoted a collection of things rather than a particular thing. It happened to be the case that there was only one member in that collection at the time. For example, if there happened to be only one tree in the world, that wouldn't change the way we capitalize sentences like, "John sat under a tree." Intuitively, it makes sense that the addition and removal of objects from the world shouldn't cause published books to become orthographicallly incorrect while sitting on their shelves.
Accordingly, the reason "qube" is a common noun rather than a proper noun is because it doesn't refer to any one specific thing (in this case, any one specific virtual machine). Rather, it's the term for any virtual machine in a Qubes OS installation. (Technically, while qubes are currently implemented as virtual machines, Qubes OS is independent of its underlying compartmentalization technology. Virtual machines could be replaced with a different technology, and qubes would still be called "qubes.")
I have several qubes in my Qubes OS installation, and you have several in yours. Every Qubes OS user has their own set of qubes, just as each of us lives in some neighborhood on some street. Yet we aren't tempted to treat words like "neighborhood" or "street" as proper nouns (unless, of course, they're part of a name, like "Acorn Street"). Again, while this might seem odd because "qube" is a new word that we invented, that doesn't change how English works. After all, *every* word was a new word that someone invented at some point (otherwise we wouldn't have any words at all). We treat "telephone," "computer," "network," "program," and so on as common nouns, even though those were all new technological inventions in the not-too-distant past (on a historical scale, at least). So, we shouldn't allow ourselves to be confused by irrelevant factors, like the fact that the inventors happened to be *us* or that the invention was *recent* or is not in widespread use among humanity.
### English language conventions
For the sake of consistency and uniformity, the Qubes documentation aims to follow the conventions of American English, where applicable. (Please note that this is an arbitrary convention for the sake consistency and not a value judgment about the relative merits of British versus American English.)
## Organizational guidelines
### Do not duplicate documentation
Duplicating documentation is almost always a bad idea. There are many reasons for this. The main one is that almost all documentation has to be updated as some point. When similar documentation appears in more than one place, it is very easy for it to get updated in one place but not the others (perhaps because the person updating it doesn't realize it's in more than once place). When this happens, the documentation as a whole is now inconsistent, and the outdated documentation becomes a trap, especially for novice users. Such traps are often more harmful than if the documentation never existed in the first place. The solution is to **link** to existing documentation rather than duplicating it. There are some exceptions to this policy (e.g., information that is certain not to change for a very long time), but they are rare.
### Core vs. external documentation
Core documentation resides in the [Qubes OS Project's official repositories](https://github.com/QubesOS/), mainly in [qubes-doc](https://github.com/QubesOS/qubes-doc). External documentation can be anywhere else (such as forums, community websites, and blogs), but there is an especially large collection in the [Qubes Forum](https://forum.qubes-os.org/docs). External documentation should not be submitted to [qubes-doc](https://github.com/QubesOS/qubes-doc). If you've written a piece of documentation that is not appropriate for [qubes-doc](https://github.com/QubesOS/qubes-doc), we encourage you to submit it to the [Qubes Forum](https://forum.qubes-os.org/docs) instead. However, *linking* to external documentation from [qubes-doc](https://github.com/QubesOS/qubes-doc) is perfectly fine. Indeed, the maintainers of the [Qubes Forum](https://forum.qubes-os.org/) should regularly submit PRs against the documentation index (see [How to edit the documentation index](/doc/how-to-edit-the-documentation/#how-to-edit-the-documentation-index)) to add and update Qubes Forum links in the ["External documentation"](/doc/#external-documentation) section of the documentation table of contents.
The main difference between **core** (or **official**) and **external** (or **community** or **unofficial**) documentation is whether it documents software that is officially written and maintained by the Qubes OS Project. The purpose of this distinction is to keep the core docs maintainable and high-quality by limiting them to the software output by the Qubes OS Project. In other words, we take responsibility for documenting all of the software we put out into the world, but it doesn't make sense for us to take on the responsibility of documenting or maintaining documentation for anything else. For example, Qubes OS may use a popular Linux distribution for an official [TemplateVM](/doc/templates/). However, it would not make sense for a comparatively small project like ours, with modest funding and a lean workforce, to attempt to document software belonging to a large, richly-funded project with an army of paid and volunteer contributors, especially when they probably already have documentation of their own. This is particularly true when it comes to Linux in general. Although many users who are new to Qubes are also new to Linux, it makes absolutely no sense for our comparatively tiny project to try to document Linux in general when there is already a plethora of documentation out there.
Many contributors do not realize that there is a significant amount of work involved in *maintaining* documentation after it has been written. They may wish to write documentation and submit it to the core docs, but they see only their own writing process and fail to consider that it will have to be kept up-to-date and consistent with the rest of the docs for years afterward. Submissions to the core docs also have to [undergo a review process](/doc/how-to-edit-the-documentation/#security) to ensure accuracy before being merged, which takes up valuable time from the team. We aim to maintain high quality standards for the core docs (style and mechanics, formatting), which also takes up a lot of time. If the documentation involves anything external to the Qubes OS Project (such as a website, platform, program, protocol, framework, practice, or even a reference to a version number), the documentation is likely to become outdated when that external thing changes. It's also important to periodically review and update this documentation, especially when a new Qubes release comes out. Periodically, there may be technical or policy changes that affect all the core documentation. The more documentation there is relative to maintainers, the harder all of this will be. Since there are many more people who are willing to write documentation than to maintain it, these individually small incremental additions amount to a significant maintenance burden for the project.
On the positive side, we consider the existence of community documentation to be a sign of a healthy ecosystem, and this is quite common in the software world. The community is better positioned to write and maintain documentation that applies, combines, and simplifies the official documentation, e.g., tutorials that explain how to install and use various programs in Qubes, how to create custom VM setups, and introductory tutorials that teach basic Linux concepts and commands in the context of Qubes. In addition, just because the Qubes OS Project has officially written and maintains some flexible framework, such as `qrexec`, it does not make sense to include every tutorial that says "here's how to do something cool with `qrexec`" in the core docs. Such tutorials generally also belong in the community documentation.
See [#4693](https://github.com/QubesOS/qubes-issues/issues/4693) for more background information.
### Release-specific documentation
*See [#5308](https://github.com/QubesOS/qubes-issues/issues/5308) for pending changes to this policy.*
We maintain only one set of documentation for Qubes OS. We do not maintain a different set of documentation for each release of Qubes. Our single set of Qubes OS documentation is updated on a continual, rolling basis. Our first priority is to document all **current, stable releases** of Qubes. Our second priority is to document the next, upcoming release (if any) that is currently in the beta or release candidate stage.
In cases where a documentation page covers functionality that differs considerably between Qubes OS releases, the page should be subdivided into clearly-labeled sections that cover the different functionality in different releases (examples below).
In general, avoid mentioning specific Qubes versions in the body text of documentation, as these references rapidly go out of date and become misleading to readers.
#### Incorrect Example
```
## How to Foo
Fooing is the process by which one foos. There are both general and specific
versions of fooing, which vary in usefulness depending on your goals, but for
the most part, all fooing is fooing.
To foo in Qubes 3.2:
$ qvm-foo <foo-bar>
Note that this does not work in Qubes 4.0, where there is a special widget
for fooing, which you can find in the lower-right corner of the screen in
the Foo Manager. Alternatively, you can use the more general `qubes-baz`
command introduced in 4.0:
$ qubes-baz --foo <bar>
Once you foo, make sure to close the baz before fooing the next bar.
```
#### Correct Example
```
## Qubes 3.2
### How to Foo
Fooing is the process by which one foos. There are both general and specific
versions of fooing, which vary in usefulness depending on your goals, but for
the most part, all fooing is fooing.
To foo:
$ qvm-foo <foo-bar>
Once you foo, make sure to close the baz before fooing the next bar.
## Qubes 4.0
### How to Foo
Fooing is the process by which one foos. There are both general and specific
versions of fooing, which vary in usefulness depending on your goals, but for
the most part, all fooing is fooing.
There is a special widget for fooing, which you can find in the lower-right
corner of the screen in the Foo Manager. Alternatively, you can use the
general `qubes-baz` command:
$ qubes-baz --foo <bar>
Once you foo, make sure to close the baz before fooing the next bar.
```
Subdividing the page into clearly-labeled sections for each release has several benefits:
- It preserves good content for older (but still supported) releases. Many documentation contributors are also people who prefer to use the latest release. Many of them are tempted to *replace* existing content that applies to an older, supported release with content that applies only to the latest release. This is somewhat understandable. Since they only use the latest release, they may be focused on their own experience, and they may even regard the older release as deprecated, even when it's actually still supported. However, allowing this replacement of content would do a great disservice to those who still rely on the older, supported release. In many cases, these users value the stability and reliability of the older, supported release. With the older, supported release, there has been more time to fix bugs and make improvements in both the software and the documentation. Consequently, much of the documentation content for this release may have gone through several rounds of editing, review, and revision. It would be a tragedy for this content to vanish while the very set of users who most prize stability and reliability are depending on it.
- It's easy for readers to quickly find the information they're looking for, since they can go directly to the section that applies to their release.
- It's hard for readers to miss information they need, since it's all in one place. In the incorrect example, information that the reader needs could be in any paragraph in the entire document, and there's no way to tell without reading the entire page. In the correct example, the reader can simply skim the headings in order to know which parts of the page need to be read and which can be safely ignored. The fact that some content is repeated in the two release-specific sections is not a problem, since no reader has to read the same thing twice. Moreover, as one release gets updated, it's likely that the documentation for that release will also be updated. Therefore, content that is initially duplicated between release-specific sections will not necessarily stay that way, and this is a good thing: We want the documentation for a release that *doesn't* change to stay the same, and we want the documentation for a release that *does* change to change along with the software.
- It's easy for documentation contributors and maintainers to know which file to edit and update, since there's only one page for all Qubes OS releases. Initially creating the new headings and duplicating content that applies to both is only a one-time cost for each page, and many pages don't even require this treatment, since they apply to all currently-supported Qubes OS releases.
By contrast, an alternative approach, such as segregating the documentation into two different branches, would mean that contributions that apply to both Qubes releases would only end up in one branch, unless someone remembered to manually submit the same thing to the other branch and actually made the effort to do so. Most of the time, this wouldn't happen. When it did, it would mean a second pull request that would have to be reviewed. Over time, the different branches would diverge in non-release-specific content. Good general content that was submitted only to one branch would effectively disappear once that release was deprecated. (Even if it were still on the website, no one would look at it, since it would explicitly be in the subdirectory of a deprecated release, and there would be a motivation to remove it from the website so that search results wouldn't be populated with out-of-date information.)
For further discussion about release-specific documentation in Qubes, see [here](https://groups.google.com/d/topic/qubes-users/H9BZX4K9Ptk/discussion).
## Git conventions
Please follow our [Git commit message guidelines](/doc/coding-style/#commit-message-guidelines).

735
developer/general/documentation-style-guide.rst Normal file
View File

@ -0,0 +1,735 @@
=========================
Documentation style guide
=========================
*Also see* :doc:`how to edit the documentation </developer/general/how-to-edit-the-documentation>` *.*
Qubes OS documentation pages are stored as plain text Markdown files in
the `qubes-doc <https://github.com/QubesOS/qubes-doc>`__ repository. By
cloning and regularly pulling from this repo, users can maintain their
own up-to-date offline copy of all Qubes documentation rather than
relying solely on the web.
The documentation is a volunteer community effort. People like you are
constantly working to make it better. If you notice something that can
be fixed or improved, please :doc:`edit the documentation </developer/general/how-to-edit-the-documentation>`!
This page explains the standards we follow for writing, formatting, and
organizing the documentation. Please follow these guidelines and
conventions when editing the documentation. For the standards governing
the website as a whole, please see the :doc:`website style guide </developer/general/website-style-guide>`.
Markdown conventions
--------------------
All the documentation is written in Markdown for maximum accessibility.
When making contributions, please observe the following style
conventions. If you’re not familiar with Markdown syntax,
`this <https://daringfireball.net/projects/markdown/>`__ is a great
resource.
Hyperlink syntax
^^^^^^^^^^^^^^^^
Use non-reference-style links like `` `website <https://example.com/>`__``.
Do *not* use reference-style links like ``[website][example]``,
``[website][]`` or ``[website]``. This facilitates the localization
process.
Relative vs. absolute links
^^^^^^^^^^^^^^^^^^^^^^^^^^^
Always use relative rather than absolute paths for internal website
links. For example, use ``/doc/documentation-style-guide/`` instead of
``https://www.qubes-os.org/doc/documentation-style-guide/``.
You may use absolute URLs in the following cases:
- External links
- URLs that appear inside code blocks (e.g., in comments and document
templates, and the plain text reproductions of
`QSBs <https://www.qubes-os.org/security/qsb/>`__ and `Canaries <https://www.qubes-os.org/security/canary/>`__),
since they’re not hyperlinks
- Git repo files like ``README.md`` and ``CONTRIBUTING.md``, since
they’re not part of the website itself but rather of the auxiliary
infrastructure supporting the website
This rule is important because using absolute URLs for internal website
links breaks:
- Serving the website offline
- Website localization
- Generating offline documentation
- Automatically redirecting Tor Browser visitors to the correct page on
the onion service mirror
Image linking
^^^^^^^^^^^^^
See :ref:`how to add images <developer/general/how-to-edit-the-documentation:how to add images>` for
the required syntax. This will make the image a hyperlink to the image
file, allowing the reader to click on the image in order to view the
full image by itself. This is important. Following best practices, our
website has a responsive design, which allows the website to render
appropriately across all screen sizes. When viewing this page on a
smaller screen, such as on a mobile device, the image will automatically
shrink down to fit the screen. If visitors cannot click on the image to
view it in full size, then, depending on their device, they may have no
way see the details in the image clearly.
In addition, make sure to link only to images in the
`qubes-attachment <https://github.com/QubesOS/qubes-attachment>`__
repository. Do not attempt to link to images hosted on other websites.
HTML and CSS
^^^^^^^^^^^^
Do not write HTML inside Markdown documents (except in rare, unavoidable
cases, such as `alerts <#alerts>`__). In particular, never include HTML
or CSS for styling, formatting, or white space control. That belongs in
the (S)CSS files instead.
Headings
^^^^^^^^
Do not use ``h1`` headings (single ``#`` or ``======`` underline). These
are automatically generated from the ``title:`` line in the YAML front
matter.
Use Atx-style syntax for headings: ``##h2``, ``### h3``, etc. Do not use
underlining syntax (``-----``).
Indentation
^^^^^^^^^^^
Use spaces instead of tabs. Use hanging indentations where appropriate.
Lists
^^^^^
If appropriate, make numerals in numbered lists match between Markdown
source and HTML output. Some users read the Markdown source directly,
and this makes numbered lists easier to follow.
Code blocks
^^^^^^^^^^^
When writing code blocks, use `syntax highlighting <https://github.github.com/gfm/#info-string>`__ where
possible (see
`here <https://github.com/jneen/rouge/wiki/List-of-supported-languages-and-lexers>`__
for a list of supported languages). Use ``[...]`` for anything omitted.
Line wrapping
^^^^^^^^^^^^^
Do not hard wrap text, except where necessary (e.g., inside code
blocks).
Do not use Markdown syntax for styling
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
For example, there is a common temptation to use block quotations
(created by beginning lines with the ``>`` character) in order to
stylistically distinguish some portion of text from the rest of the
document, e.g.:
.. code:: bash
> Note: This is an important note!
This renders as:
Note: This is an important note!
There are two problems with this:
1. It is a violation of the `separation of content and presentation <https://en.wikipedia.org/wiki/Separation_of_content_and_presentation>`__,
since it abuses markup syntax in order to achieve unintended
stylistic results. The Markdown (and HTML, if any) should embody the
*content* of the documentation, while the *presentation* is handled
by (S)CSS.
2. It is an abuse of quotation syntax for text that is not actually a
quotation. (You are not quoting anyone here. You’re just telling the
reader to note something and trying to draw their attention to your
note visually.)
Instead, an example of an appropriate way to stylistically distinguish a
portion of text is by using `alerts <#alerts>`__. Consider also that
extra styling and visual distinction may not even be necessary. In most
cases, traditional writing methods are perfectly sufficient, e.g.,:
.. code:: bash
**Note:** This is an important note.
This renders as:
**Note:** This is an important note.
Alerts
^^^^^^
Alerts are sections of HTML used to draw the reader’s attention to
important information, such as warnings, and for stylistic purposes.
They are typically styled as colored text boxes, usually accompanied by
icons. Alerts should generally be used somewhat sparingly, so as not to
cause `alert fatigue <https://en.wikipedia.org/wiki/Alarm_fatigue>`__
and since they must be written in HTML instead of Markdown, which makes
the source less readable and more difficult to work with for
localization and automation purposes. Here are examples of several types
of alerts and their recommended icons:
.. code:: bash
<div class="alert alert-success" role="alert">
<i class="fa fa-check-circle"></i>
<b>Did you know?</b> The Qubes OS installer is completely offline. It doesn't
even load any networking drivers, so there is no possibility of
internet-based data leaks or attacks during the installation process.
</div>
<div class="alert alert-info" role="alert">
<i class="fa fa-info-circle"></i>
<b>Note:</b> Using Rufus to create the installation medium means that you <a
href="https://github.com/QubesOS/qubes-issues/issues/2051">won't be able</a>
to choose the "Test this media and install Qubes OS" option mentioned in the
example below. Instead, choose the "Install Qubes OS" option.
</div>
<div class="alert alert-warning" role="alert">
<i class="fa fa-exclamation-circle"></i>
<b>Note:</b> Qubes OS is not meant to be installed inside a virtual machine
as a guest hypervisor. In other words, <b>nested virtualization</b> is not
supported. In order for a strict compartmentalization to be enforced, Qubes
OS needs to be able to manage the hardware directly.
</div>
<div class="alert alert-danger" role="alert">
<i class="fa fa-exclamation-triangle"></i>
<b>Warning:</b> Qubes has no control over what happens on your computer
before you install it. No software can provide security if it is installed on
compromised hardware. Do not install Qubes on a computer you don't trust. See
<a href="/doc/install-security/">installation security</a> for more
information.
</div>
These render as:
.. note::
Did you know? The Qubes OS installer is completely offline. It
doesn’t even load any networking drivers, so there is no possibility
of internet-based data leaks or attacks during the installation
process.
.. note::
Note: Using Rufus to create the installation medium means that you
won’t be able to choose the “Test this media and install Qubes OS”
option mentioned in the example below. Instead, choose the “Install
Qubes OS” option.
.. warning::
Note: Qubes OS is not meant to be installed inside a virtual machine
as a guest hypervisor. In other words, nested virtualization is not
supported. In order for a strict compartmentalization to be enforced,
Qubes OS needs to be able to manage the hardware directly.
.. DANGER::
Warning: Qubes has no control over what happens on your computer
before you install it. No software can provide security if it is
installed on compromised hardware. Do not install Qubes on a computer
you don’t trust. See installation security for more information.
Writing guidelines
------------------
Correct use of terminology
^^^^^^^^^^^^^^^^^^^^^^^^^^
Familiarize yourself with the terms defined in the
:doc:`glossary </user/reference/glossary>`. Use these terms consistently and
accurately throughout your writing.
Sentence case in headings
^^^^^^^^^^^^^^^^^^^^^^^^^
Use sentence case (rather than title case) in headings for the reasons
explained
`here <https://www.sallybagshaw.com.au/articles/sentence-case-v-title-case/>`__.
In particular, since the authorship of the Qubes documentation is
decentralized and widely distributed among users from around the world,
many contributors come from regions with different conventions for
implementing title case, not to mention that there are often differing
style guide recommendations even within a single region. It is much
easier for all of us to implement sentence case consistently across our
growing body of pages, which is very important for managing the ongoing
maintenance burden and sustainability of the documentation.
Writing command-line examples
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
When providing command-line examples:
- Tell the reader where to open a terminal (dom0 or a specific domU),
and show the command along with its output (if any) in a code block,
e.g.:
.. code:: bash
Open a terminal in dom0 and run:
```
$ cd test
$ echo Hello
Hello
```
- Precede each command with the appropriate command prompt: At a
minimum, the prompt should contain a trailing ``#`` (for the user
``root``) or ``$`` (for other users) on Linux systems and ``>`` on
Windows systems, respectively.
- Don’t try to add comments inside the code block. For example, *don’t*
do this:
.. code:: bash
Open a terminal in dom0 and run:
```
# Navigate to the new directory
$ cd test
# Generate a greeting
$ echo Hello
Hello
```
The ``#`` symbol preceding each comment is ambiguous with a root
command prompt. Instead, put your comments *outside* of the code
block in normal prose.
Variable names in commands
^^^^^^^^^^^^^^^^^^^^^^^^^^
Syntactically distinguish variables in commands. For example, this is
ambiguous:
.. code:: bash
$ qvm-run --dispvm=disposable-template --service qubes.StartApp+xterm
It should instead be:
.. code:: bash
$ qvm-run --dispvm=<DISPOSABLE_TEMPLATE> --service qubes.StartApp+xterm
Note that we syntactically distinguish variables in three ways:
1. Surrounding them in angled brackets (``< >``)
2. Using underscores (``_``) instead of spaces between words
3. Using all capital letters
We have observed that many novices make the mistake of typing the
surrounding angled brackets (``< >``) on the command line, even after
substituting the desired real value between them. Therefore, in
documentation aimed at novices, we also recommend clarifying that the
angled brackets should not be typed. This can be accomplished in one of
several ways:
- Explicitly say something like “without the angled brackets.”
- Provide an example command using real values that excludes the angled
brackets.
- If you know that almost all users will want to use (or should use) a
specific command containing all real values and no variables, you
might consider providing exactly that command and forgoing the
version with variables. Novices may not realize which parts of the
command they can substitute with different values, but if you’ve
correctly judged that they should use the command you’ve provided as
is, then this shouldn’t matter.
Capitalization of "qube"
^^^^^^^^^^^^^^^^^^^^^^^^
We introduced the term :ref:`“qube” <user/reference/glossary:qube>` as a
user-friendly alternative to the term :ref:`“virtual machine” (“VM”) <user/reference/glossary:vm>` in the context of Qubes OS. Nonetheless,
“qube” is a common noun like the words “compartment” and “container.”
Therefore, in English, “qube” follows the standard capitalization rules
for common nouns. For example, “I have three qubes” is correct, while “I
have three Qubes” is incorrect. Like other common nouns, “qube” should
still be capitalized at the beginnings of sentences, the beginnings of
sentence-case headings, and in title-case headings. Note, however, that
starting a sentence with the plural of “qube” (e.g., “Qubes can be shut
down…”) can be ambiguous, since it may not be clear whether the referent
is a plurality of qubes, :ref:`Qubes OS <user/reference/glossary:qubes os>`, or even
the Qubes OS Project itself. Hence, it is generally a good idea to
rephrase such sentences in order to avoid this ambiguity.
Many people feel a strong temptation to capitalize the word “qube” all
the time, like a proper noun, perhaps because it’s a new and unfamiliar
term that’s closely associated with a particular piece of software
(namely, Qubes OS). However, these factors are not relevant to the
capitalization rules of English. In fact, it’s not unusual for new
common nouns to be introduced into English, especially in the context of
technology. For example, “blockchain” is a relatively recent technical
term that’s a common noun. Why is it a common noun rather than a proper
noun? Because proper nouns refer to *particular* people, places, things,
and ideas. There are many different blockchains. However, even when
there was just one, the word still denoted a collection of things rather
than a particular thing. It happened to be the case that there was only
one member in that collection at the time. For example, if there
happened to be only one tree in the world, that wouldn’t change the way
we capitalize sentences like, “John sat under a tree.” Intuitively, it
makes sense that the addition and removal of objects from the world
shouldn’t cause published books to become orthographicallly incorrect
while sitting on their shelves.
Accordingly, the reason “qube” is a common noun rather than a proper
noun is because it doesn’t refer to any one specific thing (in this
case, any one specific virtual machine). Rather, it’s the term for any
virtual machine in a Qubes OS installation. (Technically, while qubes
are currently implemented as virtual machines, Qubes OS is independent
of its underlying compartmentalization technology. Virtual machines
could be replaced with a different technology, and qubes would still be
called “qubes.”)
I have several qubes in my Qubes OS installation, and you have several
in yours. Every Qubes OS user has their own set of qubes, just as each
of us lives in some neighborhood on some street. Yet we aren’t tempted
to treat words like “neighborhood” or “street” as proper nouns (unless,
of course, they’re part of a name, like “Acorn Street”). Again, while
this might seem odd because “qube” is a new word that we invented, that
doesn’t change how English works. After all, *every* word was a new word
that someone invented at some point (otherwise we wouldn’t have any
words at all). We treat “telephone,” “computer,” “network,” “program,”
and so on as common nouns, even though those were all new technological
inventions in the not-too-distant past (on a historical scale, at
least). So, we shouldn’t allow ourselves to be confused by irrelevant
factors, like the fact that the inventors happened to be *us* or that
the invention was *recent* or is not in widespread use among humanity.
English language conventions
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
For the sake of consistency and uniformity, the Qubes documentation aims
to follow the conventions of American English, where applicable. (Please
note that this is an arbitrary convention for the sake consistency and
not a value judgment about the relative merits of British versus
American English.)
Organizational guidelines
-------------------------
Do not duplicate documentation
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Duplicating documentation is almost always a bad idea. There are many
reasons for this. The main one is that almost all documentation has to
be updated as some point. When similar documentation appears in more
than one place, it is very easy for it to get updated in one place but
not the others (perhaps because the person updating it doesn’t realize
it’s in more than once place). When this happens, the documentation as a
whole is now inconsistent, and the outdated documentation becomes a
trap, especially for novice users. Such traps are often more harmful
than if the documentation never existed in the first place. The solution
is to **link** to existing documentation rather than duplicating it.
There are some exceptions to this policy (e.g., information that is
certain not to change for a very long time), but they are rare.
Core vs. external documentation
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Core documentation resides in the `Qubes OS Project’s official repositories <https://github.com/QubesOS/>`__, mainly in
`qubes-doc <https://github.com/QubesOS/qubes-doc>`__. External
documentation can be anywhere else (such as forums, community websites,
and blogs), but there is an especially large collection in the `Qubes Forum <https://forum.qubes-os.org/docs>`__. External documentation
should not be submitted to
`qubes-doc <https://github.com/QubesOS/qubes-doc>`__. If you’ve written
a piece of documentation that is not appropriate for
`qubes-doc <https://github.com/QubesOS/qubes-doc>`__, we encourage you
to submit it to the `Qubes Forum <https://forum.qubes-os.org/docs>`__
instead. However, *linking* to external documentation from
`qubes-doc <https://github.com/QubesOS/qubes-doc>`__ is perfectly fine.
Indeed, the maintainers of the `Qubes Forum <https://forum.qubes-os.org/>`__ should regularly submit PRs
against the documentation index (see :ref:`How to edit the documentation index <developer/general/how-to-edit-the-documentation:how to edit the documentation index>`)
to add and update Qubes Forum links in the :ref:`“External documentation” <index:external documentation>` section of the
documentation table of contents.
The main difference between **core** (or **official**) and **external**
(or **community** or **unofficial**) documentation is whether it
documents software that is officially written and maintained by the
Qubes OS Project. The purpose of this distinction is to keep the core
docs maintainable and high-quality by limiting them to the software
output by the Qubes OS Project. In other words, we take responsibility
for documenting all of the software we put out into the world, but it
doesn’t make sense for us to take on the responsibility of documenting
or maintaining documentation for anything else. For example, Qubes OS
may use a popular Linux distribution for an official
:doc:`TemplateVM </user/templates/templates>`. However, it would not make sense for a
comparatively small project like ours, with modest funding and a lean
workforce, to attempt to document software belonging to a large,
richly-funded project with an army of paid and volunteer contributors,
especially when they probably already have documentation of their own.
This is particularly true when it comes to Linux in general. Although
many users who are new to Qubes are also new to Linux, it makes
absolutely no sense for our comparatively tiny project to try to
document Linux in general when there is already a plethora of
documentation out there.
Many contributors do not realize that there is a significant amount of
work involved in *maintaining* documentation after it has been written.
They may wish to write documentation and submit it to the core docs, but
they see only their own writing process and fail to consider that it
will have to be kept up-to-date and consistent with the rest of the docs
for years afterward. Submissions to the core docs also have to :ref:`undergo a review process <developer/general/how-to-edit-the-documentation:security>` to
ensure accuracy before being merged, which takes up valuable time from
the team. We aim to maintain high quality standards for the core docs
(style and mechanics, formatting), which also takes up a lot of time. If
the documentation involves anything external to the Qubes OS Project
(such as a website, platform, program, protocol, framework, practice, or
even a reference to a version number), the documentation is likely to
become outdated when that external thing changes. It’s also important to
periodically review and update this documentation, especially when a new
Qubes release comes out. Periodically, there may be technical or policy
changes that affect all the core documentation. The more documentation
there is relative to maintainers, the harder all of this will be. Since
there are many more people who are willing to write documentation than
to maintain it, these individually small incremental additions amount to
a significant maintenance burden for the project.
On the positive side, we consider the existence of community
documentation to be a sign of a healthy ecosystem, and this is quite
common in the software world. The community is better positioned to
write and maintain documentation that applies, combines, and simplifies
the official documentation, e.g., tutorials that explain how to install
and use various programs in Qubes, how to create custom VM setups, and
introductory tutorials that teach basic Linux concepts and commands in
the context of Qubes. In addition, just because the Qubes OS Project has
officially written and maintains some flexible framework, such as
``qrexec``, it does not make sense to include every tutorial that says
“here’s how to do something cool with ``qrexec``” in the core docs. Such
tutorials generally also belong in the community documentation.
See `#4693 <https://github.com/QubesOS/qubes-issues/issues/4693>`__ for
more background information.
Release-specific documentation
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
*See* `#5308 <https://github.com/QubesOS/qubes-issues/issues/5308>`__
*for pending changes to this policy.*
We maintain only one set of documentation for Qubes OS. We do not
maintain a different set of documentation for each release of Qubes. Our
single set of Qubes OS documentation is updated on a continual, rolling
basis. Our first priority is to document all **current, stable releases** of Qubes. Our second priority is to document the next,
upcoming release (if any) that is currently in the beta or release
candidate stage.
In cases where a documentation page covers functionality that differs
considerably between Qubes OS releases, the page should be subdivided
into clearly-labeled sections that cover the different functionality in
different releases (examples below).
In general, avoid mentioning specific Qubes versions in the body text of
documentation, as these references rapidly go out of date and become
misleading to readers.
Incorrect Example
^^^^^^^^^^^^^^^^^
.. code:: bash
## How to Foo
Fooing is the process by which one foos. There are both general and specific
versions of fooing, which vary in usefulness depending on your goals, but for
the most part, all fooing is fooing.
To foo in Qubes 3.2:
$ qvm-foo <foo-bar>
Note that this does not work in Qubes 4.0, where there is a special widget
for fooing, which you can find in the lower-right corner of the screen in
the Foo Manager. Alternatively, you can use the more general `qubes-baz`
command introduced in 4.0:
$ qubes-baz --foo <bar>
Once you foo, make sure to close the baz before fooing the next bar.
Correct Example
^^^^^^^^^^^^^^^
.. code:: bash
## Qubes 3.2
### How to Foo
Fooing is the process by which one foos. There are both general and specific
versions of fooing, which vary in usefulness depending on your goals, but for
the most part, all fooing is fooing.
To foo:
$ qvm-foo <foo-bar>
Once you foo, make sure to close the baz before fooing the next bar.
## Qubes 4.0
### How to Foo
Fooing is the process by which one foos. There are both general and specific
versions of fooing, which vary in usefulness depending on your goals, but for
the most part, all fooing is fooing.
There is a special widget for fooing, which you can find in the lower-right
corner of the screen in the Foo Manager. Alternatively, you can use the
general `qubes-baz` command:
$ qubes-baz --foo <bar>
Once you foo, make sure to close the baz before fooing the next bar.
Subdividing the page into clearly-labeled sections for each release has
several benefits:
- It preserves good content for older (but still supported) releases.
Many documentation contributors are also people who prefer to use the
latest release. Many of them are tempted to *replace* existing
content that applies to an older, supported release with content that
applies only to the latest release. This is somewhat understandable.
Since they only use the latest release, they may be focused on their
own experience, and they may even regard the older release as
deprecated, even when it’s actually still supported. However,
allowing this replacement of content would do a great disservice to
those who still rely on the older, supported release. In many cases,
these users value the stability and reliability of the older,
supported release. With the older, supported release, there has been
more time to fix bugs and make improvements in both the software and
the documentation. Consequently, much of the documentation content
for this release may have gone through several rounds of editing,
review, and revision. It would be a tragedy for this content to
vanish while the very set of users who most prize stability and
reliability are depending on it.
- It’s easy for readers to quickly find the information they’re looking
for, since they can go directly to the section that applies to their
release.
- It’s hard for readers to miss information they need, since it’s all
in one place. In the incorrect example, information that the reader
needs could be in any paragraph in the entire document, and there’s
no way to tell without reading the entire page. In the correct
example, the reader can simply skim the headings in order to know
which parts of the page need to be read and which can be safely
ignored. The fact that some content is repeated in the two
release-specific sections is not a problem, since no reader has to
read the same thing twice. Moreover, as one release gets updated,
it’s likely that the documentation for that release will also be
updated. Therefore, content that is initially duplicated between
release-specific sections will not necessarily stay that way, and
this is a good thing: We want the documentation for a release that
*doesn’t* change to stay the same, and we want the documentation for
a release that *does* change to change along with the software.
- It’s easy for documentation contributors and maintainers to know
which file to edit and update, since there’s only one page for all
Qubes OS releases. Initially creating the new headings and
duplicating content that applies to both is only a one-time cost for
each page, and many pages don’t even require this treatment, since
they apply to all currently-supported Qubes OS releases.
By contrast, an alternative approach, such as segregating the
documentation into two different branches, would mean that contributions
that apply to both Qubes releases would only end up in one branch,
unless someone remembered to manually submit the same thing to the other
branch and actually made the effort to do so. Most of the time, this
wouldn’t happen. When it did, it would mean a second pull request that
would have to be reviewed. Over time, the different branches would
diverge in non-release-specific content. Good general content that was
submitted only to one branch would effectively disappear once that
release was deprecated. (Even if it were still on the website, no one
would look at it, since it would explicitly be in the subdirectory of a
deprecated release, and there would be a motivation to remove it from
the website so that search results wouldn’t be populated with
out-of-date information.)
For further discussion about release-specific documentation in Qubes,
see
`here <https://groups.google.com/d/topic/qubes-users/H9BZX4K9Ptk/discussion>`__.
Git conventions
---------------
Please follow our :ref:`Git commit message guidelines <developer/code/coding-style:commit message guidelines>`.

660
developer/general/gsoc.md
View File

@ -1,660 +0,0 @@
---
lang: en
layout: doc
permalink: /gsoc/
redirect_from:
- /GSoC/
ref: 33
title: Google Summer of Code (GSoC)
---
## Information for Students
Thank you for your interest in participating in the [Google Summer of Code program](https://summerofcode.withgoogle.com/) with the [Qubes OS team](/team/). You can read more about the Google Summer of Code program at the [official website](https://summerofcode.withgoogle.com/) and the [official FAQ](https://developers.google.com/open-source/gsoc/faq).
Being accepted as a Google Summer of Code contributor is quite competitive. If you are interested in participating in the Summer of Code please be aware that you must be able to produce code for Qubes OS for 3-5 months. Your mentors, Qubes developers, will dedicate a portion of their time towards mentoring you. Therefore, we seek candidates who are committed to helping Qubes long-term and are willing to do quality work and be proactive in communicating with your mentor.
You don't have to be a proven developer -- in fact, this whole program is meant to facilitate joining Qubes and other free and open source communities. The Qubes community maintains information about [contributing to Qubes development](/doc/contributing/#contributing-code) and [how to send patches](/doc/source-code/#how-to-send-patches). In order to contribute code to the Qubes project, you must be able to [sign your code](/doc/code-signing/).
You should start learning the components that you plan on working on before the start date. Qubes developers are available on the [mailing lists](/support/#qubes-devel) for help. The GSoC timeline reserves a lot of time for bonding with the project -- use that time wisely. Good communication is key, you should plan to communicate with your team daily and formally report progress and plans weekly. Students who neglect active communication will be failed.
### Overview of Steps
- Join the [qubes-devel list](/support/#qubes-devel) and introduce yourself, and meet your fellow developers
- Read [Google's instructions for participating](https://developers.google.com/open-source/gsoc/) and the [GSoC Student Manual](https://google.github.io/gsocguides/student/)
- Take a look at the list of ideas below
- Come up with a project that you are interested in (and feel free to propose your own! Don't feel limited by the list below.)
- Read the Contributor Proposal guidelines below
- Write a first draft proposal and send it to the qubes-devel mailing list for review
- Submit proposal using [Google's web interface](https://summerofcode.withgoogle.com/) ahead of the deadline (this requires a Google Account!)
- Submit proof of enrollment well ahead of the deadline
Coming up with an interesting idea that you can realistically achieve in the time available to you (one summer) is probably the most difficult part. We strongly recommend getting involved in advance of the beginning of GSoC, and we will look favorably on applications from prospective contributors who have already started to act like free and open source developers.
Before the summer starts, there are some preparatory tasks which are highly encouraged. First, if you aren't already, definitely start using Qubes as your primary OS as soon as possible! Also, it is encouraged that you become familiar and comfortable with the Qubes development workflow sooner than later. A good way to do this (and also a great way to stand out as an awesome applicant and make us want to accept you!) might be to pick up some issues from [qubes-issues](https://github.com/QubesOS/qubes-issues/issues) (our issue-tracking repo) and submit some patches addressing them. Some suitable issues might be those with tags ["help wanted" and "P: minor"](https://github.com/QubesOS/qubes-issues/issues?q=is%3Aissue%20is%3Aopen%20label%3A%22P%3A%20minor%22%20label%3A%22help%20wanted%22) (although more significant things are also welcome, of course). Doing this will get you some practice with [qubes-builder](/doc/qubes-builder/), our code-signing policies, and some familiarity with our code base in general so you are ready to hit the ground running come summer.
### Contributor proposal guidelines
A project proposal is what you will be judged upon. Write a clear proposal on what you plan to do, the scope of your project, and why we should choose you to do it. Proposals are the basis of the GSoC projects and therefore one of the most important things to do well.
Below is the application template:
```
# Introduction
Every software project should solve a problem. Before offering the solution (your Google Summer of Code project), you should first define the problem. What’s the current state of things? What’s the issue you wish to solve and why? Then you should conclude with a sentence or two about your solution. Include links to discussions, features, or bugs that describe the problem further if necessary.
# Project goals
Be short and to the point, and perhaps format it as a list. Propose a clear list of deliverables, explaining exactly what you promise to do and what you do not plan to do. “Future developments” can be mentioned, but your promise for the Google Summer of Code term is what counts.
# Implementation
Be detailed. Describe what you plan to do as a solution for the problem you defined above. Include technical details, showing that you understand the technology. Illustrate key technical elements of your proposed solution in reasonable detail.
# Timeline
Show that you understand the problem, have a solution, have also broken it down into manageable parts, and that you have a realistic plan on how to accomplish your goal. Here you set expectations, so don’t make promises you can’t keep. A modest, realistic and detailed timeline is better than promising the impossible.
If you have other commitments during GSoC, such as a job, vacation, exams, internship, seminars, or papers to write, disclose them here. GSoC should be treated like a full-time job, and we will expect approximately 40 hours of work per week. If you have conflicts, explain how you will work around them. If you are found to have conflicts which you did not disclose, you may be failed.
Open and clear communication is of utmost importance. Include your plans for communication in your proposal; daily if possible. You will need to initiate weekly formal communications such as a detailed email to the qubes-devel mailing list. Lack of communication will result in you being failed.
# About me
Provide your contact information and write a few sentences about you and why you think you are the best for this job. Prior contributions to Qubes are helpful; list your commits. Name people (other developers, students, professors) who can act as a reference for you. Mention your field of study if necessary. Now is the time to join the relevant mailing lists. We want you to be a part of our community, not just contribute your code.
Tell us if you are submitting proposals to other organizations, and whether or not you would choose Qubes if given the choice.
Other things to think about:
* Are you comfortable working independently under a supervisor or mentor who is several thousand miles away, and perhaps 12 time zones away? How will you work with your mentor to track your work? Have you worked in this style before?
* If your native language is not English, are you comfortable working closely with a supervisor whose native language is English? What is your native language, as that may help us find a mentor who has the same native language?
* After you have written your proposal, you should get it reviewed. Do not rely on the Qubes mentors to do it for you via the web interface, although we will try to comment on every proposal. It is wise to ask a colleague or a developer to critique your proposal. Clarity and completeness are important.
```
## Project Ideas
These project ideas were contributed by our developers and may be incomplete. If you are interested in submitting a proposal based on these ideas, you should contact the [qubes-devel mailing list](/support/#qubes-devel) and associated GitHub issue to learn more about the idea.
```
### Adding a Proposal
**Project**: Something that you're totally excited about
**Brief explanation**: What is the project, where does the code live?
**Expected results**: What is the expected result in the timeframe given
**Difficulty**: easy / medium / hard
**Knowledge prerequisite**: Pre-requisites for working on the project. What coding language and knowledge is needed?
If applicable, links to more information or discussions
**Size of the project**: either 175 hours (medium) or 350 hours (large)
**Mentor**: Name and email address.
```
### Qubes as a Vagrant provider
**Project**: Qubes as a Vagrant provider
**Brief explanation**: Currently using Vagrant on Qubes requires finding an image that uses Docker as isolation provider and running Docker in a qube, or downloading the Vagrantfile and manually setting up a qube according to the Vagrantfile. This project aims at simplifying this workflow. Since introduction of Admin API, it's possible for a qube to provision another qube - which is exactly what is needed for Vagrant. [Related discussion](https://groups.google.com/d/msgid/qubes-devel/535299ca-d16a-4a70-8223-a4ac6be4be41%40googlegroups.com)
**Expected results**:
- Design how Vagrant Qubes provider should look like, including:
- [box format](https://www.vagrantup.com/docs/plugins/providers.html#box-format)
- method for running commands inside (ssh vs qvm-run)
- Write a Vagrant provider able to create/start/stop/etc a VM
- Document how to configure and use the provider, including required qrexec policy changes and possibly firewall rules
- Write integration tests
**Difficulty**: medium
**Knowledge prerequisite**:
- Ruby
- Vagrant concepts
**Size of the project**: 350 hours
**Mentor**: [Wojtek Porczyk](/team/), [Marek Marczykowski-Górecki](/team/)
### System health monitor
**Project**: System health monitor
**Brief explanation**: A tool that informs the user about common system and configuration issues. Some of this is already available, but scattered across different places. See related issues: [6663](https://github.com/QubesOS/qubes-issues/issues/6663), [2134](https://github.com/QubesOS/qubes-issues/issues/2134)
**Expected results**:
- a tool / service that checks for common issues and things needing user attention, for example:
- some updates to be applied (separate widget already exists)
- running out of disk space (separate widget already exists)
- insecure USB configuration (USB in dom0)
- some system VM crashed
- ...
- a GUI that provides terse overview of the system state, and notifies the user if something bad happens
**Difficulty**: medium
**Knowledge prerequisite**:
- Python
- basic knowledge about systemd services
- PyGTK (optional)
**Size of the project**: 350 hours
**Mentor**: [Marta Marczykowska-Górecka](/team/)
### Mechanism for maintaining in-VM configuration
**Project**: Mechanism for maintaining in-VM configuration
**Brief explanation**: Large number of VMs is hard to maintain. Templates helps with keeping them updated, but many applications have configuration in user home directory, which is not synchronized.
**Expected results**:
- Design a mechanism how to _safely_ synchronize application configuration living in user home directory (`~/.config`, some other "dotfiles"). Mechanism should be resistant against malicious VM forcing its configuration on other VMs. Some approach could be a strict control which VM can send what changes (whitelist approach, not blacklist).
- Implementation of the above mechanism.
- Documentation how to configure it securely.
**Difficulty**: medium
**Knowledge prerequisite**:
- shell and/or python scripting
- Qubes OS qrexec services
**Size of the project**: 175 hours
**Mentor**: [Frédéric Pierret](/team/)
<!--
REMOVED as of February 2022: work is being done on this
### Wayland support in GUI agent and/or GUI daemon
**Project**: Wayland support in GUI agent and/or GUI daemon
**Brief explanation**: Currently both GUI agent (VM side of the GUI virtualization) and GUI daemon (dom0 side of GUI virtualization) support X11 protocol only. It may be useful to add support for Wayland there. Note that those are in fact two independent projects:
1. GUI agent - make it work as Wayland compositor, instead of extracting window's composition buffers using custom X11 driver
2. GUI daemon - act as Wayland application, showing windows retrieved from VMs, keeping zero-copy display path (window content is directly mapped from application running in VM, not copied)
**Expected results**:
Choose either of GUI agent, GUI daemon. Both are of similar complexity and each separately looks like a good task for GSoC time period.
- design relevant GUI agent/daemon changes, the GUI protocol should not be affected
- consider window decoration handling - VM should have no way of spoofing those, so it must be enforced by GUI daemon (either client-side - by GUI daemon itself, or server-side, based on hints given by GUI daemon)
- implement relevant GUI agent/daemon changes
- implement tests for new GUI handling, similar to existing tests for X11 based GUI
Relevant links:
- [Low level GUI documentation](/doc/gui/)
- [qubes-gui-agent-linux](https://github.com/qubesos/qubes-gui-agent-linux)
- [qubes-gui-daemon](https://github.com/qubesos/qubes-gui-daemon)
- [Use Wayland instead of X11 to increase performance](https://github.com/qubesos/qubes-issues/issues/3366)
**Knowledge prerequisite**:
- Wayland architecture
- basics of X11 (for understanding existing code)
- C language
- using shared memory (synchronization methods etc)
**Mentor**: [Marek Marczykowski-Górecki](/team/).
-->
### Qubes Live USB
**Project**: Revive Qubes Live USB, integrate it with installer
**Brief explanation**: Qubes Live USB is based on Fedora tools to build live
distributions. But for Qubes we need some adjustments: starting Xen instead of
Linux kernel, smarter copy-on-write handling (we run there multiple VMs, so a
lot more data to save) and few more. Additionally in Qubes 3.2 we have
so many default VMs that default installation does not fit in 16GB image
(default value) - some subset of those VMs should be chosen. Ideally we'd like
to have just one image being both live system and installation image. More
details: [#1552](https://github.com/QubesOS/qubes-issues/issues/1552),
[#1965](https://github.com/QubesOS/qubes-issues/issues/1965).
**Expected results**:
- Adjust set of VMs and templates included in live edition.
- Update and fix build scripts for recent Qubes OS version.
- Update startup script to mount appropriate directories as either
copy-on-write (device-mapper snapshot), or tmpfs.
- Optimize memory usage: should be possible to run sys-net, sys-firewall, and
at least two more VMs on 4GB machine. This include minimizing writes to
copy-on-write layer and tmpfs (disable logging etc).
- Research option to install the system from live image. If feasible add
this option.
**Difficulty**: hard
**Knowledge prerequisite**:
- System startup sequence: bootloaders (isolinux, syslinux, grub, UEFI), initramfs, systemd.
- Python and Bash scripting
- Filesystems and block devices: loop devices, device-mapper, tmpfs, overlayfs, sparse files.
**Size of the project**: 350 hours
**Mentor**: [Frédéric Pierret](/team/)
<!--
### Unikernel-based firewallvm with Qubes firewall settings support
REMOVED as of January 2020: work is being done on this
**Project**: Unikernel based firewallvm with Qubes firewall settings support
**Brief explanation**: [blog post](https://roscidus.com/blog/blog/2016/01/01/a-unikernel-firewall-for-qubesos/), [repo](https://github.com/talex5/qubes-mirage-firewall)
**Expected results**: A firewall implemented as a unikernel which supports all the networking-related functionality as the default sys-firewall VM, including configuration via Qubes Manager. Other duties currently assigned to sys-firewall such as the update proxy may need to be appropriately migrated first.
**Knowledge prerequisite**:
- [OCaml](https://ocaml.org/) + [MirageOS](https://mirage.io/) or other unikernel framework,
- Xen network stack,
- Qubes networking model & firewall semantics.
**Mentor**: [Thomas Leonard](mailto:talex5@gmail.com), [Marek Marczykowski-Górecki](/team/)
-->
### LogVM(s)
**Project**: LogVM(s)
**Brief explanation**: Qubes AppVMs do not have persistent /var (on purpose).
It would be useful to send logs generated by various VMs to a dedicated
log-collecting VM. This way logs will not only survive VM shutdown, but also be
immune to altering past entries. See
[#830](https://github.com/QubesOS/qubes-issues/issues/830) for details.
**Expected results**:
- Design a _simple_ protocol for transferring logs. The less metadata (parsed
in log-collecting VM) the better.
- Implement log collecting service. Besides logs itself, should save
information about logs origin (VM name) and timestamp. The service should
_not_ trust sending VM in any of those.
- Implement log forwarder compatible with systemd-journald and rsyslog. A
mechanism (service/plugin) fetching logs in real time from those and sending
to log-collecting VM over qrexec service.
- Document the protocol.
- Write unit tests and integration tests.
**Difficulty**: easy
**Knowledge prerequisite**:
- syslog
- systemd
- Python/Bash scripting
**Size of the project**: 175 hours
**Mentor**: [Frédéric Pierret](/team/)
### Whonix IPv6 and nftables support
**Project**: Whonix IPv6 and nftables support
**Brief explanation**: [T509](https://phabricator.whonix.org/T509)
**Expected results**:
- Work at upstream Tor: An older version of [TransparentProxy](https://trac.torproject.org/projects/tor/wiki/doc/TransparentProxy) page was the origin of Whonix. Update that page for nftables / IPv6 support without mentioning Whonix. Then discuss that on the tor-talk mailing list for wider input. [here](https://trac.torproject.org/projects/tor/ticket/21397)
- implement corridor feature request add IPv6 support / port to nftables - [issue](https://github.com/rustybird/corridor/issues/39)
- port [whonix-firewall](https://github.com/Whonix/whonix-firewall) to nftables
- make connections to IPv6 Tor relays work
- make connections to IPv6 destinations work
**Difficulty**: medium
**Knowledge prerequisite**:
- nftables
- iptables
- IPv6
**Size of the project**: 175 hours
**Mentor**: [Patrick Schleizer](/team/)
### GUI agent for Windows 8/10
**Project**: GUI agent for Windows 8/10
**Brief explanation**: Add support for Windows 8+ to the Qubes GUI agent and video driver. Starting from Windows 8, Microsoft requires all video drivers to conform to the WDDM display driver model which is incompatible with the current Qubes video driver. Unfortunately the WDDM model is much more complex than the old XPDM one and officially *requires* a physical GPU device (which may be emulated). Some progress has been made to create a full WDDM driver that *doesn't* require a GPU device, but the driver isn't working correctly yet. Alternatively, WDDM model supports display-only drivers which are much simpler but don't have access to system video memory and rendering surfaces (a key feature that would simplify seamless GUI mode). [#1861](https://github.com/QubesOS/qubes-issues/issues/1861)
**Expected results**: Working display-only WDDM video driver or significant progress towards making the full WDDM driver work correctly.
**Difficulty**: hard
**Knowledge prerequisite**: C/C++ languages, familiarity with Windows API, familiarity with the core Windows WDM driver model. Ideally familiarity with the WDDM display driver model.
**Size of the project**: 175 hours
**Mentor**: [Rafał Wojdyła](/team/)
### GNOME support in dom0 / GUI VM
**Project**: GNOME support in dom0
**Brief explanation**: Integrating GNOME into Qubes dom0. This include:
- patching window manager to add colorful borders
- removing stuff not needed in dom0 (file manager(s), indexing services etc)
- adjusting menu for easy navigation (same applications in different VMs and such problems, dom0-related entries in one place)
- More info: [#1806](https://github.com/QubesOS/qubes-issues/issues/1806)
**Expected results**:
- Review existing support for other desktop environments (KDE, Xfce4, i3, awesome).
- Patch window manager to draw colorful borders (we use only server-side
decorations), there is already very similar patch in
[Cappsule project](https://github.com/cappsule/cappsule-gui).
- Configure GNOME to not make use of dom0 user home in visible way (no search
in files there, no file manager, etc).
- Configure GNOME to not look into external devices plugged in (no auto
mounting, device notifications etc).
- Package above modifications as RPMs, preferably as extra configuration files
and/or plugins than overwriting existing files. Exceptions to this rule may
apply if no other option.
- Adjust comps.xml (in installer-qubes-os repo) to define package group with
all required packages.
- Document installation procedure.
**Difficulty**: hard
**Knowledge prerequisite**:
- GNOME architecture
- C language (patching metacity)
- Probably also javascript - for modifying GNOME shell extensions
**Size of the project**: 175 hours
**Mentor**: [Frédéric Pierret](/team/), [Marek Marczykowski-Górecki](/team/)
### Generalize the Qubes PDF Converter to other types of files
**Project**: Qubes Converters
**Brief explanation**: One of the pioneering ideas of Qubes is to use disposable virtual machines to convert untrustworthy files (such as documents given to journalists by unknown and potentially malicious whistleblowers) into trustworthy files. See [Joanna's blog on the Qubes PDF Convert](https://theinvisiblethings.blogspot.co.uk/2013/02/converting-untrusted-pdfs-into-trusted.html) for details of the idea. Joanna has implemented a prototype for PDF documents. The goal of this project would be to generalize beyond the simple prototype to accommodate a wide variety of file formats, including Word documents, audio files, video files, spreadsheets, and so on. The converters should prioritise safety over faithful conversion. For example the Qubes PDF converter typically leads to lower quality PDFs (e.g. cut and paste is no longer possible), because this makes the conversion process safer.
**Expected results**: We expect that in the timeframe, it will be possible to implement many converters for many file formats. However, if any unexpected difficulties arise, we would prioritise a small number of safe and high quality converters over a large number of unsafe or unuseful converters.
**Difficulty**: easy
**Knowledge prerequisite**: Most of the coding will probably be implemented as shell scripts to interface with pre-existing converters (such as ImageMagick in the Qubes PDF converter). However, shell scripts are not safe for processing untrusted data, so any extra processing will need to be implemented in another language -- probably Python.
**Size of the project**: 175 hours
**Mentors**: Andrew Clausen and Jean-Philippe Ouellet
### Progress towards reproducible builds
**Project**: Progress towards reproducible builds
**Brief explanation**: A long-term goal is to be able to build the entire OS and installation media in a completely bit-wise deterministic manner, but there are many baby steps to be taken along that path. See:
- "[Security challenges for the Qubes build process](/news/2016/05/30/build-security/)"
- [This mailing list post](https://groups.google.com/d/msg/qubes-devel/gq-wb9wTQV8/mdliS4P2BQAJ)
- and [reproducible-builds.org](https://reproducible-builds.org/)
for more information and qubes-specific background.
**Expected results**: Significant progress towards making the Qubes build process deterministic. This would likely involve cooperation with and hacking on several upstream build tools to eliminate sources of variability.
**Difficulty**: medium
**Knowledge prerequisite**: qubes-builder [[1]](/doc/qubes-builder/) [[2]](/doc/qubes-builder-details/) [[3]](https://github.com/QubesOS/qubes-builder/tree/master/doc), and efficient at introspecting complex systems: comfortable with tracing and debugging tools, ability to quickly identify and locate issues within a large codebase (upstream build tools), etc.
**Size of the project**: 350 hours
**Mentor**: [Marek Marczykowski-Górecki](/team/)
### Porting Qubes to ARM/aarch64
**Project**: Porting Qubes to ARM/aarch64
**Brief explanation**:
Qubes currently only supports the x86_64 CPU architecture. Xen currently has additional support for ARM32/ARM64 processors, however work needs to be done to integrate this into the Qubes build process, as well as work in integrating this with the Qubes toolstack and security model. This may also be beneficial in simplifying the process of porting to other architectures.
Some related discussion:
- [#4318](https://github.com/QubesOS/qubes-issues/issues/4318) on porting to ppc64.
- [#3894](https://github.com/QubesOS/qubes-issues/issues/3894) on porting to L4 microkernel.
**Expected results**:
- Add cross-compilation support to qubes-builder and related components.
- Make aarch64 specific adjustments to Qubes toolstacks/manager (including passthrough of devices from device tree to guest domains).
- Aarch64 specific integration and unit tests.
- Production of generic u-boot or uefi capable image/iso for target hardware.
**Difficulty**: hard
**Knowledge prerequisite**:
- Libvirt and Qubes toolstacks (C and python languages).
- Xen debugging.
- General ARM architecture knowledge.
**Size of the project**: 350 hours
**Mentor**: [Marek Marczykowski-Górecki](/team/)
<!--
REMOVED as of February 2021: work is being done on this
### Porting Qubes to POWER9/PPC64
**Project**: Porting Qubes to POWER9/ppc64
**Brief explanation**:
Qubes currently supports the x86_64 CPU architecture. PowerPC is desirable for security purposes as it is the only architecture where one can get performant hardware with entirely open source firmware. Xen has **deprecated** support for Power9/PPC64 processors. Here are two directions to tackle this project from:
- Port Qubes to KVM then work on ppc64 specifics
- Implement some missing functionality in KVM then implement KVM support in the Qubes Hypervisor Abstraction Layer and build process. Improving the HAL will also be beneficial for simplifying the process of porting to further architectures and hypervisors.
- Port Xen to ppc64 then work on Qubes specifics
- For more information on porting Xen see [this thread](https://markmail.org/message/vuk7atnyqfq52epp).
More information and further links can be found in the related issue:
[#4318](https://github.com/QubesOS/qubes-issues/issues/4318).
**Expected results**:
- Add cross-compilation support to qubes-builder and related components.
- Make ppc64 specific adjustments to Qubes toolstacks/manager (including passthrough of devices from device tree to guest domains).
- ppc64 specific integration and unit tests.
- Production of generic u-boot or uefi capable image/iso for target hardware.
**Knowledge prerequisite**:
- Libvirt and Qubes toolstacks (C and python languages).
- KVM or XEN internals
- General ppc64 architecture knowledge.
**Mentor**: [Marek Marczykowski-Górecki](/team/)
-->
### Android development in Qubes
**Project**: Research running Android in Qubes VM (probably HVM) and connecting it to Android Studio
**Brief explanation**: The goal is to enable Android development (and testing!)
on Qubes OS. Currently it's only possible using qemu-emulated Android for ARM.
Since it's software emulation it's rather slow.
Details, reference: [#2233](https://github.com/QubesOS/qubes-issues/issues/2233)
**Expected results**:
- a simple way of setting up Android qubes with hardware emulation
(distributed as a template or as a salt, handling various modern Android versions)
- figuring out and implementing an easy and secure way to connect an Android
qube to a development qube with Android studio
- documentation and tests
**Difficulty**: hard
**Knowledge prerequisite**:
**Size of the project**: 350 hours
**Mentor**: Inquire on [qubes-devel](/support/#qubes-devel).
### Admin API Fuzzer
**Project**: Develop a [Fuzzer](https://en.wikipedia.org/wiki/Fuzzing) for the
[Qubes OS Admin API](/doc/admin-api/).
**Brief explanation**: The [Qubes OS Admin API](/doc/admin-api/)
enables VMs to execute privileged actions on other VMs or dom0 - if allowed by the Qubes OS RPC policy.
Programming errors in the Admin API however may cause these access rights to be more permissive
than anticipated by the programmer.
Since the Admin API is continuously growing and changing, continuous security assessments are required.
A [Fuzzer](https://en.wikipedia.org/wiki/Fuzzing) would help to automate part of these assessments.
**Expected results**:
- fully automated & extensible Fuzzer for parts of the Admin API
- user & developer documentation
**Difficulty**: medium
**Prerequisites**:
- basic Python understanding
- some knowledge about fuzzing & existing fuzzing frameworks (e.g. [oss-fuzz](https://github.com/google/oss-fuzz/tree/master/projects/qubes-os))
- a hacker's curiosity
**Size of the project**: 175 hours
**Mentor**: Inquire on [qubes-devel](/support/#qubes-devel).
### Secure Boot support
**Project**: Add support for protecting boot binaries with Secure Boot technology, using user-generated keys.
**Brief explanation**: Since recently, Xen supports "unified EFI boot" which allows to sign not only Xen binary itself, but also dom0 kernel and their parameters. While the base technology is there, enabling it is a painful and complex process. The goal of this project is to integrate configuration of this feature into Qubes, automating as much as possible. See discussion in [issue #4371](https://github.com/QubesOS/qubes-issues/issues/4371)
**Expected results**:
- a tool to prepare relevant boot files for unified Xen EFI boot - this includes collecting Xen, dom0 kernel, initramfs, config file, and possibly few more (ucode update?); the tool should then sign the file with user provided key (preferably propose to generate it too)
- integrate it with updates mechanism, so new Xen or dom0 kernel will be picked up automatically
- include a fallback configuration that can be used for troubleshooting (main unified Xen EFI intentionally does not allow to manipulate parameters at boot time)
**Difficulty**: hard
**Knowledge prerequisite**:
- basic understanding of Secure Boot
- Bash and Python scripting
**Size of the project**: 175 hours
**Mentor**: [Marek Marczykowski-Górecki](/team/)
### Reduce logging of Disposable VMs
**Project**: Reduce logging of Disposable VMs
**Brief explanation**: Partial metadata of a DisposableVM is stored in the dom0 filesystem. This applies to various logs, GUI status files etc. There should be an option to hide as much of that as possible - including bypassing some logging, and removing various state files, or at the very least obfuscating any hints what is running inside DisposableVM. More details at [issue #4972](https://github.com/QubesOS/qubes-issues/issues/4972)
**Expected results**: A DisposableVM should not leave logs hinting what was running inside.
**Difficulty**: medium
**Knowledge prerequisite**:
- Python scripting
- Basic knowledge of Linux system services management (systemd, syslog etc)
**Size of the project**: 350 hours
**Mentor**: [Marek Marczykowski-Górecki](/team/)
## Past Projects
You can view the projects we had in 2017 in the [GSoC 2017 archive](https://summerofcode.withgoogle.com/archive/2017/organizations/5074771758809088/). We also participated in GSoC 2020 and GSoC 2021, and you can see the project in the [GSoC 2020 archive](https://summerofcode.withgoogle.com/archive/2020/organizations/4924517870206976/) and [GSoC 2021 archive](https://summerofcode.withgoogle.com/archive/2021/organizations/5682513023860736).
Here are some successful projects which have been implemented in the past by Google Summer of Code participants.
### Template manager, new template distribution mechanism
**Project**: Template manager, new template distribution mechanism
**Brief explanation**: Template VMs currently are distributed using RPM
packages. There are multiple problems with that, mostly related to static
nature of RPM package (what files belong to the package). This means such
Template VM cannot be renamed, migrated to another storage (like LVM), etc.
Also we don't want RPM to automatically update template package itself (which
would override all the user changes there). More details:
[#2064](https://github.com/QubesOS/qubes-issues/issues/2064),
[#2534](https://github.com/QubesOS/qubes-issues/issues/2534),
[#3573](https://github.com/QubesOS/qubes-issues/issues/3573).
**Expected results**:
- Design new mechanism for distributing templates (possibly including some
package format - either reuse something already existing, or design
new one). The mechanism needs to handle:
- integrity protection (digital signatures), not parsing any data in dom0
prior to signature verification
- efficient handling of large sparse files
- ability to deploy the template into various storage mechanisms (sparse
files, LVM thin volumes etc).
- template metadata, templates repository - enable the user to browse
available templates (probably should be done in dedicated VM, or DisposableVM)
- manual template removal by users (without it, see problems such
as [#5509](https://github.com/QubesOS/qubes-issues/issues/5509)
- Implement the above mechanism:
- tool to download named template - should perform download operation in
some VM (as dom0 have no network access), then transfer the data to dom0,
verify its integrity and then create Template VM and feed it's root
filesystem image with downloaded data.
- tool to browse templates repository - both CLI and GUI (preferably integrated
with existing Template Manager tool)
- integrate both tools - user should be able to choose some template to be
installed from repository browsing tool - see
[#1705](https://github.com/QubesOS/qubes-issues/issues/1705) for some idea
(this one lacks integrity verification, but a similar service could
be developed with that added)
- If new "package" format is developed, add support for it into
[linux-template-builder](https://github.com/QubesOS/qubes-linux-template-builder).
- Document the mechanism.
- Write unit tests and integration tests.
**Knowledge prerequisite**:
- Large files (disk images) handling (sparse files, archive formats)
- Bash and Python scripting
- Data integrity handling - digital signatures (gpg2, gpgv2)
- PyGTK
- RPM package format, (yum) repository basics
**Mentor**: [Marek Marczykowski-Górecki](/team/)
----
We adapted some of the language here about GSoC from the [KDE GSoC page](https://community.kde.org/GSoC).

991
developer/general/gsoc.rst Normal file
View File

@ -0,0 +1,991 @@
============================
Google Summer of Code (GSoC)
============================
Information for Students
------------------------
Thank you for your interest in participating in the `Google Summer of Code program <https://summerofcode.withgoogle.com/>`__ with the `Qubes OS team <https://www.qubes-os.org/team/>`__. You can read more about the Google Summer of Code
program at the `official website <https://summerofcode.withgoogle.com/>`__ and the `official FAQ <https://developers.google.com/open-source/gsoc/faq>`__.
Being accepted as a Google Summer of Code contributor is quite
competitive. If you are interested in participating in the Summer of
Code please be aware that you must be able to produce code for Qubes OS
for 3-5 months. Your mentors, Qubes developers, will dedicate a portion
of their time towards mentoring you. Therefore, we seek candidates who
are committed to helping Qubes long-term and are willing to do quality
work and be proactive in communicating with your mentor.
You don’t have to be a proven developer – in fact, this whole program is
meant to facilitate joining Qubes and other free and open source
communities. The Qubes community maintains information about
:ref:`contributing to Qubes development <introduction/contributing:contributing code>` and :ref:`how to send patches <developer/code/source-code:how to send patches>`. In order to
contribute code to the Qubes project, you must be able to :doc:`sign your code </developer/code/code-signing>`.
You should start learning the components that you plan on working on
before the start date. Qubes developers are available on the :ref:`mailing lists <introduction/support:qubes-devel>` for help. The GSoC timeline reserves a
lot of time for bonding with the project – use that time wisely. Good
communication is key, you should plan to communicate with your team
daily and formally report progress and plans weekly. Students who
neglect active communication will be failed.
Overview of Steps
^^^^^^^^^^^^^^^^^
- Join the :ref:`qubes-devel list <introduction/support:qubes-devel>` and introduce
yourself, and meet your fellow developers
- Read `Google’s instructions for participating <https://developers.google.com/open-source/gsoc/>`__
and the `GSoC Student Manual <https://google.github.io/gsocguides/student/>`__
- Take a look at the list of ideas below
- Come up with a project that you are interested in (and feel free to
propose your own! Don’t feel limited by the list below.)
- Read the Contributor Proposal guidelines below
- Write a first draft proposal and send it to the qubes-devel mailing
list for review
- Submit proposal using `Google’s web interface <https://summerofcode.withgoogle.com/>`__ ahead of the
deadline (this requires a Google Account!)
- Submit proof of enrollment well ahead of the deadline
Coming up with an interesting idea that you can realistically achieve in
the time available to you (one summer) is probably the most difficult
part. We strongly recommend getting involved in advance of the beginning
of GSoC, and we will look favorably on applications from prospective
contributors who have already started to act like free and open source
developers.
Before the summer starts, there are some preparatory tasks which are
highly encouraged. First, if you aren’t already, definitely start using
Qubes as your primary OS as soon as possible! Also, it is encouraged
that you become familiar and comfortable with the Qubes development
workflow sooner than later. A good way to do this (and also a great way
to stand out as an awesome applicant and make us want to accept you!)
might be to pick up some issues from
`qubes-issues <https://github.com/QubesOS/qubes-issues/issues>`__ (our
issue-tracking repo) and submit some patches addressing them. Some
suitable issues might be those with tags `“help wanted” and “P: minor” <https://github.com/QubesOS/qubes-issues/issues?q=is%3Aissue%20is%3Aopen%20label%3A%22P%3A%20minor%22%20label%3A%22help%20wanted%22>`__
(although more significant things are also welcome, of course). Doing
this will get you some practice with
:doc:`qubes-builder </developer/building/qubes-builder>`, our code-signing policies, and
some familiarity with our code base in general so you are ready to hit
the ground running come summer.
Contributor proposal guidelines
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
A project proposal is what you will be judged upon. Write a clear
proposal on what you plan to do, the scope of your project, and why we
should choose you to do it. Proposals are the basis of the GSoC projects
and therefore one of the most important things to do well.
Below is the application template:
.. code:: bash
# Introduction
Every software project should solve a problem. Before offering the solution (your Google Summer of Code project), you should first define the problem. What’s the current state of things? What’s the issue you wish to solve and why? Then you should conclude with a sentence or two about your solution. Include links to discussions, features, or bugs that describe the problem further if necessary.
# Project goals
Be short and to the point, and perhaps format it as a list. Propose a clear list of deliverables, explaining exactly what you promise to do and what you do not plan to do. “Future developments” can be mentioned, but your promise for the Google Summer of Code term is what counts.
# Implementation
Be detailed. Describe what you plan to do as a solution for the problem you defined above. Include technical details, showing that you understand the technology. Illustrate key technical elements of your proposed solution in reasonable detail.
# Timeline
Show that you understand the problem, have a solution, have also broken it down into manageable parts, and that you have a realistic plan on how to accomplish your goal. Here you set expectations, so don’t make promises you can’t keep. A modest, realistic and detailed timeline is better than promising the impossible.
If you have other commitments during GSoC, such as a job, vacation, exams, internship, seminars, or papers to write, disclose them here. GSoC should be treated like a full-time job, and we will expect approximately 40 hours of work per week. If you have conflicts, explain how you will work around them. If you are found to have conflicts which you did not disclose, you may be failed.
Open and clear communication is of utmost importance. Include your plans for communication in your proposal; daily if possible. You will need to initiate weekly formal communications such as a detailed email to the qubes-devel mailing list. Lack of communication will result in you being failed.
# About me
Provide your contact information and write a few sentences about you and why you think you are the best for this job. Prior contributions to Qubes are helpful; list your commits. Name people (other developers, students, professors) who can act as a reference for you. Mention your field of study if necessary. Now is the time to join the relevant mailing lists. We want you to be a part of our community, not just contribute your code.
Tell us if you are submitting proposals to other organizations, and whether or not you would choose Qubes if given the choice.
Other things to think about:
* Are you comfortable working independently under a supervisor or mentor who is several thousand miles away, and perhaps 12 time zones away? How will you work with your mentor to track your work? Have you worked in this style before?
* If your native language is not English, are you comfortable working closely with a supervisor whose native language is English? What is your native language, as that may help us find a mentor who has the same native language?
* After you have written your proposal, you should get it reviewed. Do not rely on the Qubes mentors to do it for you via the web interface, although we will try to comment on every proposal. It is wise to ask a colleague or a developer to critique your proposal. Clarity and completeness are important.
Project Ideas
-------------
These project ideas were contributed by our developers and may be
incomplete. If you are interested in submitting a proposal based on
these ideas, you should contact the :ref:`qubes-devel mailing list <introduction/support:qubes-devel>` and associated GitHub issue to learn
more about the idea.
.. code:: bash
### Adding a Proposal
**Project**: Something that you're totally excited about
**Brief explanation**: What is the project, where does the code live?
**Expected results**: What is the expected result in the timeframe given
**Difficulty**: easy / medium / hard
**Knowledge prerequisite**: Pre-requisites for working on the project. What coding language and knowledge is needed?
If applicable, links to more information or discussions
**Size of the project**: either 175 hours (medium) or 350 hours (large)
**Mentor**: Name and email address.
Qubes as a Vagrant provider
^^^^^^^^^^^^^^^^^^^^^^^^^^^
**Project**: Qubes as a Vagrant provider
**Brief explanation**: Currently using Vagrant on Qubes requires finding
an image that uses Docker as isolation provider and running Docker in a
qube, or downloading the Vagrantfile and manually setting up a qube
according to the Vagrantfile. This project aims at simplifying this
workflow. Since introduction of Admin API, it’s possible for a qube to
provision another qube - which is exactly what is needed for Vagrant.
`Related discussion <https://groups.google.com/d/msgid/qubes-devel/535299ca-d16a-4a70-8223-a4ac6be4be41%40googlegroups.com>`__
**Expected results**:
- Design how Vagrant Qubes provider should look like, including:
- `box format <https://www.vagrantup.com/docs/plugins/providers.html#box-format>`__
- method for running commands inside (ssh vs qvm-run)
- Write a Vagrant provider able to create/start/stop/etc a VM
- Document how to configure and use the provider, including required
qrexec policy changes and possibly firewall rules
- Write integration tests
**Difficulty**: medium
**Knowledge prerequisite**:
- Ruby
- Vagrant concepts
**Size of the project**: 350 hours
**Mentor**: `Wojtek Porczyk <https://www.qubes-os.org/team/>`__, `Marek Marczykowski-Górecki <https://www.qubes-os.org/team/>`__
System health monitor
^^^^^^^^^^^^^^^^^^^^^
**Project**: System health monitor
**Brief explanation**: A tool that informs the user about common system
and configuration issues. Some of this is already available, but
scattered across different places. See related issues:
`6663 <https://github.com/QubesOS/qubes-issues/issues/6663>`__,
`2134 <https://github.com/QubesOS/qubes-issues/issues/2134>`__
**Expected results**:
- a tool / service that checks for common issues and things needing
user attention, for example:
- some updates to be applied (separate widget already exists)
- running out of disk space (separate widget already exists)
- insecure USB configuration (USB in dom0)
- some system VM crashed
- …
- a GUI that provides terse overview of the system state, and notifies
the user if something bad happens
**Difficulty**: medium
**Knowledge prerequisite**:
- Python
- basic knowledge about systemd services
- PyGTK (optional)
**Size of the project**: 350 hours
**Mentor**: `Marta Marczykowska-Górecka <https://www.qubes-os.org/team/>`__
Mechanism for maintaining in-VM configuration
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
**Project**: Mechanism for maintaining in-VM configuration
**Brief explanation**: Large number of VMs is hard to maintain.
Templates helps with keeping them updated, but many applications have
configuration in user home directory, which is not synchronized.
**Expected results**:
- Design a mechanism how to *safely* synchronize application
configuration living in user home directory (``~/.config``, some
other “dotfiles”). Mechanism should be resistant against malicious VM
forcing its configuration on other VMs. Some approach could be a
strict control which VM can send what changes (whitelist approach,
not blacklist).
- Implementation of the above mechanism.
- Documentation how to configure it securely.
**Difficulty**: medium
**Knowledge prerequisite**:
- shell and/or python scripting
- Qubes OS qrexec services
**Size of the project**: 175 hours
**Mentor**: `Frédéric Pierret <https://www.qubes-os.org/team/>`__
<!--
REMOVED as of February 2022: work is being done on this
### Wayland support in GUI agent and/or GUI daemon
**Project**: Wayland support in GUI agent and/or GUI daemon
**Brief explanation**: Currently both GUI agent (VM side of the GUI virtualization) and GUI daemon (dom0 side of GUI virtualization) support X11 protocol only. It may be useful to add support for Wayland there. Note that those are in fact two independent projects:
1. GUI agent - make it work as Wayland compositor, instead of extracting window's composition buffers using custom X11 driver
2. GUI daemon - act as Wayland application, showing windows retrieved from VMs, keeping zero-copy display path (window content is directly mapped from application running in VM, not copied)
**Expected results**:
Choose either of GUI agent, GUI daemon. Both are of similar complexity and each separately looks like a good task for GSoC time period.
- design relevant GUI agent/daemon changes, the GUI protocol should not be affected
- consider window decoration handling - VM should have no way of spoofing those, so it must be enforced by GUI daemon (either client-side - by GUI daemon itself, or server-side, based on hints given by GUI daemon)
- implement relevant GUI agent/daemon changes
- implement tests for new GUI handling, similar to existing tests for X11 based GUI
Relevant links:
- `Low level GUI documentation <https://ark.intel.com/content/www/us/en/ark.html#@Processors>`__
- `qubes-gui-agent-linux <https://github.com/qubesos/qubes-gui-agent-linux>`__
- `qubes-gui-daemon <https://github.com/qubesos/qubes-gui-daemon>`__
- `Use Wayland instead of X11 to increase performance <https://github.com/qubesos/qubes-issues/issues/3366>`__
**Knowledge prerequisite**:
- Wayland architecture
- basics of X11 (for understanding existing code)
- C language
- using shared memory (synchronization methods etc)
**Mentor**: `Marek Marczykowski-Górecki <https://ark.intel.com/content/www/us/en/ark.html#@Processors>`__.
-->
Qubes Live USB
^^^^^^^^^^^^^^
**Project**: Revive Qubes Live USB, integrate it with installer
**Brief explanation**: Qubes Live USB is based on Fedora tools to build
live distributions. But for Qubes we need some adjustments: starting Xen
instead of Linux kernel, smarter copy-on-write handling (we run there
multiple VMs, so a lot more data to save) and few more. Additionally in
Qubes 3.2 we have so many default VMs that default installation does not
fit in 16GB image (default value) - some subset of those VMs should be
chosen. Ideally we’d like to have just one image being both live system
and installation image. More details:
`#1552 <https://github.com/QubesOS/qubes-issues/issues/1552>`__,
`#1965 <https://github.com/QubesOS/qubes-issues/issues/1965>`__.
**Expected results**:
- Adjust set of VMs and templates included in live edition.
- Update and fix build scripts for recent Qubes OS version.
- Update startup script to mount appropriate directories as either
copy-on-write (device-mapper snapshot), or tmpfs.
- Optimize memory usage: should be possible to run sys-net,
sys-firewall, and at least two more VMs on 4GB machine. This include
minimizing writes to copy-on-write layer and tmpfs (disable logging
etc).
- Research option to install the system from live image. If feasible
add this option.
**Difficulty**: hard
**Knowledge prerequisite**:
- System startup sequence: bootloaders (isolinux, syslinux, grub,
UEFI), initramfs, systemd.
- Python and Bash scripting
- Filesystems and block devices: loop devices, device-mapper, tmpfs,
overlayfs, sparse files.
**Size of the project**: 350 hours
**Mentor**: `Frédéric Pierret <https://www.qubes-os.org/team/>`__
<!--
### Unikernel-based firewallvm with Qubes firewall settings support
REMOVED as of January 2020: work is being done on this
**Project**: Unikernel based firewallvm with Qubes firewall settings support
**Brief explanation**: `blog post <https://roscidus.com/blog/blog/2016/01/01/a-unikernel-firewall-for-qubesos/>`__, `repo <https://github.com/talex5/qubes-mirage-firewall>`__
**Expected results**: A firewall implemented as a unikernel which supports all the networking-related functionality as the default sys-firewall VM, including configuration via Qubes Manager. Other duties currently assigned to sys-firewall such as the update proxy may need to be appropriately migrated first.
**Knowledge prerequisite**:
- `OCaml <https://ocaml.org/>`__ + `MirageOS <https://mirage.io/>`__ or other unikernel framework,
- Xen network stack,
- Qubes networking model & firewall semantics.
**Mentor**: `Thomas Leonard`_, `Marek Marczykowski-Górecki <https://ark.intel.com/content/www/us/en/ark.html#@Processors>`__
-->
LogVM(s)
^^^^^^^^
**Project**: LogVM(s)
**Brief explanation**: Qubes AppVMs do not have persistent /var (on
purpose). It would be useful to send logs generated by various VMs to a
dedicated log-collecting VM. This way logs will not only survive VM
shutdown, but also be immune to altering past entries. See
`#830 <https://github.com/QubesOS/qubes-issues/issues/830>`__ for
details.
**Expected results**:
- Design a *simple* protocol for transferring logs. The less metadata
(parsed in log-collecting VM) the better.
- Implement log collecting service. Besides logs itself, should save
information about logs origin (VM name) and timestamp. The service
should *not* trust sending VM in any of those.
- Implement log forwarder compatible with systemd-journald and rsyslog.
A mechanism (service/plugin) fetching logs in real time from those
and sending to log-collecting VM over qrexec service.
- Document the protocol.
- Write unit tests and integration tests.
**Difficulty**: easy
**Knowledge prerequisite**:
- syslog
- systemd
- Python/Bash scripting
**Size of the project**: 175 hours
**Mentor**: `Frédéric Pierret <https://www.qubes-os.org/team/>`__
Whonix IPv6 and nftables support
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
**Project**: Whonix IPv6 and nftables support
**Brief explanation**: `T509 <https://phabricator.whonix.org/T509>`__
**Expected results**:
- Work at upstream Tor: An older version of
`TransparentProxy <https://trac.torproject.org/projects/tor/wiki/doc/TransparentProxy>`__
page was the origin of Whonix. Update that page for nftables / IPv6
support without mentioning Whonix. Then discuss that on the tor-talk
mailing list for wider input.
`here <https://trac.torproject.org/projects/tor/ticket/21397>`__
- implement corridor feature request add IPv6 support / port to
nftables -
`issue <https://github.com/rustybird/corridor/issues/39>`__
- port `whonix-firewall <https://github.com/Whonix/whonix-firewall>`__
to nftables
- make connections to IPv6 Tor relays work
- make connections to IPv6 destinations work
**Difficulty**: medium
**Knowledge prerequisite**:
- nftables
- iptables
- IPv6
**Size of the project**: 175 hours
**Mentor**: `Patrick Schleizer <https://www.qubes-os.org/team/>`__
GUI agent for Windows 8/10
^^^^^^^^^^^^^^^^^^^^^^^^^^
**Project**: GUI agent for Windows 8/10
**Brief explanation**: Add support for Windows 8+ to the Qubes GUI agent
and video driver. Starting from Windows 8, Microsoft requires all video
drivers to conform to the WDDM display driver model which is
incompatible with the current Qubes video driver. Unfortunately the WDDM
model is much more complex than the old XPDM one and officially
*requires* a physical GPU device (which may be emulated). Some progress
has been made to create a full WDDM driver that *doesn’t* require a GPU
device, but the driver isn’t working correctly yet. Alternatively, WDDM
model supports display-only drivers which are much simpler but don’t
have access to system video memory and rendering surfaces (a key feature
that would simplify seamless GUI mode).
`#1861 <https://github.com/QubesOS/qubes-issues/issues/1861>`__
**Expected results**: Working display-only WDDM video driver or
significant progress towards making the full WDDM driver work correctly.
**Difficulty**: hard
**Knowledge prerequisite**: C/C++ languages, familiarity with Windows
API, familiarity with the core Windows WDM driver model. Ideally
familiarity with the WDDM display driver model.
**Size of the project**: 175 hours
**Mentor**: `Rafał Wojdyła <https://www.qubes-os.org/team/>`__
GNOME support in dom0 / GUI VM
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
**Project**: GNOME support in dom0
**Brief explanation**: Integrating GNOME into Qubes dom0. This include:
- patching window manager to add colorful borders
- removing stuff not needed in dom0 (file manager(s), indexing services
etc)
- adjusting menu for easy navigation (same applications in different
VMs and such problems, dom0-related entries in one place)
- More info:
`#1806 <https://github.com/QubesOS/qubes-issues/issues/1806>`__
**Expected results**:
- Review existing support for other desktop environments (KDE, Xfce4,
i3, awesome).
- Patch window manager to draw colorful borders (we use only
server-side decorations), there is already very similar patch in
`Cappsule project <https://github.com/cappsule/cappsule-gui>`__.
- Configure GNOME to not make use of dom0 user home in visible way (no
search in files there, no file manager, etc).
- Configure GNOME to not look into external devices plugged in (no auto
mounting, device notifications etc).
- Package above modifications as RPMs, preferably as extra
configuration files and/or plugins than overwriting existing files.
Exceptions to this rule may apply if no other option.
- Adjust comps.xml (in installer-qubes-os repo) to define package group
with all required packages.
- Document installation procedure.
**Difficulty**: hard
**Knowledge prerequisite**:
- GNOME architecture
- C language (patching metacity)
- Probably also javascript - for modifying GNOME shell extensions
**Size of the project**: 175 hours
**Mentor**: `Frédéric Pierret <https://www.qubes-os.org/team/>`__, `Marek Marczykowski-Górecki <https://www.qubes-os.org/team/>`__
Generalize the Qubes PDF Converter to other types of files
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
**Project**: Qubes Converters
**Brief explanation**: One of the pioneering ideas of Qubes is to use
disposable virtual machines to convert untrustworthy files (such as
documents given to journalists by unknown and potentially malicious
whistleblowers) into trustworthy files. See `Joanna’s blog on the Qubes PDF Convert <https://theinvisiblethings.blogspot.co.uk/2013/02/converting-untrusted-pdfs-into-trusted.html>`__
for details of the idea. Joanna has implemented a prototype for PDF
documents. The goal of this project would be to generalize beyond the
simple prototype to accommodate a wide variety of file formats,
including Word documents, audio files, video files, spreadsheets, and so
on. The converters should prioritise safety over faithful conversion.
For example the Qubes PDF converter typically leads to lower quality
PDFs (e.g. cut and paste is no longer possible), because this makes the
conversion process safer.
**Expected results**: We expect that in the timeframe, it will be
possible to implement many converters for many file formats. However, if
any unexpected difficulties arise, we would prioritise a small number of
safe and high quality converters over a large number of unsafe or
unuseful converters.
**Difficulty**: easy
**Knowledge prerequisite**: Most of the coding will probably be
implemented as shell scripts to interface with pre-existing converters
(such as ImageMagick in the Qubes PDF converter). However, shell scripts
are not safe for processing untrusted data, so any extra processing will
need to be implemented in another language – probably Python.
**Size of the project**: 175 hours
**Mentors**: Andrew Clausen and Jean-Philippe Ouellet
Progress towards reproducible builds
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
**Project**: Progress towards reproducible builds
**Brief explanation**: A long-term goal is to be able to build the
entire OS and installation media in a completely bit-wise deterministic
manner, but there are many baby steps to be taken along that path. See:
- “`Security challenges for the Qubes build process <https://www.qubes-os.org/news/2016/05/30/build-security/>`__”
- `This mailing list post <https://groups.google.com/d/msg/qubes-devel/gq-wb9wTQV8/mdliS4P2BQAJ>`__
- and `reproducible-builds.org <https://reproducible-builds.org/>`__
for more information and qubes-specific background.
**Expected results**: Significant progress towards making the Qubes
build process deterministic. This would likely involve cooperation with
and hacking on several upstream build tools to eliminate sources of
variability.
**Difficulty**: medium
**Knowledge prerequisite**: qubes-builder :doc:`[1] </developer/building/qubes-builder>`
:doc:`[2] </developer/building/qubes-builder-details>`
`[3] <https://github.com/QubesOS/qubes-builder/tree/master/doc>`__, and
efficient at introspecting complex systems: comfortable with tracing and
debugging tools, ability to quickly identify and locate issues within a
large codebase (upstream build tools), etc.
**Size of the project**: 350 hours
**Mentor**: `Marek Marczykowski-Górecki <https://www.qubes-os.org/team/>`__
Porting Qubes to ARM/aarch64
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
**Project**: Porting Qubes to ARM/aarch64
**Brief explanation**:
Qubes currently only supports the x86_64 CPU architecture. Xen currently
has additional support for ARM32/ARM64 processors, however work needs to
be done to integrate this into the Qubes build process, as well as work
in integrating this with the Qubes toolstack and security model. This
may also be beneficial in simplifying the process of porting to other
architectures.
Some related discussion:
- `#4318 <https://github.com/QubesOS/qubes-issues/issues/4318>`__ on
porting to ppc64.
- `#3894 <https://github.com/QubesOS/qubes-issues/issues/3894>`__ on
porting to L4 microkernel.
**Expected results**:
- Add cross-compilation support to qubes-builder and related
components.
- Make aarch64 specific adjustments to Qubes toolstacks/manager
(including passthrough of devices from device tree to guest domains).
- Aarch64 specific integration and unit tests.
- Production of generic u-boot or uefi capable image/iso for target
hardware.
**Difficulty**: hard
**Knowledge prerequisite**:
- Libvirt and Qubes toolstacks (C and python languages).
- Xen debugging.
- General ARM architecture knowledge.
**Size of the project**: 350 hours
**Mentor**: `Marek Marczykowski-Górecki <https://www.qubes-os.org/team/>`__
<!--
REMOVED as of February 2021: work is being done on this
### Porting Qubes to POWER9/PPC64
**Project**: Porting Qubes to POWER9/ppc64
**Brief explanation**:
Qubes currently supports the x86_64 CPU architecture. PowerPC is desirable for security purposes as it is the only architecture where one can get performant hardware with entirely open source firmware. Xen has **deprecated** support for Power9/PPC64 processors. Here are two directions to tackle this project from:
- Port Qubes to KVM then work on ppc64 specifics
- Implement some missing functionality in KVM then implement KVM support in the Qubes Hypervisor Abstraction Layer and build process. Improving the HAL will also be beneficial for simplifying the process of porting to further architectures and hypervisors.
- Port Xen to ppc64 then work on Qubes specifics
- For more information on porting Xen see `this thread <https://markmail.org/message/vuk7atnyqfq52epp>`__.
More information and further links can be found in the related issue:
`#4318 <https://github.com/QubesOS/qubes-issues/issues/4318>`__.
**Expected results**:
- Add cross-compilation support to qubes-builder and related components.
- Make ppc64 specific adjustments to Qubes toolstacks/manager (including passthrough of devices from device tree to guest domains).
- ppc64 specific integration and unit tests.
- Production of generic u-boot or uefi capable image/iso for target hardware.
**Knowledge prerequisite**:
- Libvirt and Qubes toolstacks (C and python languages).
- KVM or XEN internals
- General ppc64 architecture knowledge.
**Mentor**: `Marek Marczykowski-Górecki <https://ark.intel.com/content/www/us/en/ark.html#@Processors>`__
-->
Android development in Qubes
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
**Project**: Research running Android in Qubes VM (probably HVM) and
connecting it to Android Studio
**Brief explanation**: The goal is to enable Android development (and
testing!) on Qubes OS. Currently it’s only possible using qemu-emulated
Android for ARM. Since it’s software emulation it’s rather slow.
Details, reference:
`#2233 <https://github.com/QubesOS/qubes-issues/issues/2233>`__
**Expected results**:
- a simple way of setting up Android qubes with hardware emulation
(distributed as a template or as a salt, handling various modern
Android versions)
- figuring out and implementing an easy and secure way to connect an
Android qube to a development qube with Android studio
- documentation and tests
**Difficulty**: hard
**Knowledge prerequisite**:
**Size of the project**: 350 hours
**Mentor**: Inquire on :ref:`qubes-devel <introduction/support:qubes-devel>`.
Admin API Fuzzer
^^^^^^^^^^^^^^^^
**Project**: Develop a
`Fuzzer <https://en.wikipedia.org/wiki/Fuzzing>`__ for the :doc:`Qubes OS Admin API </developer/services/admin-api>`.
**Brief explanation**: The :doc:`Qubes OS Admin API </developer/services/admin-api>`
enables VMs to execute privileged actions on other VMs or dom0 - if
allowed by the Qubes OS RPC policy. Programming errors in the Admin API
however may cause these access rights to be more permissive than
anticipated by the programmer.
Since the Admin API is continuously growing and changing, continuous
security assessments are required. A
`Fuzzer <https://en.wikipedia.org/wiki/Fuzzing>`__ would help to
automate part of these assessments.
**Expected results**: - fully automated & extensible Fuzzer for parts of
the Admin API - user & developer documentation
**Difficulty**: medium
**Prerequisites**: - basic Python understanding - some knowledge about
fuzzing & existing fuzzing frameworks
(e.g. `oss-fuzz <https://github.com/google/oss-fuzz/tree/master/projects/qubes-os>`__)
- a hacker’s curiosity
**Size of the project**: 175 hours
**Mentor**: Inquire on :ref:`qubes-devel <introduction/support:qubes-devel>`.
Secure Boot support
^^^^^^^^^^^^^^^^^^^
**Project**: Add support for protecting boot binaries with Secure Boot
technology, using user-generated keys.
**Brief explanation**: Since recently, Xen supports “unified EFI boot”
which allows to sign not only Xen binary itself, but also dom0 kernel
and their parameters. While the base technology is there, enabling it is
a painful and complex process. The goal of this project is to integrate
configuration of this feature into Qubes, automating as much as
possible. See discussion in `issue #4371 <https://github.com/QubesOS/qubes-issues/issues/4371>`__
**Expected results**: - a tool to prepare relevant boot files for
unified Xen EFI boot - this includes collecting Xen, dom0 kernel,
initramfs, config file, and possibly few more (ucode update?); the tool
should then sign the file with user provided key (preferably propose to
generate it too) - integrate it with updates mechanism, so new Xen or
dom0 kernel will be picked up automatically - include a fallback
configuration that can be used for troubleshooting (main unified Xen EFI
intentionally does not allow to manipulate parameters at boot time)
**Difficulty**: hard
**Knowledge prerequisite**: - basic understanding of Secure Boot - Bash
and Python scripting
**Size of the project**: 175 hours
**Mentor**: `Marek Marczykowski-Górecki <https://www.qubes-os.org/team/>`__
Reduce logging of Disposable VMs
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
**Project**: Reduce logging of Disposable VMs
**Brief explanation**: Partial metadata of a DisposableVM is stored in
the dom0 filesystem. This applies to various logs, GUI status files etc.
There should be an option to hide as much of that as possible -
including bypassing some logging, and removing various state files, or
at the very least obfuscating any hints what is running inside
DisposableVM. More details at `issue #4972 <https://github.com/QubesOS/qubes-issues/issues/4972>`__
**Expected results**: A DisposableVM should not leave logs hinting what
was running inside.
**Difficulty**: medium
**Knowledge prerequisite**: - Python scripting - Basic knowledge of
Linux system services management (systemd, syslog etc)
**Size of the project**: 350 hours
**Mentor**: `Marek Marczykowski-Górecki <https://www.qubes-os.org/team/>`__
Past Projects
-------------
You can view the projects we had in 2017 in the `GSoC 2017 archive <https://summerofcode.withgoogle.com/archive/2017/organizations/5074771758809088/>`__.
We also participated in GSoC 2020 and GSoC 2021, and you can see the
project in the `GSoC 2020 archive <https://summerofcode.withgoogle.com/archive/2020/organizations/4924517870206976/>`__
and `GSoC 2021 archive <https://summerofcode.withgoogle.com/archive/2021/organizations/5682513023860736>`__.
Here are some successful projects which have been implemented in the
past by Google Summer of Code participants.
Template manager, new template distribution mechanism
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
**Project**: Template manager, new template distribution mechanism
**Brief explanation**: Template VMs currently are distributed using RPM
packages. There are multiple problems with that, mostly related to
static nature of RPM package (what files belong to the package). This
means such Template VM cannot be renamed, migrated to another storage
(like LVM), etc. Also we don’t want RPM to automatically update template
package itself (which would override all the user changes there). More
details:
`#2064 <https://github.com/QubesOS/qubes-issues/issues/2064>`__,
`#2534 <https://github.com/QubesOS/qubes-issues/issues/2534>`__,
`#3573 <https://github.com/QubesOS/qubes-issues/issues/3573>`__.
**Expected results**:
- Design new mechanism for distributing templates (possibly including
some package format - either reuse something already existing, or
design new one). The mechanism needs to handle:
- integrity protection (digital signatures), not parsing any data in
dom0 prior to signature verification
- efficient handling of large sparse files
- ability to deploy the template into various storage mechanisms
(sparse files, LVM thin volumes etc).
- template metadata, templates repository - enable the user to
browse available templates (probably should be done in dedicated
VM, or DisposableVM)
- manual template removal by users (without it, see problems such as
`#5509 <https://github.com/QubesOS/qubes-issues/issues/5509>`__
- Implement the above mechanism:
- tool to download named template - should perform download
operation in some VM (as dom0 have no network access), then
transfer the data to dom0, verify its integrity and then create
Template VM and feed it’s root filesystem image with downloaded
data.
- tool to browse templates repository - both CLI and GUI (preferably
integrated with existing Template Manager tool)
- integrate both tools - user should be able to choose some template
to be installed from repository browsing tool - see
`#1705 <https://github.com/QubesOS/qubes-issues/issues/1705>`__
for some idea (this one lacks integrity verification, but a
similar service could be developed with that added)
- If new “package” format is developed, add support for it into
`linux-template-builder <https://github.com/QubesOS/qubes-linux-template-builder>`__.
- Document the mechanism.
- Write unit tests and integration tests.
**Knowledge prerequisite**:
- Large files (disk images) handling (sparse files, archive formats)
- Bash and Python scripting
- Data integrity handling - digital signatures (gpg2, gpgv2)
- PyGTK
- RPM package format, (yum) repository basics
**Mentor**: `Marek Marczykowski-Górecki <https://www.qubes-os.org/team/>`__
----
We adapted some of the language here about GSoC from the `KDE GSoC page <https://community.kde.org/GSoC>`__.
.. _Thomas Leonard: talex5@gmail.com

187
developer/general/gsod.md
View File

@ -1,187 +0,0 @@
---
lang: en
layout: doc
permalink: /gsod/
ref: 242
title: Google Season of Docs (GSoD)
---
Thank you for your interest in participating in the [2024 Google Season of Docs](https://developers.google.com/season-of-docs/) program with the [Qubes OS team](/team/). This page details our 2024 project idea as well as completed past projects. You can read more about the Google Season of Docs in the official [guides](https://developers.google.com/season-of-docs/docs/) and [FAQ](https://developers.google.com/season-of-docs/docs/faq).
## Update and Expand How-To Guides -- Qubes OS
### About the Qubes OS Project
Qubes OS is a security-focused operating system that allows you to organize your digital life into compartments called "qubes." If one qube is compromised, the others remain safe, so a single cyberattack can no longer take down your entire digital life in one fell swoop. You can think of using Qubes OS as having many different computers on your desk for different activities but with the convenience of a single physical machine, a single unified desktop environment, and a set of tools for using qubes together securely as parts of a unified system.
Qubes OS was launched in 2011 and has [received praise from security experts and organizations](/endorsements/) like Edward Snowden, the Freedom of the Press Foundation, Micah Lee, and Let's Encrypt. Qubes has [over 40,000 active users](/statistics/). From network-level to software-level protections, as well as protections against firmware and hardware attacks, Qubes OS is trying to protect the user from the most significant attacks they encounter so that they can get their work done safely.
### The project's problem
- Some of the existing content is stale. It refers to previous Qubes releases and has not yet been updated to the cover the latest stable release.
- There are important topic areas that the current guides do not completely cover, such as "understanding Qubes OS," "using Qubes OS in practice," "Qubes OS workflow for coding," "printing," and "audio."
- The guides do not consistently address users of all skill and experience levels (beginner, intermediate, and expert).
- Some of the guides lack relevant illustrations and screenshots.
- When the developers update a software tool, they should update all the guides that mentions this tool. However, there is currently no way for the developers to know which guides mention which tools.
### The project's scope
The project is estimated to need around six months to complete (see the timeline below). Qubes team members, including Michael Carbone, Andrew Wong, Marta Marczykowska-Górecka, and Marek Marczykowski-Górecki, will supervise and support the writer.
### Measuring the project's success
We will consider the project successful if:
- Existing guides are updated to describe currently supported Qubes OS version
- Missing common guides are identified
- 2-3 new guides are written
### Timeline
| Dates | Action items |
| --------------- | --------------------------------------- |
| May | Orientation |
| June - November | Update & extend how-to guides |
| December | Final project evaluation and case study |
### Project budget
| Expense | Amount |
| --------------------------------------- | ------- |
| writer (10 hours/week, 6 months) | $12,000 |
| TOTAL | $12,000 |
### Additional information
Qubes OS regularly participates in Google Summer of Code and Google Season of Docs. This is our third time participating in Google Season of Docs. Our mentorships for GSoD 2019, 2020, and 2023 were successes, and the projects were completed within the times allotted. The past Google Season of Docs projects have given us experience in working with technical writers and have helped us to understand the benefits that technical writers can bring to our project.
## Past Projects
You can view the project we had in 2019 in the [2019 GSoD archive](https://developers.google.com/season-of-docs/docs/2019/participants/project-qubes) and the [2019 writer's report](https://refre.ch/report-qubesos/).
You can view the project we had in 2020 in the [2020 GSoD archive](https://developers.google.com/season-of-docs/docs/2020/participants/project-qubesos-c1e0) and the [2020 writer's report](https://gist.github.com/PROTechThor/bfe9b8b28295d88c438b6f6c754ae733).
You can view the results of the project we had in 2023 [here](https://www.youtube.com/playlist?list=PLjwSYc73nX6aHcpqub-6lzJbL0vhLleTB).
Here are some successful projects which have been implemented in the past by Google Season of Docs participants.
### Instructional video series
#### The project's problem
There is user demand for high-quality, up-to-date video guides that take users from zero Linux knowledge to using Qubes as a daily driver and performing specific tasks inside of Qubes, but almost no such videos exist. Although most of the required knowledge is documented, many users report that they would prefer to watch videos rather than read text or that they would find videos easier to understand and follow along with.
#### The project's scope
This project consists of creating a series of instructional videos that satisfy the following criteria:
- Prospective users who are not yet familiar with Linux or Qubes OS can easily understand and follow the videos.
- The videos make a good effort to catch and keep the attention of their target audience.
- Users can follow the videos step-by-step to install Qubes OS and accomplish various tasks.
- The videos show the actual software being used (i.e., Qubes OS and any relevant software running inside of it).
- The videos are technically accurate, include security warnings where appropriate, and use terminology in a way that is consistent with the rest of the documentation (also see the [glossary](/doc/glossary/)).
- The video series is comprehensive enough that users do not need to consult the documentation or ask questions (e.g., on the forum) in order to accomplish the most popular tasks and activities.
- The videos include voice narration. (Showing the speaker is optional.)
- The quality of the videos is consistent with current standards regarding things like editing, transitions, animations, lighting, and audio quality.
- The videos are in high definition (minimum 1080p, preferably 4k).
- The videos are separated into a series, where each video is an appropriate length and is appropriately connected to the other videos in the series.
- The videos are suitable for upload and sharing on popular video-sharing and social-media platforms, such as YouTube and Twitter. (The account or channel under which the videos are uploaded is open to discussion on platforms where the Qubes OS Project does not already have a significant established presence, such as YouTube.)
- The videos are suitable for embedding in appropriate places in the Qubes documentation. (E.g., a video on how to update Qubes OS should be appropriate for appearing on the [how to update](/doc/how-to-update/) page.)
- Where possible, the videos should strive to be version-independent. (For example, a video explaining the template system should still be relevant many releases from now if the template system has not changed.)
Below is an example of the content (which is already [documented](/doc/)) that the video series is likely to cover. The precise scope of content is to be determined in consultation with the video creator.
- Introduction to Qubes
- Selecting appropriate hardware
- How to install Qubes OS
- First steps after installing
- How to organize your qubes
- How to update
- How to back up, restore, and migrate
- How to copy and paste text (including dom0)
- How to copy and move files (including dom0)
- How to install software
- How to use and customize disposables
- How to enter fullscreen mode
- How to use devices (including block storage, USB, PCI, and optical)
- Templates: understanding, installing, uninstalling, reinstalling, etc.
- Common troubleshooting (preferably included in previous videos at appropriate points)
- The Qubes firewall
- Passwordless root
- Anti Evil Maid
- Split GPG
- CTAP proxy
- YubiKey
- Whonix
- How to install and use a VPN in Qubes
- How to install and use Windows in Qubes
- Other popular topics, as time permits
The project is estimated to need around six months to complete (see the timeline below). Qubes team members, including Michael Carbone, Andrew Wong, and Marek Marczykowski-Górecki, will supervise and support the creator.
#### Measuring the project's success
We will consider the project successful if, after publication of the video series:
- Actual prospective users with no prior familiarity with Linux or Qubes OS are able to successfully install and use Qubes OS as intended by following along with the videos.
- The reception to the videos is generally positive and complaints about quality and accuracy are minimal.
- Appropriate analytics (e.g., YouTube metrics) are average or better for videos of this type (to be determined in consultation with the creator).
### Consolidate troubleshooting guides
**Project**: Consolidate troubleshooting guides
**Brief explanation**: Troubleshooting guides are scattered across many pages and sometimes incomplete, leading to repeatedly posting the same instruction over and over when helping users to diagnose problems.
This could be helped by writing a consolidated guide with a clear list of symptom-action layout.
**Expected results**:
- Review existing [troubleshooting guides](/doc/#troubleshooting)
- Review [issues](https://github.com/QubesOS/qubes-issues/issues?q=is%3Aopen+is%3Aissue+label%3A%22C%3A+doc%22) containing common troubleshooting steps (checking specific logs etc)
- Propose updated, consolidated troubleshooting documentation, including its layout
**Knowledge prerequisite**:
- [Markdown](https://daringfireball.net/projects/markdown/)
**Mentor**: [Marek Marczykowski-Górecki](/team/)
### Improve Getting Started page
**Project**: Improve Getting Started page
**Brief explanation**: The [Getting Started page](/doc/getting-started/) is the place a new user would go to understand better how to use Qubes. It is currently has old screenshots not using the default desktop environment and could have much better flow. In addition, this improved page content may end up being served more directly to the user via the [offline documentation](https://github.com/QubesOS/qubes-issues/issues/1019) or the firstboot guide.
**Expected results**:
- Review the existing page and website, similar pages for other OSes
- Provide visual mock-ups and proposed text
**Knowledge prerequisite**:
- basic Qubes OS knowledge
- [Markdown](https://daringfireball.net/projects/markdown/)
**Mentor**: [Michael Carbone](/team/)
### Rewrite qrexec documentation
**Project**: Rewrite qrexec documentation
**Brief explanation**: Current qrexec (qubes remote exec) documentation is hard to follow, important information is hidden within a wall of text.
Some parts are split into multiple sections, for example version specific to avoid duplication, but it doesn't help reading it.
Additionally, protocol documentation describes only few specific use cases, instead of being clear and precise protocol specification.
Fixing this last point may require very close cooperation with developers, as the current documentation doesn't multiple corner cases (that's one of the issue with its current shape).
**Expected results**:
- Review existing [qrexec documentation](/doc/qrexec3/) and an [issue about it](https://github.com/QubesOS/qubes-issues/issues/1392)
- Propose updated, consolidated admin documentation (policy writing, adding services)
- Propose consolidated protocol specification, based on the current documentation, and cooperation with developers
**Knowledge prerequisite**:
- [Markdown](https://daringfireball.net/projects/markdown/)
**Mentor**: [Marek Marczykowski-Górecki](/team/)

424
developer/general/gsod.rst Normal file
View File

@ -0,0 +1,424 @@
============================
Google Season of Docs (GSoD)
============================
Thank you for your interest in participating in the `2024 Google Season of Docs <https://developers.google.com/season-of-docs/>`__ program with
the `Qubes OS team <https://www.qubes-os.org/team/>`__. This page details our 2024 project idea
as well as completed past projects. You can read more about the Google
Season of Docs in the official
`guides <https://developers.google.com/season-of-docs/docs/>`__ and
`FAQ <https://developers.google.com/season-of-docs/docs/faq>`__.
Update and Expand How-To Guides – Qubes OS
------------------------------------------
About the Qubes OS Project
^^^^^^^^^^^^^^^^^^^^^^^^^^
Qubes OS is a security-focused operating system that allows you to
organize your digital life into compartments called “qubes.” If one qube
is compromised, the others remain safe, so a single cyberattack can no
longer take down your entire digital life in one fell swoop. You can
think of using Qubes OS as having many different computers on your desk
for different activities but with the convenience of a single physical
machine, a single unified desktop environment, and a set of tools for
using qubes together securely as parts of a unified system.
Qubes OS was launched in 2011 and has `received praise from security experts and organizations <https://www.qubes-os.org/endorsements/>`__ like Edward Snowden, the
Freedom of the Press Foundation, Micah Lee, and Let’s Encrypt. Qubes has
:doc:`over 40,000 active users </introduction/statistics>`. From network-level to
software-level protections, as well as protections against firmware and
hardware attacks, Qubes OS is trying to protect the user from the most
significant attacks they encounter so that they can get their work done
safely.
The project's problem
^^^^^^^^^^^^^^^^^^^^^
- Some of the existing content is stale. It refers to previous Qubes
releases and has not yet been updated to the cover the latest stable
release.
- There are important topic areas that the current guides do not
completely cover, such as “understanding Qubes OS,” “using Qubes OS
in practice,” “Qubes OS workflow for coding,” “printing,” and
“audio.”
- The guides do not consistently address users of all skill and
experience levels (beginner, intermediate, and expert).
- Some of the guides lack relevant illustrations and screenshots.
- When the developers update a software tool, they should update all
the guides that mentions this tool. However, there is currently no
way for the developers to know which guides mention which tools.
The project's scope
^^^^^^^^^^^^^^^^^^^
The project is estimated to need around six months to complete (see the
timeline below). Qubes team members, including Michael Carbone, Andrew
Wong, Marta Marczykowska-Górecka, and Marek Marczykowski-Górecki, will
supervise and support the writer.
Measuring the project's success
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
We will consider the project successful if:
- Existing guides are updated to describe currently supported Qubes OS
version
- Missing common guides are identified
- 2-3 new guides are written
Timeline
^^^^^^^^
.. list-table::
:widths: 15 15
:align: center
:header-rows: 1
* - Dates
- Action items
* - May
- Orientation
* - June - November
- Update & extend how-to guides
* - December
- Final project evaluation and case study
Project budget
^^^^^^^^^^^^^^
.. list-table::
:widths: 32 32
:align: center
:header-rows: 1
* - Expense
- Amount
* - writer (10 hours/week, 6 months)
- $12,000
* - TOTAL
- $12,000
Additional information
^^^^^^^^^^^^^^^^^^^^^^
Qubes OS regularly participates in Google Summer of Code and Google
Season of Docs. This is our third time participating in Google Season of
Docs. Our mentorships for GSoD 2019, 2020, and 2023 were successes, and
the projects were completed within the times allotted. The past Google
Season of Docs projects have given us experience in working with
technical writers and have helped us to understand the benefits that
technical writers can bring to our project.
Past Projects
-------------
You can view the project we had in 2019 in the `2019 GSoD archive <https://developers.google.com/season-of-docs/docs/2019/participants/project-qubes>`__
and the `2019 writer’s report <https://refre.ch/report-qubesos/>`__.
You can view the project we had in 2020 in the `2020 GSoD archive <https://developers.google.com/season-of-docs/docs/2020/participants/project-qubesos-c1e0>`__
and the `2020 writer’s report <https://gist.github.com/PROTechThor/bfe9b8b28295d88c438b6f6c754ae733>`__.
You can view the results of the project we had in 2023
`here <https://www.youtube.com/playlist?list=PLjwSYc73nX6aHcpqub-6lzJbL0vhLleTB>`__.
Here are some successful projects which have been implemented in the
past by Google Season of Docs participants.
Instructional video series
^^^^^^^^^^^^^^^^^^^^^^^^^^
.. _the-projects-problem-1:
The project's problem
^^^^^^^^^^^^^^^^^^^^^
There is user demand for high-quality, up-to-date video guides that take
users from zero Linux knowledge to using Qubes as a daily driver and
performing specific tasks inside of Qubes, but almost no such videos
exist. Although most of the required knowledge is documented, many users
report that they would prefer to watch videos rather than read text or
that they would find videos easier to understand and follow along with.
.. _the-projects-scope-1:
The project's scope
^^^^^^^^^^^^^^^^^^^
This project consists of creating a series of instructional videos that
satisfy the following criteria:
- Prospective users who are not yet familiar with Linux or Qubes OS can
easily understand and follow the videos.
- The videos make a good effort to catch and keep the attention of
their target audience.
- Users can follow the videos step-by-step to install Qubes OS and
accomplish various tasks.
- The videos show the actual software being used (i.e., Qubes OS and
any relevant software running inside of it).
- The videos are technically accurate, include security warnings where
appropriate, and use terminology in a way that is consistent with the
rest of the documentation (also see the
:doc:`glossary </user/reference/glossary>`).
- The video series is comprehensive enough that users do not need to
consult the documentation or ask questions (e.g., on the forum) in
order to accomplish the most popular tasks and activities.
- The videos include voice narration. (Showing the speaker is
optional.)
- The quality of the videos is consistent with current standards
regarding things like editing, transitions, animations, lighting, and
audio quality.
- The videos are in high definition (minimum 1080p, preferably 4k).
- The videos are separated into a series, where each video is an
appropriate length and is appropriately connected to the other videos
in the series.
- The videos are suitable for upload and sharing on popular
video-sharing and social-media platforms, such as YouTube and
Twitter. (The account or channel under which the videos are uploaded
is open to discussion on platforms where the Qubes OS Project does
not already have a significant established presence, such as
YouTube.)
- The videos are suitable for embedding in appropriate places in the
Qubes documentation. (E.g., a video on how to update Qubes OS should
be appropriate for appearing on the :doc:`:doc:`how to update </user/how-to-guides/how-to-update>`` page.)
- Where possible, the videos should strive to be version-independent.
(For example, a video explaining the template system should still be
relevant many releases from now if the template system has not
changed.)
Below is an example of the content (which is already
:doc:`documented </index>`) that the video series is likely to cover. The
precise scope of content is to be determined in consultation with the
video creator.
- Introduction to Qubes
- Selecting appropriate hardware
- How to install Qubes OS
- First steps after installing
- How to organize your qubes
- How to update
- How to back up, restore, and migrate
- How to copy and paste text (including dom0)
- How to copy and move files (including dom0)
- How to install software
- How to use and customize disposables
- How to enter fullscreen mode
- How to use devices (including block storage, USB, PCI, and optical)
- Templates: understanding, installing, uninstalling, reinstalling,
etc.
- Common troubleshooting (preferably included in previous videos at
appropriate points)
- The Qubes firewall
- Passwordless root
- Anti Evil Maid
- Split GPG
- CTAP proxy
- YubiKey
- Whonix
- How to install and use a VPN in Qubes
- How to install and use Windows in Qubes
- Other popular topics, as time permits
The project is estimated to need around six months to complete (see the
timeline below). Qubes team members, including Michael Carbone, Andrew
Wong, and Marek Marczykowski-Górecki, will supervise and support the
creator.
.. _measuring-the-projects-success-1:
Measuring the project's success
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
We will consider the project successful if, after publication of the
video series:
- Actual prospective users with no prior familiarity with Linux or
Qubes OS are able to successfully install and use Qubes OS as
intended by following along with the videos.
- The reception to the videos is generally positive and complaints
about quality and accuracy are minimal.
- Appropriate analytics (e.g., YouTube metrics) are average or better
for videos of this type (to be determined in consultation with the
creator).
Consolidate troubleshooting guides
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
**Project**: Consolidate troubleshooting guides
**Brief explanation**: Troubleshooting guides are scattered across many
pages and sometimes incomplete, leading to repeatedly posting the same
instruction over and over when helping users to diagnose problems. This
could be helped by writing a consolidated guide with a clear list of
symptom-action layout.
**Expected results**:
- Review existing :ref:`troubleshooting guides <index:troubleshooting>`
- Review
`issues <https://github.com/QubesOS/qubes-issues/issues?q=is%3Aopen+is%3Aissue+label%3A%22C%3A+doc%22>`__
containing common troubleshooting steps (checking specific logs etc)
- Propose updated, consolidated troubleshooting documentation,
including its layout
**Knowledge prerequisite**:
- `Markdown <https://daringfireball.net/projects/markdown/>`__
**Mentor**: `Marek Marczykowski-Górecki <https://www.qubes-os.org/team/>`__
Improve Getting Started page
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
**Project**: Improve Getting Started page
**Brief explanation**: The :doc:`Getting Started page </introduction/getting-started>` is the place a new user would go to
understand better how to use Qubes. It is currently has old screenshots
not using the default desktop environment and could have much better
flow. In addition, this improved page content may end up being served
more directly to the user via the `offline documentation <https://github.com/QubesOS/qubes-issues/issues/1019>`__
or the firstboot guide.
**Expected results**:
- Review the existing page and website, similar pages for other OSes
- Provide visual mock-ups and proposed text
**Knowledge prerequisite**:
- basic Qubes OS knowledge
- `Markdown <https://daringfireball.net/projects/markdown/>`__
**Mentor**: `Michael Carbone <https://www.qubes-os.org/team/>`__
Rewrite qrexec documentation
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
**Project**: Rewrite qrexec documentation
**Brief explanation**: Current qrexec (qubes remote exec) documentation
is hard to follow, important information is hidden within a wall of
text. Some parts are split into multiple sections, for example version
specific to avoid duplication, but it doesn’t help reading it.
Additionally, protocol documentation describes only few specific use
cases, instead of being clear and precise protocol specification. Fixing
this last point may require very close cooperation with developers, as
the current documentation doesn’t multiple corner cases (that’s one of
the issue with its current shape).
**Expected results**:
- Review existing :doc:`qrexec documentation </developer/services/qrexec>` and an
`issue about it <https://github.com/QubesOS/qubes-issues/issues/1392>`__
- Propose updated, consolidated admin documentation (policy writing,
adding services)
- Propose consolidated protocol specification, based on the current
documentation, and cooperation with developers
**Knowledge prerequisite**:
- `Markdown <https://daringfireball.net/projects/markdown/>`__
**Mentor**: `Marek Marczykowski-Górecki <https://www.qubes-os.org/team/>`__

198
developer/general/how-to-edit-the-documentation.md
View File

@ -1,198 +0,0 @@
---
lang: en
layout: doc
permalink: /doc/how-to-edit-the-documentation/
title: How to edit the documentation
---
_Also see the [documentation style guide](/doc/documentation-style-guide/)._
Qubes OS documentation pages are stored as plain text Markdown files in the
[qubes-doc](https://github.com/QubesOS/qubes-doc) repository. By cloning and
regularly pulling from this repo, users can maintain their own up-to-date
offline copy of all Qubes documentation rather than relying solely on the web.
The documentation is a volunteer community effort. People like you are
constantly working to make it better. If you notice something that can be fixed
or improved, please follow the steps below to open a pull request!
## How to submit a pull request
We keep all the documentation in
[qubes-doc](https://github.com/QubesOS/qubes-doc), a dedicated Git repository
hosted on [GitHub](https://github.com/). Thanks to GitHub's easy web interface,
you can edit the documentation even if you're not familiar with Git or the
command line! All you need is a free GitHub account.
A few notes before we get started:
- Since Qubes is a security-oriented project, every documentation change will
be [reviewed](#security) before it's accepted. This allows us to maintain
quality control and protect our users.
- To give your contribution a better chance of being accepted, please follow
our [documentation style guide](/doc/documentation-style-guide/).
- We don't want you to spend time and effort on a contribution that we can't
accept. If your contribution would take a lot of time, please [file an
issue](/doc/issue-tracking/) for it first so that we can make sure we're on
the same page before significant works begins.
- Alternatively, you may already have written content that doesn't conform to
these guidelines, but you'd be willing to modify it so that it does. In this
case, you can still submit it by following the instructions below. Just make
a note in your pull request (PR) that you're aware of the changes that need
to be made and that you're just asking for the content to be reviewed before
you spend time making those changes.
- Finally, if you've written something that doesn't belong in qubes-doc but
that would be beneficial to the Qubes community, please consider adding it to
the [external
documentation](/doc/documentation-style-guide/#core-vs-external-documentation).
(**Advanced users:** If you're already familiar with GitHub or wish to work
from the command line, you can skip the rest of this section. All you need to
do to contribute is to [fork and
clone](https://guides.github.com/activities/forking/) the
[qubes-doc](https://github.com/QubesOS/qubes-doc) repo, make your changes, then
[submit a pull
request](https://help.github.com/articles/using-pull-requests/).)
Ok, let's begin. Every documentation page has a "Page Source on GitHub" button.
Depending on the size of your screen, it may either be on the side (larger
screens) or on the bottom (smaller screens).
[![page-source-button](/attachment/doc/doc-pr_01_page-source-button.png)](/attachment/doc/doc-pr_01_page-source-button.png)
When you click on it, you'll be taken to the source file --- usually a Markdown
(`.md`) file --- on GitHub. On this page, there will be a button to edit the
file.
[![github-edit](/attachment/doc/doc-pr_02_github-edit.png)](/attachment/doc/doc-pr_02_github-edit.png)
You'll be prompted to sign in with your GitHub username and password
(if you aren't already logged in). You can also create a free account from here.
[![github-sign-in](/attachment/doc/doc-pr_03_sign-in.png)](/attachment/doc/doc-pr_03_sign-in.png)
If this is your first contribution to the documentation, you need to "fork" the
repository (make your own copy). It's easy --- just click the big green button
on the next page. This step is only needed the first time you make a
contribution.
[![fork](/attachment/doc/doc-pr_04_fork.png)](/attachment/doc/doc-pr_04_fork.png)
Now you can make your modifications. You can also preview the changes to see
how they'll be formatted by clicking the "Preview changes" tab. If you want to
add images, please see [How to add images](#how-to-add-images). If you're
making formatting changes, please [render the site
locally](https://github.com/QubesOS/qubesos.github.io#instructions) to verify
that everything looks correct before submitting any changes.
[![edit](/attachment/doc/doc-pr_05_edit.png)](/attachment/doc/doc-pr_05_edit.png)
Once you're finished, describe your changes at the bottom and click "Propose
file change".
[![commit](/attachment/doc/doc-pr_06_commit-msg.png)](/attachment/doc/doc-pr_06_commit-msg.png)
After that, you'll see exactly what modifications you've made. At this stage,
those changes are still in your own copy of the documentation ("fork"). If
everything looks good, send those changes to us by pressing the "Create pull
request" button.
[![pull-request](/attachment/doc/doc-pr_07_review-changes.png)](/attachment/doc/doc-pr_07_review-changes.png)
You will be able to adjust the pull request message and title there. In most
cases, the defaults are ok, so you can just confirm by pressing the "Create
pull request" button again. However, if you're not ready for your PR to be
reviewed or merged yet, please [make a draft PR
instead](https://github.blog/2019-02-14-introducing-draft-pull-requests/).
[![pull-request-confirm](/attachment/doc/doc-pr_08_create-pull-request.png)](/attachment/doc/doc-pr_08_create-pull-request.png)
If any of your changes should be reflected in the [documentation index (a.k.a.
table of contents)](/doc/) --- for example, if you're adding a new page,
changing the title of an existing page, or removing a page --- please see [How
to edit the documentation index](#how-to-edit-the-documentation-index).
That's all! We will review your changes. If everything looks good, we'll pull
them into the official documentation. Otherwise, we may have some questions for
you, which we'll post in a comment on your pull request. (GitHub will
automatically notify you if we do.) If, for some reason, we can't accept your
pull request, we'll post a comment explaining why we can't.
[![done](/attachment/doc/doc-pr_09_done.png)](/attachment/doc/doc-pr_09_done.png)
## How to edit the documentation index
The source file for the [documentation index (a.k.a. table of contents)](/doc/)
is
[doc-index.yml](https://github.com/QubesOS/qubesos.github.io/blob/master/_data/doc-index.yml).
Editing this file will change what appears on the documentation index. If your
pull request (PR) adds, removes, or edits anything that should be reflected in
the documentation index, please make sure you also submit an associated pull
request against this file.
## How to add images
To add an image to a page, use the following syntax in the main document (see
[here](/doc/documentation-style-guide/#image-linking) for why this syntax is
important).
```
[![Image Title](/attachment/doc/image.png)](/attachment/doc/image.png)
```
Then, submit your image(s) in a separate pull request to the
[qubes-attachment](https://github.com/QubesOS/qubes-attachment) repository
using the same path and filename. This is the only permitted way to include
images. Do not link to images on other websites.
## Serving the website locally
You can serve the website offline on your local machine by following [these
instructions](https://github.com/QubesOS/qubesos.github.io#instructions). This
can be useful for making sure that your changes render the way you expect,
especially when your changes affect formatting, images, tables, styling, etc.
## Security
*Also see: [Should I trust this website?](/faq/#should-i-trust-this-website)*
All pull requests (PRs) against
[qubes-doc](https://github.com/QubesOS/qubes-doc) must pass review prior to be
merged, except in the case of [external
documentation](/doc/#external-documentation) (see
[#4693](https://github.com/QubesOS/qubes-issues/issues/4693)). This process is
designed to ensure that contributed text is accurate and non-malicious. This
process is a best effort that should provide a reasonable degree of assurance,
but it is not foolproof. For example, all text characters are checked for ANSI
escape sequences. However, binaries, such as images, are simply checked to
ensure they appear or function the way they should when the website is
rendered. They are not further analyzed in an attempt to determine whether they
are malicious.
Once a pull request passes review, the reviewer should add a signed comment
stating, "Passed review as of `<LATEST_COMMIT>`" (or similar). The
documentation maintainer then verifies that the pull request is mechanically
sound (no merge conflicts, broken links, ANSI escapes, etc.). If so, the
documentation maintainer then merges the pull request, adds a PGP-signed tag to
the latest commit (usually the merge commit), then pushes to the remote. In
cases in which another reviewer is not required, the documentation maintainer
may review the pull request (in which case no signed comment is necessary,
since it would be redundant with the signed tag).
## Questions, problems, and improvements
If you have a question about something you read in the documentation or about
how to edit the documentation, please post it on the
[forum](https://forum.qubes-os.org/) or send it to the appropriate [mailing
list](/support/). If you see that something in the documentation should be
fixed or improved, please [contribute](#how-to-submit-a-pull-request) the
change yourself. To report an issue with the documentation, please follow our
standard [issue reporting guidelines](/doc/issue-tracking/). (If you report an
issue with the documentation, you will likely be asked to submit a pull request
for it, unless there is a clear indication in your report that you are not
willing or able to do so.)

240
developer/general/how-to-edit-the-documentation.rst Normal file
View File

@ -0,0 +1,240 @@
=============================
How to edit the documentation
=============================
*Also see the* :doc:`documentation style guide </developer/general/documentation-style-guide>` *.*
Qubes OS documentation pages are stored as plain text Markdown files in
the `qubes-doc <https://github.com/QubesOS/qubes-doc>`__ repository. By
cloning and regularly pulling from this repo, users can maintain their
own up-to-date offline copy of all Qubes documentation rather than
relying solely on the web.
The documentation is a volunteer community effort. People like you are
constantly working to make it better. If you notice something that can
be fixed or improved, please follow the steps below to open a pull
request!
How to submit a pull request
----------------------------
We keep all the documentation in
`qubes-doc <https://github.com/QubesOS/qubes-doc>`__, a dedicated Git
repository hosted on `GitHub <https://github.com/>`__. Thanks to
GitHub’s easy web interface, you can edit the documentation even if
you’re not familiar with Git or the command line! All you need is a free
GitHub account.
A few notes before we get started:
- Since Qubes is a security-oriented project, every documentation
change will be `reviewed <#security>`__ before it’s accepted. This
allows us to maintain quality control and protect our users.
- To give your contribution a better chance of being accepted, please
follow our :doc:`documentation style guide </developer/general/documentation-style-guide>`.
- We don’t want you to spend time and effort on a contribution that we
can’t accept. If your contribution would take a lot of time, please
:doc:`file an issue </introduction/issue-tracking>` for it first so that we can
make sure we’re on the same page before significant works begins.
- Alternatively, you may already have written content that doesn’t
conform to these guidelines, but you’d be willing to modify it so
that it does. In this case, you can still submit it by following the
instructions below. Just make a note in your pull request (PR) that
you’re aware of the changes that need to be made and that you’re just
asking for the content to be reviewed before you spend time making
those changes.
- Finally, if you’ve written something that doesn’t belong in qubes-doc
but that would be beneficial to the Qubes community, please consider
adding it to the :ref:`external documentation <developer/general/documentation-style-guide:core vs. external documentation>`.
(**Advanced users:** If you’re already familiar with GitHub or wish to
work from the command line, you can skip the rest of this section. All
you need to do to contribute is to `fork and clone <https://guides.github.com/activities/forking/>`__ the
`qubes-doc <https://github.com/QubesOS/qubes-doc>`__ repo, make your
changes, then `submit a pull request <https://help.github.com/articles/using-pull-requests/>`__.)
Ok, let’s begin. Every documentation page has a “Page Source on GitHub”
button. Depending on the size of your screen, it may either be on the
side (larger screens) or on the bottom (smaller screens).
|page-source-button|
When you click on it, you’ll be taken to the source file — usually a
Markdown (``.md``) file — on GitHub. On this page, there will be a
button to edit the file.
|github-edit|
You’ll be prompted to sign in with your GitHub username and password (if
you aren’t already logged in). You can also create a free account from
here.
|github-sign-in|
If this is your first contribution to the documentation, you need to
“fork” the repository (make your own copy). It’s easy — just click the
big green button on the next page. This step is only needed the first
time you make a contribution.
|fork|
Now you can make your modifications. You can also preview the changes to
see how they’ll be formatted by clicking the “Preview changes” tab. If
you want to add images, please see `How to add images <#how-to-add-images>`__. If you’re making formatting changes,
please `render the site locally <https://github.com/QubesOS/qubesos.github.io#instructions>`__
to verify that everything looks correct before submitting any changes.
|edit|
Once you’re finished, describe your changes at the bottom and click
“Propose file change”.
|commit|
After that, you’ll see exactly what modifications you’ve made. At this
stage, those changes are still in your own copy of the documentation
(“fork”). If everything looks good, send those changes to us by pressing
the “Create pull request” button.
|pull-request|
You will be able to adjust the pull request message and title there. In
most cases, the defaults are ok, so you can just confirm by pressing the
“Create pull request” button again. However, if you’re not ready for
your PR to be reviewed or merged yet, please `make a draft PR instead <https://github.blog/2019-02-14-introducing-draft-pull-requests/>`__.
|pull-request-confirm|
If any of your changes should be reflected in the :doc:`documentation index (a.k.a. table of contents) </index>` — for example, if you’re adding a
new page, changing the title of an existing page, or removing a page —
please see `How to edit the documentation index <#how-to-edit-the-documentation-index>`__.
That’s all! We will review your changes. If everything looks good, we’ll
pull them into the official documentation. Otherwise, we may have some
questions for you, which we’ll post in a comment on your pull request.
(GitHub will automatically notify you if we do.) If, for some reason, we
can’t accept your pull request, we’ll post a comment explaining why we
can’t.
|done|
How to edit the documentation index
-----------------------------------
The source file for the :doc:`documentation index (a.k.a. table of contents) </index>` is
`doc-index.yml <https://github.com/QubesOS/qubesos.github.io/blob/master/_data/doc-index.yml>`__.
Editing this file will change what appears on the documentation index.
If your pull request (PR) adds, removes, or edits anything that should
be reflected in the documentation index, please make sure you also
submit an associated pull request against this file.
How to add images
-----------------
To add an image to a page, use the following syntax in the main document
(see :ref:`here <developer/general/documentation-style-guide:image linking>` for why
this syntax is important).
.. code:: bash
[![Image Title](/attachment/doc/image.png)](/attachment/doc/image.png)
Then, submit your image(s) in a separate pull request to the
`qubes-attachment <https://github.com/QubesOS/qubes-attachment>`__
repository using the same path and filename. This is the only permitted
way to include images. Do not link to images on other websites.
Serving the website locally
---------------------------
You can serve the website offline on your local machine by following
`these instructions <https://github.com/QubesOS/qubesos.github.io#instructions>`__.
This can be useful for making sure that your changes render the way you
expect, especially when your changes affect formatting, images, tables,
styling, etc.
Security
--------
*Also see:* :ref:`Should I trust this website? <introduction/faq:should i trust this website?>`
All pull requests (PRs) against
`qubes-doc <https://github.com/QubesOS/qubes-doc>`__ must pass review
prior to be merged, except in the case of :ref:`external documentation <index:external documentation>` (see
`#4693 <https://github.com/QubesOS/qubes-issues/issues/4693>`__). This
process is designed to ensure that contributed text is accurate and
non-malicious. This process is a best effort that should provide a
reasonable degree of assurance, but it is not foolproof. For example,
all text characters are checked for ANSI escape sequences. However,
binaries, such as images, are simply checked to ensure they appear or
function the way they should when the website is rendered. They are not
further analyzed in an attempt to determine whether they are malicious.
Once a pull request passes review, the reviewer should add a signed
comment stating, “Passed review as of ``<LATEST_COMMIT>``” (or similar).
The documentation maintainer then verifies that the pull request is
mechanically sound (no merge conflicts, broken links, ANSI escapes,
etc.). If so, the documentation maintainer then merges the pull request,
adds a PGP-signed tag to the latest commit (usually the merge commit),
then pushes to the remote. In cases in which another reviewer is not
required, the documentation maintainer may review the pull request (in
which case no signed comment is necessary, since it would be redundant
with the signed tag).
Questions, problems, and improvements
-------------------------------------
If you have a question about something you read in the documentation or
about how to edit the documentation, please post it on the
`forum <https://forum.qubes-os.org/>`__ or send it to the appropriate
:doc:`mailing list </introduction/support>`. If you see that something in the
documentation should be fixed or improved, please
`contribute <#how-to-submit-a-pull-request>`__ the change yourself. To
report an issue with the documentation, please follow our standard
:doc:`issue reporting guidelines </introduction/issue-tracking>`. (If you report an
issue with the documentation, you will likely be asked to submit a pull
request for it, unless there is a clear indication in your report that
you are not willing or able to do so.)
.. |page-source-button| image:: /attachment/doc/doc-pr_01_page-source-button.png
.. |github-edit| image:: /attachment/doc/doc-pr_02_github-edit.png
.. |github-sign-in| image:: /attachment/doc/doc-pr_03_sign-in.png
.. |fork| image:: /attachment/doc/doc-pr_04_fork.png
.. |edit| image:: /attachment/doc/doc-pr_05_edit.png
.. |commit| image:: /attachment/doc/doc-pr_06_commit-msg.png
.. |pull-request| image:: /attachment/doc/doc-pr_07_review-changes.png
.. |pull-request-confirm| image:: /attachment/doc/doc-pr_08_create-pull-request.png
.. |done| image:: /attachment/doc/doc-pr_09_done.png

102
developer/general/package-contributions.md
View File

@ -1,102 +0,0 @@
---
lang: en
layout: doc
permalink: /doc/package-contributions/
ref: 29
title: Package contributions
---
_This page is for developers who wish to contribute packages.
If you want to install contributed packages, please see [installing contributed packages](/doc/installing-contributed-packages/)._
We're very grateful to the talented and hard-working community members who contribute software packages to Qubes OS.
This page explains the inclusion criteria and procedures for such packages, as well as the roles and responsibilities of those involved.
Inclusion Criteria
------------------
In order to be accepted, packages must:
* In no way weaken the security of Qubes OS.
* Be published under an open-source license (read about the [Qubes OS License](/doc/license/)).
* Follow our [coding guidelines](/doc/coding-style/).
* Be thoroughly tested.
* Have a clearly-defined use case for Qubes users.
* Not be unduly burdensome to review.
(Please note that we always reserve the right to add criteria to this list.)
Contribution Procedure
----------------------
Before you start putting serious work into a package, we recommend that you discuss your idea with the Qubes developers and the broader community on the [qubes-devel mailing list](/support/#qubes-devel).
Once you have a package that's ready to become part of Qubes OS, please follow this procedure:
1. Ensure that your package satisfies the [Inclusion Criteria](#inclusion-criteria).
2. If your code isn't already on GitHub, create a GitHub repo that contains your code. You can have a look to an example package called [qubes-skeleton](https://github.com/QubesOS-contrib/qubes-skeleton).
3. If you haven't already, [sign your code](/doc/code-signing/).
4. Create an issue in [qubes-issues](https://github.com/QubesOS/qubes-issues/issues/) with the title `[Contribution] your-package-name`.
Include a link to your repo, a brief description of your package, and a brief explanation of why you think it should be included in Qubes.
Please note that the Qubes core developers are very busy.
If they are under heavy load when you submit your contribution, it may be a very long time before they have time to review your package.
If this happens, please do not be discouraged.
If you think they may have forgotten about your pending contribution, you may "bump" your request by commenting on your issue, but please do this *very* sparingly (i.e., no more than once a month).
We appreciate your understanding!
5. You may be asked followup questions.
If we decide to accept your contribution, you will be invited to join the [QubesOS-contrib](https://github.com/QubesOS-contrib) organization on GitHub as public recognition of your contribution (but without push access; see [Review Procedure](#review-procedure)), and [QubesOS-contrib](https://github.com/QubesOS-contrib) will fork your repo.
If we decide not to accept your contribution, we will state the reason and close the issue.
Update Procedure
----------------
*Anyone* can provide an update (patch) to a contributed package, not just the person who contributed that package!
The update procedure is the same for everyone, including the original package contributor.
If you would like to update an already-contributed package (specifically, a fork owned by [QubesOS-contrib](https://github.com/QubesOS-contrib)), please submit a [signed](/doc/code-signing/), fast-forwardable pull request to that repo with your changes.
Please note that your pull request **must** be both [signed](/doc/code-signing/) and fast-forwardable, or else it will be closed without further review.
One or more reviewers may post comments on your pull request.
Please be prepared to read and respond to these comments.
Review Procedure
----------------
This review procedure covers both original package contributions (see [Contribution Procedure](#contribution-procedure)) and all subsequent updates to those packages, including updates from the original package contributor (see [Update Procedure](#update-procedure)).
All changes will be reviewed by a Qubes Core Reviewer (QCR) and the [Package Maintainer](#package-maintainers) (PM).
In all cases, the QCR will be a core Qubes developer.
In some cases, the QCR and the PM will be the same person.
For example, if someone contributes a package, then disappears, and no suitable replacement has been found, then it is likely that a core Qubes developer will play both the QCR and PM roles for that package, at least until another suitable candidate volunteers to become the PM for that package.
The review procedure is as follows:
1. Someone, S, wishes to make a change to a package, P.
2. S submits a fast-forwardable pull request against the fork of P's repo owned by [QubesOS-contrib](https://github.com/QubesOS-contrib).
3. The PM reviews the pull request.
If the pull request passes the PM's review, the PM adds a [signed](/doc/code-signing/) *comment* on the pull request stating that it has passed review.
(In cases in which S = PM, the PM can simply add a [signed](/doc/code-signing/) *tag* to the HEAD commit prior to submitting the pull request.)
If the pull request does not pass the PM's review, the PM leaves a comment on the pull request explaining why not.
4. The QCR reviews the pull request.
If the pull request passes the QCR's review, the QCR pushes a [signed](/doc/code-signing/) tag to the HEAD commit stating that it has passed review and fast-forward merges the pull request.
If the pull request does not pass the QCR's review, the QCR leaves a comment on the pull request explaining why not, and the QCR may decide to close the pull request.
In all the cases, the first condition to be validated by the QCR's review is to ensure that the contribution **will not** hijack any core packages of [QubesOS](https://github.com/QubesOS) and of course, none of the [QubesOS-contrib](https://github.com/QubesOS-contrib) packages too. More precisely, particular attention to the whole build pipeline will be made with a specific review of:
* Package dependencies,
* Build scripts (including downloaded ones),
* All downloaded components should be verified against static hash,
* RPM/DEB installation scripts (e.g. looking at constraints who would hijack other packages),
* Makefiles,
* Package build [reproducible](https://reproducible-builds.org/)
and any steps which would result in partial/total compromise of legitimate components. For this part, you can have a look to an example package called [qubes-skeleton](https://github.com/QubesOS-contrib/qubes-skeleton).
Package Maintainers
-------------------
If you contribute a package, we assume that you will be the maintainer of that package, unless you tell us otherwise.
As the maintainer of the package, it is your privilege and responsibility to:
* [Review](#review-procedure) each pull request made against the package.
* Decide when the package has reached a new version, and notify the Qubes core developers when this occurs.
If you do not wish to be the maintainer of your package, please let us know.
If you do not act on your maintainer duties for a given package for an extended period of time and after at least one reminder, we will assume that you no longer wish to be the maintainer for that package.

183
developer/general/package-contributions.rst Normal file
View File

@ -0,0 +1,183 @@
=====================
Package contributions
=====================
*This page is for developers who wish to contribute packages. If you want to install contributed packages, please see* :doc:`installing contributed packages </user/advanced-topics/installing-contributed-packages>` *.*
We’re very grateful to the talented and hard-working community members
who contribute software packages to Qubes OS. This page explains the
inclusion criteria and procedures for such packages, as well as the
roles and responsibilities of those involved.
Inclusion Criteria
------------------
In order to be accepted, packages must:
- In no way weaken the security of Qubes OS.
- Be published under an open-source license (read about the :doc:`Qubes OS License </developer/code/license>`).
- Follow our :doc:`coding guidelines </developer/code/coding-style>`.
- Be thoroughly tested.
- Have a clearly-defined use case for Qubes users.
- Not be unduly burdensome to review.
(Please note that we always reserve the right to add criteria to this
list.)
Contribution Procedure
----------------------
Before you start putting serious work into a package, we recommend that
you discuss your idea with the Qubes developers and the broader
community on the :ref:`qubes-devel mailing list <introduction/support:qubes-devel>`.
Once you have a package that’s ready to become part of Qubes OS, please
follow this procedure:
1. Ensure that your package satisfies the `Inclusion Criteria <#inclusion-criteria>`__.
2. If your code isn’t already on GitHub, create a GitHub repo that
contains your code. You can have a look to an example package called
`qubes-skeleton <https://github.com/QubesOS-contrib/qubes-skeleton>`__.
3. If you haven’t already, :doc:`sign your code </developer/code/code-signing>`.
4. Create an issue in
`qubes-issues <https://github.com/QubesOS/qubes-issues/issues/>`__
with the title ``[Contribution] your-package-name``. Include a link
to your repo, a brief description of your package, and a brief
explanation of why you think it should be included in Qubes. Please
note that the Qubes core developers are very busy. If they are under
heavy load when you submit your contribution, it may be a very long
time before they have time to review your package. If this happens,
please do not be discouraged. If you think they may have forgotten
about your pending contribution, you may “bump” your request by
commenting on your issue, but please do this *very* sparingly (i.e.,
no more than once a month). We appreciate your understanding!
5. You may be asked followup questions. If we decide to accept your
contribution, you will be invited to join the
`QubesOS-contrib <https://github.com/QubesOS-contrib>`__ organization
on GitHub as public recognition of your contribution (but without
push access; see `Review Procedure <#review-procedure>`__), and
`QubesOS-contrib <https://github.com/QubesOS-contrib>`__ will fork
your repo. If we decide not to accept your contribution, we will
state the reason and close the issue.
Update Procedure
----------------
*Anyone* can provide an update (patch) to a contributed package, not
just the person who contributed that package! The update procedure is
the same for everyone, including the original package contributor.
If you would like to update an already-contributed package
(specifically, a fork owned by
`QubesOS-contrib <https://github.com/QubesOS-contrib>`__), please submit
a :doc:`signed </developer/code/code-signing>`, fast-forwardable pull request to that
repo with your changes. Please note that your pull request **must** be
both :doc:`signed </developer/code/code-signing>` and fast-forwardable, or else it
will be closed without further review. One or more reviewers may post
comments on your pull request. Please be prepared to read and respond to
these comments.
Review Procedure
----------------
This review procedure covers both original package contributions (see
`Contribution Procedure <#contribution-procedure>`__) and all subsequent
updates to those packages, including updates from the original package
contributor (see `Update Procedure <#update-procedure>`__). All changes
will be reviewed by a Qubes Core Reviewer (QCR) and the `Package Maintainer <#package-maintainers>`__ (PM). In all cases, the QCR will be
a core Qubes developer. In some cases, the QCR and the PM will be the
same person. For example, if someone contributes a package, then
disappears, and no suitable replacement has been found, then it is
likely that a core Qubes developer will play both the QCR and PM roles
for that package, at least until another suitable candidate volunteers
to become the PM for that package.
The review procedure is as follows:
1. Someone, S, wishes to make a change to a package, P.
2. S submits a fast-forwardable pull request against the fork of P’s
repo owned by
`QubesOS-contrib <https://github.com/QubesOS-contrib>`__.
3. The PM reviews the pull request. If the pull request passes the PM’s
review, the PM adds a :doc:`signed </developer/code/code-signing>` *comment* on
the pull request stating that it has passed review. (In cases in
which S = PM, the PM can simply add a :doc:`signed </developer/code/code-signing>`
*tag* to the HEAD commit prior to submitting the pull request.) If
the pull request does not pass the PM’s review, the PM leaves a
comment on the pull request explaining why not.
4. The QCR reviews the pull request. If the pull request passes the
QCR’s review, the QCR pushes a :doc:`signed </developer/code/code-signing>` tag to
the HEAD commit stating that it has passed review and fast-forward
merges the pull request. If the pull request does not pass the QCR’s
review, the QCR leaves a comment on the pull request explaining why
not, and the QCR may decide to close the pull request.
In all the cases, the first condition to be validated by the QCR’s
review is to ensure that the contribution **will not** hijack any core
packages of `QubesOS <https://github.com/QubesOS>`__ and of course, none
of the `QubesOS-contrib <https://github.com/QubesOS-contrib>`__ packages
too. More precisely, particular attention to the whole build pipeline
will be made with a specific review of:
- Package dependencies,
- Build scripts (including downloaded ones),
- All downloaded components should be verified against static hash,
- RPM/DEB installation scripts (e.g. looking at constraints who would
hijack other packages),
- Makefiles,
- Package build `reproducible <https://reproducible-builds.org/>`__
and any steps which would result in partial/total compromise of
legitimate components. For this part, you can have a look to an example
package called
`qubes-skeleton <https://github.com/QubesOS-contrib/qubes-skeleton>`__.
Package Maintainers
-------------------
If you contribute a package, we assume that you will be the maintainer
of that package, unless you tell us otherwise. As the maintainer of the
package, it is your privilege and responsibility to:
- `Review <#review-procedure>`__ each pull request made against the
package.
- Decide when the package has reached a new version, and notify the
Qubes core developers when this occurs.
If you do not wish to be the maintainer of your package, please let us
know. If you do not act on your maintainer duties for a given package
for an extended period of time and after at least one reminder, we will
assume that you no longer wish to be the maintainer for that package.

17
developer/general/research.md
View File

@ -1,17 +0,0 @@
---
lang: en
layout: site
permalink: /research/
redirect_from:
- /doc/qubes-research/
- /en/doc/qubes-research/
- /doc/QubesResearch/
- /wiki/QubesResearch/
ref: 139
title: Research
---
Here are links to various research papers, projects, videos, and blog posts
related to Qubes OS.
{% include research.html %}

66
developer/general/research.rst Normal file
View File

@ -0,0 +1,66 @@
========
Research
========
Here are links to various research papers, projects, videos, and blog
posts related to Qubes OS.
Secure Software Development
===========================
- `Security challenges for the Qubes build process <https://blog.invisiblethings.org/2016/05/30/build-security.html>`__ by Joanna Rutkowska, May 2016
Towards Trusted Hardware
========================
- `Thoughts on the "physically secure" ORWL computer <https://blog.invisiblethings.org/2016/09/03/thoughts-about-orwl.html>`__ by Joanna Rutkowska, September 2016
- `State considered harmful <https://blog.invisiblethings.org/papers/2015/state_harmful.pdf>`__ by Joanna Rutkowska, December 2015
- `Intel x86 considered harmful <https://blog.invisiblethings.org/papers/2015/x86_harmful.pdf>`__ by Joanna Rutkowska, October 2015
Intel's Safeguard Extensions
============================
- `Thoughts on Intel's upcoming SoftwareGuard Extensions (Part 2) <https://blog.invisiblethings.org/2013/09/23/thoughts-on-intels-upcoming-software.html>`__ by Joanna Rutkowska, September 2013
- `Thoughts on Intel's upcoming SoftwareGuard Extensions (Part 1) <https://blog.invisiblethings.org/2013/08/30/thoughts-on-intels-upcoming-software.html>`__ by Joanna Rutkowska, August 2013
Attacks on Intel TXT
====================
- `Attacking Intel TXT® via SINIT code execution hijacking <https://invisiblethingslab.com/resources/2011/Attacking_Intel_TXT_via_SINIT_hijacking.pdf>`__ by Rafal Wojtczuk and Joanna Rutkowska, November 2011
- `Another Way to Circumvent Intel® Trusted Execution Technology <https://invisiblethingslab.com/resources/misc09/Another%20TXT%20Attack.pdf>`__ by Rafal Wojtczuk, Joanna Rutkowska, Alex Tereshkin, December 2009
- `ACPI: Design Principles and Concerns <https://www.ssi.gouv.fr/IMG/pdf/article_acpi.pdf>`__ by Loic Duflot, Olivier Levillain, and Benjamin Morin, 2009
- `Attacking Intel® Trusted Execution Technology <https://invisiblethingslab.com/resources/bh09dc/Attacking%20Intel%20TXT%20-%20paper.pdf>`__ by Rafal Wojtczuk and Joanna Rutkowska
Software Attacks Coming Through Devices
=======================================
- `Following the White Rabbit: Software Attacks against Intel® VT-d <https://invisiblethingslab.com/resources/2011/Software%20Attacks%20on%20Intel%20VT-d.pdf>`__ by Rafal Wojtczuk and Joanna Rutkowska, April 2011
- `On Formally Verified Microkernels (and on attacking them) <https://blog.invisiblethings.org/2010/05/03/on-formally-verified-microkernels-and.html>`__ by Joanna Rutkowska, May 2010
- `Remotely Attacking Network Cards (or why we do need VT-d and TXT) <https://blog.invisiblethings.org/2010/04/30/remotely-attacking-network-cards-or-why.html>`__ by Joanna Rutkowska, April 2010
- `Can you still trust your network card? <https://www.ssi.gouv.fr/IMG/pdf/csw-trustnetworkcard.pdf>`__ by Loïc Duflot, Yves-Alexis Perez and others
Application-level Security
==========================
- `Virtics: A System for Privilege Separation of Legacy Desktop Applications <https://www2.eecs.berkeley.edu/Pubs/TechRpts/2010/EECS-2010-70.pdf>`__ by Matt Piotrowski, May 2010
Compartmentalization, Isolation, and Separation
===============================================
- `Software compartmentalization vs. physical separation <https://invisiblethingslab.com/resources/2014/Software_compartmentalization_vs_physical_separation.pdf>`__ by Joanna Rutkowska, August 2014
- `Breaking Up is Hard to Do: Security and Functionality in a Commodity Hypervisor <https://tjd.phlegethon.org/words/sosp11-xoar.pdf>`__ by Patrick Colp at el., October 2011
The Qubes Architecture
======================
- `Qubes virtual mini-summit 2021 <https://www.qubes-os.org/news/2021/07/30/minisummit-agenda/>`__ by 3mdeb and the Qubes team, August 2021
- `Qubes Air: Generalizing the Qubes Architecture <https://www.qubes-os.org/news/2018/01/22/qubes-air/>`__ by Joanna Rutkowska, January 2018
- `Introducing the Next Generation Qubes Core Stack <https://www.qubes-os.org/news/2017/10/03/core3/>`__ by Joanna Rutkowska, October 2017
- `Introducing the Qubes Admin API <https://www.qubes-os.org/news/2017/06/27/qubes-admin-api/>`__ by Joanna Rutkowska, June 2017
- `Qubes OS Architecture Spec v0.3 <https://www.qubes-os.org/attachment/doc/arch-spec-0.3.pdf>`__ by Joanna Rutkowska and Rafal Wojtczuk, January 2010

230
developer/general/usability-ux.md
View File

@ -1,230 +0,0 @@
---
lang: en
layout: doc
permalink: /doc/usability-ux/
ref: 31
title: Usability & UX
---
Software that is too complicated to use, is often unused. Because we want as many people as possible to benefit from its unique security properties, the usability and user experience of Qubes OS is an utmost priority!
We ask anyone developing for Qubes OS to please read through this guide to better understand the user experience we strive to achieve. We also ask them to review [our visual style guide](/doc/visual-style-guide/) for other design related information.
---
## Easy To Use
An ideal user experience is friendly, and it beckons a new user to explore the interface. In this process, they can naturally discover how to use the software. Below are some guidelines that will help you design a user interface that accomplishes this goal.
<div class="focus">
<i class="fa fa-times"></i> <strong>Interfaces Should Not</strong>
</div>
- Require extensive configuration before a user can *begin* doing things
- Make it possible to break provided features or actions in unrecoverable ways
- Perform actions which compromise security and data
- Overwhelm the user with too much information and cognitive load
Perhaps the most common cause of mistakes is complexity. If there is a configuration setting that will significantly affect the user's experience, choose a safe and smart default then tuck this setting in an `Advanced Settings` panel.
<div class="focus">
<i class="fa fa-check"></i> <strong>Interfaces Should</strong>
</div>
- Make it easy to discover features and available actions
- Provide some understanding of what discovered features do
- Offer the ability to easily undo mistakes
- Choose intelligent defaults for settings
In making software easy to use, it is crucial to be mindful of [cognitive load](https://en.wikipedia.org/wiki/Cognitive_load) which dictates that *"humans are generally able to hold only seven +/- two units of information in short-term memory."* Making sure your interfaces don't pass this short-term memory limit is perhaps the most important factor in helping a user feel comfortable instead of overwhelmed.
---
## Easy to Understand
There will always be the need to communicate things to users. In these cases, an interface should aim to make this information easy to understand. The following are simple guides to help achieve this - none of these are absolute maxims!
<div class="focus">
<i class="fa fa-times"></i> <strong>Avoid Acronyms</strong>
</div>
Acronyms are compact and make good names for command line tools. They do not make graphical user interfaces more intuitive for non-technical users. Until one learns an acronym's meaning, it is gibberish. Avoid acronyms in your interfaces whenever possible!
- `DVM` - Disposable Virtual Machine
- `GUID` - Global Unique Identifier
- `PID` - Process Identification Number
- `NetVM` - Networking Virtual Machine
Despite this rule, some acronyms like `USB` are widely used and understood due to being in common use for over a decade. It is good to use these acronyms when the full words like `Universal Serial Bus` are more likely to confuse users.
<div class="focus">
<i class="fa fa-check"></i> <strong> Use Simple Words</strong>
</div>
Use the minimum amount of words needed to be descriptive, but also informative. Go with common words that are as widely understood. Sometimes, inventing a word such as `Qube` to describe a `virtual machine` makes the life of the user much easier.
- Use `Disposable Qube` instead of `DVM` or `Disposable Virtual Machine`
- Use `interface` instead of `GUI` or `Graphical User Interface`
- Use `application number` instead of `PID` or `Process Identification Number`
- Use `Networking` or `Networking Qube` instead of `NetVM` given context
---
<div class="focus">
<i class="fa fa-times"></i> <strong> Avoid Technical Words</strong>
</div>
Technical words are usually more accurate, but they often *only* make sense to technical users and are confusing and unhelpful to non-technical users. Examples of technical words that might show up in Qubes OS are:
- `root.img`
- `savefile`
- `qrexec-daemon`
These are all terms that have at some point showed up in users' notification messages. Each term is very specific, but requires the user to understand virtualization to interpret.
<div class="focus">
<i class="fa fa-check"></i> <strong> Use Common Concepts</strong>
</div>
Large amounts of the global population have been using computers for one or two decades and have formed some mental models of how things work. Leveraging these mental models are a huge gain.
- Use `disk space` instead of `root.img`, since while not quite accurate, it makes contextual sense
- Use `saving` instead of `savefile` as the former is the action trying to be completed
- Use `Qubes` instead of `qrexec-daemon` as it gives better context on what is happening
These words are more abstract and user relevant- they help a user understand what is happening based on already known concepts (disk space) or start to form a mental model of something new (Qubes).
---
<div class="focus">
<i class="fa fa-times"></i> <strong>Avoid Inconsistencies</strong>
</div>
It is easy to start abbreviating (or making acronyms) of long terms like `Disposable Virtual Machine` depending on where the term shows up in an interface.
- `DVM`
- `DispVM`
- `DisposableVM`
This variation in terms can cause new users to question or second guess what the three different variations mean, which can lead to inaction or mistakes.
<div class="focus">
<i class="fa fa-check"></i> <strong> Make Things Consistent</strong>
</div>
Always strive to keep things consistent in the interfaces as well as documentation and other materials.
- Use `Disposable Qube` at all times as it meets other criteria as well.
By using the same term throughout an interface, a user can create a mental model and relationship with that term allowing them to feel empowered.
---
<div class="focus">
<i class="fa fa-times"></i> <strong>Avoid Duplicate Words</strong>
</div>
It is easy to add words like `Domain` before items in a list or menu in an attempt to be descriptive, such as:
~~~
Menu
- Domain: work
- Domain: banking
- Domain: personal
~~~
The repeated use of the word `Domain` requires a user to read it for each item in the list, which makes extra work for the eye in parsing out the relevant word like `work, banking, or personal`. This also affects horizontal space on fixed width lines.
<div class="focus">
<i class="fa fa-check"></i> <strong> Create Groups & Categories</strong>
</div>
It is more efficient to group things under headings instead as this allows the eye to easily scan the uniqueness of the items. (As per our previous example:)
~~~
Domains
- Work
- Banking
- Personal
~~~
---
## Easy To Complete
Lastly, expected (and unexpected) situations often require user actions or input. Make resolving these occurences as easy as possible to complete the action.
<div class="focus">
<i class="fa fa-times"></i><strong>Don't Leave Users Stranded</strong>
</div>
Consider the following notifications:
- `The disk space of your Qube "Work" is full`
- `There was an error saving Qube "Personal"`
Instead of displaying solvable errors like these and neglecting to provide a fix:
<div class="focus">
<i class="fa fa-check"></i><strong>Offer Actionable Solutions</strong>
</div>
Error messages and limits such as those in the previous example can be greatly improved by adding buttons or links to helpful information.
- Add a button to `Increase Disk Space`
- Add a link to a documentation page called `Troubleshoot saving data`
In adhering to these principles, you'll make undesirable situations more manageable for users instead of feeling stranded.
---
<div class="focus">
<i class="fa fa-check"></i><strong>Minimize Repetitive Steps</strong>
</div>
There are many cases where a user wants to perform an action on more than one file or folder. However in order to do the action, the user must repeat certain steps such as:
1. Click on `Open File` from a menu or button
2. Navigate through file system
- Click Folder One
- Click Folder Two
- Click Folder Three
- Click Folder Four
3. Select proper file
4. Complete task on file
That subtle act of clicking through a file system can prove to be significant if a user needs to open more than a couple files in the same directory. We can alleviate some of the work by changing the process:
1. Click on `Open File` from a menu or button
2. Remember last open folder/file system
3. Select proper file
4. Complete task
Clearly, cutting out something as simple as navigating through the file system can save a user quite a bit of time. Alternatively, adding a button or menu item like `Open Multiple Files` might be even better, because remembering and using relevant hotkeys is often something only power users know how to do!
---
## GNOME, KDE, and Xfce
The desktop GUIs that QubesOS versions 1 - 4.1 offer are [KDE](https://kde.org) and [Xfce](https://xfce.org). We are currently migrating towards using [GNOME](https://www.gnome.org). We know some people prefer KDE, but we believe Gnome is easier to use for average non-technical users. Xfce will always be supported, and technical users will always have the choice to use KDE or other desktop environments.
This change means you should use [GTK](https://gtk.org/) rather than Qt for new GUIs.
All three of these mentioned desktop environments have their own [human interface guidelines](https://en.wikipedia.org/wiki/Human_interface_guidelines), and we suggest you familiarize yourself with the platform you developing for.
- [GNOME Human Interface Guidelines](https://developer.gnome.org/hig/)
- [KDE HIG](https://hig.kde.org/)
- [Xfce UI Guidlines](https://wiki.xfce.org/dev/hig/general)
---
## Further Learning & Inspiration
Learning to make well designing intuitive interfaces and software is specialized skillset that can take years to cultivate, but if you are interested in furthering your understanding, we suggest the following resources:
- [Learn Design Principles](https://web.archive.org/web/20180101172357/http://learndesignprinciples.com/) by Melissa Mandelbaum
- [Usability in Free Software](https://jancborchardt.net/usability-in-free-software) by Jan C. Borchardt
- [Superheroes & Villains in Design](https://vimeo.com/70030549) by Aral Balkan
- [First Rule of Usability? Don’t Listen to Users](https://www.nngroup.com/articles/first-rule-of-usability-dont-listen-to-users/) by Jakob Nielsen
- [10 Usability Heuristics for User Interface Design](https://www.nngroup.com/articles/ten-usability-heuristics/) by Jakob Nielsen
- [Hack Design](https://hackdesign.org/) - online learning program

420
developer/general/usability-ux.rst Normal file
View File

@ -0,0 +1,420 @@
==============
Usability & UX
==============
Software that is too complicated to use, is often unused. Because we
want as many people as possible to benefit from its unique security
properties, the usability and user experience of Qubes OS is an utmost
priority!
We ask anyone developing for Qubes OS to please read through this guide
to better understand the user experience we strive to achieve. We also
ask them to review :doc:`our visual style guide </developer/general/visual-style-guide>`
for other design related information.
----
Easy To Use
-----------
An ideal user experience is friendly, and it beckons a new user to
explore the interface. In this process, they can naturally discover how
to use the software. Below are some guidelines that will help you design
a user interface that accomplishes this goal.
.. important::
Interfaces Should Not
- Require extensive configuration before a user can *begin* doing
things
- Make it possible to break provided features or actions in
unrecoverable ways
- Perform actions which compromise security and data
- Overwhelm the user with too much information and cognitive load
Perhaps the most common cause of mistakes is complexity. If there is a
configuration setting that will significantly affect the user’s
experience, choose a safe and smart default then tuck this setting in an
``Advanced Settings`` panel.
.. important::
Interfaces Should
- Make it easy to discover features and available actions
- Provide some understanding of what discovered features do
- Offer the ability to easily undo mistakes
- Choose intelligent defaults for settings
In making software easy to use, it is crucial to be mindful of
`cognitive load <https://en.wikipedia.org/wiki/Cognitive_load>`__ which
dictates that *“humans are generally able to hold only seven +/- two units of information in short-term memory.”* Making sure your interfaces
don’t pass this short-term memory limit is perhaps the most important
factor in helping a user feel comfortable instead of overwhelmed.
----
Easy to Understand
------------------
There will always be the need to communicate things to users. In these
cases, an interface should aim to make this information easy to
understand. The following are simple guides to help achieve this - none
of these are absolute maxims!
.. important::
Avoid Acronyms
Acronyms are compact and make good names for command line tools. They do
not make graphical user interfaces more intuitive for non-technical
users. Until one learns an acronym’s meaning, it is gibberish. Avoid
acronyms in your interfaces whenever possible!
- ``DVM`` - Disposable Virtual Machine
- ``GUID`` - Global Unique Identifier
- ``PID`` - Process Identification Number
- ``NetVM`` - Networking Virtual Machine
Despite this rule, some acronyms like ``USB`` are widely used and
understood due to being in common use for over a decade. It is good to
use these acronyms when the full words like ``Universal Serial Bus`` are
more likely to confuse users.
.. important::
Use Simple Words
Use the minimum amount of words needed to be descriptive, but also
informative. Go with common words that are as widely understood.
Sometimes, inventing a word such as ``Qube`` to describe a
``virtual machine`` makes the life of the user much easier.
- Use ``Disposable Qube`` instead of ``DVM`` or
``Disposable Virtual Machine``
- Use ``interface`` instead of ``GUI`` or ``Graphical User Interface``
- Use ``application number`` instead of ``PID`` or
``Process Identification Number``
- Use ``Networking`` or ``Networking Qube`` instead of ``NetVM`` given
context
----
.. important::
Avoid Technical Words
Technical words are usually more accurate, but they often *only* make
sense to technical users and are confusing and unhelpful to
non-technical users. Examples of technical words that might show up in
Qubes OS are:
- ``root.img``
- ``savefile``
- ``qrexec-daemon``
These are all terms that have at some point showed up in users’
notification messages. Each term is very specific, but requires the user
to understand virtualization to interpret.
.. important::
Use Common Concepts
Large amounts of the global population have been using computers for one
or two decades and have formed some mental models of how things work.
Leveraging these mental models are a huge gain.
- Use ``disk space`` instead of ``root.img``, since while not quite
accurate, it makes contextual sense
- Use ``saving`` instead of ``savefile`` as the former is the action
trying to be completed
- Use ``Qubes`` instead of ``qrexec-daemon`` as it gives better context
on what is happening
These words are more abstract and user relevant- they help a user
understand what is happening based on already known concepts (disk
space) or start to form a mental model of something new (Qubes).
----
.. important::
Avoid Inconsistencies
It is easy to start abbreviating (or making acronyms) of long terms like
``Disposable Virtual Machine`` depending on where the term shows up in
an interface.
- ``DVM``
- ``DispVM``
- ``DisposableVM``
This variation in terms can cause new users to question or second guess
what the three different variations mean, which can lead to inaction or
mistakes.
.. important::
Make Things Consistent
Always strive to keep things consistent in the interfaces as well as
documentation and other materials.
- Use ``Disposable Qube`` at all times as it meets other criteria as
well.
By using the same term throughout an interface, a user can create a
mental model and relationship with that term allowing them to feel
empowered.
----
.. important::
Avoid Duplicate Words
It is easy to add words like ``Domain`` before items in a list or menu
in an attempt to be descriptive, such as:
.. code:: bash
Menu
- Domain: work
- Domain: banking
- Domain: personal
The repeated use of the word ``Domain`` requires a user to read it for
each item in the list, which makes extra work for the eye in parsing out
the relevant word like ``work, banking, or personal``. This also affects
horizontal space on fixed width lines.
.. important::
Create Groups & Categories
It is more efficient to group things under headings instead as this
allows the eye to easily scan the uniqueness of the items. (As per our
previous example:)
.. code:: bash
Domains
- Work
- Banking
- Personal
----
Easy To Complete
----------------
Lastly, expected (and unexpected) situations often require user actions
or input. Make resolving these occurences as easy as possible to
complete the action.
.. important::
Don’t Leave Users Stranded
Consider the following notifications:
- ``The disk space of your Qube "Work" is full``
- ``There was an error saving Qube "Personal"``
Instead of displaying solvable errors like these and neglecting to
provide a fix:
.. important::
Offer Actionable Solutions
Error messages and limits such as those in the previous example can be
greatly improved by adding buttons or links to helpful information.
- Add a button to ``Increase Disk Space``
- Add a link to a documentation page called
``Troubleshoot saving data``
In adhering to these principles, you’ll make undesirable situations more
manageable for users instead of feeling stranded.
----
.. important::
Minimize Repetitive Steps
There are many cases where a user wants to perform an action on more
than one file or folder. However in order to do the action, the user
must repeat certain steps such as:
1. Click on ``Open File`` from a menu or button
2. Navigate through file system
- Click Folder One
- Click Folder Two
- Click Folder Three
- Click Folder Four
3. Select proper file
4. Complete task on file
That subtle act of clicking through a file system can prove to be
significant if a user needs to open more than a couple files in the same
directory. We can alleviate some of the work by changing the process:
1. Click on ``Open File`` from a menu or button
2. Remember last open folder/file system
3. Select proper file
4. Complete task
Clearly, cutting out something as simple as navigating through the file
system can save a user quite a bit of time. Alternatively, adding a
button or menu item like ``Open Multiple Files`` might be even better,
because remembering and using relevant hotkeys is often something only
power users know how to do!
----
GNOME, KDE, and Xfce
--------------------
The desktop GUIs that QubesOS versions 1 - 4.1 offer are
`KDE <https://kde.org>`__ and `Xfce <https://xfce.org>`__. We are
currently migrating towards using `GNOME <https://www.gnome.org>`__. We
know some people prefer KDE, but we believe Gnome is easier to use for
average non-technical users. Xfce will always be supported, and
technical users will always have the choice to use KDE or other desktop
environments.
This change means you should use `GTK <https://gtk.org/>`__ rather than
Qt for new GUIs.
All three of these mentioned desktop environments have their own `human interface guidelines <https://en.wikipedia.org/wiki/Human_interface_guidelines>`__,
and we suggest you familiarize yourself with the platform you developing
for.
- `GNOME Human Interface Guidelines <https://developer.gnome.org/hig/>`__
- `KDE HIG <https://hig.kde.org/>`__
- `Xfce UI Guidlines <https://wiki.xfce.org/dev/hig/general>`__
----
Further Learning & Inspiration
------------------------------
Learning to make well designing intuitive interfaces and software is
specialized skillset that can take years to cultivate, but if you are
interested in furthering your understanding, we suggest the following
resources:
- `Learn Design Principles <https://web.archive.org/web/20180101172357/http://learndesignprinciples.com/>`__
by Melissa Mandelbaum
- `Usability in Free Software <https://jancborchardt.net/usability-in-free-software>`__ by
Jan C. Borchardt
- `Superheroes & Villains in Design <https://vimeo.com/70030549>`__ by
Aral Balkan
- `First Rule of Usability? Don’t Listen to Users <https://www.nngroup.com/articles/first-rule-of-usability-dont-listen-to-users/>`__
by Jakob Nielsen
- `10 Usability Heuristics for User Interface Design <https://www.nngroup.com/articles/ten-usability-heuristics/>`__
by Jakob Nielsen
- `Hack Design <https://hackdesign.org/>`__ - online learning program

9
developer/general/visual-style-guide.md
View File

@ -1,9 +0,0 @@
---
lang: en
layout: doc
permalink: /doc/visual-style-guide/
redirect_from:
- /doc/style-guide/
ref: 27
title: Visual style guide
---

4
developer/general/visual-style-guide.rst Normal file
View File

@ -0,0 +1,4 @@
==================
Visual style guide
==================

78
developer/general/website-style-guide.md
View File

@ -1,78 +0,0 @@
---
lang: en
layout: doc
permalink: /doc/website-style-guide/
title: Website style guide
---
This page explains the standards we follow for building and maintaining the
website. Please follow these guidelines and conventions when modifying the
website. For the standards governing the documentation in particular, please
see the [documentation style guide](/doc/documentation-style-guide/).
## Coding conventions
The following conventions apply to the website as a whole, including everything
written in HTML, CSS, YAML, and Liquid. These conventions are intended to keep
the codebase consistent when multiple collaborators are working on it. They
should be understood as a practical set of rules for maintaining order in this
specific codebase rather than as a statement of what is objectively right or
good.
### General practices
- Use comments to indicate the purposes of different blocks of code. This makes
the file easier to understand and navigate.
- Use descriptive variable names. Never use one or two letter variable names.
Avoid obscure abbreviations and made-up words.
- In general, make it easy for others to read your code. Your future self will
thank you, and so will your collaborators!
- [Don't Repeat Yourself
(DRY)!](https://en.wikipedia.org/wiki/Don%27t_repeat_yourself) Instead of
repeating the same block of code multiple times, abstract it out into a
separate file and `include` that file where you need it.
### Whitespace
- Always use spaces. Never use tabs.
- Each indentation step should be exactly two (2) spaces.
- Whenever you add an opening tag, indent the following line. (Exception: If
you open and close the tag on the same line, do not indent the following
line.)
- Indent Liquid the same way as HTML.
- In general, the starting columns of every adjacent pair of lines should be no
more than two spaces apart (example below).
- No blank or empty lines. (Hint: When you feel you need one, add a comment on
that line instead.)
#### Indentation example
Here's an example that follows the indentation rules:
{% raw %}
```html
<table>
<tr>
<th title="Anchor Link"><span class="fa fa-link"></span></th>
{% for item in secs.htmlsections[0].columns %}
<th>{{ item.title }}</th>
{% endfor %}
</tr>
{% for canary in site.data.sec-canary reversed %}
<tr id="{{ canary.canary }}">
<td><a href="#{{ canary.canary }}" class="fa fa-link black-icon" title="Anchor link to Qubes Canary row: Qubes Canary #{{ canary.canary }}"></a></td>
<td>{{ canary.date }}</td>
<td><a href="https://github.com/QubesOS/qubes-secpack/blob/master/canaries/canary-{{ canary.canary }}-{{ canary.date | date: '%Y' }}.txt">Qubes Canary #{{ canary.canary }}</a></td>
</tr>
{% endfor %}
</table>
```
{% endraw %}

91
developer/general/website-style-guide.rst Normal file
View File

@ -0,0 +1,91 @@
===================
Website style guide
===================
This page explains the standards we follow for building and maintaining
the website. Please follow these guidelines and conventions when
modifying the website. For the standards governing the documentation in
particular, please see the :doc:`documentation style guide </developer/general/documentation-style-guide>`.
Coding conventions
------------------
The following conventions apply to the website as a whole, including
everything written in HTML, CSS, YAML, and Liquid. These conventions are
intended to keep the codebase consistent when multiple collaborators are
working on it. They should be understood as a practical set of rules for
maintaining order in this specific codebase rather than as a statement
of what is objectively right or good.
General practices
^^^^^^^^^^^^^^^^^
- Use comments to indicate the purposes of different blocks of code.
This makes the file easier to understand and navigate.
- Use descriptive variable names. Never use one or two letter variable
names. Avoid obscure abbreviations and made-up words.
- In general, make it easy for others to read your code. Your future
self will thank you, and so will your collaborators!
- `Don’t Repeat Yourself (DRY)! <https://en.wikipedia.org/wiki/Don%27t_repeat_yourself>`__
Instead of repeating the same block of code multiple times, abstract
it out into a separate file and ``include`` that file where you need
it.
Whitespace
^^^^^^^^^^
- Always use spaces. Never use tabs.
- Each indentation step should be exactly two (2) spaces.
- Whenever you add an opening tag, indent the following line.
(Exception: If you open and close the tag on the same line, do not
indent the following line.)
- Indent Liquid the same way as HTML.
- In general, the starting columns of every adjacent pair of lines
should be no more than two spaces apart (example below).
- No blank or empty lines. (Hint: When you feel you need one, add a
comment on that line instead.)
Indentation example
^^^^^^^^^^^^^^^^^^^
Here’s an example that follows the indentation rules:
.. code:: html
<table>
<tr>
<th title="Anchor Link"><span class="fa fa-link"></span></th>
{% for item in secs.htmlsections[0].columns %}
<th>{{ item.title }}</th>
{% endfor %}
</tr>
{% for canary in site.data.sec-canary reversed %}
<tr id="{{ canary.canary }}">
<td><a href="#{{ canary.canary }}" class="fa fa-link black-icon" title="Anchor link to Qubes Canary row: Qubes Canary #{{ canary.canary }}"></a></td>
<td>{{ canary.date }}</td>
<td><a href="https://github.com/QubesOS/qubes-secpack/blob/master/canaries/canary-{{ canary.canary }}-{{ canary.date | date: '%Y' }}.txt">Qubes Canary #{{ canary.canary }}</a></td>
</tr>
{% endfor %}
</table>

50
developer/releases/1_0/release-notes.md
View File

@ -1,50 +0,0 @@
---
lang: en
layout: doc
permalink: /doc/releases/1.0/release-notes/
redirect_from:
- /en/doc/releases/1.0/release-notes/
ref: 18
title: Qubes R1.0 release notes
---
Detailed release notes in [this blog post](https://blog.invisiblethings.org/2012/09/03/introducing-qubes-10.html).
## Known issues
- Installer might not support some USB keyboards (\#230). This seems to include all the Mac Book keyboards (most PC laptops have PS2 keyboards and are not affected).
- If you don't enable Composition (System Setting -\> Desktop -\> Enable desktop effects), which you really should do, then the KDE task bar might get ugly (e.g. half of it might be black). This is some KDE bug that we don't plan to fix.
- Some keyboard layout set by KDE System Settings can cause [keyboard not working at all](https://groups.google.com/group/qubes-devel/browse_thread/thread/77d076b65dda7226). If you hit this issue, you can switch to console (by console login option) and manually edit `/etc/X11/xorg.conf.d/00-system-setup-keyboard.conf` (and `/etc/sysconfig/keyboard`) and place correct keyboard layout settings (details in linked thread). You can check if specific keyboard layout settings are proper using `setxkbmap` tool.
- On systems with more than 8GB of RAM there is problem with DisposableVM. To fix it, limit maximum memory allocation for DispVM to 3GB
~~~
qvm-prefs -s fedora-17-x64-dvm maxmem 3072
qvm-create-default-dvm --default-template --default-script
~~~
- On some systems the KDE Window Manager might freeze upon resuming from S3 sleep when compositing is enabled (and the only method to log in to the system if this happens is to switch to a text console, enter your user's password, kill the kwin process, go back to the Xorg console, log in, and start a new instance of kwin using Konsole application :) If you experience such problems, make sure to disable compositing before putting the system into sleep by pressing Alt-Ctrl-F12 (and then enabling it back once you log in after resume) -- this way you should never see this problem again.
## Downloads
See [Qubes Downloads](/doc/QubesDownloads/).
## Installation instructions
See [Installation Guide](/doc/installation-guide/).
## Upgrading
### From Qubes 1.0-rc1
If you're already running Qubes 1.0-rc1, you don't need to reinstall, it's just enough to update the packages in your Dom0 and the template VM(s). The easiest way for doing this is to click on the Update Button in the Qubes Manger -- one click when you selected Dom0, and one click for each of your template VM (by default there is just one template).
### From Qubes 1.0 Beta 3
If you have Qubes Beta 3 currently installed on your system, you must reinstall from scratch, as we offer no direct upgrade option in the installer (sorry). However, we do offer tools for smooth migration of your AppVMs. In order to do that, please backup your AppVMs using the `qvm-backup` tool [as usual](/doc/backup-restore/). Then, after you install Qubes 1.0 rc1, you can restore them using `qvm-backup-restore` tool. However, because we have changed the default template in RC1, you should tell qvm-back-restore about that by passing `--replace-template` option:
~~~
qvm-backup-restore <backup_dir> --replace-template=fedora-15-x64:fedora-17-x64
~~~

96
developer/releases/1_0/release-notes.rst Normal file
View File

@ -0,0 +1,96 @@
========================
Qubes R1.0 release notes
========================
Detailed release notes in `this blog post <https://blog.invisiblethings.org/2012/09/03/introducing-qubes-10.html>`__.
Known issues
------------
- Installer might not support some USB keyboards (#230). This seems to
include all the Mac Book keyboards (most PC laptops have PS2
keyboards and are not affected).
- If you don’t enable Composition (System Setting -> Desktop -> Enable
desktop effects), which you really should do, then the KDE task bar
might get ugly (e.g. half of it might be black). This is some KDE bug
that we don’t plan to fix.
- Some keyboard layout set by KDE System Settings can cause `keyboard not working at all <https://groups.google.com/group/qubes-devel/browse_thread/thread/77d076b65dda7226>`__.
If you hit this issue, you can switch to console (by console login
option) and manually edit
``/etc/X11/xorg.conf.d/00-system-setup-keyboard.conf`` (and
``/etc/sysconfig/keyboard``) and place correct keyboard layout
settings (details in linked thread). You can check if specific
keyboard layout settings are proper using ``setxkbmap`` tool.
- On systems with more than 8GB of RAM there is problem with
DisposableVM. To fix it, limit maximum memory allocation for DispVM
to 3GB
.. code:: bash
qvm-prefs -s fedora-17-x64-dvm maxmem 3072
qvm-create-default-dvm --default-template --default-script
- On some systems the KDE Window Manager might freeze upon resuming
from S3 sleep when compositing is enabled (and the only method to log
in to the system if this happens is to switch to a text console,
enter your user’s password, kill the kwin process, go back to the
Xorg console, log in, and start a new instance of kwin using Konsole
application :) If you experience such problems, make sure to disable
compositing before putting the system into sleep by pressing
Alt-Ctrl-F12 (and then enabling it back once you log in after resume)
– this way you should never see this problem again.
Downloads
---------
See :doc:`Qubes Downloads </user/downloading-installing-upgrading/downloads>`.
Installation instructions
-------------------------
See :doc:`Installation Guide </user/downloading-installing-upgrading/installation-guide>`.
Upgrading
---------
From Qubes 1.0-rc1
^^^^^^^^^^^^^^^^^^
If you’re already running Qubes 1.0-rc1, you don’t need to reinstall,
it’s just enough to update the packages in your Dom0 and the template
VM(s). The easiest way for doing this is to click on the Update Button
in the Qubes Manger – one click when you selected Dom0, and one click
for each of your template VM (by default there is just one template).
From Qubes 1.0 Beta 3
^^^^^^^^^^^^^^^^^^^^^
If you have Qubes Beta 3 currently installed on your system, you must
reinstall from scratch, as we offer no direct upgrade option in the
installer (sorry). However, we do offer tools for smooth migration of
your AppVMs. In order to do that, please backup your AppVMs using the
``qvm-backup`` tool :doc:`as usual </user/how-to-guides/how-to-back-up-restore-and-migrate>`. Then, after you
install Qubes 1.0 rc1, you can restore them using ``qvm-backup-restore``
tool. However, because we have changed the default template in RC1, you
should tell qvm-back-restore about that by passing
``--replace-template`` option:
.. code:: bash
qvm-backup-restore <backup_dir> --replace-template=fedora-15-x64:fedora-17-x64

85
developer/releases/2_0/release-notes.md
View File

@ -1,85 +0,0 @@
---
lang: en
layout: doc
permalink: /doc/releases/2.0/release-notes/
redirect_from:
- /en/doc/releases/2.0/release-notes/
ref: 25
title: Qubes R2.0 release notes
---
Detailed release notes in [this blog post](https://blog.invisiblethings.org/2014/09/26/announcing-qubes-os-release-2.html)
## New features since 1.0
* Support for generic fully virtualized VMs (without qemu in the TCB!)
* Support for Windows-based AppVMs integration (clipboard, file exchange, qrexec, pv drivers)
* Secure audio input to select AppVMs (Hello Skype users!)
* Clipboard is now also controlled by central policies, unified with other qrexec policies.
* Out of the box TorVM support
* Experimental support for PVUSB
* Updated Xorg packages in Dom0 to support new GPUs
* DisposableVM customization support
* Introduced Xfce 4.10 environment for Dom0 as an alternative to KDE
* Advanced infrastructure for system backups
* Ability to autostart selected VM at system startup
* Support for dynamic screen resolution change
* Dom0 distribution upgraded to Fedora 20
## Known issues
* On some graphics cards the Xfce4 Window Manager (one of the two supported Dom0 Windows Managers in Qubes R2, the other being KDE) might behave "strangely", e.g. decorations might not be drawn sometimes. Also the accompanying lightdm login manager might incorrectly display the wallpaper. If you're facing those problems, it's advisable to use the KDE Window Manager and kdm instead of Xfce4 and lightdm (this is default if one chooses the KDE only installation option in the installer).
* Some icons in the Qubes Manager application might not be drawn correctly when using the Xfce4 environment in Dom0. If this bothers you, please use the KDE environment instead.
* If your GPU is not correctly supported by the Dom0 kernel (e.g. the 3D desktop effects do not run smoothly) then you might experience "heaviness" with Windows 7-based AppVMs. In that case, please solve the problem with your GPU support in Dom0 in the first place (by using a different kernel), or install Qubes OS on a different system.
* Under some circumstances, Qubes backup can create broken backup, without any visible message (\#902). It is advisable to verify a backup to spot the problem. If you encounter this problem, backup VM directory manually.
* System shutdown sometimes is very slow (\#903). To mitigate the problem, shutdown all the VMs first.
* For other known issues take a look at [our trac tickets](https://wiki.qubes-os.org/query?status=accepted&status=assigned&status=new&status=reopened&type=defect&milestone=Release+2.1+(post+R2)&col=id&col=summary&col=status&col=type&col=priority&col=milestone&col=component&order=priority)
It is advised to install updates just after system installation to apply bug fixes for (some of) the above problems.
## Downloads
See [Qubes Downloads](/doc/QubesDownloads/).
## Installation instructions
See [Installation Guide](/doc/installation-guide/).
## Upgrading
### From Qubes R2 rc1
Upgrading from Qubes R2 rc1 should be a simple matter of installing updates for [dom0](/doc/how-to-install-software-in-dom0/) and [VMs](/doc/software-update-vm/).
### From Qubes R2 beta 3 and older
The easiest and safest way to upgrade to Qubes R2 (especially from older releases) is to install it from scratch and use [qubes backup and restore tools](/doc/backup-restore/) for migrating of all of the user VMs.
Users of R2 beta 3 can upgrade using procedure that has been described [here](/doc/upgrade-to-r2/).
Note: if the user has custom Template VMs (i.e. other than the default template, e.g. created from it by cloning), or Standalone VMs, then the user should perform manual upgrade from R2B3 to R2rc1, as described under the link given above.
### Migrating between beta releases
#### From Qubes R1 to R2 beta1
If you're already running Qubes Release 1, you don't need to reinstall, it's just enough to update the packages in your Dom0 and the template VM(s). This procedure is described [here?](/doc/upgrade-to-r2/).
#### From Qubes R1 or R2 Beta 1 to R2 beta2
Because of the distribution change in R2B2 (from fc13 to fc18) it's preferred that users reinstall Qubes R2B2 from scratch, and use [qubes backup and restore tools](/doc/backup-restore/) for migrating of all of the user VMs.
Advanced users (and advanced users only) can also try a manual upgrade procedure that has been described [here](/doc/upgrade-to-r2b2/). It's advisable to backup your VMs before proceeding anyway!
#### Upgrading from Qubes R1 or R2 Beta 2 to R2 beta 3
The easiest and safest way to upgrade to Qubes R2B3 is to install it from scratch and use [qubes backup and restore tools](/doc/backup-restore/) for migrating of all of the user VMs.
Users can also try a manual upgrade procedure that has been described [here](/doc/upgrade-to-r2b3/).
Note: if the user has custom Template VMs (i.e. other than the default template, e.g. created from it by cloning), or Standalone VMs, then the user should perform manual upgrade from R2B2 to R2B3, as described under the link given above.

157
developer/releases/2_0/release-notes.rst Normal file
View File

@ -0,0 +1,157 @@
========================
Qubes R2.0 release notes
========================
Detailed release notes in `this blog post <https://blog.invisiblethings.org/2014/09/26/announcing-qubes-os-release-2.html>`__
New features since 1.0
----------------------
- Support for generic fully virtualized VMs (without qemu in the TCB!)
- Support for Windows-based AppVMs integration (clipboard, file
exchange, qrexec, pv drivers)
- Secure audio input to select AppVMs (Hello Skype users!)
- Clipboard is now also controlled by central policies, unified with
other qrexec policies.
- Out of the box TorVM support
- Experimental support for PVUSB
- Updated Xorg packages in Dom0 to support new GPUs
- DisposableVM customization support
- Introduced Xfce 4.10 environment for Dom0 as an alternative to KDE
- Advanced infrastructure for system backups
- Ability to autostart selected VM at system startup
- Support for dynamic screen resolution change
- Dom0 distribution upgraded to Fedora 20
Known issues
------------
- On some graphics cards the Xfce4 Window Manager (one of the two
supported Dom0 Windows Managers in Qubes R2, the other being KDE)
might behave “strangely”, e.g. decorations might not be drawn
sometimes. Also the accompanying lightdm login manager might
incorrectly display the wallpaper. If you’re facing those problems,
it’s advisable to use the KDE Window Manager and kdm instead of Xfce4
and lightdm (this is default if one chooses the KDE only installation
option in the installer).
- Some icons in the Qubes Manager application might not be drawn
correctly when using the Xfce4 environment in Dom0. If this bothers
you, please use the KDE environment instead.
- If your GPU is not correctly supported by the Dom0 kernel (e.g. the
3D desktop effects do not run smoothly) then you might experience
“heaviness” with Windows 7-based AppVMs. In that case, please solve
the problem with your GPU support in Dom0 in the first place (by
using a different kernel), or install Qubes OS on a different system.
- Under some circumstances, Qubes backup can create broken backup,
without any visible message (#902). It is advisable to verify a
backup to spot the problem. If you encounter this problem, backup VM
directory manually.
- System shutdown sometimes is very slow (#903). To mitigate the
problem, shutdown all the VMs first.
- For other known issues take a look at `our trac tickets <https://wiki.qubes-os.org/query?status=accepted&status=assigned&status=new&status=reopened&type=defect&milestone=Release+2.1+(post+R2)&col=id&col=summary&col=status&col=type&col=priority&col=milestone&col=component&order=priority>`__
It is advised to install updates just after system installation to apply
bug fixes for (some of) the above problems.
Downloads
---------
See :doc:`Qubes Downloads </user/downloading-installing-upgrading/downloads>`.
Installation instructions
-------------------------
See :doc:`Installation Guide </user/downloading-installing-upgrading/installation-guide>`.
Upgrading
---------
From Qubes R2 rc1
^^^^^^^^^^^^^^^^^
Upgrading from Qubes R2 rc1 should be a simple matter of installing
updates for :doc:`dom0 </user/advanced-topics/how-to-install-software-in-dom0>` and
:doc:`VMs </user/how-to-guides/how-to-install-software>`.
From Qubes R2 beta 3 and older
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The easiest and safest way to upgrade to Qubes R2 (especially from older
releases) is to install it from scratch and use :doc:`qubes backup and restore tools </user/how-to-guides/how-to-back-up-restore-and-migrate>` for migrating of all of the user
VMs.
Users of R2 beta 3 can upgrade using procedure that has been described
:doc:`here </user/downloading-installing-upgrading/upgrade/2>`.
Note: if the user has custom Template VMs (i.e. other than the default
template, e.g. created from it by cloning), or Standalone VMs, then the
user should perform manual upgrade from R2B3 to R2rc1, as described
under the link given above.
Migrating between beta releases
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
From Qubes R1 to R2 beta1
^^^^^^^^^^^^^^^^^^^^^^^^^
If you’re already running Qubes Release 1, you don’t need to reinstall,
it’s just enough to update the packages in your Dom0 and the template
VM(s). This procedure is described :doc:`here? </user/downloading-installing-upgrading/upgrade/2>`.
From Qubes R1 or R2 Beta 1 to R2 beta2
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Because of the distribution change in R2B2 (from fc13 to fc18) it’s
preferred that users reinstall Qubes R2B2 from scratch, and use :doc:`qubes backup and restore tools </user/how-to-guides/how-to-back-up-restore-and-migrate>` for migrating of all
of the user VMs.
Advanced users (and advanced users only) can also try a manual upgrade
procedure that has been described :doc:`here </user/downloading-installing-upgrading/upgrade/2b2>`. It’s
advisable to backup your VMs before proceeding anyway!
Upgrading from Qubes R1 or R2 Beta 2 to R2 beta 3
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The easiest and safest way to upgrade to Qubes R2B3 is to install it
from scratch and use :doc:`qubes backup and restore tools </user/how-to-guides/how-to-back-up-restore-and-migrate>` for migrating of all of the user VMs.
Users can also try a manual upgrade procedure that has been described
:doc:`here </user/downloading-installing-upgrading/upgrade/2b3>`.
Note: if the user has custom Template VMs (i.e. other than the default
template, e.g. created from it by cloning), or Standalone VMs, then the
user should perform manual upgrade from R2B2 to R2B3, as described under
the link given above.

62
developer/releases/3_0/release-notes.md
View File

@ -1,62 +0,0 @@
---
lang: en
layout: doc
permalink: /doc/releases/3.0/release-notes/
redirect_from:
- /en/doc/releases/3.0/release-notes/
ref: 19
title: Qubes R3.0 release notes
---
### Qubes R3.0 Release Notes
This Qubes OS release is dedicated to the memory of Caspar Bowden.
## New features since 2.0
* HAL (Hypervisor Abstraction Layer) - based on libvirt, opens a whole new
possibilities of using different hypervisors. Currently Qubes OS uses Xen.
* Xen 4.4 - many new features, but for us the most important is much more
mature libxl toolstack.
* Qrexec 3 - greatly improved performance by using direct VM-VM connections and
bigger buffers.
* Debian templates gets official support.
* Whonix templates
* Build system improvements - especially support for distribution-specific
plugins (makes supporting multiple distributions much easier) and building
templates using DispVM.
* Automated tests - makes much easier to find bugs, before its even shipped to users
## Known issues
* Windows Tools: `qvm-block` does not work
* UEFI is not supported, you need to enable "legacy boot" in BIOS before installing Qubes OS
* Some icons in the Qubes Manager application might not be drawn correctly when using the Xfce4 environment in Dom0. If this bothers you, please use the KDE environment instead.
* If your GPU is not correctly supported by the Dom0 kernel (e.g. the 3D desktop effects do not run smoothly) then you might experience "heaviness" with Windows 7-based AppVMs. In that case, please solve the problem with your GPU support in Dom0 in the first place (by using a different kernel), or install Qubes OS on a different system.
* For other known issues take a look at [our tickets](https://github.com/QubesOS/qubes-issues/issues?q=is%3Aopen+is%3Aissue+milestone%3A%22Release+3.0%22+label%3Abug)
It is advised to install updates just after system installation to apply bug fixes for (some of) the above problems.
## Downloads
See [Qubes Downloads](/doc/QubesDownloads/).
## Installation instructions
See [Installation Guide](/doc/installation-guide/).
## Upgrading
### From R3.0 release candidate
If you are using Qubes R3.0rc1, R3.0rc2 or R3.0rc3, just install system updates, there is no special steps required.
### From R2.0 or earlier
The easiest and safest way to upgrade to Qubes R3.0 is to install it from scratch and use [qubes backup and restore tools](/doc/backup-restore/) for migrating of all of the user VMs.
Users of Qubes R2 can upgrade using [experimental procedure](/doc/upgrade-to-r3.0/).

97
developer/releases/3_0/release-notes.rst Normal file
View File

@ -0,0 +1,97 @@
========================
Qubes R3.0 release notes
========================
Qubes R3.0 Release Notes
------------------------
This Qubes OS release is dedicated to the memory of Caspar Bowden.
New features since 2.0
^^^^^^^^^^^^^^^^^^^^^^
- HAL (Hypervisor Abstraction Layer) - based on libvirt, opens a whole
new possibilities of using different hypervisors. Currently Qubes OS
uses Xen.
- Xen 4.4 - many new features, but for us the most important is much
more mature libxl toolstack.
- Qrexec 3 - greatly improved performance by using direct VM-VM
connections and bigger buffers.
- Debian templates gets official support.
- Whonix templates
- Build system improvements - especially support for
distribution-specific plugins (makes supporting multiple
distributions much easier) and building templates using DispVM.
- Automated tests - makes much easier to find bugs, before its even
shipped to users
Known issues
^^^^^^^^^^^^
- Windows Tools: ``qvm-block`` does not work
- UEFI is not supported, you need to enable “legacy boot” in BIOS
before installing Qubes OS
- Some icons in the Qubes Manager application might not be drawn
correctly when using the Xfce4 environment in Dom0. If this bothers
you, please use the KDE environment instead.
- If your GPU is not correctly supported by the Dom0 kernel (e.g. the
3D desktop effects do not run smoothly) then you might experience
“heaviness” with Windows 7-based AppVMs. In that case, please solve
the problem with your GPU support in Dom0 in the first place (by
using a different kernel), or install Qubes OS on a different system.
- For other known issues take a look at `our tickets <https://github.com/QubesOS/qubes-issues/issues?q=is%3Aopen+is%3Aissue+milestone%3A%22Release+3.0%22+label%3Abug>`__
It is advised to install updates just after system installation to apply
bug fixes for (some of) the above problems.
Downloads
^^^^^^^^^
See :doc:`Qubes Downloads </user/downloading-installing-upgrading/downloads>`.
Installation instructions
^^^^^^^^^^^^^^^^^^^^^^^^^
See :doc:`Installation Guide </user/downloading-installing-upgrading/installation-guide>`.
Upgrading
^^^^^^^^^
From R3.0 release candidate
---------------------------
If you are using Qubes R3.0rc1, R3.0rc2 or R3.0rc3, just install system
updates, there is no special steps required.
From R2.0 or earlier
--------------------
The easiest and safest way to upgrade to Qubes R3.0 is to install it
from scratch and use :doc:`qubes backup and restore tools </user/how-to-guides/how-to-back-up-restore-and-migrate>` for migrating of all of the user VMs.
Users of Qubes R2 can upgrade using :doc:`experimental procedure </user/downloading-installing-upgrading/upgrade/3_0>`.

15
developer/releases/3_0/schedule.md
View File

@ -1,15 +0,0 @@
---
lang: en
layout: doc
permalink: /doc/releases/3.0/schedule/
redirect_from:
- /en/doc/releases/3.0/schedule/
ref: 20
title: Qubes R3.0 release schedule
---
| Date | Stage |
| -----------:| ------------------------------------- |
| 5 Sep 2015 | current-testing freeze before 3.0-rc3 |
| 15 Sep 2015 | 3.0-rc3 release |
| 1 Oct 2015 | 3.0 release |

20
developer/releases/3_0/schedule.rst Normal file
View File

@ -0,0 +1,20 @@
===========================
Qubes R3.0 release schedule
===========================
.. list-table::
:widths: 11 11
:align: center
:header-rows: 1
* - Date
- Stage
* - 5 Sep 2015
- current-testing freeze before 3.0-rc3
* - 15 Sep 2015
- 3.0-rc3 release
* - 1 Oct 2015
- 3.0 release

65
developer/releases/3_1/release-notes.md
View File

@ -1,65 +0,0 @@
---
lang: en
layout: doc
permalink: /doc/releases/3.1/release-notes/
ref: 16
title: Qubes R3.1 release notes
---
## New features since 3.0
* Management Stack based of Salt Stack in dom0 - [documentation](/doc/salt/)
* Out of the box Whonix setup
* UEFI support
* LIVE edition (still alpha, not part of R3.1-rc1)
* Updated GPU drivers in dom0
* Colorful window application icons (instead of just colorful lock icon)
* PV Grub support ([documentation](/doc/managing-vm-kernels/))
* Out of the box USB VM setup, including [handling USB mouse](https://github.com/QubesOS/qubes-app-linux-input-proxy/blob/master/README.md)
* Xen upgraded to 4.6, for better hardware support (especially Skylake platform)
* Improve updates proxy flexibility - especially repositories served over HTTPS
You can get detailed description in [completed github issues](https://github.com/QubesOS/qubes-issues/issues?q=is%3Aissue+sort%3Aupdated-desc+milestone%3A%22Release+3.1%22+label%3Arelease-notes+is%3Aclosed)
## Known issues
* Installation image does not fit on DVD, requires either DVD DL, or USB stick (5GB or more)
* Windows Tools: `qvm-block` does not work
* Some icons in the Qubes Manager application might not be drawn correctly when using the Xfce4 environment in Dom0. If this bothers you, please use the KDE environment instead.
* USB mouse (in the case of USB VM) does not work at first system startup (just after completing firstboot). Workaround: restart the system.
* For other known issues take a look at [our tickets](https://github.com/QubesOS/qubes-issues/issues?q=is%3Aopen+is%3Aissue+milestone%3A%22Release+3.1%22+label%3Abug)
It is advised to install updates just after system installation to apply bug fixes for (some of) the above problems.
## Downloads
See [Qubes Downloads](/downloads/).
## Installation instructions
See [Installation Guide](/doc/installation-guide/).
## Upgrading
### From R3.0
The easiest and safest way to upgrade to Qubes R3.1 is to install it from
scratch and use [qubes backup and restore tools](/doc/backup-restore/) for
migrating of all of the user VMs.
Users of Qubes R3.0 can upgrade using [experimental
procedure](/doc/upgrade-to-r3.1/).
### From R2 or earlier
When upgrading from earlier versions the easiest and safest way is to install
it from scratch and use [qubes backup and restore tools](/doc/backup-restore/)
for migrating of all of the user VMs.
Alternatively you can [upgrade to R3.0
using](/doc/releases/3.0/release-notes/#upgrading) first, then follow the
instructions above. This will be time consuming process.

95
developer/releases/3_1/release-notes.rst Normal file
View File

@ -0,0 +1,95 @@
========================
Qubes R3.1 release notes
========================
New features since 3.0
----------------------
- Management Stack based of Salt Stack in dom0 -
:doc:`documentation </user/advanced-topics/salt>`
- Out of the box Whonix setup
- UEFI support
- LIVE edition (still alpha, not part of R3.1-rc1)
- Updated GPU drivers in dom0
- Colorful window application icons (instead of just colorful lock
icon)
- PV Grub support (:doc:`documentation </user/advanced-topics/managing-vm-kernels>`)
- Out of the box USB VM setup, including `handling USB mouse <https://github.com/QubesOS/qubes-app-linux-input-proxy/blob/master/README.md>`__
- Xen upgraded to 4.6, for better hardware support (especially Skylake
platform)
- Improve updates proxy flexibility - especially repositories served
over HTTPS
You can get detailed description in `completed github issues <https://github.com/QubesOS/qubes-issues/issues?q=is%3Aissue+sort%3Aupdated-desc+milestone%3A%22Release+3.1%22+label%3Arelease-notes+is%3Aclosed>`__
Known issues
------------
- Installation image does not fit on DVD, requires either DVD DL, or
USB stick (5GB or more)
- Windows Tools: ``qvm-block`` does not work
- Some icons in the Qubes Manager application might not be drawn
correctly when using the Xfce4 environment in Dom0. If this bothers
you, please use the KDE environment instead.
- USB mouse (in the case of USB VM) does not work at first system
startup (just after completing firstboot). Workaround: restart the
system.
- For other known issues take a look at `our tickets <https://github.com/QubesOS/qubes-issues/issues?q=is%3Aopen+is%3Aissue+milestone%3A%22Release+3.1%22+label%3Abug>`__
It is advised to install updates just after system installation to apply
bug fixes for (some of) the above problems.
Downloads
---------
See :doc:`Qubes Downloads </user/downloading-installing-upgrading/downloads>`.
Installation instructions
-------------------------
See :doc:`Installation Guide </user/downloading-installing-upgrading/installation-guide>`.
Upgrading
---------
From R3.0
^^^^^^^^^
The easiest and safest way to upgrade to Qubes R3.1 is to install it
from scratch and use :doc:`qubes backup and restore tools </user/how-to-guides/how-to-back-up-restore-and-migrate>` for migrating of all of the user VMs.
Users of Qubes R3.0 can upgrade using :doc:`experimental procedure </user/downloading-installing-upgrading/upgrade/3_1>`.
From R2 or earlier
^^^^^^^^^^^^^^^^^^
When upgrading from earlier versions the easiest and safest way is to
install it from scratch and use :doc:`qubes backup and restore tools </user/how-to-guides/how-to-back-up-restore-and-migrate>` for migrating of all of the user VMs.
Alternatively you can :ref:`upgrade to R3.0 using <developer/releases/3_0/release-notes:upgrading>` first, then follow
the instructions above. This will be time consuming process.

20
developer/releases/3_1/schedule.md
View File

@ -1,20 +0,0 @@
---
lang: en
layout: doc
permalink: /doc/releases/3.1/schedule/
redirect_from:
- /en/doc/releases/3.1/schedule/
ref: 17
title: Qubes R3.1 release schedule
---
This schedule is based on [Version Scheme](/doc/version-scheme/#release-schedule).
| Date | Stage |
| -----------:| --------------------------------------- |
| 8 Dec 2015 | 3.1-rc1 release |
| 5 Jan 2016 | current-testing freeze before 3.1-rc2 |
| 12 Jan 2016 | 3.1-rc2 release |
| 26 Jan 2016 | decide whether 3.1-rc2 is the final 3.1 |
| 9 Feb 2016 | current-testing freeze before 3.1-rc3 |
| ~~16 Feb 2016~~ <br/> 23 Feb 2016 | 3.1-rc3 release |

28
developer/releases/3_1/schedule.rst Normal file
View File

@ -0,0 +1,28 @@
===========================
Qubes R3.1 release schedule
===========================
This schedule is based on :ref:`Version Scheme <developer/releases/version-scheme:release schedule>`.
.. list-table::
:widths: 15 15
:align: center
:header-rows: 1
* - Date
- Stage
* - 8 Dec 2015
- 3.1-rc1 release
* - 5 Jan 2016
- current-testing freeze before 3.1-rc2
* - 12 Jan 2016
- 3.1-rc2 release
* - 26 Jan 2016
- decide whether 3.1-rc2 is the final 3.1
* - 9 Feb 2016
- current-testing freeze before 3.1-rc3
* - :strike:`16 Feb 2016` 23 Feb 2016
- 3.1-rc3 release

61
developer/releases/3_2/release-notes.md
View File

@ -1,61 +0,0 @@
---
lang: en
layout: doc
permalink: /doc/releases/3.2/release-notes/
ref: 21
title: Qubes R3.2 release notes
---
## New features since 3.1
* Management Stack extended to support in-VM configuration - [documentation](/doc/salt/)
* PV USB - [documentation](/doc/usb/)
* Dom0 update to Fedora 23 for better hardware support
* Kernel 4.4.x
* Default desktop environment switched to Xfce4
* KDE 5 support (but it is no longer the default one)
* Tiling window managers support: awesome, [i3](/doc/i3/)
* More flexible Qubes RPC services - [related ticket](https://github.com/QubesOS/qubes-issues/issues/1876), [documentation](/doc/qrexec/#service-policies-with-arguments)
You can get detailed description in [completed github issues](https://github.com/QubesOS/qubes-issues/issues?q=is%3Aissue+sort%3Aupdated-desc+milestone%3A%22Release+3.2%22+label%3Arelease-notes+is%3Aclosed)
## Known issues
* [Fedora 23 reached EOL in December 2016](https://fedoraproject.org/wiki/End_of_life). There is a [manual procedure to upgrade your VMs](/news/2018/01/06/fedora-26-upgrade/).
* Windows Tools: `qvm-block` does not work
* Some icons in the Qubes Manager application might not be drawn correctly when using the Xfce4 environment in Dom0. If this bothers you, please use the KDE environment instead.
* For other known issues take a look at [our tickets](https://github.com/QubesOS/qubes-issues/issues?q=is%3Aopen+is%3Aissue+milestone%3A%22Release+3.2%22+label%3Abug)
It is advised to install updates just after system installation to apply bug fixes for (some of) the above problems.
## Downloads
See [Qubes Downloads](/downloads/).
## Installation instructions
See [Installation Guide](/doc/installation-guide/).
After installation, [manually upgrade to Fedora 26](/news/2018/01/06/fedora-26-upgrade/).
## Upgrading
### From R3.1
The easiest and safest way to upgrade to Qubes R3.2 is to install it from
scratch and use [qubes backup and restore tools](/doc/backup-restore/) for
migrating of all of the user VMs.
Users of Qubes R3.1 can also upgrade using [this
procedure](/doc/upgrade-to-r3.2/).
### From R3.0 or earlier
When upgrading from earlier versions the easiest and safest way is to install
it from scratch and use [qubes backup and restore tools](/doc/backup-restore/)
for migrating of all of the user VMs.
Alternatively you can [upgrade to R3.1 using](/doc/releases/3.1/release-notes/#upgrading) first, then follow
the instructions above. This will be time consuming process.

86
developer/releases/3_2/release-notes.rst Normal file
View File

@ -0,0 +1,86 @@
========================
Qubes R3.2 release notes
========================
New features since 3.1
----------------------
- Management Stack extended to support in-VM configuration -
:doc:`documentation </user/advanced-topics/salt>`
- PV USB - :doc:`documentation </user/how-to-guides/how-to-use-usb-devices>`
- Dom0 update to Fedora 23 for better hardware support
- Kernel 4.4.x
- Default desktop environment switched to Xfce4
- KDE 5 support (but it is no longer the default one)
- Tiling window managers support: awesome, :doc:`i3 </user/advanced-topics/i3>`
- More flexible Qubes RPC services - `related ticket <https://github.com/QubesOS/qubes-issues/issues/1876>`__,
:ref:`documentation <developer/services/qrexec:service policies with arguments>`
You can get detailed description in `completed github issues <https://github.com/QubesOS/qubes-issues/issues?q=is%3Aissue+sort%3Aupdated-desc+milestone%3A%22Release+3.2%22+label%3Arelease-notes+is%3Aclosed>`__
Known issues
------------
- `Fedora 23 reached EOL in December 2016 <https://fedoraproject.org/wiki/End_of_life>`__. There is a
`manual procedure to upgrade your VMs <https://www.qubes-os.org/news/2018/01/06/fedora-26-upgrade/>`__.
- Windows Tools: ``qvm-block`` does not work
- Some icons in the Qubes Manager application might not be drawn
correctly when using the Xfce4 environment in Dom0. If this bothers
you, please use the KDE environment instead.
- For other known issues take a look at `our tickets <https://github.com/QubesOS/qubes-issues/issues?q=is%3Aopen+is%3Aissue+milestone%3A%22Release+3.2%22+label%3Abug>`__
It is advised to install updates just after system installation to apply
bug fixes for (some of) the above problems.
Downloads
---------
See :doc:`Qubes Downloads </user/downloading-installing-upgrading/downloads>`.
Installation instructions
-------------------------
See :doc:`Installation Guide </user/downloading-installing-upgrading/installation-guide>`. After
installation, `manually upgrade to Fedora 26 <https://www.qubes-os.org/news/2018/01/06/fedora-26-upgrade/>`__.
Upgrading
---------
From R3.1
^^^^^^^^^
The easiest and safest way to upgrade to Qubes R3.2 is to install it
from scratch and use :doc:`qubes backup and restore tools </user/how-to-guides/how-to-back-up-restore-and-migrate>` for migrating of all of the user VMs.
Users of Qubes R3.1 can also upgrade using :doc:`this procedure </user/downloading-installing-upgrading/upgrade/3_2>`.
From R3.0 or earlier
^^^^^^^^^^^^^^^^^^^^
When upgrading from earlier versions the easiest and safest way is to
install it from scratch and use :doc:`qubes backup and restore tools </user/how-to-guides/how-to-back-up-restore-and-migrate>` for migrating of all of the user VMs.
Alternatively you can :ref:`upgrade to R3.1 using <developer/releases/3_1/release-notes:upgrading>` first, then follow
the instructions above. This will be time consuming process.

22
developer/releases/3_2/schedule.md
View File

@ -1,22 +0,0 @@
---
lang: en
layout: doc
permalink: /doc/releases/3.2/schedule/
redirect_from:
- /en/doc/releases/3.2/schedule/
ref: 22
title: Qubes R3.2 release schedule
---
This schedule is based on [Version Scheme](/doc/version-scheme/#release-schedule).
| Date | Stage |
| -----------:| --------------------------------------- |
| 18 Jun 2016 | 3.2-rc1 release |
| 2 Jul 2016 | decide whether 3.2-rc1 is the final 3.2 |
| ~~16 Jul 2016~~ <br/> 20 Jul 2016 | current-testing freeze before 3.2-rc2 |
| ~~23 Jul 2016~~ <br/> 27 Jul 2016 | 3.2-rc2 release |
| ~~5 Aug 2016~~ <br/> 9 Aug 2016 | decide whether 3.2-rc2 is the final 3.2 |
| 24 Aug 2016 | current-testing freeze before 3.2-rc3 |
| 31 Aug 2016 | 3.2-rc3 release |
| 29 Sep 2016 | 3.2 release |

32
developer/releases/3_2/schedule.rst Normal file
View File

@ -0,0 +1,32 @@
===========================
Qubes R3.2 release schedule
===========================
This schedule is based on :ref:`Version Scheme <developer/releases/version-scheme:release schedule>`.
.. list-table::
:widths: 15 15
:align: center
:header-rows: 1
* - Date
- Stage
* - 18 Jun 2016
- 3.2-rc1 release
* - 2 Jul 2016
- decide whether 3.2-rc1 is the final 3.2
* - :strike:`16 Jul 2016` 20 Jul 2016
- current-testing freeze before 3.2-rc2
* - :strike:`23 Jul 2016` 27 Jul 2016
- 3.2-rc2 release
* - :strike:`5 Aug 2016` 9 Aug 2016
- decide whether 3.2-rc2 is the final 3.2
* - 24 Aug 2016
- current-testing freeze before 3.2-rc3
* - 31 Aug 2016
- 3.2-rc3 release
* - 29 Sep 2016
- 3.2 release

107
developer/releases/4_0/release-notes.md
View File

@ -1,107 +0,0 @@
---
lang: en
layout: doc
permalink: /doc/releases/4.0/release-notes/
ref: 23
title: Qubes R4.0 release notes
---
New features since 3.2
----------------------
* Core management scripts rewrite with better structure and extensibility, [API documentation](https://dev.qubes-os.org/projects/qubes-core-admin/en/latest/)
* [Admin API](/news/2017/06/27/qubes-admin-api/) allowing strictly controlled managing from non-dom0
* All `qvm-*` command-line tools rewritten, some options have changed
* Renaming VM directly is prohibited, there is GUI to clone under new name and remove old VM
* Use [PVH](https://github.com/QubesOS/qubes-secpack/blob/master/QSBs/qsb-037-2018.txt) and [HVM](https://github.com/QubesOS/qubes-issues/issues/2185) by default to [mitigate Meltdown & Spectre](https://github.com/QubesOS/qubes-secpack/blob/master/QSBs/qsb-037-2018.txt) and lower the [attack surface on Xen](https://github.com/QubesOS/qubes-secpack/blob/master/QSBs/qsb-024-2016.txt)
* Create USB VM by default
* [Multiple DisposableVMs templates support](https://github.com/QubesOS/qubes-issues/issues/2253)
* New [backup format](/doc/backup-emergency-restore-v4/) using scrypt key-derivation function
* Non-encrypted backups no longer supported
* [split VM packages](https://github.com/QubesOS/qubes-issues/issues/2771), for better support minimal, specialized templates
* [Qubes Manager decomposition](https://github.com/QubesOS/qubes-issues/issues/2132) - domains and devices widgets instead of full Qubes Manager; devices widget support also USB
* [More flexible firewall interface](/doc/vm-interface/) for ease unikernel integration
* Template VMs do not have network interface by default, [qrexec-based updates proxy](https://github.com/QubesOS/qubes-issues/issues/1854) is used instead
* More flexible IP addressing for VMs - [custom IP](https://github.com/QubesOS/qubes-issues/issues/1477), [hidden from the IP](https://github.com/QubesOS/qubes-issues/issues/1143)
* More flexible Qubes RPC policy - [related ticket](https://github.com/QubesOS/qubes-issues/issues/865), [documentation](/doc/qrexec/#specifying-vms-tags-types-targets-etc)
* [New Qubes RPC confirmation window](https://github.com/QubesOS/qubes-issues/issues/910), including option to specify destination VM
* [New storage subsystem design](https://github.com/QubesOS/qubes-issues/issues/1842)
* Dom0 update to Fedora 25 for better hardware support
* Kernel 4.9.x
You can get detailed description in [completed github issues](https://github.com/QubesOS/qubes-issues/issues?q=is%3Aissue+sort%3Aupdated-desc+milestone%3A%22Release+4.0%22+label%3Arelease-notes+is%3Aclosed)
Security Notes
--------------
* PV VMs migrated from 3.2 to 4.0-rc4 or later are automatically set to PVH mode in order to protect against Meltdown (see [QSB #37](https://github.com/QubesOS/qubes-secpack/blob/master/QSBs/qsb-037-2018.txt)).
However, PV VMs migrated from any earlier 4.0 release candidate (RC1, RC2, or RC3) are not automatically set to PVH mode.
These must be set manually.
* The following steps may need to be applied in dom0 and Fedora 26 TemplateVMs in order to receive updates (see [#3737](https://github.com/QubesOS/qubes-issues/issues/3737)).
Steps for dom0 updates:
1. Open the Qubes Menu by clicking on the "Q" icon in the top-left corner of the screen.
2. Select `Terminal Emulator`.
3. In the window that opens, enter this command:
```
sudo nano /etc/yum.repos.d/qubes-dom0.repo
```
4. This opens the nano text editor. Change all four instances of `http` to `https`.
5. Press `CTRL+X`, then `Y`, then `ENTER` to save changes and exit.
6. Check for updates normally.
Steps for Fedora 26 TemplateVM updates:
1. Open the Qubes Menu by clicking on the "Q" icon in the top-left corner of the screen.
2. Select `Template: fedora-26`, then `fedora-26: Terminal`.
3. In the window that opens, enter the command for your version:
```
[Qubes 3.2] sudo gedit /etc/yum.repos.d/qubes-r3.repo
[Qubes 4.0] sudo gedit /etc/yum.repos.d/qubes-r4.repo
```
4. This opens the gedit text editor in a window. Change all four instances of `http` to `https`.
5. Click the "Save" button in the top-right corner of the window.
6. Close the window.
7. Check for updates normally.
8. Shut down the TemplateVM.
Known issues
------------
* Locale using coma as decimal separator [crashes qubesd](https://github.com/QubesOS/qubes-issues/issues/3753). Either install with different locale (English (United States) for example), or manually apply fix explained in that issue.
* In the middle of installation, [keyboard layout reset to US](https://github.com/QubesOS/qubes-issues/issues/3352). Be careful what is the current layout while setting default user password (see upper right screen corner).
* On some laptops (for example Librem 15v2), touchpad do not work directly after installation. Reboot the system to fix the issue.
* List of USB devices may contain device identifiers instead of name
* With R4.0.1, which ships kernel-4.19, you may never reach the anaconda startup and be block on an idle black screen with blinking cursor. You can try to add `plymouth.ignore-serial-consoles` in the grub installer boot menu right after `quiet rhgb`. With legacy mode, you can do it directly when booting the DVD or USB key. In UEFI mode, follow the same procedure described for [disabling](/doc/uefi-troubleshooting/#installation-freezes-before-displaying-installer) `nouveau` module (related [solved issue](https://github.com/QubesOS/qubes-issues/issues/3849) in further version of Qubes).
* For other known issues take a look at [our tickets](https://github.com/QubesOS/qubes-issues/issues?q=is%3Aopen+is%3Aissue+milestone%3A%22Release+4.0%22+label%3Abug)
It is advised to install updates just after system installation to apply bug fixes for (some of) the above problems.
Downloads
---------
See [Qubes Downloads](/downloads/).
Installation instructions
-------------------------
See [Installation Guide](/doc/installation-guide/).
Upgrading
---------
There is no in-place upgrade path from earlier Qubes versions. The only
supported option to upgrade to Qubes R4.0 is to install it from scratch and use
[qubes backup and restore tools](/doc/backup-restore/) for migrating of all of the user VMs.
We also provide [detailed instruction](/doc/upgrade-to-r4.0/) for this procedure.

188
developer/releases/4_0/release-notes.rst Normal file
View File

@ -0,0 +1,188 @@
========================
Qubes R4.0 release notes
========================
New features since 3.2
----------------------
- Core management scripts rewrite with better structure and
extensibility, `API documentation <https://dev.qubes-os.org/projects/qubes-core-admin/en/latest/>`__
- `Admin API <https://www.qubes-os.org/news/2017/06/27/qubes-admin-api/>`__ allowing strictly
controlled managing from non-dom0
- All ``qvm-*`` command-line tools rewritten, some options have changed
- Renaming VM directly is prohibited, there is GUI to clone under new
name and remove old VM
- Use
`PVH <https://github.com/QubesOS/qubes-secpack/blob/master/QSBs/qsb-037-2018.txt>`__
and `HVM <https://github.com/QubesOS/qubes-issues/issues/2185>`__ by
default to `mitigate Meltdown & Spectre <https://github.com/QubesOS/qubes-secpack/blob/master/QSBs/qsb-037-2018.txt>`__
and lower the `attack surface on Xen <https://github.com/QubesOS/qubes-secpack/blob/master/QSBs/qsb-024-2016.txt>`__
- Create USB VM by default
- `Multiple DisposableVMs templates support <https://github.com/QubesOS/qubes-issues/issues/2253>`__
- New :doc:`backup format </user/how-to-guides/backup-emergency-restore-v4>` using
scrypt key-derivation function
- Non-encrypted backups no longer supported
- `split VM packages <https://github.com/QubesOS/qubes-issues/issues/2771>`__,
for better support minimal, specialized templates
- `Qubes Manager decomposition <https://github.com/QubesOS/qubes-issues/issues/2132>`__
- domains and devices widgets instead of full Qubes Manager; devices
widget support also USB
- :doc:`More flexible firewall interface </developer/debugging/vm-interface>` for ease
unikernel integration
- Template VMs do not have network interface by default, `qrexec-based updates proxy <https://github.com/QubesOS/qubes-issues/issues/1854>`__ is
used instead
- More flexible IP addressing for VMs - `custom IP <https://github.com/QubesOS/qubes-issues/issues/1477>`__, `hidden from the IP <https://github.com/QubesOS/qubes-issues/issues/1143>`__
- More flexible Qubes RPC policy - `related ticket <https://github.com/QubesOS/qubes-issues/issues/865>`__,
:ref:`documentation <developer/services/qrexec:specifying vms: tags, types, targets, etc.>`
- `New Qubes RPC confirmation window <https://github.com/QubesOS/qubes-issues/issues/910>`__,
including option to specify destination VM
- `New storage subsystem design <https://github.com/QubesOS/qubes-issues/issues/1842>`__
- Dom0 update to Fedora 25 for better hardware support
- Kernel 4.9.x
You can get detailed description in `completed github issues <https://github.com/QubesOS/qubes-issues/issues?q=is%3Aissue+sort%3Aupdated-desc+milestone%3A%22Release+4.0%22+label%3Arelease-notes+is%3Aclosed>`__
Security Notes
--------------
- PV VMs migrated from 3.2 to 4.0-rc4 or later are automatically set to
PVH mode in order to protect against Meltdown (see `QSB #37 <https://github.com/QubesOS/qubes-secpack/blob/master/QSBs/qsb-037-2018.txt>`__).
However, PV VMs migrated from any earlier 4.0 release candidate (RC1,
RC2, or RC3) are not automatically set to PVH mode. These must be set
manually.
- The following steps may need to be applied in dom0 and Fedora 26
TemplateVMs in order to receive updates (see
`#3737 <https://github.com/QubesOS/qubes-issues/issues/3737>`__).
Steps for dom0 updates:
1. Open the Qubes Menu by clicking on the “Q” icon in the top-left
corner of the screen.
2. Select ``Terminal Emulator``.
3. In the window that opens, enter this command:
.. code:: bash
sudo nano /etc/yum.repos.d/qubes-dom0.repo
4. This opens the nano text editor. Change all four instances of
``http`` to ``https``.
5. Press ``CTRL+X``, then ``Y``, then ``ENTER`` to save changes and
exit.
6. Check for updates normally.
Steps for Fedora 26 TemplateVM updates:
1. Open the Qubes Menu by clicking on the “Q” icon in the top-left
corner of the screen.
2. Select ``Template: fedora-26``, then ``fedora-26: Terminal``.
3. In the window that opens, enter the command for your version:
.. code:: bash
[Qubes 3.2] sudo gedit /etc/yum.repos.d/qubes-r3.repo
[Qubes 4.0] sudo gedit /etc/yum.repos.d/qubes-r4.repo
4. This opens the gedit text editor in a window. Change all four
instances of ``http`` to ``https``.
5. Click the “Save” button in the top-right corner of the window.
6. Close the window.
7. Check for updates normally.
8. Shut down the TemplateVM.
Known issues
------------
- Locale using coma as decimal separator `crashes qubesd <https://github.com/QubesOS/qubes-issues/issues/3753>`__.
Either install with different locale (English (United States) for
example), or manually apply fix explained in that issue.
- In the middle of installation, `keyboard layout reset to US <https://github.com/QubesOS/qubes-issues/issues/3352>`__. Be
careful what is the current layout while setting default user
password (see upper right screen corner).
- On some laptops (for example Librem 15v2), touchpad do not work
directly after installation. Reboot the system to fix the issue.
- List of USB devices may contain device identifiers instead of name
- With R4.0.1, which ships kernel-4.19, you may never reach the
anaconda startup and be block on an idle black screen with blinking
cursor. You can try to add ``plymouth.ignore-serial-consoles`` in the
grub installer boot menu right after ``quiet rhgb``. With legacy
mode, you can do it directly when booting the DVD or USB key. In UEFI
mode, follow the same procedure described for
:ref:`disabling <user/troubleshooting/uefi-troubleshooting:installation freezes before displaying installer>`
``nouveau`` module (related `solved issue <https://github.com/QubesOS/qubes-issues/issues/3849>`__ in
further version of Qubes).
- For other known issues take a look at `our tickets <https://github.com/QubesOS/qubes-issues/issues?q=is%3Aopen+is%3Aissue+milestone%3A%22Release+4.0%22+label%3Abug>`__
It is advised to install updates just after system installation to apply
bug fixes for (some of) the above problems.
Downloads
---------
See :doc:`Qubes Downloads </user/downloading-installing-upgrading/downloads>`.
Installation instructions
-------------------------
See :doc:`Installation Guide </user/downloading-installing-upgrading/installation-guide>`.
Upgrading
---------
There is no in-place upgrade path from earlier Qubes versions. The only
supported option to upgrade to Qubes R4.0 is to install it from scratch
and use :doc:`qubes backup and restore tools </user/how-to-guides/how-to-back-up-restore-and-migrate>` for
migrating of all of the user VMs. We also provide :doc:`detailed instruction </user/downloading-installing-upgrading/upgrade/4_0>` for this procedure.

28
developer/releases/4_0/schedule.md
View File

@ -1,28 +0,0 @@
---
lang: en
layout: doc
permalink: /doc/releases/4.0/schedule/
redirect_from:
- /en/doc/releases/4.0/schedule/
ref: 24
title: Qubes R4.0 release schedule
---
This schedule is based on [Version Scheme](/doc/version-scheme/#release-schedule).
| Date | Stage |
| -----------:| --------------------------------------- |
| 31 Jul 2017 | 4.0-rc1 release |
| ~~28 Aug 2017~~ <br/>~~11 Sep 2017~~ <br/>~~9 Oct 2017~~ <br/>18 Oct 2017 | current-testing freeze before 4.0-rc2 |
| ~~4 Sep 2017~~ <br/> ~~18 Sep 2017~~ <br/>~~16 Oct 2017~~ <br/>23 Oct 2017 | 4.0-rc2 release |
| 6 Nov 2017 | decide whether 4.0-rc2 is the final 4
| 20 Nov 2017 | current-testing freeze before 4.0-rc3 |
| 27 Nov 2017 | 4.0-rc3 release |
| 11 Dec 2017 | decide whether 4.0-rc3 is the final 4.0 |
| 1 Jan 2018 | current-testing freeze before 4.0-rc4 |
| ~~8 Jan 2018~~ <br/>31 Jan 2018 | 4.0-rc4 release |
| ~~22 Jan 2018~~ <br/>14 Feb 2018 | decide whether 4.0-rc4 is the final 4.0 |
| 27 Feb 2018 | current-testing freeze before 4.0-rc5 |
| 6 Mar 2018 | 4.0-rc5 release |
| 20 Mar 2018 | decide whether 4.0-rc5 is the final 4.0 |
| 28 Mar 2018 | final 4.0 release |

44
developer/releases/4_0/schedule.rst Normal file
View File

@ -0,0 +1,44 @@
===========================
Qubes R4.0 release schedule
===========================
This schedule is based on :ref:`Version Scheme <developer/releases/version-scheme:release schedule>`.
.. list-table::
:widths: 15 15
:align: center
:header-rows: 1
* - Date
- Stage
* - 31 Jul 2017
- 4.0-rc1 release
* - :strike:`28 Aug 2017` :strike:`11 Sep 2017` :strike:`9 Oct 2017` 18 Oct 2017
- current-testing freeze before 4.0-rc2
* - :strike:`4 Sep 2017` :strike:`18 Sep 2017` :strike:`16 Oct 2017` 23 Oct 2017
- 4.0-rc2 release
* - 6 Nov 2017
- decide whether 4.0-rc2 is the final 4
* - 20 Nov 2017
- current-testing freeze before 4.0-rc3
* - 27 Nov 2017
- 4.0-rc3 release
* - 11 Dec 2017
- decide whether 4.0-rc3 is the final 4.0
* - 1 Jan 2018
- current-testing freeze before 4.0-rc4
* - :strike:`8 Jan 2018` 31 Jan 2018
- 4.0-rc4 release
* - :strike:`22 Jan 2018` 14 Feb 2018
- decide whether 4.0-rc4 is the final 4.0
* - 27 Feb 2018
- current-testing freeze before 4.0-rc5
* - 6 Mar 2018
- 4.0-rc5 release
* - 20 Mar 2018
- decide whether 4.0-rc5 is the final 4.0
* - 28 Mar 2018
- final 4.0 release

93
developer/releases/4_1/release-notes.md
View File

@ -1,93 +0,0 @@
---
layout: doc
title: Qubes OS 4.1 release notes
permalink: /doc/releases/4.1/release-notes/
---
## New features and improvements since Qubes 4.0
- Optional qubes-remote-support package now available from repositories
(strictly opt-in, no package installed by default; no new ports or network
connections open by default; requires explicit connection initiation by the
user, then requires sharing a code word with the remote party before a
connection can be established; see
[#6364](https://github.com/QubesOS/qubes-issues/issues/6364) for more
information)
- Qubes firewall reworked to be more defensive (see
[#5540](https://github.com/QubesOS/qubes-issues/issues/5540) for details)
- Xen upgraded to version 4.14
- Dom0 operating system upgraded to Fedora 32
- Default desktop environment upgraded to Xfce 4.14
- Upgraded default template releases
- Experimental support for GUI running outside of dom0 (hybrid mode GUI domain
without real GPU passthrough; see
[#5662](https://github.com/QubesOS/qubes-issues/issues/5662) for details)
- Experimental support for audio server running outside of dom0 ("Audio domain")
- sys-firewall and sys-usb are now disposables by default
- UEFI boot now loads GRUB, which in turn loads Xen, making the boot path
similar to legacy boot and allowing the user to modify boot parameters or
choose an alternate boot menu entry
- New qrexec policy format (see
[#4370](https://github.com/QubesOS/qubes-issues/issues/4370) for details)
- qrexec protocol improvements (see
[#4909](https://github.com/QubesOS/qubes-issues/issues/4909) for details)
- New qrexec-policy daemon
- Simplified using in-qube kernels
- Windows USB and audio support courtesy of
[tabit-pro](https://github.com/tabit-pro) (see
[#5802](https://github.com/QubesOS/qubes-issues/issues/5802) and
[#2624](https://github.com/QubesOS/qubes-issues/issues/2624))
- Clarified disposable-related terminology and properties
- Default kernelopts can now be specified by a kernel package
- Improved support for high-resolution displays
- Improved notifications when a system drive runs out of free space
- Support for different cursor shapes
- "Paranoid mode" backup restore option now properly supported using
disposables
- Users can now choose between Debian and Fedora in the installer
- Certain files and applications are now opened in disposables, e.g.,
Thunderbird email attachments
- New graphical interface for managing testing repository updates
- New "Cute Qube" icon family (replaces padlock icons)
- Disposable qube types now use the disposable icon
- New Template Manager tool for installing, removing, and updating templates
(meanwhile, the tool previously known as the "Template Manager," which was
for mass template switching, has been integrated into the Qube Manager)
- The "file" storage driver has been deprecated in Qubes 4.1 and will be
removed in Qubes 4.2
- `property-del` event renamed to `property-reset` to avoid confusion
- qrexec no longer supports non-executable files in `/etc/qubes-rpc`
- qrexec components have been reorganized into the core-qrexec repository
- The `qvm-pool` argument parser has been rewritten and improved
- Removed the need for the out-of-tree u2mfn kernel module
- Qrexec services can now run as a socket server
- Improved template distribution mechanism
- Now possible to restart qrexec-agent
- The term "VM" has largely been replaced by "qube"
- GUI daemon is now configured using `qvm-features` tool,
`/etc/qubes/guid.conf` file is no longer used
- `qvm-run` tool got `--no-shell` option to run a single command without using
a shell inside the qube
- MAC Randomization for iwlwifi (see [#938](https://github.com/QubesOS/qubes-issues/issues/938))
For a full list, including more detailed descriptions, please see
[here](https://github.com/QubesOS/qubes-issues/issues?q=is%3Aissue+sort%3Aupdated-desc+milestone%3A%22Release+4.1%22+label%3A%22release+notes%22+is%3Aclosed).
## Known issues
For a full list of known 4.1 issues with open bug reports, please see
[here](https://github.com/QubesOS/qubes-issues/issues?q=is%3Aopen+is%3Aissue+milestone%3A%22Release+4.1%22+label%3A%22T%3A+bug%22).
We strongly recommend [updating Qubes OS](/doc/how-to-update/) immediately
after installation in order to apply any and all available bug fixes.
## Download
See [downloads](/downloads/).
## Installation instructions
See the [installation guide](/doc/installation-guide/).
## Upgrading
Please see [how to upgrade to Qubes 4.1](/doc/upgrade/4.1/).

153
developer/releases/4_1/release-notes.rst Normal file
View File

@ -0,0 +1,153 @@
==========================
Qubes OS 4.1 release notes
==========================
New features and improvements since Qubes 4.0
---------------------------------------------
- Optional qubes-remote-support package now available from repositories
(strictly opt-in, no package installed by default; no new ports or
network connections open by default; requires explicit connection
initiation by the user, then requires sharing a code word with the
remote party before a connection can be established; see
`#6364 <https://github.com/QubesOS/qubes-issues/issues/6364>`__ for
more information)
- Qubes firewall reworked to be more defensive (see
`#5540 <https://github.com/QubesOS/qubes-issues/issues/5540>`__ for
details)
- Xen upgraded to version 4.14
- Dom0 operating system upgraded to Fedora 32
- Default desktop environment upgraded to Xfce 4.14
- Upgraded default template releases
- Experimental support for GUI running outside of dom0 (hybrid mode GUI
domain without real GPU passthrough; see
`#5662 <https://github.com/QubesOS/qubes-issues/issues/5662>`__ for
details)
- Experimental support for audio server running outside of dom0 (“Audio
domain”)
- sys-firewall and sys-usb are now disposables by default
- UEFI boot now loads GRUB, which in turn loads Xen, making the boot
path similar to legacy boot and allowing the user to modify boot
parameters or choose an alternate boot menu entry
- New qrexec policy format (see
`#4370 <https://github.com/QubesOS/qubes-issues/issues/4370>`__ for
details)
- qrexec protocol improvements (see
`#4909 <https://github.com/QubesOS/qubes-issues/issues/4909>`__ for
details)
- New qrexec-policy daemon
- Simplified using in-qube kernels
- Windows USB and audio support courtesy of
`tabit-pro <https://github.com/tabit-pro>`__ (see
`#5802 <https://github.com/QubesOS/qubes-issues/issues/5802>`__ and
`#2624 <https://github.com/QubesOS/qubes-issues/issues/2624>`__)
- Clarified disposable-related terminology and properties
- Default kernelopts can now be specified by a kernel package
- Improved support for high-resolution displays
- Improved notifications when a system drive runs out of free space
- Support for different cursor shapes
- “Paranoid mode” backup restore option now properly supported using
disposables
- Users can now choose between Debian and Fedora in the installer
- Certain files and applications are now opened in disposables, e.g.,
Thunderbird email attachments
- New graphical interface for managing testing repository updates
- New “Cute Qube” icon family (replaces padlock icons)
- Disposable qube types now use the disposable icon
- New Template Manager tool for installing, removing, and updating
templates (meanwhile, the tool previously known as the “Template
Manager,” which was for mass template switching, has been integrated
into the Qube Manager)
- The “file” storage driver has been deprecated in Qubes 4.1 and will
be removed in Qubes 4.2
- ``property-del`` event renamed to ``property-reset`` to avoid
confusion
- qrexec no longer supports non-executable files in ``/etc/qubes-rpc``
- qrexec components have been reorganized into the core-qrexec
repository
- The ``qvm-pool`` argument parser has been rewritten and improved
- Removed the need for the out-of-tree u2mfn kernel module
- Qrexec services can now run as a socket server
- Improved template distribution mechanism
- Now possible to restart qrexec-agent
- The term “VM” has largely been replaced by “qube”
- GUI daemon is now configured using ``qvm-features`` tool,
``/etc/qubes/guid.conf`` file is no longer used
- ``qvm-run`` tool got ``--no-shell`` option to run a single command
without using a shell inside the qube
- MAC Randomization for iwlwifi (see
`#938 <https://github.com/QubesOS/qubes-issues/issues/938>`__)
For a full list, including more detailed descriptions, please see
`here <https://github.com/QubesOS/qubes-issues/issues?q=is%3Aissue+sort%3Aupdated-desc+milestone%3A%22Release+4.1%22+label%3A%22release+notes%22+is%3Aclosed>`__.
Known issues
------------
For a full list of known 4.1 issues with open bug reports, please see
`here <https://github.com/QubesOS/qubes-issues/issues?q=is%3Aopen+is%3Aissue+milestone%3A%22Release+4.1%22+label%3A%22T%3A+bug%22>`__.
We strongly recommend :doc:`updating Qubes OS </user/how-to-guides/how-to-update>`
immediately after installation in order to apply any and all available
bug fixes.
Download
--------
See :doc:`downloads </user/downloading-installing-upgrading/downloads>`.
Installation instructions
-------------------------
See the :doc:`installation guide </user/downloading-installing-upgrading/installation-guide>`.
Upgrading
---------
Please see :doc:`how to upgrade to Qubes 4.1 </user/downloading-installing-upgrading/upgrade/4_1>`.

25
developer/releases/4_1/schedule.md
View File

@ -1,25 +0,0 @@
---
lang: en
layout: doc
permalink: /doc/releases/4.1/schedule/
redirect_from:
- /en/doc/releases/4.1/schedule/
title: Qubes R4.1 release schedule
---
The table below is based on our [release schedule
policy](/doc/version-scheme/#release-schedule).
| Date | Stage |
| ----------:| ----------------------------------------- |
| 2021-10-11 | 4.1.0-rc1 release |
| 2021-11-08 | current-testing freeze before 4.1.0-rc2 |
| 2021-11-15 | 4.1.0-rc2 release |
| 2021-11-29 | decide whether 4.1.0-rc2 is the final 4.1 |
| 2021-12-13 | current-testing freeze before 4.1.0-rc3 |
| 2021-12-20 | 4.1.0-rc3 release |
| 2022-01-03 | decide whether 4.1.0-rc3 is the final 4.1 |
| 2022-01-11 | current-testing freeze before 4.1.0-rc4 |
| 2022-01-18 | 4.1.0-rc4 release |
| 2022-01-31 | decide whether 4.1.0-rc4 is the final 4.1 |
| 2022-02-04 | final 4.1.0 release |

38
developer/releases/4_1/schedule.rst Normal file
View File

@ -0,0 +1,38 @@
===========================
Qubes R4.1 release schedule
===========================
The table below is based on our :ref:`release schedule policy <developer/releases/version-scheme:release schedule>`.
.. list-table::
:widths: 10 10
:align: center
:header-rows: 1
* - Date
- Stage
* - 2021-10-11
- 4.1.0-rc1 release
* - 2021-11-08
- current-testing freeze before 4.1.0-rc2
* - 2021-11-15
- 4.1.0-rc2 release
* - 2021-11-29
- decide whether 4.1.0-rc2 is the final 4.1
* - 2021-12-13
- current-testing freeze before 4.1.0-rc3
* - 2021-12-20
- 4.1.0-rc3 release
* - 2022-01-03
- decide whether 4.1.0-rc3 is the final 4.1
* - 2022-01-11
- current-testing freeze before 4.1.0-rc4
* - 2022-01-18
- 4.1.0-rc4 release
* - 2022-01-31
- decide whether 4.1.0-rc4 is the final 4.1
* - 2022-02-04
- final 4.1.0 release

67
developer/releases/4_2/release-notes.md
View File

@ -1,67 +0,0 @@
---
layout: doc
title: Qubes OS 4.2 release notes
permalink: /doc/releases/4.2/release-notes/
---
## New features and improvements since Qubes 4.1
- Dom0 upgraded to Fedora 37 ([#6982](https://github.com/QubesOS/qubes-issues/issues/6982))
- Xen upgraded to version 4.17
- Default Debian template upgraded to Debian 12
- Default Fedora and Debian templates use Xfce instead of GNOME ([#7784](https://github.com/QubesOS/qubes-issues/issues/7784))
- SELinux support in Fedora templates ([#4239](https://github.com/QubesOS/qubes-issues/issues/4239))
- Several GUI applications rewritten (screenshots below), including:
- Applications Menu (also available as preview in R4.1) ([#6665](https://github.com/QubesOS/qubes-issues/issues/6665)), ([#5677](https://github.com/QubesOS/qubes-issues/issues/5677))
- Qubes Global Settings ([#6898](https://github.com/QubesOS/qubes-issues/issues/6898))
- Create New Qube
- Qubes Update ([#7443](https://github.com/QubesOS/qubes-issues/issues/7443))
- Unified `grub.cfg` location for both UEFI and legacy boot ([#7985](https://github.com/QubesOS/qubes-issues/issues/7985))
- PipeWire support ([#6358](https://github.com/QubesOS/qubes-issues/issues/6358))
- fwupd integration for firmware updates ([#4855](https://github.com/QubesOS/qubes-issues/issues/4855))
- Optional automatic clipboard clearing ([#3415](https://github.com/QubesOS/qubes-issues/issues/3415))
- Official packages built using Qubes Builder v2 ([#6486](https://github.com/QubesOS/qubes-issues/issues/6486))
- Split GPG management in Qubes Global Settings
- Qrexec services use new qrexec policy format by default (but old format is still supported) ([#8000](https://github.com/QubesOS/qubes-issues/issues/8000))
- Improved keyboard layout switching
For a full list, including more detailed descriptions, please see [here](https://github.com/QubesOS/qubes-issues/issues?q=is%3Aissue+sort%3Aupdated-desc+milestone%3A%22Release+4.2%22+label%3A%22release+notes%22+is%3Aclosed). Below are some screenshots of the new and improved Qubes GUI tools.
The new Qubes OS Update tool:
[![Screenshot of the Qubes OS Update tool](/attachment/site/4-2_update.png)](/attachment/site/4-2_update.png)
The new Qubes OS Global Config tool:
[![Screenshot of the Qubes OS Global Config tool](/attachment/site/4-2_global-config_1.png)](/attachment/site/4-2_global-config_1.png)
[![Screenshot of the Qubes OS Global Config tool](/attachment/site/4-2_global-config_2.png)](/attachment/site/4-2_global-config_2.png)
The new Qubes OS Policy Editor tool:
[![Screenshot of the Qubes OS Policy Editor tool](/attachment/site/4-2_policy-editor.png)](/attachment/site/4-2_policy-editor.png)
## Known issues
- DomU firewalls have completely switched to nftables. Users should add their custom rules to the `custom-input` and `custom-forward` chains. (For more information, see issues [#5031](https://github.com/QubesOS/qubes-issues/issues/5031) and [#6062](https://github.com/QubesOS/qubes-issues/issues/6062).)
- Templates restored in 4.2 from a pre-4.2 backup continue to target their original Qubes OS release repos. If you are using fresh templates on a clean 4.2 installation, or if you performed an [in-place upgrade from 4.1 to 4.2](/doc/upgrade/4.2/#in-place-upgrade), then this does not affect you. (For more information, see issue [#8701](https://github.com/QubesOS/qubes-issues/issues/8701).)
Also see the [full list of open bug reports affecting Qubes 4.2](https://github.com/QubesOS/qubes-issues/issues?q=is%3Aissue+label%3Aaffects-4.2+label%3A%22T%3A+bug%22+is%3Aopen).
We strongly recommend [updating Qubes OS](/doc/how-to-update/) immediately after installation in order to apply all available bug fixes.
## Notes
- Qubes 4.2 does not support Debian 11 templates (see [supported template releases](/doc/supported-releases/#templates)). Please [upgrade your Debian templates](/doc/templates/debian/#upgrading) to Debian 12.
## Download
All Qubes ISOs and associated [verification files](/security/verifying-signatures/) are available on the [downloads](/downloads/) page.
## Installation instructions
See the [installation guide](/doc/installation-guide/).
## Upgrading
Please see [how to upgrade to Qubes 4.2](/doc/upgrade/4.2/).

144
developer/releases/4_2/release-notes.rst Normal file
View File

@ -0,0 +1,144 @@
==========================
Qubes OS 4.2 release notes
==========================
New features and improvements since Qubes 4.1
---------------------------------------------
- Dom0 upgraded to Fedora 37
(`#6982 <https://github.com/QubesOS/qubes-issues/issues/6982>`__)
- Xen upgraded to version 4.17
- Default Debian template upgraded to Debian 12
- Default Fedora and Debian templates use Xfce instead of GNOME
(`#7784 <https://github.com/QubesOS/qubes-issues/issues/7784>`__)
- SELinux support in Fedora templates
(`#4239 <https://github.com/QubesOS/qubes-issues/issues/4239>`__)
- Several GUI applications rewritten (screenshots below), including:
- Applications Menu (also available as preview in R4.1)
(`#6665 <https://github.com/QubesOS/qubes-issues/issues/6665>`__),
(`#5677 <https://github.com/QubesOS/qubes-issues/issues/5677>`__)
- Qubes Global Settings
(`#6898 <https://github.com/QubesOS/qubes-issues/issues/6898>`__)
- Create New Qube
- Qubes Update
(`#7443 <https://github.com/QubesOS/qubes-issues/issues/7443>`__)
- Unified ``grub.cfg`` location for both UEFI and legacy boot
(`#7985 <https://github.com/QubesOS/qubes-issues/issues/7985>`__)
- PipeWire support
(`#6358 <https://github.com/QubesOS/qubes-issues/issues/6358>`__)
- fwupd integration for firmware updates
(`#4855 <https://github.com/QubesOS/qubes-issues/issues/4855>`__)
- Optional automatic clipboard clearing
(`#3415 <https://github.com/QubesOS/qubes-issues/issues/3415>`__)
- Official packages built using Qubes Builder v2
(`#6486 <https://github.com/QubesOS/qubes-issues/issues/6486>`__)
- Split GPG management in Qubes Global Settings
- Qrexec services use new qrexec policy format by default (but old
format is still supported)
(`#8000 <https://github.com/QubesOS/qubes-issues/issues/8000>`__)
- Improved keyboard layout switching
For a full list, including more detailed descriptions, please see
`here <https://github.com/QubesOS/qubes-issues/issues?q=is%3Aissue+sort%3Aupdated-desc+milestone%3A%22Release+4.2%22+label%3A%22release+notes%22+is%3Aclosed>`__.
Below are some screenshots of the new and improved Qubes GUI tools.
The new Qubes OS Update tool:
|Screenshot of the Qubes OS Update tool|
The new Qubes OS Global Config tool:
|Screenshot of the Qubes OS Global Config tool| |image1|
The new Qubes OS Policy Editor tool:
|Screenshot of the Qubes OS Policy Editor tool|
Known issues
------------
- DomU firewalls have completely switched to nftables. Users should add
their custom rules to the ``custom-input`` and ``custom-forward``
chains. (For more information, see issues
`#5031 <https://github.com/QubesOS/qubes-issues/issues/5031>`__ and
`#6062 <https://github.com/QubesOS/qubes-issues/issues/6062>`__.)
- Templates restored in 4.2 from a pre-4.2 backup continue to target
their original Qubes OS release repos. If you are using fresh
templates on a clean 4.2 installation, or if you performed an
:ref:`in-place upgrade from 4.1 to 4.2 <user/downloading-installing-upgrading/upgrade/4_2:in-place upgrade>`, then this does not
affect you. (For more information, see issue
`#8701 <https://github.com/QubesOS/qubes-issues/issues/8701>`__.)
Also see the `full list of open bug reports affecting Qubes 4.2 <https://github.com/QubesOS/qubes-issues/issues?q=is%3Aissue+label%3Aaffects-4.2+label%3A%22T%3A+bug%22+is%3Aopen>`__.
We strongly recommend :doc:`updating Qubes OS </user/how-to-guides/how-to-update>`
immediately after installation in order to apply all available bug
fixes.
Notes
-----
- Qubes 4.2 does not support Debian 11 templates (see :ref:`supported template releases <user/downloading-installing-upgrading/supported-releases:templates>`). Please
:ref:`upgrade your Debian templates <user/templates/debian/debian:upgrading>`
to Debian 12.
Download
--------
All Qubes ISOs and associated :doc:`verification files </project-security/verifying-signatures>` are available on the
:doc:`downloads </user/downloading-installing-upgrading/downloads>` page.
Installation instructions
-------------------------
See the :doc:`installation guide </user/downloading-installing-upgrading/installation-guide>`.
Upgrading
---------
Please see :doc:`how to upgrade to Qubes 4.2 </user/downloading-installing-upgrading/upgrade/4_2>`.
.. |Screenshot of the Qubes OS Update tool| image:: /attachment/site/4-2_update.png
.. |Screenshot of the Qubes OS Global Config tool| image:: /attachment/site/4-2_global-config_1.png
.. |image1| image:: /attachment/site/4-2_global-config_2.png
.. |Screenshot of the Qubes OS Policy Editor tool| image:: /attachment/site/4-2_policy-editor.png

20
developer/releases/4_2/schedule.md
View File

@ -1,20 +0,0 @@
---
lang: en
layout: doc
permalink: /doc/releases/4.2/schedule/
redirect_from:
- /en/doc/releases/4.2/schedule/
title: Qubes R4.2 release schedule
---
_**Please note:** This page is still an unfinished draft in progress. It is being updated as Qubes 4.2 development and testing continues._
The table below is based on our [release schedule
policy](/doc/version-scheme/#release-schedule).
| Date | Stage |
| ----------:| ----------------------------------------- |
| 2023-06-02 | 4.2.0-rc1 release |
| 2023-08-28 | 4.2.0-rc2 release |
| 2023-09-03 | 4.2.0-rc3 release |
| 2023-10-13 | 4.2.0-rc4 release |

26
developer/releases/4_2/schedule.rst Normal file
View File

@ -0,0 +1,26 @@
===========================
Qubes R4.2 release schedule
===========================
**Please note:** *This page is still an unfinished draft in progress. It is being updated as Qubes 4.2 development and testing continues.*
The table below is based on our :ref:`release schedule policy <developer/releases/version-scheme:release schedule>`.
.. list-table::
:widths: 10 10
:align: center
:header-rows: 1
* - Date
- Stage
* - 2023-06-02
- 4.2.0-rc1 release
* - 2023-08-28
- 4.2.0-rc2 release
* - 2023-09-03
- 4.2.0-rc3 release
* - 2023-10-13
- 4.2.0-rc4 release

16
developer/releases/notes.md
View File

@ -1,16 +0,0 @@
---
lang: en
layout: doc
permalink: /doc/releases/notes/
ref: 13
title: Release notes
---
- [Qubes R1.0 release notes](/doc/releases/1.0/release-notes/)
- [Qubes R2.0 release notes](/doc/releases/2.0/release-notes/)
- [Qubes R3.0 release notes](/doc/releases/3.0/release-notes/)
- [Qubes R3.1 release notes](/doc/releases/3.1/release-notes/)
- [Qubes R3.2 release notes](/doc/releases/3.2/release-notes/)
- [Qubes R4.0 release notes](/doc/releases/4.0/release-notes/)
- [Qubes R4.1 release notes](/doc/releases/4.1/release-notes/)
- [Qubes R4.2 release notes](/doc/releases/4.2/release-notes/)

25
developer/releases/notes.rst Normal file
View File

@ -0,0 +1,25 @@
=============
Release notes
=============
.. toctree::
:maxdepth: 1
Qubes R1.0 release notes </developer/releases/1_0/release-notes>
Qubes R2.0 release notes </developer/releases/2_0/release-notes>
Qubes R3.0 release notes </developer/releases/3_0/release-notes>
Qubes R3.1 release notes </developer/releases/3_1/release-notes>
Qubes R3.2 release notes </developer/releases/3_2/release-notes>
Qubes R4.0 release notes </developer/releases/4_0/release-notes>
Qubes R4.1 release notes </developer/releases/4_1/release-notes>
Qubes R4.2 release notes </developer/releases/4_2/release-notes>

14
developer/releases/schedules.md
View File

@ -1,14 +0,0 @@
---
lang: en
layout: doc
permalink: /doc/releases/schedules/
ref: 15
title: Release schedules
---
- [Qubes R3.0 release schedule](/doc/releases/3.0/schedule/)
- [Qubes R3.1 release schedule](/doc/releases/3.1/schedule/)
- [Qubes R3.2 release schedule](/doc/releases/3.2/schedule/)
- [Qubes R4.0 release schedule](/doc/releases/4.0/schedule/)
- [Qubes R4.1 release schedule](/doc/releases/4.1/schedule/)
- [Qubes R4.2 release schedule](/doc/releases/4.2/schedule/)

21
developer/releases/schedules.rst Normal file
View File

@ -0,0 +1,21 @@
=================
Release schedules
=================
.. toctree::
:maxdepth: 1
Qubes R3.0 release schedule </developer/releases/3_0/schedule>
Qubes R3.1 release schedule </developer/releases/3_1/schedule>
Qubes R3.2 release schedule </developer/releases/3_2/schedule>
Qubes R4.0 release schedule </developer/releases/4_0/schedule>
Qubes R4.1 release schedule </developer/releases/4_1/schedule>
Qubes R4.2 release schedule </developer/releases/4_2/schedule>

42
developer/releases/todo.md
View File

@ -1,42 +0,0 @@
---
lang: en
layout: doc
permalink: /doc/releases/todo/
redirect_from:
- /en/doc/releases/todo/
ref: 14
title: Release checklist
---
*the checklist is probably unfinished*
On -rc1
-------
* write schedule
* create package repositories (linux-yum, linux-deb)
* update repository definition (core-agent-linux, installer-qubes-os/qubes-release)
* push all packages to `current-testing`
* draft release notes, one note per feature
* create upgrade package in previous release branch (r2->r3.0, r3.0->r3.1, etc) - core-agent-linux
* make sure that keys for the current release are included in previous release's qubes-release package (for upgrade)
* build ISO and push to mirrors
On subsequent -rc
-----------------
* push packages to `current`
* update release notes
* build ISO and push to mirrors
* notify @Rudd-O about the new ISO for new torrent hosting
On final release
----------------
* push packages to `current`
* finish release notes
* update InstallationInstructions
* build ISO and push to mirrors
* notify @Rudd-O about the new ISO for new torrent hosting
* write blog post
* announce on Twitter

65
developer/releases/todo.rst Normal file
View File

@ -0,0 +1,65 @@
=================
Release checklist
=================
*the checklist is probably unfinished*
On -rc1
-------
- write schedule
- create package repositories (linux-yum, linux-deb)
- update repository definition (core-agent-linux,
installer-qubes-os/qubes-release)
- push all packages to ``current-testing``
- draft release notes, one note per feature
- create upgrade package in previous release branch (r2->r3.0,
r3.0->r3.1, etc) - core-agent-linux
- make sure that keys for the current release are included in previous
release’s qubes-release package (for upgrade)
- build ISO and push to mirrors
On subsequent -rc
-----------------
- push packages to ``current``
- update release notes
- build ISO and push to mirrors
- notify @Rudd-O about the new ISO for new torrent hosting
On final release
----------------
- push packages to ``current``
- finish release notes
- update InstallationInstructions
- build ISO and push to mirrors
- notify @Rudd-O about the new ISO for new torrent hosting
- write blog post
- announce on Twitter

189
developer/releases/version-scheme.md
View File

@ -1,189 +0,0 @@
---
lang: en
layout: doc
permalink: /doc/version-scheme/
redirect_from:
- /en/doc/version-scheme/
- /doc/VersionScheme/
- /wiki/VersionScheme/
ref: 151
title: Version scheme
---
The Qubes OS Project uses the [semantic versioning](https://semver.org/)
standard. Version numbers are written as `<major>.<minor>.<patch>`. When
`<patch>` is omitted (e.g., `4.1`), it is usually either because `<patch>` is
zero (as in `4.1.0`) or because we are referring to a specific minor release
irrespective of any particular patch release within it. Similarly, the major
release number alone (e.g., `R4`) is sometimes used to refer to an entire
release series inclusive of all minor and patch releases within it.
In general, patch releases are for backward-compatible bug fixes, minor
releases are for backward-compatible enhancements and new features, and major
release are for any backward-incompatible changes. This means that, in general,
one should *not* try to introduce features or enhancements in patch releases or
any backward-incompatible changes in patch or minor releases. (Templates are a
notable exception, as upstream OSes almost always have their own release
schedules.) Bug fixes are allowed in all releases, and backward-compatible
changes are allowed in all major and minor releases.
Qubes OS minor releases generally include new features, new templates, and
occasionally new defaults, but they are still backward-compatible in the sense
that qubes and features that worked in the previous release still function,
though the UI may be different in some cases. In general, deprecated features
are removed only in major releases, and in-place upgrades between major versions
are not guaranteed.
Following standard practice, **version** refers to any build that has been
assigned a version name or number, e.g., `3.2-rc2`, `4.0.4`, `4.1-beta1`. By
contrast, **release** refers to any version that is intended for consumption by
the general userbase. For example, `4.0.4` was both a **version** and a
**release**, since it was stable and intended for general public use, while
`4.1-beta1` was a **version** but *not* a **release**, since it was not stable
and was intended only for [testing](/doc/testing/). All releases are
versions, but not all versions are releases.
The letter **R**, as in `R4.1`, stands for **release**. The abbreviation **RC**,
as in `3.2-rc2`, stands for **release candidate**.
## Qubes distributions and products
We intend to make it easy to make a remix of Qubes, targeting another
hypervisor or isolation provider. We may also create commercial products
intended for specific circumstances. There is one distinguished distribution
called **Qubes OS**. All source code for it is available for download under a
[free and open-source license](/doc/license/) and is openly developed on
[GitHub](https://github.com/QubesOS) and our [mailing
lists](https://www.qubes-os.org/support/). The rest of this document discusses
Qubes OS. Another remix may have its own version series.
## Release versioning
Qubes OS as a whole is released from time to time. When preparing a new release,
we decide on the `<major>.<minor>` numbers (e.g., `3.0`, which is short for
`3.0.0`). We then publish the first release candidate, e.g., `3.0.0-rc1`. When
we feel that enough progress has been made, we'll release `3.0.0-rc2` and so on.
All these versions (which are not yet releases) are considered unstable and are
not intended for production use. You are welcome to [help us
test](/doc/testing/) these versions.
When enough progress has been made, we announce the first stable release, e.g.
`3.0.0`. This is not only a version but an actual release. It is considered
stable, and we commit to supporting it according to our [support
schedule](/doc/supported-releases/). Core components are branched at this
moment, and bug fixes are backported from the master branch. Please see [help,
support, mailing lists, and forum](/support/) for places to ask questions about
stable releases. No major features or interface incompatibilities are to be
included in this release. We release bug fixes as patch releases (`3.0.1`,
`3.0.2`, and so on), while backward-compatible enhancements and new features
are introduced in the next minor release (e.g., `3.1`). Any
backward-incompatible changes are introduced in the next major release (e.g.,
`4.0`).
Please see [issue tracking](/doc/issue-tracking/) for information about how
releases are handled in the issue tracker.
## Release schedule
There is no specific schedule for releases other than a general roadmap.
When the time comes, we declare a feature freeze, tag `-rc1`, and
release an ISO. From this point on, no new features are accepted, and our
schedule begins.
Each release candidate period is as follows: For the first two weeks, we accept
and assign bug reports to be fixed before the next release candidate. For the
next two weeks, we generally focus on fixing assigned bug reports, so issues
discovered during this period may be postponed until a later RC. Finally,
there is a one week current-testing freeze, during which time no new packages
are released, in the hope that they will be installed and tested by wider user
base.
The next RC is released five weeks after the former. All packages are published
in the `current` repository, and the cycle starts over. There should always be
at least one release candidate before the final release.
| Stage | Duration |
| ------------------------ | --------- |
| initial testing | two weeks |
| bug fixing | two weeks |
| `current-testing` freeze | one week |
Starting with the second cycle (that is, after `-rc1`), two weeks into the cycle
(after the primary bug-reporting period), we decide whether there should be
another RC. If, based on the bugs that have been reported, we decide that the
latest RC will be designated as the stable release, then we decide on its
release date, which should be no more than one week later.
[![Release cycle](/attachment/doc/release-cycle.svg)](/attachment/doc/release-cycle.svg)
## Bug priorities
When deciding whether the current release candidate is the final one, the
Committee takes bug [priorities](/doc/issue-tracking/#priority) into
consideration. The meaning of them is as follows:
- `blocker` --- when any such bug is present in the current release candidate,
it can't be considered final release. Bugs with this priority must be fixed
before the next release candidate, even if that means delaying its release
(which should be considered only last resort option).
- `critical` --- when any such bug is present in the current release candidate,
it can't be considered final release. But such bugs are not qualified to
delay next release candidate release.
- `major` --- existence of such bugs do not strictly prevent the current
release candidate be considered final (but of course we should try hard to
not have them there). Fixing bugs of this priority can be delayed and
qualified as updates to the final stable release.
- `default` and `minor` --- existence of such bugs do not prevent the current
release candidate be considered final. Fixing such bugs can be delayed to the
next Qubes OS release. Eventually such fixes might be backported as an update
to the stable release(s). (`default` should really be assigned a more
specific priority, but in practice there are too many issues and not enough
time, so `default` ends up staying on many issues.)
All above is about bugs, no features should be assigned to the current release
after first `-rc`. Supreme Committee is free to adjust priorities
appropriately.
## Component version
Qubes release is defined as specific versions of components, which are
developed more or less separately. Their versions are composed of major and
minor version of target Qubes OS release followed by third component which is
just incremented. There is no apparent indication that given version is stable
or not.
There are some non-essential components like `qubes-apps-*` that are shared
between releases. Their versions indicate oldest qubes-release that is
supported. We try hard to support multiple releases by one branch to ease code
maintenance.
Different Qubes releases remixes may comprise of different components and
version are not guaranteed to be monotonic between releases. We may decide that
for newer release some component should be downgraded. There is no guarantee
that arbitrary combination of different versions of random components will
yield usable (or even install-able) compilation.
## Git tags and branches
We mark each component version in the repository by tag containing
`v<version>`. Likewise, each Qubes OS release is marked by `R<release>` tag.
At the release of some release we create branches named like `release2`. Only
bug fixes and compatible improvements are backported to these branches. These
branches should compile. All new development is done in `master` branch. This
branch is totally unsupported and may not even compile depending on maintainer
of repository.
All version and release tags should be made and signed by someone from ITL
staff. Public keys are included in `qubes-builder` and available at
<https://keys.qubes-os.org/keys/>.
## Check installed version
If you want to know which version you are running, for example to report an
issue, you can either check in the Qubes Manager menu under `About > Qubes OS`
or in the file `/etc/qubes-release` in dom0. For the latter you can use a
command like `cat /etc/qubes-release` in a dom0 terminal.

221
developer/releases/version-scheme.rst Normal file
View File

@ -0,0 +1,221 @@
==============
Version scheme
==============
The Qubes OS Project uses the `semantic versioning <https://semver.org/>`__ standard. Version numbers are
written as ``<major>.<minor>.<patch>``. When ``<patch>`` is omitted
(e.g., ``4.1``), it is usually either because ``<patch>`` is zero (as in
``4.1.0``) or because we are referring to a specific minor release
irrespective of any particular patch release within it. Similarly, the
major release number alone (e.g., ``R4``) is sometimes used to refer to
an entire release series inclusive of all minor and patch releases
within it.
In general, patch releases are for backward-compatible bug fixes, minor
releases are for backward-compatible enhancements and new features, and
major release are for any backward-incompatible changes. This means
that, in general, one should *not* try to introduce features or
enhancements in patch releases or any backward-incompatible changes in
patch or minor releases. (Templates are a notable exception, as upstream
OSes almost always have their own release schedules.) Bug fixes are
allowed in all releases, and backward-compatible changes are allowed in
all major and minor releases.
Qubes OS minor releases generally include new features, new templates,
and occasionally new defaults, but they are still backward-compatible in
the sense that qubes and features that worked in the previous release
still function, though the UI may be different in some cases. In
general, deprecated features are removed only in major releases, and
in-place upgrades between major versions are not guaranteed.
Following standard practice, **version** refers to any build that has
been assigned a version name or number, e.g., ``3.2-rc2``, ``4.0.4``,
``4.1-beta1``. By contrast, **release** refers to any version that is
intended for consumption by the general userbase. For example, ``4.0.4``
was both a **version** and a **release**, since it was stable and
intended for general public use, while ``4.1-beta1`` was a **version**
but *not* a **release**, since it was not stable and was intended only
for :doc:`testing </user/downloading-installing-upgrading/testing>`. All releases are versions, but not all
versions are releases.
The letter **R**, as in ``R4.1``, stands for **release**. The
abbreviation **RC**, as in ``3.2-rc2``, stands for **release candidate**.
Qubes distributions and products
--------------------------------
We intend to make it easy to make a remix of Qubes, targeting another
hypervisor or isolation provider. We may also create commercial products
intended for specific circumstances. There is one distinguished
distribution called **Qubes OS**. All source code for it is available
for download under a :doc:`free and open-source license </developer/code/license>`
and is openly developed on `GitHub <https://github.com/QubesOS>`__ and
our `mailing lists <https://www.qubes-os.org/support/>`__. The rest of
this document discusses Qubes OS. Another remix may have its own version
series.
Release versioning
------------------
Qubes OS as a whole is released from time to time. When preparing a new
release, we decide on the ``<major>.<minor>`` numbers (e.g., ``3.0``,
which is short for ``3.0.0``). We then publish the first release
candidate, e.g., ``3.0.0-rc1``. When we feel that enough progress has
been made, we’ll release ``3.0.0-rc2`` and so on. All these versions
(which are not yet releases) are considered unstable and are not
intended for production use. You are welcome to :doc:`help us test </user/downloading-installing-upgrading/testing>` these versions.
When enough progress has been made, we announce the first stable
release, e.g. ``3.0.0``. This is not only a version but an actual
release. It is considered stable, and we commit to supporting it
according to our :doc:`support schedule </user/downloading-installing-upgrading/supported-releases>`. Core
components are branched at this moment, and bug fixes are backported
from the master branch. Please see :doc:`help, support, mailing lists, and forum </introduction/support>` for places to ask questions about stable releases.
No major features or interface incompatibilities are to be included in
this release. We release bug fixes as patch releases (``3.0.1``,
``3.0.2``, and so on), while backward-compatible enhancements and new
features are introduced in the next minor release (e.g., ``3.1``). Any
backward-incompatible changes are introduced in the next major release
(e.g., ``4.0``).
Please see :doc:`issue tracking </introduction/issue-tracking>` for information
about how releases are handled in the issue tracker.
Release schedule
----------------
There is no specific schedule for releases other than a general roadmap.
When the time comes, we declare a feature freeze, tag ``-rc1``, and
release an ISO. From this point on, no new features are accepted, and
our schedule begins.
Each release candidate period is as follows: For the first two weeks, we
accept and assign bug reports to be fixed before the next release
candidate. For the next two weeks, we generally focus on fixing assigned
bug reports, so issues discovered during this period may be postponed
until a later RC. Finally, there is a one week current-testing freeze,
during which time no new packages are released, in the hope that they
will be installed and tested by wider user base.
The next RC is released five weeks after the former. All packages are
published in the ``current`` repository, and the cycle starts over.
There should always be at least one release candidate before the final
release.
.. list-table::
:widths: 26 26
:align: center
:header-rows: 1
* - Stage
- Duration
* - initial testing
- two weeks
* - bug fixing
- two weeks
* - current-testing freeze
- one week
Starting with the second cycle (that is, after ``-rc1``), two weeks into
the cycle (after the primary bug-reporting period), we decide whether
there should be another RC. If, based on the bugs that have been
reported, we decide that the latest RC will be designated as the stable
release, then we decide on its release date, which should be no more
than one week later.
|Release cycle|
Bug priorities
--------------
When deciding whether the current release candidate is the final one,
the Committee takes bug :ref:`priorities <introduction/issue-tracking:priority>`
into consideration. The meaning of them is as follows:
- ``blocker`` — when any such bug is present in the current release
candidate, it can’t be considered final release. Bugs with this
priority must be fixed before the next release candidate, even if
that means delaying its release (which should be considered only last
resort option).
- ``critical`` — when any such bug is present in the current release
candidate, it can’t be considered final release. But such bugs are
not qualified to delay next release candidate release.
- ``major`` — existence of such bugs do not strictly prevent the
current release candidate be considered final (but of course we
should try hard to not have them there). Fixing bugs of this priority
can be delayed and qualified as updates to the final stable release.
- ``default`` and ``minor`` — existence of such bugs do not prevent the
current release candidate be considered final. Fixing such bugs can
be delayed to the next Qubes OS release. Eventually such fixes might
be backported as an update to the stable release(s). (``default``
should really be assigned a more specific priority, but in practice
there are too many issues and not enough time, so ``default`` ends up
staying on many issues.)
All above is about bugs, no features should be assigned to the current
release after first ``-rc``. Supreme Committee is free to adjust
priorities appropriately.
Component version
-----------------
Qubes release is defined as specific versions of components, which are
developed more or less separately. Their versions are composed of major
and minor version of target Qubes OS release followed by third component
which is just incremented. There is no apparent indication that given
version is stable or not.
There are some non-essential components like ``qubes-apps-*`` that are
shared between releases. Their versions indicate oldest qubes-release
that is supported. We try hard to support multiple releases by one
branch to ease code maintenance.
Different Qubes releases remixes may comprise of different components
and version are not guaranteed to be monotonic between releases. We may
decide that for newer release some component should be downgraded. There
is no guarantee that arbitrary combination of different versions of
random components will yield usable (or even install-able) compilation.
Git tags and branches
---------------------
We mark each component version in the repository by tag containing
``v<version>``. Likewise, each Qubes OS release is marked by
``R<release>`` tag.
At the release of some release we create branches named like
``release2``. Only bug fixes and compatible improvements are backported
to these branches. These branches should compile. All new development is
done in ``master`` branch. This branch is totally unsupported and may
not even compile depending on maintainer of repository.
All version and release tags should be made and signed by someone from
ITL staff. Public keys are included in ``qubes-builder`` and available
at https://keys.qubes-os.org/keys/.
Check installed version
-----------------------
If you want to know which version you are running, for example to report
an issue, you can either check in the Qubes Manager menu under
``About > Qubes OS`` or in the file ``/etc/qubes-release`` in dom0. For
the latter you can use a command like ``cat /etc/qubes-release`` in a
dom0 terminal.
.. |Release cycle| image:: /attachment/doc/release-cycle.png

361
developer/services/admin-api.md
View File

@ -1,361 +0,0 @@
---
lang: en
layout: doc
permalink: /doc/admin-api/
redirect_from:
- /doc/qubes-admin-api/
- /doc/mgmt/
- /doc/mgmt1/
- /doc/mgmt-architecture/
- /doc/admin-api-architecture/
- /doc/admin-api/table/
ref: 36
title: Admin API
---
_You may also be interested in the article
[Introducing the Qubes Admin API](/news/2017/06/27/qubes-admin-api/)._
## Goals
The goals of the Admin API system is to provide a way for the user to manage
the domains without direct access to dom0.
Foreseen benefits include:
- Ability to remotely manage the Qubes OS.
- Possibility to create multi-user system, where different users are able to use
different sets of domains, possibly overlapping. This would also require to
have separate GUI domain.
The API would be used by:
- Qubes OS Manager (or any tools that would replace it)
- CLI tools, when run from another VM (and possibly also from dom0)
- remote management tools
- any custom tools
## Threat model
TBD
## Components
![Admin API Architecture](/attachment/doc/admin-api-architecture.svg)
A central entity in the Qubes Admin API system is a `qubesd` daemon, which
holds information about all domains in the system and mediates all actions (like
starting and stopping a qube) with `libvirtd`. The `qubesd` daemon also manages
the `qubes.xml` file, which stores all persistent state information and
dispatches events to extensions. Last but not least, `qubesd` is responsible for
querying the RPC policy for qrexec daemon.
The `qubesd` daemon may be accessed from other domains through a set of qrexec
API calls called the "Admin API". This API is the intended
management interface supported by the Qubes OS. The API is stable. When called,
the RPC handler performs basic validation and forwards the request to the
`qubesd` via UNIX domain socket. The socket API is private, unstable, and not
yet documented.
## The calls
The API should be implemented as a set of qrexec calls. This is to make it easy
to set the policy using current mechanism.
| call | dest | argument | inside | return | note |
| ------------------------------------- | --------- | --------- | ----------------------------------------- | --------------------------------------------------------- | ---- |
| `admin.vmclass.List` | `dom0` | - | - | `<class>\n` |
| `admin.vm.List` | `dom0|<vm>` | - | - | `<name> class=<class> state=<state>\n` |
| `admin.vm.Create.<class>` | `dom0` | template | `name=<name> label=<label>` | - |
| `admin.vm.CreateInPool.<class>` | `dom0` | template | `name=<name> label=<label> `<br/>`pool=<pool> pool:<volume>=<pool>` | - | either use `pool=` to put all volumes there, <br/>or `pool:<volume>=` for individual volumes - both forms are not allowed at the same time
| `admin.vm.CreateDisposable` | template | - | - | name | Create new DisposableVM, `template` is any AppVM with `dispvm_allowed` set to True, or `dom0` to use default defined in `default_dispvm` property of calling VM; VM created with this call will be automatically removed after its shutdown; the main difference from `admin.vm.Create.DispVM` is automatic (random) name generation.
| `admin.vm.Remove` | vm | - | - | - |
| `admin.label.List` | `dom0` | - | - | `<property>\n` |
| `admin.label.Create` | `dom0` | label | `0xRRGGBB` | - |
| `admin.label.Get` | `dom0` | label | - | `0xRRGGBB` |
| `admin.label.Index` | `dom0` | label | - | `<label-index>` |
| `admin.label.Remove` | `dom0` | label | - | - |
| `admin.property.List` | `dom0` | - | - | `<property>\n` |
| `admin.property.Get` | `dom0` | property | - | `default={True|False} `<br/>`type={str|int|bool|vm|label|list} <value>` | Type `list` is added in R4.1. Values are of type `str` and each entry is suffixed with newline character.
| `admin.property.GetAll` | `dom0` | - | - | `<property-name> <full-value-as-in-property.Get>\n` | Get all the properties in one call. Each property is returned on a separate line and use the same value encoding as property.Get method, with an exception that newlines are encoded as literal `\n` and literal `\` are encoded as `\\`.
| `admin.property.GetDefault` | `dom0` | property | - | `type={str|int|bool|vm|label|list} <value>` | Type `list` is added in R4.1. Values are of type `str` and each entry is suffixed with newline character.
| `admin.property.Help` | `dom0` | property | - | `help` |
| `admin.property.HelpRst` | `dom0` | property | - | `help.rst` |
| `admin.property.Reset` | `dom0` | property | - | - |
| `admin.property.Set` | `dom0` | property | value | - |
| `admin.vm.property.List` | vm | - | - | `<property>\n` |
| `admin.vm.property.Get` | vm | property | - | `default={True|False} `<br/>`type={str|int|bool|vm|label|list} <value>` | Type `list` is added in R4.1. Each list entry is suffixed with a newline character.
| `admin.vm.property.GetAll` | vm | - | - | `<property-name> <full-value-as-in-property.Get>\n` | Get all the properties in one call. Each property is returned on a separate line and use the same value encoding as property.Get method, with an exception that newlines are encoded as literal `\n` and literal `\` are encoded as `\\`.
| `admin.vm.property.GetDefault` | vm | property | - | `type={str|int|bool|vm|label|type} <value>` | Type `list` is added in R4.1. Each list entry is suffixed with a newline character.
| `admin.vm.property.Help` | vm | property | - | `help` |
| `admin.vm.property.HelpRst` | vm | property | - | `help.rst` |
| `admin.vm.property.Reset` | vm | property | - | - |
| `admin.vm.property.Set` | vm | property | value | - |
| `admin.vm.feature.List` | vm | - | - | `<feature>\n` |
| `admin.vm.feature.Get` | vm | feature | - | value |
| `admin.vm.feature.CheckWithTemplate` | vm | feature | - | value |
| `admin.vm.feature.CheckWithNetvm` | vm | feature | - | value |
| `admin.vm.feature.CheckWithAdminVM` | vm | feature | - | value |
| `admin.vm.feature.CheckWithTemplateAndAdminVM`| vm | feature | - | value |
| `admin.vm.feature.Remove` | vm | feature | - | - |
| `admin.vm.feature.Set` | vm | feature | value | - |
| `admin.vm.tag.List` | vm | - | - | `<tag>\n` |
| `admin.vm.tag.Get` | vm | tag | - | `0` or `1` | retcode? |
| `admin.vm.tag.Remove` | vm | tag | - | - |
| `admin.vm.tag.Set` | vm | tag | - | - |
| `admin.vm.firewall.Get` | vm | - | - | `<rule>\n` | rules syntax as in [firewall interface](/doc/vm-interface/#firewall-rules-in-4x) with addition of `expire=` and `comment=` options; `comment=` (if present) must be the last option
| `admin.vm.firewall.Set` | vm | - | `<rule>\n` | - | set firewall rules, see `admin.vm.firewall.Get` for syntax
| `admin.vm.firewall.Reload` | vm | - | - | - | force reload firewall without changing any rule
| `admin.vm.deviceclass.List` | `dom0` | - | - | `<class>\n` |
| `admin.vm.device.<class>.Attach` | vm | device | options | - | `device` is in form `<backend-name>+<device-ident>` <br/>optional options given in `key=value` format, separated with spaces; <br/>options can include `persistent=True` to "persistently" attach the device (default is temporary)
| `admin.vm.device.<class>.Detach` | vm | device | - | - | `device` is in form `<backend-name>+<device-ident>`
| `admin.vm.device.<class>.Set.persistent`| vm | device | `True`\|`False` | - | `device` is in form `<backend-name>+<device-ident>`
| `admin.vm.device.<class>.List` | vm | - | - | `<device> <options>\n` | options can include `persistent=True` for "persistently" attached devices (default is temporary)
| `admin.vm.device.<class>.Available` | vm | device-ident | - | `<device-ident> <properties> description=<desc>\n` | optional service argument may be used to get info about a single device, <br/>optional (device class specific) properties are in `key=value` form, <br/>`description` must be the last one and is the only one allowed to contain spaces
| `admin.pool.List` | `dom0` | - | - | `<pool>\n` |
| `admin.pool.ListDrivers` | `dom0` | - | - | `<pool-driver> <property> ...\n` | Properties allowed in `admin.pool.Add`
| `admin.pool.Info` | `dom0` | pool | - | `<property>=<value>\n` |
| `admin.pool.Add` | `dom0` | driver | `<property>=<value>\n` | - |
| `admin.pool.Set.revisions_to_keep` | `dom0` | pool | `<value>` | - |
| `admin.pool.Remove` | `dom0` | pool | - | - |
| `admin.pool.volume.List` | `dom0` | pool | - | volume id |
| `admin.pool.volume.Info` | `dom0` | pool | vid | `<property>=<value>\n` |
| `admin.pool.volume.Set.revisions_to_keep`| `dom0` | pool | `<vid> <value>` | - |
| `admin.pool.volume.ListSnapshots` | `dom0` | pool | vid | `<snapshot>\n` |
| `admin.pool.volume.Snapshot` | `dom0` | pool | vid | snapshot |
| `admin.pool.volume.Revert` | `dom0` | pool | `<vid> <snapshot>` | - |
| `admin.pool.volume.Resize` | `dom0` | pool | `<vid> <size_in_bytes>` | - |
| `admin.pool.volume.Import` | `dom0` | pool | `<vid>\n<raw volume data>` | - |
| `admin.pool.volume.CloneFrom` | `dom0` | pool | vid | token, to be used in `admin.pool.volume.CloneTo` | obtain a token to copy volume `vid` in `pool`;<br/>the token is one time use only, it's invalidated by `admin.pool.volume.CloneTo`, even if the operation fails |
| `admin.pool.volume.CloneTo` | `dom0` | pool | `<vid> <token>` | - | copy volume pointed by a token to volume `vid` in `pool` |
| `admin.vm.volume.List` | vm | - | - | `<volume>\n` | `<volume>` is per-VM volume name (`root`, `private`, etc), `<vid>` is pool-unique volume id
| `admin.vm.volume.Info` | vm | volume | - | `<property>=<value>\n` |
| `admin.vm.volume.Set.revisions_to_keep`| vm | volume | value | - |
| `admin.vm.volume.ListSnapshots` | vm | volume | - | snapshot | duplicate of `admin.pool.volume.`, but with other call params |
| `admin.vm.volume.Snapshot` | vm | volume | - | snapshot | id. |
| `admin.vm.volume.Revert` | vm | volume | snapshot | - | id. |
| `admin.vm.volume.Resize` | vm | volume | size_in_bytes | - | id. |
| `admin.vm.volume.Import` | vm | volume | raw volume data | - | id. |
| `admin.vm.volume.ImportWithSize` | vm | volume | `<size_in_bytes>\n<raw volume data>` | - | new version of `admin.vm.volume.Import`, allows new volume to be different size |
| `admin.vm.volume.Clear` | vm | volume | - | - | clear contents of volume |
| `admin.vm.volume.CloneFrom` | vm | volume | - | token, to be used in `admin.vm.volume.CloneTo` | obtain a token to copy `volume` of `vm`;<br/>the token is one time use only, it's invalidated by `admin.vm.volume.CloneTo`, even if the operation fails |
| `admin.vm.volume.CloneTo` | vm | volume | token, obtained with `admin.vm.volume.CloneFrom` | - | copy volume pointed by a token to `volume` of `vm` |
| `admin.vm.CurrentState` | vm | - | - | `<state-property>=<value>\n` | state properties: `power_state`, `mem`, `mem_static_max`, `cputime`
| `admin.vm.Start` | vm | - | - | - |
| `admin.vm.Shutdown` | vm | - | - | - |
| `admin.vm.Pause` | vm | - | - | - |
| `admin.vm.Unpause` | vm | - | - | - |
| `admin.vm.Kill` | vm | - | - | - |
| `admin.backup.Execute` | `dom0` | config id | - | - | config in `/etc/qubes/backup/<id>.conf`, only one backup operation of given `config id` can be running at once |
| `admin.backup.Info` | `dom0` | config id | - | backup info | info what would be included in the backup
| `admin.backup.Cancel` | `dom0` | config id | - | - | cancel running backup operation
| `admin.Events` | `dom0|vm` | - | - | events |
| `admin.vm.Stats` | `dom0|vm` | - | - | `vm-stats` events, see below | emit VM statistics (CPU, memory usage) in form of events
Volume properties:
- `pool`
- `vid`
- `size`
- `usage`
- `rw`
- `source`
- `save_on_stop`
- `snap_on_start`
- `revisions_to_keep`
- `is_outdated`
Method `admin.vm.Stats` returns `vm-stats` events every `stats_interval` seconds, for every running VM. Parameters of `vm-stats` events:
- `memory_kb` - memory usage in kB
- `cpu_time` - absolute CPU time (in milliseconds) spent by the VM since its startup, normalized for one CPU
- `cpu_usage` - CPU usage in percents
## Returned messages
First byte of a message is a message type. This is 8 bit non-zero integer.
Values start at 0x30 (48, `'0'`, zero digit in ASCII) for readability in hexdump.
Next byte must be 0x00 (a separator).
This alternatively can be thought of as zero-terminated string containing
single ASCII digit.
### OK (0)
```
30 00 <content>
```
Server will close the connection after delivering single message.
### EVENT (1)
```
31 00 <subject> 00 <event> 00 ( <key> 00 <value> 00 )* 00
```
Events are returned as stream of messages in selected API calls. Normally server
will not close the connection.
A method yielding events will not ever return a `OK` or `EXCEPTION` message.
When calling such method, it will produce an artificial event
`connection-established` just after connection, to help avoiding race
conditions during event handler registration.
### EXCEPTION (2)
```
32 00 <type> 00 ( <traceback> )? 00 <format string> 00 ( <field> 00 )*
```
Server will close the connection.
Traceback may be empty, can be enabled server-side as part of debug mode.
Delimiting zero-byte is always present.
Fields are should substituted into `%`-style format string, possibly after
client-side translation, to form final message to be displayed unto user. Server
does not by itself support translation.
## Tags
The tags provided can be used to write custom policies. They are not used in
a&nbsp;default Qubes OS installation. However, they are created anyway.
- `created-by-<vm>` &mdash;&nbsp;Created in an extension to qubesd at the
moment of creation of the VM. Cannot be changed via API, which is also
enforced by this extension.
- `managed-by-<vm>` &mdash;&nbsp;Can be used for the same purpose, but it is
not created automatically, nor is it forbidden to set or reset this tag.
## Backup profile
Backup-related calls do not allow (yet) to specify what should be included in
the backup. This needs to be configured separately in dom0, with a backup
profile, stored in `/etc/qubes/backup/<profile>.conf`. The file use yaml syntax
and have following settings:
- `include` - list of VMs to include, can also contains tags using
`$tag:some-tag` syntax or all VMs of given type using `$type:AppVM`, known
from qrexec policy
- `exclude` - list of VMs to exclude, after evaluating `include` setting
- `destination_vm` - VM to which the backup should be send
- `destination_path` - path to which backup should be written in
`destination_vm`. This setting is given to `qubes.Backup` service and
technically it's up to it how to interpret it. In current implementation it is
interpreted as a directory where a new file should be written (with a name
based on the current timestamp), or a command where the backup should
be piped to
- `compression` - should the backup be compressed (default: True)? The value can be either
`False` or `True` for default compression, or a compression command (needs to
accept `-d` argument for decompression)
- `passphrase_text` - passphrase used to encrypt and integrity protect the backup
- `passphrase_vm` - VM which should be asked what backup passphrase should be
used. The asking is performed using `qubes.BackupPassphrase+profile_name`
service, which is expected to output chosen passphrase to its stdout. Empty
output cancel the backup operation. This service can be used either to ask
the user interactively, or to have some automated passphrase handling (for
example: generate randomly, then encrypt with a public key and send
somewhere)
Not all settings needs to be set.
Example backup profile:
```yaml
# Backup only selected VMs
include:
- work
- personal
- vault
- banking
# Store the backup on external disk
destination_vm: sys-usb
destination_path: /media/my-backup-disk
# Use static passphrase
passphrase_text: "My$Very!@Strong23Passphrase"
```
And slightly more advanced one:
```yaml
# Include all VMs with a few exceptions
include:
- $type:AppVM
- $type:TemplateVM
- $type:StandaloneVM
exclude:
- untrusted
- $tag:do-not-backup
# parallel gzip for faster backup
compression: pigz
# ask 'vault' VM for the backup passphrase
passphrase_vm: vault
# send the (encrypted) backup directly to remote server
destination_vm: sys-net
destination_path: ncftpput -u my-ftp-username -p my-ftp-pass -c my-ftp-server /directory/for/backups
```
## General notes
- there is no provision for `qvm-run`, but there already exists `qubes.VMShell` call
- generally actions `*.List` return a list of objects and have "object
identifier" as first word in a row. Such action can be also called with "object
identifier" in argument to get only a single entry (in the same format).
- closing qrexec connection normally does _not_ interrupt running operation; this is important to avoid leaving the system in inconsistent state
- actual operation starts only after caller send all the parameters (including a payload), signaled by sending EOF mark; there is no support for interactive protocols, to keep the protocol reasonable simple
## Policy admin API
There is also an API to view and update [Qubes RPC policy files](/doc/qrexec) in dom0. All of the following calls have dom0 as destination:
| call | argument | inside | return |
| ---------------------------------------------- | ---- | -------------------- | ----------------------- |
| `policy.List` <br> `policy.include.List` | - | - | `<name1>\n<name2>\n...` |
| `policy.Get` <br> `policy.include.Get` | name | - | `<token>\n<content>` |
| `policy.Replace` <br> `policy.include.Replace` | name | `<token>\n<content>` | - |
| `policy.Remove` <br> `policy.include.Remove` | name | `<token>` | - |
The `policy.*` calls refer to main policy files (`/etc/qubes/policy.d/`), and
the `policy.include.*` calls refer to the include directory
(`/etc/qubes/policy.d/include/`). The `.policy` extension for files in the main
directory is always omitted.
The responses do not follow admin API protocol, but signal error using an exit
code and a message on stdout.
The changes are validated before saving, so that the policy cannot end up in an
invalid state (e.g. syntax error, missing include file).
In addition, there is a mechanism to prevent concurrent modifications of the policy files:
- A `*.Get` call returns a file along with a *token* (currently implemented as
a hash of the file).
- When calling `Replace` or `Remove`, you need to include the current token as
first line. If the token does not match, the modification will fail.
- When adding a new file using `Replace`, pass `new` as token. This will ensure
that the file does not exist before adding.
- To skip the check, pass `any` as token.
## TODO
- notifications
- how to constrain the events?
- how to pass the parameters? maybe XML, since this is trusted anyway and
parser may be complicated
- how to constrain the possible values for `admin.vm.property.Set` etc, like
"you can change `netvm`, but you have to pick from this set"; this currently
can be done by writing an extension
- a call for executing `*.desktop` file from `/usr/share/applications`, for use
with appmenus without giving access to `qubes.VMShell`; currently this can be
done by writing custom qrexec calls
- maybe some generator for `.desktop` for appmenus, which would wrap calls in
`qrexec-client-vm`
<!-- vim: set ts=4 sts=4 sw=4 et : -->

845
developer/services/admin-api.rst Normal file
View File

@ -0,0 +1,845 @@
=========
Admin API
=========
*You may also be interested in the article*\ `Introducing the Qubes Admin API <https://www.qubes-os.org/news/2017/06/27/qubes-admin-api/>`__\ *.*
Goals
=====
The goals of the Admin API system is to provide a way for the user to
manage the domains without direct access to dom0.
Foreseen benefits include:
- Ability to remotely manage the Qubes OS.
- Possibility to create multi-user system, where different users are
able to use different sets of domains, possibly overlapping. This
would also require to have separate GUI domain.
The API would be used by:
- Qubes OS Manager (or any tools that would replace it)
- CLI tools, when run from another VM (and possibly also from dom0)
- remote management tools
- any custom tools
Threat model
============
TBD
Components
==========
.. figure:: /attachment/doc/admin-api-architecture.png
:alt: Admin API Architecture
Admin API Architecture
A central entity in the Qubes Admin API system is a ``qubesd`` daemon,
which holds information about all domains in the system and mediates all
actions (like starting and stopping a qube) with ``libvirtd``. The
``qubesd`` daemon also manages the ``qubes.xml`` file, which stores all
persistent state information and dispatches events to extensions. Last
but not least, ``qubesd`` is responsible for querying the RPC policy for
qrexec daemon.
The ``qubesd`` daemon may be accessed from other domains through a set
of qrexec API calls called the “Admin API”. This API is the intended
management interface supported by the Qubes OS. The API is stable. When
called, the RPC handler performs basic validation and forwards the
request to the ``qubesd`` via UNIX domain socket. The socket API is
private, unstable, and not yet documented.
The calls
=========
The API should be implemented as a set of qrexec calls. This is to make
it easy to set the policy using current mechanism.
.. list-table:: i
:widths: 15 8 8 10 20 30
:align: left
:header-rows: 1
* - call
- dest
- argument
- inside
- return
- note
* - ``admin.vmclass.List``
- ``dom0``
- `-`
- `-`
- ``<class>``
-
* - ``admin.vm.List``
- ``dom0|<vm>``
- `-`
- `-`
- ``<name> class=<class> state=<state>``
-
* - ``admin.vm.Create.<class>``
- ``dom0``
- template
- ``name=<name> label=<label>``
- `-`
-
* - ``admin.vm.CreateInPool.<class>``
- ``dom0``
- template
- ``name=<name> label=<label>``, ``pool=<pool> pool:<volume>=<pool>``
- `-`
- either use ``pool=`` to put all volumes there, or ``pool:<volume>=`` for individual volumes - both forms are not allowed at the same time
* - ``admin.vm.CreateDisposable``
- template
- `-`
- `-`
- name
- Create new DisposableVM, ``template`` is any AppVM with ``dispvm_allowed`` set to True, or ``dom0`` to use default defined in ``default_dispvm`` property of calling VM; VM created with this call will be automatically removed after its shutdown; the main difference from ``admin.vm.Create.DispVM`` is automatic (random) name generation.
* - ``admin.vm.Remove``
- vm
- `-`
- `-`
- `-`
-
* - ``admin.label.List``
- ``dom0``
- `-`
- `-`
- ``<property>``
-
* - ``admin.label.Create``
- ``dom0``
- label
- ``0xRRGGBB``
- `-`
-
* - ``admin.label.Get``
- ``dom0``
- label
- `-`
- ``0xRRGGBB``
-
* - ``admin.label.Index``
- ``dom0``
- label
- `-`
- ``<label-index>``
-
* - ``admin.label.Remove``
- ``dom0``
- label
- `-`
- `-`
-
* - ``admin.property.List``
- ``dom0``
- `-`
- `-`
- ``<property>``
-
* - ``admin.property.Get``
- ``dom0``
- property
- `-`
- ``default={True|False}`` ``type={str|int|bool|vm|label|list} <value>``
- Type ``list`` is added in R4.1. Values are of type ``str`` and each entry is suffixed with newline character.
* - ``admin.property.GetAll``
- ``dom0``
- `-`
- `-`
- ``<property-name> <full-value-as-in-property.Get>``
- Get all the properties in one call. Each property is returned on a separate line and use the same value encoding as property.Get method, with an exception that newlines are encoded as literal ``\n`` and literal ``\`` are encoded as ``\\``.
* - ``admin.property.GetDefault``
- ``dom0``
- propety
- `-`
- ``type={str|int|bool|vm|label|list} <value>``
- Type ``list`` is added in R4.1. Values are of type ``str`` and each entry is suffixed with newline character.
* - ``admin.property.Help``
- ``dom0``
- property
- `-`
- ``help``
-
* - ``admin.property.HelpRst``
- ``dom0``
- property
- `-`
- ``help.rst``
-
* - ``admin.property.Reset``
- ``dom0``
- property
- `-`
- `-`
-
* - ``admin.property.Set``
- ``dom0``
- property
- value
- `-`
-
* - ``admin.vm.property.List``
- ``vm``
- `-`
- `-`
- ``<property>``
-
* - ``admin.vm.property.Get``
- ``vm``
- property
- `-`
- ``default={True|False}`` ``type={str|int|bool|vm|label|list} <value>``
-
* - ``admin.vm.property.GetAll``
- ``vm``
- `-`
- `-`
- ``<property-name> <full-value-as-in-property.Get>``
- Get all the properties in one call. Each property is returned on a separate line and use the same value encoding as property.Get method, with an exception that newlines are encoded as literal ``\n`` and literal ``\`` are encoded as ``\\``.
* - ``admin.vm.property.GetDefault``
- ``vm``
- property
- `-`
- ``type={str|int|bool|vm|label|type} <value>``
- Type ``list`` is added in R4.1. Each list entry is suffixed with a newline character
* - ``admin.vm.property.Help``
- ``vm``
- property
- `-`
- ``help``
-
* - ``admin.vm.property.HelpRst``
- ``vm``
- property
- `-`
- ``help.rst``
-
* - ``admin.vm.property.Reset``
- ``vm``
- property
- `-`
- `-`
-
* - ``admin.vm.property.Set``
- ``vm``
- property
- value
- `-`
-
* - ``admin.vm.feature.List``
- ``vm``
- `-`
- `-`
- ``<feature>``
-
* - ``admin.vm.feature.Get``
- ``vm``
- feature
- `-`
- value
-
* - ``admin.vm.feature.CheckWithTemplate``
- ``vm``
- feature
- `-`
- value
-
* - ``admin.vm.feature.CheckWithNetvm``
- ``vm``
- feature
- `-`
- value
-
* - ``admin.vm.feature.CheckWithAdminVM``
- ``vm``
- feature
- `-`
- value
-
* - ``admin.vm.feature.CheckWithTemplateAndAdminVM``
- ``vm``
- feature
- `-`
- value
-
* - ``admin.vm.feature.Remove``
- vm
- feature
- `-`
- `-`
-
* - ``admin.vm.feature.Set``
- vm
- feature
- value
- `-`
-
* - ``admin.vm.tag.List``
- vm
- `-`
- `-`
- ``<tag>``
-
* - ``admin.vm.tag.Get``
- vm
- tag
- `-`
- ``0`` or ``1``
- retcode?
* - ``admin.vm.tag.Remove``
- vm
- tag
- `-`
- `-`
-
* - ``admin.vm.tag.Set``
- vm
- tag
- `-`
- `-`
-
* - ``admin.vm.firewall.Get``
- vm
- `-`
- `-`
- ``<rule>``
- rules syntax as in :doc:`firewall interface </developer/debugging/vm-interface>` (Firewall Rules in 4x) with addition of ``expire=`` and ``comment=`` options; ``comment=`` (if present) must be the last option
* - ``admin.vm.firewall.Set``
- vm
- `-`
- ``<rule>``
- `-`
- set firewall rules, see ``admin.vm.firewall.Get`` for syntax
* - ``admin.vm.firewall.Reload``
- vm
- `-`
- `-`
- `-`
- force reload firewall without changing any rule
* - ``admin.vm.device.<class>.Attach``
- vm
- device
- options
- `-`
- ``device`` is in form ``<backend-name>+<device-ident>`` optional options given in ``key=value`` format, separated with spaces; options can include ``persistent=True`` to "persistently" attach the device (default is temporary)
* - ``admin.vm.device.<class>.Detach``
- vm
- device
- `-`
- `-`
- ``device`` is in form ``<backend-name>+<device-ident>``
* - ``admin.vm.device.<class>.Set.persistent``
- vm
- device
- ``True|False``
- `-`
- ``device`` is in form ``<backend-name>+<device-ident>``
* - ``admin.vm.device.<class>.List``
- vm
- `-`
- `-`
- ``<device> <options>``
- options can include ``persistent=True`` for "persistently" attached devices (default is temporary)
* - ``admin.vm.device.<class>.Available``
- vm
- device-ident
- `-`
- ``<device-ident> <properties> description=<desc>``
- optional service argument may be used to get info about a single device, optional (device class specific) properties are in ``key=value`` form, `description` must be the last one and is the only one allowed to contain spaces
* - ``admin.pool.List``
- ``dom0``
- `-`
- `-`
- ``<pool>``
-
* - ``admin.pool.ListDrivers``
- ``dom0``
- `-`
- `-`
- ``<pool-driver> <property> ...``
- Properties allowed in ``admin.pool.Add``
* - ``admin.pool.Info``
- ``dom0``
- pool
- `-`
- ``<property>=<value>``
-
* - ``admin.pool.Add``
- ``dom0``
- driver
- ``<property>=<value>``
- `-`
-
* - ``admin.pool.Set.revisions_to_keep``
- ``dom0``
- pool
- ``<value>``
- `-`
-
* - ``admin.pool.Remove``
- ``dom0``
- pool
- `-`
- `-`
-
* - ``admin.pool.volume.List``
- ``dom0``
- pool
- `-`
- volume id
-
* - ``admin.pool.volume.Info``
- ``dom0``
- pool
- vid
- ``<property>=<value>``
-
* - ``admin.pool.volume.Set.revisions_to_keep``
- ``dom0``
- pool
- ``<vid> <value>``
- `-`
-
* - ``admin.pool.volume.ListSnapshots``
- ``dom0``
- pool
- vid
- ``<snapshot>``
-
* - ``admin.pool.volume.Snapshot``
- ``dom0``
- pool
- vid
- snapshot
-
* - ``admin.pool.volume.Revert``
- ``dom0``
- pool
- ``<vid> <snapshot>``
- `-`
-
* - ``admin.pool.volume.Resize``
- ``dom0``
- pool
- ``<vid> <size_in_bytes>``
- `-`
-
* - ``admin.pool.volume.Import``
- ``dom0``
- pool
- ``<vid> <raw volume data>``
- `-`
-
* - ``admin.pool.volume.CloneFrom``
- ``dom0``
- pool
- vid
- token, to be used in ``admin.pool.volume.CloneTo``
- obtain a token to copy volume ``vid`` in ``pool``; the token is one time use only, it's invalidated by ``admin.pool.volume.CloneTo``, even if the operation fails
* - ``admin.pool.volume.CloneTo``
- ``dom0``
- pool
- ``<vid> <token>``
- `-`
- copy volume pointed by a token to volume ``vid`` in ``pool``
* - ``admin.vm.volume.List``
- vm
- `-`
- `-`
- ``<volume>``
- ``<volume>`` is per-VM volume name (``root``, ``private``, etc), ``<vid>`` is pool-unique volume id
* - ``admin.vm.volume.Info``
- vm
- volume
- `-`
- ``<property>=<value>``
-
* - ``admin.vm.volume.Set.revisions_to_keep``
- vm
- volume
- value
- `-`
-
* - ``admin.vm.volume.ListSnapshots``
- vm
- volume
- `-`
- snapshot
- duplicate of ``admin.pool.volume.``, but with other call params
* - ``admin.vm.volume.Snapshot``
- vm
- volume
- `-`
- snapshot
- id.
* - ``admin.vm.volume.Revert``
- vm
- volume
- snapshot
- `-`
- id.
* - ``admin.vm.volume.Resize``
- vm
- volume
- size_in_bytes
- `-`
- id.
* - ``admin.vm.volume.Import``
- vm
- volume
- raw volume data
- `-`
- id.
* - ``admin.vm.volume.ImportWithSize``
- vm
- volume
- ``<size_in_bytes> <raw volume data>``
- `-`
- new version of ``admin.vm.volume.Import``, allows new volume to be different size
* - ``admin.vm.volume.Clear``
- vm
- volume
- `-`
- `-`
- clear contents of volume
* - ``admin.vm.volume.CloneFrom``
- vm
- volume
- `-`
- token, to be used in ``admin.vm.volume.CloneTo``
- obtain a token to copy ``volume`` of ``vm``; the token is one time use only, it's invalidated by ``admin.vm.volume.CloneTo``, even if the operation fails
* - ``admin.vm.volume.CloneTo``
- vm
- volume
- token, obtained with ``admin.vm.volume.CloneFrom``
- `-`
- copy volume pointed by a token to ``volume`` of ``vm``
* - ``admin.vm.CurrentState``
- vm
- `-`
- `-`
- ``<state-property>=<value>``
- state properties: ``power_state``, ``mem``, ``mem_static_max``, ``cputime``
* - ``admin.vm.Start``
- vm
- `-`
- `-`
- `-`
-
* - ``admin.vm.Shutdown``
- vm
- `-`
- `-`
- `-`
-
* - ``admin.vm.Pause``
- vm
- `-`
- `-`
- `-`
-
* - ``admin.vm.Unpause``
- vm
- `-`
- `-`
- `-`
-
* - ``admin.vm.Kill``
- vm
- `-`
- `-`
- `-`
-
* - ``admin.backup.Execute``
- ``dom0``
- config id
- `-`
- `-`
- config in ``/etc/qubes/backup/<id>.conf``, only one backup operation of given ``config id`` can be running at once
* - ``admin.backup.Info``
- ``dom0``
- config id
- `-`
- backup info
- info what would be included in the backup
* - ``admin.backup.Cancel``
- ``dom0``
- config id
- `-`
- `-`
- cancel running backup operation
* - ``admin.Events``
- ``dom0|vm``
- `-`
- `-`
- events
-
* - ``admin.vm.Stats``
- ``dom0|vm``
- `-`
- `-`
- ``vm-stats`` events, see below
- emit VM statistics (CPU, memory usage) in form of events
Volume properties:
- ``pool``
- ``vid``
- ``size``
- ``usage``
- ``rw``
- ``source``
- ``save_on_stop``
- ``snap_on_start``
- ``revisions_to_keep``
- ``is_outdated``
Method ``admin.vm.Stats`` returns ``vm-stats`` events every
``stats_interval`` seconds, for every running VM. Parameters of
``vm-stats`` events:
- ``memory_kb`` - memory usage in kB
- ``cpu_time`` - absolute CPU time (in milliseconds) spent by the VM
since its startup, normalized for one CPU
- ``cpu_usage`` - CPU usage in percents
Returned messages
=================
First byte of a message is a message type. This is 8 bit non-zero
integer. Values start at 0x30 (48, ``'0'``, zero digit in ASCII) for
readability in hexdump. Next byte must be 0x00 (a separator).
This alternatively can be thought of as zero-terminated string
containing single ASCII digit.
OK (0)
------
::
30 00 <content>
Server will close the connection after delivering single message.
EVENT (1)
---------
::
31 00 <subject> 00 <event> 00 ( <key> 00 <value> 00 )* 00
Events are returned as stream of messages in selected API calls.
Normally server will not close the connection.
A method yielding events will not ever return a ``OK`` or ``EXCEPTION``
message.
When calling such method, it will produce an artificial event
``connection-established`` just after connection, to help avoiding race
conditions during event handler registration.
EXCEPTION (2)
-------------
::
32 00 <type> 00 ( <traceback> )? 00 <format string> 00 ( <field> 00 )*
Server will close the connection.
Traceback may be empty, can be enabled server-side as part of debug
mode. Delimiting zero-byte is always present.
Fields are should substituted into ``%``-style format string, possibly
after client-side translation, to form final message to be displayed
unto user. Server does not by itself support translation.
Tags
====
The tags provided can be used to write custom policies. They are not
used in a default Qubes OS installation. However, they are created
anyway.
- ``created-by-<vm>`` — Created in an extension to qubesd at the moment
of creation of the VM. Cannot be changed via API, which is also
enforced by this extension.
- ``managed-by-<vm>`` — Can be used for the same purpose, but it is not
created automatically, nor is it forbidden to set or reset this tag.
Backup profile
==============
Backup-related calls do not allow (yet) to specify what should be
included in the backup. This needs to be configured separately in dom0,
with a backup profile, stored in ``/etc/qubes/backup/<profile>.conf``.
The file use yaml syntax and have following settings:
- ``include`` - list of VMs to include, can also contains tags using
``$tag:some-tag`` syntax or all VMs of given type using
``$type:AppVM``, known from qrexec policy
- ``exclude`` - list of VMs to exclude, after evaluating ``include``
setting
- ``destination_vm`` - VM to which the backup should be send
- ``destination_path`` - path to which backup should be written in
``destination_vm``. This setting is given to ``qubes.Backup`` service
and technically it’s up to it how to interpret it. In current
implementation it is interpreted as a directory where a new file
should be written (with a name based on the current timestamp), or a
command where the backup should be piped to
- ``compression`` - should the backup be compressed (default: True)?
The value can be either ``False`` or ``True`` for default
compression, or a compression command (needs to accept ``-d``
argument for decompression)
- ``passphrase_text`` - passphrase used to encrypt and integrity
protect the backup
- ``passphrase_vm`` - VM which should be asked what backup passphrase
should be used. The asking is performed using
``qubes.BackupPassphrase+profile_name`` service, which is expected to
output chosen passphrase to its stdout. Empty output cancel the
backup operation. This service can be used either to ask the user
interactively, or to have some automated passphrase handling (for
example: generate randomly, then encrypt with a public key and send
somewhere)
Not all settings needs to be set.
Example backup profile:
.. code:: yaml
# Backup only selected VMs
include:
- work
- personal
- vault
- banking
# Store the backup on external disk
destination_vm: sys-usb
destination_path: /media/my-backup-disk
# Use static passphrase
passphrase_text: "My$Very!@Strong23Passphrase"
And slightly more advanced one:
.. code:: yaml
# Include all VMs with a few exceptions
include:
- $type:AppVM
- $type:TemplateVM
- $type:StandaloneVM
exclude:
- untrusted
- $tag:do-not-backup
# parallel gzip for faster backup
compression: pigz
# ask 'vault' VM for the backup passphrase
passphrase_vm: vault
# send the (encrypted) backup directly to remote server
destination_vm: sys-net
destination_path: ncftpput -u my-ftp-username -p my-ftp-pass -c my-ftp-server /directory/for/backups
General notes
=============
- there is no provision for ``qvm-run``, but there already exists
``qubes.VMShell`` call
- generally actions ``*.List`` return a list of objects and have
“object identifier” as first word in a row. Such action can be also
called with “object identifier” in argument to get only a single
entry (in the same format).
- closing qrexec connection normally does *not* interrupt running
operation; this is important to avoid leaving the system in
inconsistent state
- actual operation starts only after caller send all the parameters
(including a payload), signaled by sending EOF mark; there is no
support for interactive protocols, to keep the protocol reasonable
simple
Policy admin API
================
There is also an API to view and update :doc:`Qubes RPC policy files </developer/services/qrexec>` in dom0. All of the following calls have dom0 as
destination:
+------------------+----------+------------------+------------------+
| call | argument | inside | return |
+==================+==========+==================+==================+
| ``policy.List`` | - | - | ``<name1> |
| ``polic | | | \n<name2>\n...`` |
| y.include.List`` | | | |
+------------------+----------+------------------+------------------+
| ``policy.Get`` | name | - | ``<tok |
| ``poli | | | en>\n<content>`` |
| cy.include.Get`` | | | |
+------------------+----------+------------------+------------------+
| `` | name | ``<tok | - |
| policy.Replace`` | | en>\n<content>`` | |
| ``policy.i | | | |
| nclude.Replace`` | | | |
+------------------+----------+------------------+------------------+
| ` | name | ``<token>`` | - |
| `policy.Remove`` | | | |
| ``policy. | | | |
| include.Remove`` | | | |
+------------------+----------+------------------+------------------+
The ``policy.*`` calls refer to main policy files
(``/etc/qubes/policy.d/``), and the ``policy.include.*`` calls refer to
the include directory (``/etc/qubes/policy.d/include/``). The
``.policy`` extension for files in the main directory is always omitted.
The responses do not follow admin API protocol, but signal error using
an exit code and a message on stdout.
The changes are validated before saving, so that the policy cannot end
up in an invalid state (e.g. syntax error, missing include file).
In addition, there is a mechanism to prevent concurrent modifications of
the policy files:
- A ``*.Get`` call returns a file along with a *token* (currently
implemented as a hash of the file).
- When calling ``Replace`` or ``Remove``, you need to include the
current token as first line. If the token does not match, the
modification will fail.
- When adding a new file using ``Replace``, pass ``new`` as token. This
will ensure that the file does not exist before adding.
- To skip the check, pass ``any`` as token.
TODO
====
- notifications
- how to constrain the events?
- how to pass the parameters? maybe XML, since this is trusted
anyway and parser may be complicated
- how to constrain the possible values for ``admin.vm.property.Set``
etc, like “you can change ``netvm``, but you have to pick from this
set”; this currently can be done by writing an extension
- a call for executing ``*.desktop`` file from
``/usr/share/applications``, for use with appmenus without giving
access to ``qubes.VMShell``; currently this can be done by writing
custom qrexec calls
- maybe some generator for ``.desktop`` for appmenus, which would wrap
calls in ``qrexec-client-vm``
.. raw:: html
<!-- vim: set ts=4 sts=4 sw=4 et : -->

58
developer/services/disposablevm-implementation.md
View File

@ -1,58 +0,0 @@
---
lang: en
layout: doc
permalink: /doc/disposablevm-implementation/
redirect_from:
- /doc/dvm-impl/
- /en/doc/dvm-impl/
- /doc/DVMimpl/
- /wiki/DVMimpl/
ref: 34
title: Disposable implementation
---
**Note: The content below applies to Qubes R3.2.**
DisposableVM image preparation
------------------------------
DisposableVM is not started like other VMs, by executing equivalent of `xl create` - it would be too slow. Instead, DisposableVM are started by restore from a savefile.
Preparing a savefile is done by `/usr/lib/qubes/qubes_prepare_saved_domain.sh` script. It takes two mandatory arguments, appvm name (APPVM) and the savefile name, and optional path to "prerun" script. The script executes the following steps:
1. APPVM is started by `qvm-start`
2. xenstore key `/local/domain/appvm_domain_id/qubes_save_request` is created
3. if prerun script was specified, copy it to `qubes_save_script` xenstore key
4. wait for the `qubes_used_mem` key to appear
5. (in APPVM) APPVM boots normally, up to the point in `/etc/init.d/qubes_core` script when the presence of `qubes_save_request` key is tested. If it exists, then
1. (in APPVM) if exists, prerun script is retrieved from the respective xenstore key and executed. This preloads filesystem cache with useful applications, so that they will start faster.
2. (in APPVM) the amount of used memory is stored to `qubes_used_mem` xenstore key
3. (in APPVM) busy-waiting for `qubes_restore_complete` xenstore key to appear
6. when `qubes_used_mem` key appears, the domain memory is reduced to this amount, to make the savefile smaller.
7. APPVM private image is detached
8. the domain is saved via `xl save`
9. the COW file volatile.img (cow for root fs and swap) is packed to `saved_cows.tar` archive
The `qubes_prepare_saved_domain.sh` script is lowlevel. It is usually called by `qvm-create-default-dvm` script, that takes care of creating a special AppVM (named template\_name-dvm) to be passed to `qubes_prepare_saved_domain.sh`, as well as copying the savefile to /dev/shm (the latter action is not done if the `/var/lib/qubes/dvmdata/dont_use_shm` file exists).
Restoring a DisposableVM from the savefile
------------------------------------------
Normally, disposable VM is created when qubes rpc request with target *\$dispvm* is received. Then, as a part of rpc connection setup, the `qfile-daemon-dvm` program is executed; it executes `/usr/lib/qubes/qubes_restore` program. It is crucial that this program executes quickly, to make DisposableVM creation overhead bearable for the user. Its main steps are:
1. modify the savefile so that the VM name, VM UUID, MAC address and IP address are unique
2. restore the COW files from the `saved_cows.tar`
3. create the `/var/run/qubes/fast_block_attach` file, whose presence tells the `/etc/xen/scripts/block` script to bypass some redundant checks and execute as fast as possible.
4. execute `xl restore` in order to restore a domain.
5. create the same xenstore keys as normally created when AppVM boots (e.g. `qubes_ip`)
6. create the `qubes_restore_complete` xenstore key. This allows the boot process in DisposableVM to continue.
The actual passing of files between AppVM and a DisposableVM is implemented via qubes rpc.
Validating the DisposableVM savefile
------------------------------------
DisposableVM savefile contains references to template rootfs and to COW files. The COW files are restored before each DisposableVM start, so they cannot change. On the other hand, if templateVM is started, the template rootfs will change, and it may not be coherent with the COW files.
Therefore, the check for template rootfs modification time being older than DisposableVM savefile modification time is required. It is done in `qfilexchgd` daemon, just before restoring DisposableVM. If necessary, an attempt is made to recreate the DisposableVM savefile, using the last template used (or default template, if run for the first time) and the default prerun script, residing at `/var/lib/qubes/vm-templates/templatename/dispvm_prerun.sh`. Unfortunately, the prerun script takes a lot of time to execute - therefore, after template rootfs modification, the next DisposableVM creation can be longer by about 2.5 minutes.

119
developer/services/disposablevm-implementation.rst Normal file
View File

@ -0,0 +1,119 @@
=========================
Disposable implementation
=========================
**Note: The content below applies to Qubes R3.2.**
DisposableVM image preparation
------------------------------
DisposableVM is not started like other VMs, by executing equivalent of
``xl create`` - it would be too slow. Instead, DisposableVM are started
by restore from a savefile.
Preparing a savefile is done by
``/usr/lib/qubes/qubes_prepare_saved_domain.sh`` script. It takes two
mandatory arguments, appvm name (APPVM) and the savefile name, and
optional path to “prerun” script. The script executes the following
steps:
1. APPVM is started by ``qvm-start``
2. xenstore key ``/local/domain/appvm_domain_id/qubes_save_request`` is
created
3. if prerun script was specified, copy it to ``qubes_save_script``
xenstore key
4. wait for the ``qubes_used_mem`` key to appear
5. (in APPVM) APPVM boots normally, up to the point in
``/etc/init.d/qubes_core`` script when the presence of
``qubes_save_request`` key is tested. If it exists, then
1. (in APPVM) if exists, prerun script is retrieved from the
respective xenstore key and executed. This preloads filesystem
cache with useful applications, so that they will start faster.
2. (in APPVM) the amount of used memory is stored to
``qubes_used_mem`` xenstore key
3. (in APPVM) busy-waiting for ``qubes_restore_complete`` xenstore
key to appear
6. when ``qubes_used_mem`` key appears, the domain memory is reduced to
this amount, to make the savefile smaller.
7. APPVM private image is detached
8. the domain is saved via ``xl save``
9. the COW file volatile.img (cow for root fs and swap) is packed to
``saved_cows.tar`` archive
The ``qubes_prepare_saved_domain.sh`` script is lowlevel. It is usually
called by ``qvm-create-default-dvm`` script, that takes care of creating
a special AppVM (named template_name-dvm) to be passed to
``qubes_prepare_saved_domain.sh``, as well as copying the savefile to
/dev/shm (the latter action is not done if the
``/var/lib/qubes/dvmdata/dont_use_shm`` file exists).
Restoring a DisposableVM from the savefile
------------------------------------------
Normally, disposable VM is created when qubes rpc request with target
*$dispvm* is received. Then, as a part of rpc connection setup, the
``qfile-daemon-dvm`` program is executed; it executes
``/usr/lib/qubes/qubes_restore`` program. It is crucial that this
program executes quickly, to make DisposableVM creation overhead
bearable for the user. Its main steps are:
1. modify the savefile so that the VM name, VM UUID, MAC address and IP
address are unique
2. restore the COW files from the ``saved_cows.tar``
3. create the ``/var/run/qubes/fast_block_attach`` file, whose presence
tells the ``/etc/xen/scripts/block`` script to bypass some redundant
checks and execute as fast as possible.
4. execute ``xl restore`` in order to restore a domain.
5. create the same xenstore keys as normally created when AppVM boots
(e.g. ``qubes_ip``)
6. create the ``qubes_restore_complete`` xenstore key. This allows the
boot process in DisposableVM to continue.
The actual passing of files between AppVM and a DisposableVM is
implemented via qubes rpc.
Validating the DisposableVM savefile
------------------------------------
DisposableVM savefile contains references to template rootfs and to COW
files. The COW files are restored before each DisposableVM start, so
they cannot change. On the other hand, if templateVM is started, the
template rootfs will change, and it may not be coherent with the COW
files.
Therefore, the check for template rootfs modification time being older
than DisposableVM savefile modification time is required. It is done in
``qfilexchgd`` daemon, just before restoring DisposableVM. If necessary,
an attempt is made to recreate the DisposableVM savefile, using the last
template used (or default template, if run for the first time) and the
default prerun script, residing at
``/var/lib/qubes/vm-templates/templatename/dispvm_prerun.sh``.
Unfortunately, the prerun script takes a lot of time to execute -
therefore, after template rootfs modification, the next DisposableVM
creation can be longer by about 2.5 minutes.

49
developer/services/dom0-secure-updates.md
View File

@ -1,49 +0,0 @@
---
lang: en
layout: doc
permalink: /doc/dom0-secure-updates/
redirect_from:
- /en/doc/dom0-secure-updates/
- /doc/Dom0SecureUpdates/
- /wiki/Dom0SecureUpdates/
ref: 43
title: Dom0 secure updates
---
Reasons for Dom0 updates
------------------------
Normally there should be few reasons for updating software in Dom0. This is because there is no networking in Dom0, which means that even if some bugs will be discovered e.g. in the Dom0 Desktop Manager, this really is not a problem for Qubes, because all the third-party software running in Dom0 is not accessible from VMs or network in any way. Some exceptions to the above include: Qubes GUI daemon, Xen store daemon, and disk back-ends (we plan to move the disk backends to untrusted domain in Qubes 2.0). Of course we believe this software is reasonably secure and we hope it will not need patching.
However, we anticipate some other situations when updating Dom0 software might be required:
- Updating drivers/libs for new hardware support
- Correcting non-security related bugs (e.g. new buttons for qubes manager)
- Adding new features (e.g. GUI backup tool)
Problems with traditional network-based update mechanisms
---------------------------------------------------------
Dom0 is the most trusted domain on Qubes OS, and for this reason we decided to design Qubes in such a way that Dom0 is not connected to any network. In fact only certain domains can be connected to a network via so-called network domains. There can also be more than one network domain, e.g. in case the user is connected to more than one physically or logically separated networks.
Secure update mechanism we use in Qubes (starting from Beta 2)
-------------------------------------------------------------
Keeping Dom0 not connected to any network makes it hard, however, to provide updates for software in Dom0. For this reason we have come up with the following mechanism for Dom0 updates, which minimizes the amount of untrusted input processed by Dom0 software:
The update process is initiated by [qubes-dom0-update script](https://github.com/QubesOS/qubes-core-admin-linux/blob/release2/dom0-updates/qubes-dom0-update), running in Dom0.
Updates (`*.rpm` files) are checked and downloaded by UpdateVM, which by default is the same as the firewall VM, but can be configured to be any other, network-connected VM. This is done by [qubes-download-dom0-updates.sh script](https://github.com/QubesOS/qubes-core-agent-linux/blob/release2/misc/qubes-download-dom0-updates.sh) (this script is executed using qrexec by the previously mentioned qubes-dom0-update). Note that we assume that this script might get compromised and fetch maliciously compromised downloads -- this is not a problem as Dom0 verifies digital signatures on updates later. The downloaded rpm files are placed in a ~~~/var/lib/qubes/dom0-updates~~~ directory on UpdateVM filesystem (again, they might get compromised while being kept there, still this isn't a problem). This directory is passed to yum using the ~~~--installroot=~~~ option.
Once updates are downloaded, the update script that runs in UpdateVM requests an RPM service [qubes.ReceiveUpdates](https://github.com/QubesOS/qubes-core-admin-linux/blob/release2/dom0-updates/qubes.ReceiveUpdates) to be executed in Dom0. This service is implemented by [qubes-receive-updates script](https://github.com/QubesOS/qubes-core-admin-linux/blob/release2/dom0-updates/qubes-receive-updates) running in Dom0. The Dom0's qubes-dom0-update script (which originally initiated the whole update process) waits until qubes-receive-updates finished.
The qubes-receive-updates script processes the untrusted input from Update VM: it first extracts the received `*.rpm` files (that are sent over qrexec data connection) and then verifies digital signature on each file. The qubes-receive-updates script is a security-critical component of the Dom0 update process (as is the [qfile-dom0-unpacker.c](https://github.com/QubesOS/qubes-core-admin-linux/blob/release2/dom0-updates/qfile-dom0-unpacker.c) and the rpm utility, both used by the qubes-receive-updates for processing the untrusted input).
Once qubes-receive-updates finished unpacking and verifying the updates, the updates are placed in ``qubes-receive-updates`` directory in Dom0 filesystem. Those updates are now trusted. Dom0 is configured (see /etc/yum.conf in Dom0) to use this directory as a default (and only) [yum repository](https://github.com/QubesOS/qubes-core-admin-linux/blob/release2/dom0-updates/qubes-cached.repo).
Finally, qubes-dom0-update runs ``yum update`` that fetches the RPMs from qubes-cached repo and installs them as usual.
Security benefit of our update mechanism
----------------------------------------
The benefit of the above scheme is that one doesn't need to trust the TCP/IP stack, the HTTP stack, and wget implementation in order to safely deliver and install updates in Dom0. One only needs to trust a few hundred lines of code as discussed above, as well as the rpm utility for properly checking digital signature (but this is required in any update scheme).

105
developer/services/dom0-secure-updates.rst Normal file
View File

@ -0,0 +1,105 @@
===================
Dom0 secure updates
===================
Reasons for Dom0 updates
------------------------
Normally there should be few reasons for updating software in Dom0. This
is because there is no networking in Dom0, which means that even if some
bugs will be discovered e.g. in the Dom0 Desktop Manager, this really is
not a problem for Qubes, because all the third-party software running in
Dom0 is not accessible from VMs or network in any way. Some exceptions
to the above include: Qubes GUI daemon, Xen store daemon, and disk
back-ends (we plan to move the disk backends to untrusted domain in
Qubes 2.0). Of course we believe this software is reasonably secure and
we hope it will not need patching.
However, we anticipate some other situations when updating Dom0 software
might be required:
- Updating drivers/libs for new hardware support
- Correcting non-security related bugs (e.g. new buttons for qubes
manager)
- Adding new features (e.g. GUI backup tool)
Problems with traditional network-based update mechanisms
---------------------------------------------------------
Dom0 is the most trusted domain on Qubes OS, and for this reason we
decided to design Qubes in such a way that Dom0 is not connected to any
network. In fact only certain domains can be connected to a network via
so-called network domains. There can also be more than one network
domain, e.g. in case the user is connected to more than one physically
or logically separated networks.
Secure update mechanism we use in Qubes (starting from Beta 2)
--------------------------------------------------------------
Keeping Dom0 not connected to any network makes it hard, however, to
provide updates for software in Dom0. For this reason we have come up
with the following mechanism for Dom0 updates, which minimizes the
amount of untrusted input processed by Dom0 software:
The update process is initiated by `qubes-dom0-update script <https://github.com/QubesOS/qubes-core-admin-linux/blob/release2/dom0-updates/qubes-dom0-update>`__,
running in Dom0.
Updates (``*.rpm`` files) are checked and downloaded by UpdateVM, which
by default is the same as the firewall VM, but can be configured to be
any other, network-connected VM. This is done by
`qubes-download-dom0-updates.sh script <https://github.com/QubesOS/qubes-core-agent-linux/blob/release2/misc/qubes-download-dom0-updates.sh>`__
(this script is executed using qrexec by the previously mentioned
qubes-dom0-update). Note that we assume that this script might get
compromised and fetch maliciously compromised downloads – this is not a
problem as Dom0 verifies digital signatures on updates later. The
downloaded rpm files are placed in a
``/var/lib/qubes/dom0-updates`` directory on UpdateVM
filesystem (again, they might get compromised while being kept there,
still this isn’t a problem). This directory is passed to yum using the
``–installroot=`` option.
Once updates are downloaded, the update script that runs in UpdateVM
requests an RPM service
`qubes.ReceiveUpdates <https://github.com/QubesOS/qubes-core-admin-linux/blob/release2/dom0-updates/qubes.ReceiveUpdates>`__
to be executed in Dom0. This service is implemented by
`qubes-receive-updates script <https://github.com/QubesOS/qubes-core-admin-linux/blob/release2/dom0-updates/qubes-receive-updates>`__
running in Dom0. The Dom0’s qubes-dom0-update script (which originally
initiated the whole update process) waits until qubes-receive-updates
finished.
The qubes-receive-updates script processes the untrusted input from
Update VM: it first extracts the received ``*.rpm`` files (that are sent
over qrexec data connection) and then verifies digital signature on each
file. The qubes-receive-updates script is a security-critical component
of the Dom0 update process (as is the
`qfile-dom0-unpacker.c <https://github.com/QubesOS/qubes-core-admin-linux/blob/release2/dom0-updates/qfile-dom0-unpacker.c>`__
and the rpm utility, both used by the qubes-receive-updates for
processing the untrusted input).
Once qubes-receive-updates finished unpacking and verifying the updates,
the updates are placed in ``qubes-receive-updates`` directory in Dom0
filesystem. Those updates are now trusted. Dom0 is configured (see
/etc/yum.conf in Dom0) to use this directory as a default (and only)
`yum repository <https://github.com/QubesOS/qubes-core-admin-linux/blob/release2/dom0-updates/qubes-cached.repo>`__.
Finally, qubes-dom0-update runs ``yum update`` that fetches the RPMs
from qubes-cached repo and installs them as usual.
Security benefit of our update mechanism
----------------------------------------
The benefit of the above scheme is that one doesn’t need to trust the
TCP/IP stack, the HTTP stack, and wget implementation in order to safely
deliver and install updates in Dom0. One only needs to trust a few
hundred lines of code as discussed above, as well as the rpm utility for
properly checking digital signature (but this is required in any update
scheme).

29
developer/services/qfilecopy.md
View File

@ -1,29 +0,0 @@
---
lang: en
layout: doc
permalink: /doc/qfilecopy/
redirect_from:
- /en/doc/qfilecopy/
- /doc/Qfilecopy/
- /wiki/Qfilecopy/
ref: 35
title: Inter-qube file copying (qfilecopy)
---
There are two cases when we need a mechanism to copy files between VMs:
- "regular" file copy - when user instructs file manager to copy a given files/directories to a different VM
- DispVM copy - user selects "open in DispVM" on a file; this file must be copied to a DisposableVM, edited by user, and possibly a modified file copied back from DispVM to VM.
In the early days of Qubes OS, for both cases, a block device (backed by a file in dom0 with a vfat filesystem on it) was attached to VM, file(s) copied there, and then the device was detached and attached to target VM. In the DispVM case, if a edited file has been modified, another block device is passed to requester VM in order to update the source file.
This has the following disadvantages:
- performance - dom0 has to prepare and attach/detach block devices, which is slow because of hotplug scripts involvement.
- security - VM kernel parses partition table and filesystem metadata from the block device; they are controlled by (potentially untrusted) sender VM.
In modern Qubes OS releases, we have reimplemented interVM file copy using qrexec, which addresses the above mentioned disadvantages. Nowadays, even more generic solution (qubes rpc) is used. See the developer docs on qrexec and qubes rpc. In a nutshell, the file sender and the file receiver just read/write from stdin/stdout, and the qubes rpc layer passes data properly - so, no block devices are used.
The rpc action for regular file copy is *qubes.Filecopy*, the rpc client is named *qfile-agent*, the rpc server is named *qfile-unpacker*. For DispVM copy, the rpc action is *qubes.OpenInVM*, the rpc client is named *qopen-in-vm*, rpc server is named *vm-file-editor*. Note that the qubes.OpenInVM action can be done on a normal AppVM, too.
Being a rpc server, *qfile-unpacker* must be coded securely, as it processes potentially untrusted data format. Particularly, we do not want to use external tar or cpio and be prone to all vulnerabilities in them; we want a simplified, small utility, that handles only directory/file/symlink file type, permissions, mtime/atime, and assume user/user ownership. In the current implementation, the code that actually parses the data from srcVM has ca 100 lines of code and executes chrooted to the destination directory. The latter is hardcoded to `~user/QubesIncoming/srcVM`; because of chroot, there is no possibility to alter files outside of this directory.

57
developer/services/qfilecopy.rst Normal file
View File

@ -0,0 +1,57 @@
===================================
Inter-qube file copying (qfilecopy)
===================================
There are two cases when we need a mechanism to copy files between VMs:
- “regular” file copy - when user instructs file manager to copy a
given files/directories to a different VM
- DispVM copy - user selects “open in DispVM” on a file; this file must
be copied to a DisposableVM, edited by user, and possibly a modified
file copied back from DispVM to VM.
In the early days of Qubes OS, for both cases, a block device (backed by
a file in dom0 with a vfat filesystem on it) was attached to VM, file(s)
copied there, and then the device was detached and attached to target
VM. In the DispVM case, if a edited file has been modified, another
block device is passed to requester VM in order to update the source
file.
This has the following disadvantages:
- performance - dom0 has to prepare and attach/detach block devices,
which is slow because of hotplug scripts involvement.
- security - VM kernel parses partition table and filesystem metadata
from the block device; they are controlled by (potentially untrusted)
sender VM.
In modern Qubes OS releases, we have reimplemented interVM file copy
using qrexec, which addresses the above mentioned disadvantages.
Nowadays, even more generic solution (qubes rpc) is used. See the
developer docs on qrexec and qubes rpc. In a nutshell, the file sender
and the file receiver just read/write from stdin/stdout, and the qubes
rpc layer passes data properly - so, no block devices are used.
The rpc action for regular file copy is *qubes.Filecopy*, the rpc client
is named *qfile-agent*, the rpc server is named *qfile-unpacker*. For
DispVM copy, the rpc action is *qubes.OpenInVM*, the rpc client is named
*qopen-in-vm*, rpc server is named *vm-file-editor*. Note that the
qubes.OpenInVM action can be done on a normal AppVM, too.
Being a rpc server, *qfile-unpacker* must be coded securely, as it
processes potentially untrusted data format. Particularly, we do not
want to use external tar or cpio and be prone to all vulnerabilities in
them; we want a simplified, small utility, that handles only
directory/file/symlink file type, permissions, mtime/atime, and assume
user/user ownership. In the current implementation, the code that
actually parses the data from srcVM has ca 100 lines of code and
executes chrooted to the destination directory. The latter is hardcoded
to ``~user/QubesIncoming/srcVM``; because of chroot, there is no
possibility to alter files outside of this directory.

Some files were not shown because too many files have changed in this diff Show More