Clean up and sort developer documentation

This commit is contained in:
Axon 2015-09-26 03:16:18 +00:00
parent bd877ddc54
commit 6345b676cd
No known key found for this signature in database
GPG key ID: 8CE137352A019A17
35 changed files with 55 additions and 73 deletions

View file

@ -0,0 +1,230 @@
---
layout: doc
title: BuildingArchlinuxTemplate
permalink: /doc/BuildingArchlinuxTemplate/
redirect_from: /wiki/BuildingArchlinuxTemplate/
---
Template building
=================
The archlinux VM is now almost working as a NetVM. Based on qubes-builder code, you could find below how to build it and problem that could arise from template building to using archlinux as a netvm:
Download qubes-builder git code
-------------------------------
Prefer marmarek git repository as it is the most recent one
Change your builder.conf
------------------------
Change the following variables GIT\_SUBDIR=marmarek DISTS\_VM=archlinux
Get all required sources
------------------------
```
make get-sources
```
Note that make get-sources sometimes fails because it didn't download packages that are not used by archlinux such as xfce or kde packages.
You can ignore the repositories that are failing by adding the following line to your builder.conf:
```
COMPONENTS:=$(filter-out desktop-linux-kde desktop-linux-xfce,$(COMPONENTS))
```
Just don't forget that you need to comment this line again if you want to build the whole Qubes-OS install CD.
Make all required qubes components
----------------------------------
The first use of the builder can take several hours depending on your bandwidth as it will install an archlinux chroot:
```
make vmm-xen-vm
make core-vchan-xen-vm
make linux-utils-vm
make core-agent-linux-vm
make gui-common-vm
make gui-agent-linux-vm
```
Now build the template itself
-----------------------------
This can take again several hours, especially the first time you built and archlinux template:
```
make linux-template-builder
```
Retrieve your template
----------------------
You can now find your template in qubes-src/linux-template-builder/rpm/noarch. Install it in dom0 (just take care as it will replace your current archlinux-x64 template)
* * * * *
Known problems during building or when running the VM
=====================================================
Can't open file archlinux-2013.02.01-dual.iso
---------------------------------------------
Archlinux ISO files are sometimes removed from mirrors. Check the last version available on the archlinux mirror (eg: [http://mir.archlinux.fr/iso/](http://mir.archlinux.fr/iso/)), and update qubes-src/linux-template-builder/scripts\_archlinux/00\_prepare.sh accordingly:
```
ISO_VERSION=2013.06.01
```
You will also need to download the signature matching this ISO version inside qubes-src/linux-template-builder/scripts\_archlinux/:
```
wget http://mir.archlinux.fr/iso/2013.06.01/archlinux-2013.06.01-dual.iso.sig
```
The nm-applet (network manager icon) fails to start when archlinux is defined as a template-vm:
-----------------------------------------------------------------------------------------------
In fact /etc/dbus-1/system.d/org.freedesktop.NetworkManager.conf does not allow a standard user to run network manager clients. To allow this, one need to change inside \<policy context="default"\>:
```
<deny send_destination="org.freedesktop.NetworkManager"/>
```
to
```
<allow send_destination="org.freedesktop.NetworkManager"/>
```
DispVM, Yum proxy and most Qubes addons (thunderbird ...) have not been tested at all.
--------------------------------------------------------------------------------------
The sound does not work in AppVMs and there are messages related to pulse segfault in glibc when running dmesg (FIXED)
----------------------------------------------------------------------------------------------------------------------
This is apparently a bug in Archlinux between glibc and pulseaudio package 4.0-6. The packages pulseaudio-4.0-2 and libpulse-4.0-2 are known to work and can be downloaded and reinstalled manually.
Error when building the gui-agent-linux with pulsecore error
------------------------------------------------------------
```
module-vchan-sink.c:62:34: fatal error: pulsecore/core-error.h: No such file or directory
#include <pulsecore/core-error.h>
```
This error is because Archlinux update package too quickly. Probably, a new version of pulseaudio has been released, but the qubes team has not imported the new development headers yet.
You can create fake new headers just by copying the old headers:
```
cd qubes-builder/qubes-src/gui-agent-linux/pulse
ls
cp -r pulsecore-#lastversion pulsecore-#archlinuxversion
```
You can check the current archlinux pulseaudio version like this:
```
sudo chroot chroot-archlinux/ pacman -Qi pulseaudio
```
chroot-archlinux/dev/pts has not been unmounted
-----------------------------------------------
This is a known problem when there are errors during building. Check what is mounted using the command mount (with no parameters). Just unmount what you can (or reboot your vm if you are too lazy :) )
Known problems in Qubes R2-B2
=============================
xen-vmm-vm fail to build with a PARSETUPLE related error (FIXED):
-----------------------------------------------------------------
Commenting out "\#define HAVE\_ATTRIBUTE\_FORMAT\_PARSETUPLE" from chroot\_archlinux/usr/include/python2.7/pyconfig.h fixes the problem, but it isn't the right solution [1]...
A better fix is planned for the next python release (the bug is considered release blocking), and will be updated in archlinux chroot as soon as available.
[1] [http://bugs.python.org/issue17547](http://bugs.python.org/issue17547)
The boot process fails without visible errors in the logs, but spawn a recovery shell
-------------------------------------------------------------------------------------
The problem is a new conflict between systemd and the old sysvinit style. To fix this, you can change the master xen template in dom0 to remove sysvinit remains: Edit **INSIDE DOM0** /usr/share/qubes/vm-template.conf, and change the variable 'extra' that contains the kernel variables: from:
```
extra="ro nomodeset 3 console=hvc0 rd_NO_PLYMOUTH {kernelopts}"
```
to:
```
extra="ro nomodeset console=hvc0 rd_NO_PLYMOUTH {kernelopts}"
```
Qubes-OS is now using different xenstore variable names, which makes to archlinux VM failing to boot
----------------------------------------------------------------------------------------------------
Apply the following fix in the template to revert the variable name to the old Qubes version.
You can edit the template the following way:
```
sudo mkdir /mnt/vm
sudo mount /var/lib/qubes/vm-templates/archlinux-x64/root.img /mnt/vm
sudo chroot /mnt/vm
```
Then apply the fix:
```
sudo sed 's:qubes-keyboard:qubes_keyboard:g' -i /etc/X11/xinit/xinitrc.d/qubes-keymap.sh
sudo sed 's:qubes-netvm-domid:qubes_netvm_domid:g' -i /etc/NetworkManager/dispatcher.d/30-qubes-external-ip
sudo sed 's:qubes-netvm-external-ip:qubes_netvm_external_ip:g' -i /etc/NetworkManager/dispatcher.d/30-qubes-external-ip
sudo sed 's:qubes-netvm-network:qubes_netvm_network:g' -i /usr/lib/qubes/init/network-proxy-setup.sh
sudo sed 's:qubes-netvm-gateway:qubes_netvm_gateway:g' -i /usr/lib/qubes/init/network-proxy-setup.sh
sudo sed 's:qubes-netvm-netmask:qubes_netvm_netmask:g' -i /usr/lib/qubes/init/network-proxy-setup.sh
sudo sed 's:qubes-netvm-secondary-dns:qubes_netvm_secondary_dns:g' -i /usr/lib/qubes/init/network-proxy-setup.sh
sudo sed 's:qubes-vm-type:qubes_vm_type:g' -i /usr/lib/qubes/init/qubes-sysinit.sh
sudo sed 's:qubes-ip:qubes_ip:g' -i /usr/lib/qubes/setup-ip
sudo sed 's:qubes-netmask:qubes_netmask:g' -i /usr/lib/qubes/setup-ip
sudo sed 's:qubes-gateway:qubes_gateway:g' -i /usr/lib/qubes/setup-ip
sudo sed 's:qubes-secondary-dns:qubes_secondary_dns:g' -i /usr/lib/qubes/setup-ip
sudo sed 's:qubes-netvm-network:qubes_netvm_network:g' -i /usr/lib/qubes/setup-ip
sudo sed 's:qubes-netvm-gateway:qubes_netvm_gateway:g' -i /usr/lib/qubes/setup-ip
sudo sed 's:qubes-netvm-netmask:qubes_netvm_netmask:g' -i /usr/lib/qubes/setup-ip
sudo sed 's:qubes-netvm-secondary-dns:qubes_netvm_secondary_dns:g' -i /usr/lib/qubes/setup-ip
sudo sed 's:qubes-iptables-domainrules:qubes_iptables_domainrules:g' -i /usr/bin/qubes-firewall
sudo sed 's:qubes-iptables-header:qubes_iptables_header:g' -i /usr/bin/qubes-firewall
sudo sed 's:qubes-iptables-error:qubes_iptables_error:g' -i /usr/bin/qubes-firewall
sudo sed 's:qubes-iptables:qubes_iptables:g' -i /usr/bin/qubes-firewall
sudo sed 's:qubes-netvm-domid:qubes_netvm_domid:g' -i /usr/bin/qubes-netwatcher
sudo sed 's:qubes-netvm-external-ip:qubes_netvm_external_ip:g' -i /usr/bin/qubes-netwatcher
sudo sed 's:qubes-vm-updateable:qubes_vm_updateable:g' -i /usr/lib/qubes/qubes_trigger_sync_appmenus.sh
sudo sed 's:qubes-vm-type:qubes_vm_type:g' -i /usr/bin/qubes-session
sudo sed 's:qubes-vm-updateable:qubes_vm_updateable:g' -i /usr/bin/qubes-session
```
Do not forgot to:
```
umount /mnt/vm
```
Installing the template in dom0 fails because of a missing dependency (qubes-core-dom0-linux)
---------------------------------------------------------------------------------------------
Again you built a template based on a recent Qubes API which has not been released yet. So skip the dependency for now:
```
sudo rpm -U --nodeps yourpackage.rpm
```

View file

@ -0,0 +1,157 @@
---
layout: doc
title: BuildingNonFedoraTemplate
permalink: /doc/BuildingNonFedoraTemplate/
redirect_from: /wiki/BuildingNonFedoraTemplate/
---
Building a TemplateVM for ArchLinux (or another non fedora OS)
==============================================================
If you don't like using Fedora because of specific administration or package management / building needs, you could build a VM Template for your Distribution of choice.
This article shows how to build a template for a different OS, taking ArchLinux as an example.
Qubes builder scripts
=====================
You can start creating Qubes builder scripts for your new OS. Just note that it will probably make your testing process harder than trying to build the package directly in an HVM on which you already installed this new OS.
chroot initialization
---------------------
You need to install your OS inside a chroot that will be used to build all the required qubes agents of tools.
The scripts you will be interested in will be inside the qubes-src/linux-template-builder project:
```
scripts_fedora
scripts_archlinux
scripts_yourOSname
```
### 00\_prepare.sh
The goal of the first script 00\_prepare.sh is to download and verify the signature of the installation cd and tools. You can use the \$CACHEDIR directory variable to store files that could be reused (such as downloaded scripts or iso files)
### 01\_install\_core.sh
The goal of this script is to install a base environment of your target OS inside the \$INSTALLDIR directory variable. Generally you need to bootstrap/install your package manager inside the \$INSTALLDIR directory and install the base packages.
### Testing the installation process
Edit the builder.conf file to change the variable DISTS\_VM to your OS name (DISTS\_VM=your\_os\_name). The try to make the template to check that at least these to first scripts are working correctly:
```
make linux-template-builder
```
Qubes builder Makefiles
-----------------------
Now you need to create Makefiles specific to your OS. You will find the required scripts directly inside qubes-builder:
```
prepare-chroot-yourOSname
Makefile.yourOSname
```
### prepare-chroot-yourOSname
The goal of this file is to prepare a development environment of your target OS inside a chroot. You will reuse there the 00\_prepare.sh and 01\_install\_core.sh scripts. Additionally, the following things will be necessary to use to chroot environment:
- the \$1 variable will contain the installation directory (INSTALLDIR should contain the same value than \$1 when you run 00\_prepare or 01\_install\_core)
- after your base system is installed, you should install development tools and libraries (gcc, make, ...)
- create a user called 'user' inside your chroot, and give him enought right to run the command sudo without any password
- register all the repository that could be necessary and synchronize the package database
- register a custom repository that will be used to store qubes packages
### Makefile.yourOSname
This file will be used to define the action required when installing a package. The most important one are:
- dist-prepare-chroot: that's where you will call prepare-chroot-yourOSname if the chroot has not been initialized
- dist-package: that's where you will chroot the development environment and run the command used to build a package.
- dist-build-dep: that's where you will create the custom repository for your target OS based on already compiled packages.
These additional target need to exist once you created your first packages:
- dist-copy-out: that's where you will retrieve the package you just built and put it with all the other package you prepared.
- update-repo: that's where you will retrieve the package that have been built and that need to be installed inside the template
### Testing the development chroot
You will be able to test these script when making the first qubes packages. Don't forget that the first things that run when running 'make somcomponent-vm' will be these two scripts, and that you will need to debug it at this point.
Qubes packages
--------------
### vmm-xen-vm
### core-vchan-xen-vm
### linux-utils-vm
### core-agent-linux-vm
### gui-common-vm
### gui-agent-linux-vm
Additional Installation scripts
-------------------------------
Again you need to work on scripts inside the qubes-src/linux-template-builder project:
```
scripts_fedora
scripts_archlinux
scripts_yourOSname
```
### 02\_install\_groups.sh
The goal of this script is to install all the package that you want to use in your template (eg: firefox, thunderbird, a file manager, Xorg...)
### 04\_install\_qubes.sh
The goal of this script is to install in your template all the packages you built previously. Also you need to edit the fstab file of your template to mount qubes virtual hard drives.
### 09\_cleanup.sh
This script is use to finalize and to remove unnecessary things from your template, such as cached packages, unused development packages ...
Starting with an HVM
====================
If no Qubes packages are available for your selected OS. You could start to install an HVM with your OS. Your goals will be:
- to identify how to install the OS using command lines
- to create required Qubes packages
- to identify potential issue making all Qubes agents and scripts working correctly.
As soon as you manage to make qrexec and qubes-gui-agent working, it should be sufficient to start preparing a template VM.
### Xen libraries
Several XEN libraries are required for Qubes to work correctly. In fact, you need to make xenstore commands working before anything else. For this, Qubes git can be used as several patches have been selected by Qubes developpers that could impact the activity inside a VM. Start be retrieving a recent git and identify how you can build a package from it: `git clone git://git.qubes-os.org/marmarek/xen`
Find the .spec file in the git repository (this is the file being used to build rpm packages), and try to adapt it to your OS in order to build a package similar to the target 'xen-vm'. For example, a PKGBUILD has been created for [ArchLinux](/doc/Templates/Archlinux/) and can be found on [http://aur.archlinux.org/packages/qu/qubes-vm-xen/PKGBUILD](http://aur.archlinux.org/packages/qu/qubes-vm-xen/PKGBUILD).
Don't be afraid with the complexity of the PKGBUILD, most of the code is almost a copy/paste of required sources and patches found in the .spec file provided in the git repository.
Note once the package has been successfully compiled and installed, you need to setup XEN filesystem. Add the folowing line to your fstab (you can create this line in your package install script): `xen /proc/xen xenfs defaults 0 0`
Now install the package you built and mount /proc/xen. Verify that xenstore-read works by running: `xenstore-read qubes_vm_type` That should give you the current VM type such as HVM or AppVM.
### Qubes-OS core agents (qrexec...)
[https://aur.archlinux.org/packages/qu/qubes-vm-core/PKGBUILD](https://aur.archlinux.org/packages/qu/qubes-vm-core/PKGBUILD)
### Qubes-OS kernel modules
[https://aur.archlinux.org/packages/qu/qubes-vm-kernel-modules/PKGBUILD](https://aur.archlinux.org/packages/qu/qubes-vm-kernel-modules/PKGBUILD)
### Qubes-OS gui agents
[https://aur.archlinux.org/packages/qu/qubes-vm-gui/PKGBUILD](https://aur.archlinux.org/packages/qu/qubes-vm-gui/PKGBUILD)

View file

@ -0,0 +1,202 @@
---
layout: doc
title: DevelopmentWorkflow
permalink: /doc/DevelopmentWorkflow/
redirect_from: /wiki/DevelopmentWorkflow/
---
Development Workflow
====================
A workflow for developing Qubes OS+
First things first, setup [QubesBuilder](/doc/QubesBuilder/). This guide assumes you're using qubes-builder to build Qubes.
Repositories and committing Code
--------------------------------
Qubes is split into a bunch of git repos. This are all contained in the `qubes-src` directory under qubes-builder.
FIXME(ypid): Not on github?
The best way to write and contribute code is to create a git repo somewhere (e.g., github) for the repo you are interested in editing (e.g., `qubes-manager`, `core`, etc). To integrate your repo with the rest of Qubes, cd to the repo directory and add your repository as a remote in git
**Example:**
```
$ cd qubes-builder/qubes-src/qubes-manager
$ git remote add abel git@github.com:abeluck/qubes-manager.git
```
You can then proceed to easily develop in your own branches, pull in new commits from the dev branches, merge them, and eventually push to your own repo on github.
When you are ready to submit your changes to Qubes to be merged, push your changes, then create a signed git tag (using `git tag -s`). Finally, send a letter to the Qubes listserv describing the changes and including the link to your repository. Don't forget to include your public PGP key you use to sign your tags.
### Kernel-specific notes
#### Prepare fresh version of kernel sources, with Qubes-specific patches applied
In qubes-builder/qubes-src/kernel:
```
make prep
```
The resulting tree will be in kernel-\<VERSION\>/linux-\<VERSION\>:
```
ls -ltrd kernel*/linux*
```
```
drwxr-xr-x 23 user user 4096 Nov 5 09:50 kernel-3.4.18/linux-3.4.18
drwxr-xr-x 6 user user 4096 Nov 21 20:48 kernel-3.4.18/linux-obj
```
#### Go to the kernel tree and update the version
In qubes-builder/qubes-src/kernel:
```
cd kernel-3.4.18/linux-3.4.18
```
#### Changing the config
In kernel-3.4.18/linux-3.4.18:
```
cp ../../config-pvops .config
make oldconfig
```
Now change the configuration. For example, in kernel-3.4.18/linux-3.4.18:
```
make menuconfig
```
Copy the modified config back into the kernel tree:
```
cp .config ../../../config-pvops
```
#### Patching the code
TODO: describe the workflow for patching the code, below are some random notes, not working well
```
ln -s ../../patches.xen
export QUILT_PATCHES=patches.xen
export QUILT_REFRESH_ARGS="-p ab --no-timestamps --no-index"
export QUILT_SERIES=../../series-pvops.conf
quilt new patches.xen/pvops-3.4-0101-usb-xen-pvusb-driver-bugfix.patch
quilt add drivers/usb/host/Kconfig drivers/usb/host/Makefile \
drivers/usb/host/xen-usbback/* drivers/usb/host/xen-usbfront.c \
include/xen/interface/io/usbif.h
*edit something*
quilt refresh
cd ../..
vi series-pvops.conf
```
#### Building RPMS
TODO: Is this step generic for all subsystems?
Now it is a good moment to make sure you have changed kernel release name in rel-pvops file. For example, if you change it to '1debug201211116c' the resulting RPMs will be named 'kernel-3.4.18-1debug20121116c.pvops.qubes.x86\_64.rpm'. This will help distinguish between different versions of the same package.
You might want to take a moment here to review (git diff, git status), commit your changes locally.
To actually build RPMS, in qubes-src/kernel:
```
make rpms
```
RPMS will appear in qubes-src/kernel/rpm/x86\_64:
```
-rw-rw-r-- 1 user user 42996126 Nov 17 04:08 kernel-3.4.18-1debug20121116c.pvops.qubes.x86_64.rpm
-rw-rw-r-- 1 user user 43001450 Nov 17 05:36 kernel-3.4.18-1debug20121117a.pvops.qubes.x86_64.rpm
-rw-rw-r-- 1 user user 8940138 Nov 17 04:08 kernel-devel-3.4.18-1debug20121116c.pvops.qubes.x86_64.rpm
-rw-rw-r-- 1 user user 8937818 Nov 17 05:36 kernel-devel-3.4.18-1debug20121117a.pvops.qubes.x86_64.rpm
-rw-rw-r-- 1 user user 54490741 Nov 17 04:08 kernel-qubes-vm-3.4.18-1debug20121116c.pvops.qubes.x86_64.rpm
-rw-rw-r-- 1 user user 54502117 Nov 17 05:37 kernel-qubes-vm-3.4.18-1debug20121117a.pvops.qubes.x86_64.rpm
```
### Useful [QubesBuilder](/doc/QubesBuilder/) commands
1. *make check* - will check if all the code was commited into repository and if all repository are tagged with signed tag.
2. *make show-vtags* - show version of each component (based on git tags) - mostly useful just before building ISO. **Note:** this will not show version for components containing changes since last version tag
3. *make push* - push change from **all** repositories to git server. You must set proper remotes (see above) for all repositories first.
4. *make prepare-merge* - fetch changes from remote repositories (can be specified on commandline via GIT\_SUBDIR or GIT\_REMOTE vars), (optionally) verify tags and show the changes. This do not merge the changes - there are left for review as FETCH\_HEAD ref. You can merge them using "git merge FETCH\_HEAD" (in each repo directory).
Copying Code to dom0
--------------------
When developing it is convenient to be able to rapidly test changes. Assuming you're developing Qubes on Qubes, you should be working in a special VM for Qubes and occasionally you will want to transfer code or rpms back to dom0 for testing.
Here are some handy scripts Marek has shared to facilitate this.
You may also like to run your [test environment on separate machine](/doc/TestBench/).
### Syncing dom0 files
TODO: edit this script to be more generic
```
#!/bin/sh
set -x
set -e
QUBES_PY_DIR=/usr/lib64/python2.6/site-packages/qubes
QUBES_PY=$QUBES_PY_DIR/qubes.py
QUBESUTILS_PY=$QUBES_PY_DIR/qubesutils.py
qvm-run -p qubes-devel 'cd qubes-builder/qubes-src/core/dom0; tar c qmemman/qmemman*.py qvm-core/*.py qvm-tools/* misc/vm-template-hvm.conf misc/qubes-start.desktop ../misc/block-snapshot aux-tools ../qrexec' |tar xv
cp $QUBES_PY qubes.py.bak$$
cp $QUBESUTILS_PY qubesutils.py.bak$$
cp /etc/xen/scripts/block-snapshot block-snapshot.bak$$
sudo cp qvm-core/qubes.py $QUBES_PY
sudo cp qvm-core/qubesutils.py $QUBESUTILS_PY
sudo cp qvm-core/guihelpers.py $QUBES_PY_DIR/
sudo cp qmemman/qmemman*.py $QUBES_PY_DIR/
sudo cp misc/vm-template-hvm.conf /usr/share/qubes/
sudo cp misc/qubes-start.desktop /usr/share/qubes/
sudo cp misc/block-snapshot /etc/xen/scripts/
sudo cp aux-tools/qubes-dom0-updates.cron /etc/cron.daily/
# FIXME(Abel Luck): I hope to
```
### Apply qvm-tools
TODO: make it more generic
```
#!/bin/sh
BAK=qvm-tools.bak$$
mkdir -p $BAK
cp -a /usr/bin/qvm-* /usr/bin/qubes-* $BAK/
sudo cp qvm-tools/qvm-* qvm-tools/qubes-* /usr/bin/
```
### Copy from dom0 to an appvm
```
#/bin/sh
#
# usage ./cp-domain <vm_name> <file_to_copy>
#
domain=$1
file=$2
fname=`basename $file`
qvm-run $domain 'mkdir /home/user/incoming/dom0 -p'
cat $file| qvm-run --pass-io $domain "cat > /home/user/incoming/dom0/$fname"
```

View file

@ -0,0 +1,86 @@
---
layout: doc
title: InstallationIsoBuilding
permalink: /doc/InstallationIsoBuilding/
redirect_from: /wiki/InstallationIsoBuilding/
---
How to build Qubes installation ISO
===================================
Qubes uses [Fedora Unity Revisor](http://revisor.fedoraunity.org/) to build the installation ISO.
You may want to get familiar with [Revisor documentation](http://revisor.fedoraunity.org/documentation).
Build installer packages
------------------------
Get [Qubes Installer repository](http://git.qubes-os.org/?p=smoku/installer) and build its packages:
```
cd installer
make rpms
```
Packages will be in `rpm/noarch` and `rpm/x86_64`.
Install Revisor
---------------
Next install the freshly built revisor and anaconda:
```
yum install rpm/noarch/revisor*.rpm
yum install rpm/x86_64/anaconda*.rpm
```
Review configuration files
--------------------------
All configuration files for Qubes Revisor are kept in the ```conf/``` directory:
- ```conf/qubes-install.conf``` - Main Revisor configuration file. This configures Revisor to build Qubes Installation image based on Fedora 13. All other configuration files and working directories are pointed here.
- ```conf/qubes-x86_64.conf``` - This file describes all repositories needed to build Qubes for x86\_64 architecture.
- ```conf/qubes-kickstart.cfg``` - Fedora Kickstart formatted file describing which packages should land in the ISO `/Packages` repository. This describes basically what will be available for installation. The packages list built using this file will be further filtered by the comps file.
- ```conf/comps-qubes.xml``` - Repository Comps file for ISO `/Packages` repository, describing packages and package groups of the installer repository. Package groups are used to select which of the packages are mandatory to install, which are optional and which are to be just available on the ISO but not installed by default (not used on Qubes).
Create/Update local repository
------------------------------
Revisor fetches all RPM packages from YUM repositories. We currently use 5 repositories:
- ```yum/installer``` (installer-related rpms)
- ```yum/qubes-dom0``` (all the Qubes stuff)
- ```yum/dom0-updates``` (for select 3rd party packages, e.g. Xorg)
- ```yum/fedora13-repo``` (local fedora 13 repo, copy from DVD)
- remote fedora repo for extra packages (usually deps for qubes-dom0)
You need to manually copy the Fedora 13 installation DVD contents (```Packages/``` and ```repodata/``` directories) into ```build/fedora13-repo```.
Also, you need to copy all the qubes dom0 rpms into ```build/yum/qubes-dom0/rpm``` and run the ```yum/update_repo.sh``` script afterwards.
In order to fill the ```build/yum/installer``` repo one can just use ```make update-repo```.
The ```build/yum/dom0-updates``` is to be used for select rpms that should also be used instead of those from the fedora (loacal and remote) repos.
Update your local repos:
```
make update-repo
```
Build ISO
---------
Now you're finally ready to build the ISO image:
```
make iso
```
and wait...
You may add `-d 1` (or `-d 99` if you're a masochist) in the Makefile at the end of the revisor command to get (a ton of) debugging information.

View file

@ -0,0 +1,54 @@
---
layout: doc
title: KdeDom0
permalink: /doc/KdeDom0/
redirect_from: /wiki/KdeDom0/
---
Qubes-customized KDE packages for Dom0
======================================
The Qubes kde-dom0 project (see [Source Code](/doc/SourceCode/)) contains the source code needed for building the customized KDE packages for use in Qubes Dom0 (the user desktop). The packages are based on Fedora 12 KDE packages, but are heavily slimmed down (Qubes doesn't need lots of KDE functionality in Dom0, such as most of the KDE apps). In the near future those KDE packages will also get some Qubes specific extensions, such as coloured titlebars/frames nicely integrated into the KDE Window Manager. And, of course, custom themes, e.g. for KDM :)
Getting the sources
-------------------
```
git clone git://qubes-os.org/mainstream/kde-dom0.git kde-dom0
```
Building the packages
---------------------
It's best to use Fedora 12 or 13 as a development system.
First, you should download and verify the original KDE sources (not part of the kde-dom0 repository):
```
make get-sources verify-sources
```
Now, check if you have all the required build dependencies:
```
make prep
```
Install any required packages that `make` might have complained about. Then you're ready to build the rpms (you might want to adjust the release of each rpm package by editing the `rel` variable at the beginning of each `.spec` file):
```
make rpms
```
**Note:** The `kdebase-*` packages build process requires corresponding `kdelibs-devel` package to be installed first. If your build system is based on Fedora 12/13, and if the `kdelibs-devel` package exist in Fedora repo that is based the same KDE software version (e.g. 4.4.3) as the KDE packages you're building (see the `version` file), than you should be able to use the Fedora package:
```
yum install kdelibs-devel-{version}
```
If not, then you should build your `kdelibs-devel` first (`cd kdelibs-devel && make rpms`), then install it on your build system, and then you can build all the rest (`make rpms`).
Installing KDE packages from Qubes repository
---------------------------------------------
TODO

View file

@ -0,0 +1,152 @@
---
layout: doc
title: QubesBuilder
permalink: /doc/QubesBuilder/
redirect_from: /wiki/QubesBuilder/
---
Building Qubes from scratch
===========================
We have a fully automated build system for Qubes, that downloads, builds and
packages all the Qubes components, and finally should spit out a ready-to-use
installation ISO.
In order to use it one should use an rpm-based distro, like Fedora :) and should ensure the following packages are installed:
- git
- createrepo
- rpm-build
- make
- wget
- rpmdevtools
- python-sh
- dialog
- rpm-sign
Unusually one can install those packages by just issuing:
```
sudo yum install git createrepo rpm-build make wget rpmdevtools python-sh dialog rpm-sign
```
The build system creates build environments in chroots and so no other packages are needed on the host. All files created by the build system are contained within the qubes-builder directory. The full build requires some 25GB of free space, so keep that in mind when deciding where to place this directory.
The build system is configured via builder.conf file -- one should copy the attached builder.conf.default, and modify it as needed, e.g.:
```
cp builder.conf.default builder.conf
# edit the builder.conf file and set the following variables:
# (make sure to leave no spaces around '=' sign!)
NO_SIGN=1
# As time of writing this, the default for Qubes 2 Dom0 is fc20
# and VMs is fc20 so if you want to build Qubes 2
DIST_DOM0=fc20
DISTS_VM=fc20
```
One additional useful requirement is that 'sudo root' work without any prompt, which is default on most distros (e.g. 'sudo bash' brings you the root shell without asking for any password). This is important as the builder needs to switch to root and then back to user several times during the build process.
Additionally, if building with signing enabled (so NO\_SIGN is not set), one must adjust \~/.rpmmacro file so that it point to the GPG key used for package signing, e.g.:
```
%_signature gpg
%_gpg_path /home/user/.gnupg
%_gpg_name AC1BF9B3 # <-- Key ID used for signing
```
It is also recommended to use an empty passphrase for the private key used for signing. Contrary to a popular belief, this doesn't affect your key or sources security -- if somebody compromised your system, then the game is over, whether you use additional passphrase for the key or not.
So, to build Qubes one would do:
```
# Import the Qubes master key
gpg --recv-keys 0x36879494
# Verify its fingerprint, set as 'trusted'.
# This is described here:
# https://www.qubes-os.org/doc/VerifyingSignatures
wget http://keys.qubes-os.org/keys/qubes-developers-keys.asc
gpg --import qubes-developers-keys.asc
git clone git://github.com/QubesOS/qubes-builder.git qubes-builder
cd qubes-builder
cp example-configs/qubes-os-master.conf builder.conf
# edit the builder.conf file and set the following variables:
# (make sure to leave no spaces around '=' sign!)
# NO_SIGN="1"
# Download all components:
make get-sources
# And now to build all Qubes rpms (this will take a few hours):
make qubes
# ... and then to build the ISO
make iso
```
And this should produce a shiny new ISO.
You can also build selected component separately. Eg. to compile only gui virtualization agent/daemon:
make gui-daemon
Full list you can get from make help. For advanced use and preparing sources
for use with [QubesBuilder](/doc/QubesBuilder/) take a look at [doc directory
in QubesBuilder](https://github.com/marmarek/qubes-builder/tree/master/doc) or
[QubesBuilderDetails](/doc/QubesBuilderDetails/) page.
Making customized build
-----------------------
### Manual source modification
If you want to somehow modify sources, you can also do it, here are some basic steps:
1. Download qubes-builder as described above (if you want to use marmarek's branches, you should also download qubes-builder from his repo - replace 'QubesOS' with 'marmarek' in above git clone command)
2. Edit builder.conf (still the same as above), some useful additions:
- You can also set GIT\_PREFIX="marmarek/qubes-" to use my repo instead of "mainstream" - it contains newer (but less tested) versions
1. Download unmodified sources
make get-sources
1. **Make your modifications here**
1. Build the Qubes
`make qubes` actually is just meta target which build all required
components in correct order. List of components is configured in
builder.conf. You can also check the current value at the end of `make
help`, or using `make build-info`.
> `get-sources` is already done, so continue with the next one. You can skip `sign-all` if you've disabled signing
>
> ```
> make xen core kernel gui addons docs template kde-dom0 installer qubes-manager dom0-updates
> ```
1. build iso installation image
make iso
Code verification keys management
=================================
[QubesBuilder](/doc/QubesBuilder/) by default verifies signed tags on every downloaded code. Public keys used for that are stored in `keyrings/git`. By default Qubes developers' keys are imported automatically, but if you need some additional keys (for example your own), you can add them using:
```
GNUPGHOME=$PWD/keyrings/git gpg --import /path/to/key.asc
GNUPGHOME=$PWD/keyrings/git gpg --edit-key ID_OF_JUST_IMPORTED_KEY
# here use "trust" command to set key fully or ultimately trusted - only those keys are accepted by QubesBuilder
```
All Qubes developers' keys are signed by the Qubes Master Signing Key (which is set as ultimately trusted key), so are trusted automatically.
If you are the owner of Master key and want to revoke such signature, use the `revsig` gpg key edit command and update the key in qubes-developers-keys.asc - now the key will be no longer trusted (unless manually set as such).

View file

@ -0,0 +1,46 @@
---
layout: doc
title: QubesBuilderDetails
permalink: /doc/QubesBuilderDetails/
redirect_from: /wiki/QubesBuilderDetails/
---
[QubesBuilder](/doc/QubesBuilder/) "API"
========================================
Components Makefile.builder file
--------------------------------
[QubesBuilder](/doc/QubesBuilder/) expects that each component have *Makefile.builder* file in its root directory. This file specifies what should be done to build the package. As name suggests, this is normal makefile, which is included by builder as its configuration. Its main purpose is to set some variables. Generally all available variables/settings are described as comments at the beginning of Makefile.\* in [QubesBuilder](/doc/QubesBuilder/).
Variables for Linux build:
- `RPM_SPEC_FILES` List (space separated) of spec files for RPM package build. Path should be relative to component root directory. [QubesBuilder](/doc/QubesBuilder/) will install all BuildRequires (in chroot environment) before the build. In most Qubes components all spec files are kept in *rpm\_spec* directory. This is mainly used for Fedora packages build.
- `ARCH_BUILD_DIRS` List (space separated) of directories with PKGBUILD files for Archlinux package build. Similar to RPM build, [QubesBuilder](/doc/QubesBuilder/) will install all makedepends, then build the package.
Most components uses *archlinux* directory for this purpose, so its good to keep this style.
Variables for Windows build:
- `WIN_COMPILER` Choose which compiler should be used for this component, thus which build scripts. Currently two options available:
- `WDK` - Windows Driver Kit (default). Command used to build: *build -cZg*.
- `mingw` - MinGW (Windows gcc port). Command used to build: *make all*
- `WIN_SOURCE_SUBDIRS` List of directories in which above command should be run. In most cases it will be only one entry: current directory (*.*).
- `WIN_PREBUILD_CMD` Command to run before build, mostly useful for WDK build (in mingw case, you can use makefile for this purpose). Can be used to set some variables, preprocess some files etc.
- `WIN_SIGN_CMD` Command used to sign resulting binaries. Note that default value is *sign.bat*. If you don't want to sign binaries, specify some placeholder here (eg. *true*). Check existing components (eg. vmm-xen-windows-pvdrivers) for example scripts. This command will be run with certain environment variables:
- `CERT_FILENAME` Path to key file (pfx format)
- `CERT_PASSWORD` Key password
- `CERT_PUBLIC_FILENAME` Certificate path in case of self-signed cert
- `CERT_CROSS_CERT_FILENAME` Certificate path in case of correct autheticode cert
- `SIGNTOOL` Path to signtool
- `WIN_PACKAGE_CMD` Command used to produce installation package (msi or msm). Default value is *wix.bat*, similar to above - use *true* if you don't want this command.
- `WIN_OUTPUT_HEADERS` Directory (relative to `WIN_SOURCE_SUBDIRS` element) with public headers of the package - for use in other components.
- `WIN_OUTPUT_LIBS` Directory (relative to `WIN_SOURCE_SUBDIRS` element) with libraries (both DLL and implib) of the package - for use in other components. Note that [QubesBuilder](/doc/QubesBuilder/) will copy files specified as *\$(WIN\_OUTPUT\_LIBS)/\*/\** to match WDK directory layout (*\<specified directory\>/\<arch directory\>/\<actual libraries\>*), so you in mingw build you need to place libraries in some additional subdirectory.
- `WIN_BUILD_DEPS` List of components required to build this one. [QubesBuilder](/doc/QubesBuilder/) will copy files specified with `WIN_OUTPUT_HEADERS` and `WIN_OUTPUT_LIBS` of those components to some directory and provide its path with `QUBES_INCLUDES` and `QUBES_LIBS` variables. Use those variables in your build scripts (*sources* or *Makefile* - depending on selected compiler). You can assume that the variables are always set and directories always exists, even if empty.
builder.conf settings
---------------------
Most settings are documented in *builder.conf.default* file, which can be used as template the actual configuration.
**TODO**

View file

@ -0,0 +1,56 @@
---
layout: doc
title: QubesR3Building
permalink: /doc/QubesR3Building/
redirect_from: /wiki/QubesR3Building/
---
Building Qubes OS 3.0 ISO
=========================
Ensure your system is rpm-based and that you have necessary dependencies installed (see [QubesBuilder](/doc/QubesBuilder/) for more info):
```
sudo yum install git createrepo rpm-build make wget rpmdevtools pandoc
```
Get the necessary keys to verify the sources:
```
$ wget https://keys.qubes-os.org/keys/qubes-master-signing-key.asc
$ gpg --import qubes-master-signing-key.asc
$ gpg --edit-key 36879494
# Verify fingerprint!, set trust to *ultimate*
$ wget https://keys.qubes-os.org/keys/qubes-developers-keys.asc
$ gpg --import qubes-developers-keys.asc
```
Note we do *not* relay above on the security of our server (keys.qubes-os.org) nor the connection (ssl, cert) -- we only rely on you getting the Qubes Master Signing Key fingerprint *somehow* and ensure they match!
Now lets bootstrap the builder. Unfortunately the builder cannot verify itself (the classic Chicken and Egg problem), so we need to verify the signature manually:
```
$ git clone git://github.com/QubesOS/qubes-builder.git
$ cd qubes-builder
$ git describe --exact-match HEAD
<some tag>
$ git tag -v <some tag>
```
Assuming the verification went fine, we're good to go with all the rest without ever thinking more about verifying digital signatures on all the rest of the components, as the builder will do that for us, for each component, every time we, even for all aux files (e.g. Xen or Linux kernel sources).
Let's configure the builder first (we can use one of the example configs, either for R2 or "master", which currently means pre-released R3):
```
cp example-configs/qubes-os-master.conf builder.conf
```
You can take a loot at the `builder.conf.default` for a description of all available options. Nevertheless, the default config should be enough for start:
```
$ make get-sources qubes
$ make sign-all # this requires setting SIGN_KEY in the builder.conf, can be skipped for test builds.
$ make iso
```
Enjoy your new ISO!

View file

@ -0,0 +1,35 @@
---
layout: doc
title: USBVM
permalink: /doc/USBVM/
redirect_from: /wiki/USBVM/
---
USB Pass through: USBVM
-----------------------
**WARNING:** This is experimental and very broken.
Source: [https://groups.google.com/d/msg/qubes-devel/4AKulABh2Jc/\_R7SRSC4peYJ](https://groups.google.com/d/msg/qubes-devel/4AKulABh2Jc/_R7SRSC4peYJ)
You'll need the patch tagged abb\_e58b432 from [git://github.com/grwl/qubes-core.git](git://github.com/grwl/qubes-core.git).
The rest is in RPMs, yes. I roughly follow this procedure to have pvusb on Qubes 1.0:
- rebuild kernel, core, and xen from marmarek's repo (devel-3.4 branch for kernel),
- make dedicated usbvm domain, passthrough USB controller PCI devices there
- in usbvm: "rpm -Uhv qubes-core-vm-2.1.1-1.fc17.x86\_64.rpm xen-libs-2000:4.1.3-18.fc17.x86\_64.rpm xen-qubes-vm-essentials-2000:4.1.3-18.fc17.x86\_64.rpm"
- in appvm where you want to attach the device to: "rpm -Uhv qubes-core-vm-2.1.1-1.fc17.x86\_64.rpm"
- in dom0 I think the following packages need to be updated, don't have exact list now.qubes-core-dom0 is actually important, the rest are dependencies:
> kernel-qubes-vm-3.4.18-1.pvops.qubes.x86\_64.rpm
> qubes-core-dom0-2.1.1-1.fc13.x86\_64.rpm
> xen-4.1.3-18.fc13.x86\_64.rpm
> xen-hvm-4.1.3gui2.0.9-18.fc13.x86\_64.rpm
> xen-hypervisor-4.1.3-18.fc13.x86\_64.rpm
> xen-libs-4.1.3-18.fc13.x86\_64.rpm
> xen-licenses-4.1.3-18.fc13.x86\_64.rpm
> xen-runtime-4.1.3-18.fc13.x86\_64.rpm
- plug or replug some USB device -- it will be picked up by usbvm, and in dom0 it will appear in the output of qvm-usb