Git-Mirrors/qubes-doc
1
0
Fork 0
mirror of https://github.com/QubesOS/qubes-doc.git synced 2025-05-02 14:56:13 -04:00
Code Issues Projects Releases Packages Wiki Activity

Move current pages to /en/ subdirectory

Browse source
This commit is contained in:
Axon 2015-10-11 07:29:00 +00:00
parent f3ee78f223
commit a91496e9d6
No known key found for this signature in database
GPG key ID: 8CE137352A019A17
159 changed files with 2 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

171
developers/CodingStyle.md
View file

@ -1,171 +0,0 @@
---
layout: doc
title: CodingStyle
permalink: /en/doc/coding-style/
redirect_from:
- /doc/CodingStyle/
- /wiki/CodingStyle/
- /trac/wiki/CodingStyle/
---
Coding Guidelines for Qubes Developers
======================================
Rationale
---------
Maintaining proper coding style is very important for any larger software project, such as Qubes. Here's why:
- It eases maintenance, such as adding new functionality or generalization later,
- It allows others (as well as the original author after some time!) to easily understand fragments of code, what they were supposed to do, and so makes it easier to later extend them with newer functionality or bug fixes,
- It allows others to easily review the code and catch various bugs,
- It provides for an aesthetically pleasing experience when one reads the code...
Often, developers, usually smart developers, neglect the value of proper coding style, thinking that it's most important how their code works, and expecting that if it solves some problem using a nice and neat trick, then it's all that is really required. Such thinking shows, however, immaturity and is a signal that the developer, however bright and smart, might not be a good fit for any larger project. Writing a clever exploit, that is to be used at one Black Hat show is one thing, while writing a useful software that is to be used and maintained for years, is quite a different story. If you want to show off what a smart programmer you are, then you should become a researcher and write exploits. If, on the other hand, you want to be part of a team that makes real, useful software, you should ensure your coding style is impeccable. We often, at Qubes project, often took shortcuts, and often wrote nasty code, and this always back fired at us, sometime months, sometime years later, the net result being we had to spend time fixing code, rather than implementing new functionality.
And here's a [link to the real case](https://groups.google.com/forum/#!msg/qubes-devel/XgTo6L8-5XA/JLOadvBqnqMJ) (one Qubes Security Bulletin) demonstrating how the above described problem lead to a real security bug. Never assume you're smart enough that you can disregard clean and rigorous coding!
General typographic conventions
-------------------------------
- **Use space-expanded tabs that equal 4 spaces.** Yes, we know, there are many arguments for using "real" tabs, instead of space-expanded tabs, but we need to pick one convention to make the project consistent. One argument for using space-expanded tabs is that this way the programmer is in control of how the code will look like, despite how other users have configured their editors to visualize the tabs (of course, we assume any sane person uses a fixed-width font for viewing the source code). Anyway, if this makes you feel better, assume this is just an arbitrary choice.
- **Maintain max. line length of 80 characters**. Even though today's monitors often are very wide and it's often not a problem to have 120 characters displayed in an editor, still maintaining shorter line lengths improves readability. It also allows to have two parallel windows open, side by side, each with different parts of the source code.
- Class, functions, variables, and arguments naming convention for any OS other than Windows:
- `ClassName`
- `some_variable`, `some_function`, `some_argument`
- Class, functions, variables, and arguments naming convention for **Windows OS** -- exceptionally to preserve Windows conventions please use the following:
- `ClassName`, `FunctionName`
- `pszArgumentOne`, `hPipe` -- use hungerian notation for argument and variables
- Horizontal spacing -- maintain at least decent amount of horizontal spacing, such as e.g. add obligatory space after `if` or before `{` in C, and similar in other languages. Whether to also use spaces within expressions, such as (x\*2+5) vs. (x \* 2 + 5) is left to the developer's judgment. Do not put spaces immediately after and before the brackets in expressions, so avoid constructs like this: `if ( condition )` and use `if (condition)` instead.
- **Use single new lines** ('\\n' aka LF) in any non-Windows source code. On Windows, exceptionally, use the CRLF line endings -- this will allow the source code to be easily view-able in various Windows-based programs.
- **Use descriptive names for variables and functions**! Really, these days, when most editors have auto-completion feature, there is no excuse for using short variable names.
- Comments should be indent together with the code, e.g. like this:
~~~
for (...) {
// The following code finds PGP private key matching the given public key in O(1)
while (key_found) {
(...)
}
}
~~~
File naming conventions
-----------------------
- All file names written with small letters, use dash to separate words, rather than underscores, e.g. `qubes-dom0-update`. Never use spaces!
**File naming in Linux/Unix-like systems:**
- User commands that operate on particular VMs (also those accessible in VMs): `/usr/bin/qvm-*`
- User commands that apply to the whole system (Dom0 only): `/usr/bin/qubes-*`
- Auxiliary executables and scripts in `/usr/libexec/qubes/` (Note: previously we used `/usr/lib/qubes` but decided to change that)
- Helper, non-executable files in `/usr/share/qubes/`
- Various config files in `/etc/qubes`
- Qubes RPC services in `/etc/qubes-rpc`. Qubes RPC Policy definitions (only in Dom0) in `/etc/qubes-rpc/policy/`
- All VM-related configs, images, and other files in `/var/lib/qubes/`
- System-wide temporary files which reflect the current state of system in `/var/run/qubes`
- Logs: either log to the system-wide messages, or to `/var/log/qubes/`
**File naming in Windows systems:**
- All base qubes-related files in `C:\Program Files\Invisible Things Lab\Qubes\` (Exceptionally spaces are allowed here to adhere to Windows naming conventions)
- Other, 3rd party files, not Qubes-specific, such as e.g. Xen PV drivers might be in different vendor subdirs, e.g. `C:\Program Files\Xen PV Drivers`
General programming style guidelines
------------------------------------
- Do not try to impress with your coding kung-fu, do not use tricks to save 2 lines of code, always prefer readability over trickiness!
- Make sure your code compiles and builds without warnings.
- Always think first about interfaces (e.g. function arguments, or class methods) and data structures before you start writing the actual code.
- Use comments to explain non-trivial code fragments, or expected behavior of more complex functions, if it is not clear from their name.
- Do **not** use comments for code fragments where it is immediately clear what the code does. E.g. avoid constructs like this:
~~~
// Return window id
int get_window_id (...) {
(...)
return id;
}
~~~
- Do **not** use comments to disable code fragments. In a production code there should really be no commented or disabled code fragments. If you really, really have a good reason to retain some fragment of unused code, use \#if or \#ifdef to disable it, e.g.:
~~~
#if 0
(...) // Some unused code here
#endif
~~~
... and preferably use some descriptive macro instead of just `0`, e.g.:
~~~
#if USE_OLD_WINDOW_TRAVERSING
(...) // Some unused code here
#endif
~~~
Not sure how to do similar thing in Python... Anyone?
> But generally, there is little excuse to keep old, unused code fragments in the code. One should really use the functionality provided by the source code management system, such as git, instead. E.g. create a special branch for storing the old, unused code -- this way you will always be able to merge this code into upstream in the future.
- Do not hardcode values in the code! The only three numbers that are an exception here and for which it is acceptable to hardcode them are: `0`, `1` and `-1`, and frankly the last two are still controversial...
Source Code management (Git) guidelines
---------------------------------------
- Use git to maintain all code for Qubes project.
- Before you start using git, make sure you understand that git is a decentralized Source Code Management system, and that it doesn't behave like traditional, centralized source code management systems, such as SVN. Here's a good [introductory book on git](http://git-scm.com/book). Read it.
- Qubes code is divided into many git repositories. There are several reasons for that:
- This creates natural boundaries between different code blocks, enforcing proper interfaces, and easing independent development to be conducted on various code parts at the same time, without the fear of running into conflicts.
- By maintaining relatively small git repositories, it is easy for new developers to understand the code and contribute new patches, without the need to understand all the other code.
- Code repositories represent also licensing boundaries. So, e.g. because `core-agent-linux` and `core-agent-windows` are maintained in two different repositories, it is possible to have the latter under a proprietary, non-GPL license, while keeping the former fully open source.
- We have drastically changes the layout and naming of the code repositories shortly after Qubes OS R2 Beta 2 release. For details on the current code layout, please read [this article](http://theinvisiblethings.blogspot.com/2013/03/introducing-qubes-odyssey-framework.html).
Security coding guidelines
--------------------------
- As a general rule: **untrusted input** is our \#1 enemy!
- Any input that comes from untrusted, or less trusted, or just differently-trusted, entity should always be considered as malicious and should always be sanitized and verified. So, if your software runs in Dom0 and processes some input from any of the VMs, this input should be considered to be malicious. Even if your software runs in a VM, and processes input from some other VM, you should also assume that the input is malicious and verify it.
- Use `untrusted_` prefix for all variables that hold values read from untrusted party and which have not yet been verified to be decent, e.g.:
~~~
read_struct(untrusted_conf);
/* sanitize start */
if (untrusted_conf.width > MAX_WINDOW_WIDTH)
untrusted_conf.width = MAX_WINDOW_WIDTH;
if (untrusted_conf.height > MAX_WINDOW_HEIGHT)
untrusted_conf.height = MAX_WINDOW_HEIGHT;
width = untrusted_conf.width;
height = untrusted_conf.height;
~~~
- Use another variables, without the `untrusted_` prefix to hold the sanitized values, as shown above.
Python-specific guidelines
--------------------------
- Please follow the guidelines [here](http://www.python.org/dev/peps/pep-0008/), unless they were in conflict with what is written on this page.
C and C++ specific guidelines
-----------------------------
- Do not place code in `*.h` files.
- Use `const` whenever possible, e.g. in function arguments passed via pointers.
- Do not mix procedural and objective code together -- if you write in C++, use classes and objects.
- Think about classes hierarchy, before start implementing specific methods.
Bash-specific guidelines
------------------------
- Avoid writing scripts in bash whenever possible. Use python instead. Bash-scripts are Unix-specific and will not work under Windows VMs, or in Windows admin domain, or Windows gui domain.

28
developers/DevelBooks.md
View file

@ -1,28 +0,0 @@
---
layout: doc
title: DevelBooks
permalink: /en/doc/devel-books/
redirect_from:
- /doc/DevelBooks/
- /wiki/DevelBooks/
---
Below is a list of various books that might be useful in learning some basics needed for Qubes development.
- A must-read about Xen internals:
- [http://www.amazon.com/Definitive-Guide-Xen-Hypervisor/dp/013234971X](http://www.amazon.com/Definitive-Guide-Xen-Hypervisor/dp/013234971X)
- Some good books about Linux kernel:
- [http://www.amazon.com/Linux-Kernel-Development-Robert-Love/dp/0672327201](http://www.amazon.com/Linux-Kernel-Development-Robert-Love/dp/0672327201)
- [http://www.amazon.com/Linux-Device-Drivers-Jonathan-Corbet/dp/0596005903](http://www.amazon.com/Linux-Device-Drivers-Jonathan-Corbet/dp/0596005903)
- Solid intro into Trusted Computing by David Grawrock (original Intel architect for TXT):
- [http://www.amazon.com/Dynamics-Trusted-Platform-Buildin-Grawrock/dp/1934053082](http://www.amazon.com/Dynamics-Trusted-Platform-Buildin-Grawrock/dp/1934053082)
- Good book about GIT:
- [http://progit.org/book/](http://progit.org/book/)
- Useful books about Python:
- [http://www.qtrac.eu/py3book.html](http://www.qtrac.eu/py3book.html) (Note that Qubes management tools are written in Python 2.6, but this book is still a very good choice)
- [http://www.qtrac.eu/pyqtbook.html](http://www.qtrac.eu/pyqtbook.html)

45
developers/DevelFaq.md
View file

@ -1,45 +0,0 @@
---
layout: doc
title: DevelFaq
permalink: /en/doc/devel-faq/
redirect_from:
- /doc/DevelFaq/
- /wiki/DevelFaq/
---
Qubes Developers FAQ
====================
1. 1. [Q: Why does dom0 need to be 64-bit?](#q-why-does-dom0-need-to-be-64-bit)
2. [Q: Why do you use KDE in Dom0? What is the roadmap for Gnome support?](#q-why-do-you-use-kde-in-dom0-what-is-the-roadmap-for-gnome-support)
3. [Q: What is the recommended build environment?](#q-what-is-the-recommended-build-environment)
4. [Q: How to build Qubes from sources?](#q-how-to-build-qubes-from-sources)
5. [Q: How do I submit a patch?](#q-how-do-i-submit-a-patch)
### Q: Why does dom0 need to be 64-bit?
Often it is more difficult to exploit a bug on the x64 Linux than it is on x86 Linux (e.g. ASLR is sometimes harder to get around). While we designed Qubes with the emphasis on limiting any potential attack vectors in the first place, still we realize that some of the code running in Dom0, e.g. our GUI daemon or xen-store daemon, even though it is very simple code, might contain some bugs. Plus currently we haven't implemented a separate storage domain, so also the disk backends are in Dom0 and are "reachable" from the VMs, which adds up to the potential attack surface. So, having faced a choice between 32-bit and 64-bit OS for Dom0, it was almost a no-brainer, as the 64-bit option provides some (little perhaps, but still) more protection against some classes of attacks, and at the same time does not have any disadvantages (except that it requires a 64-bit processor, but all systems on which it makes sense to run Qubes, e.g. that have at least 3-4GB memory, they do have 64-bit CPUs anyway).
### Q: Why do you use KDE in Dom0? What is the roadmap for Gnome support?
There are a few things that are KDE-specific, but generally it should not be a big problem to also add Gnome support to Qubes (in Dom0 of course). Those KDE-specific things are:
- Qubes requires KDM (KDE Login Manager), rather than GDM, for the very simple reason that GDM doesn't obey standards and start `/usr/bin/Xorg` instead of `/usr/bin/X`. This is important for Qubes, because we need to load a special "X wrapper" (to make it possible to use Linux usermode shared memory to access Xen shared memory pages in our App Viewers -- see the sources [here](https://github.com/QubesOS/qubes-gui-daemon/tree/master/shmoverride)). So, Qubes makes the `/usr/bin/X` to be a symlink to the Qubes X Wrapper, which, in turn, executes the `/usr/bin/Xorg`. This works well with KDM (and would probably also work with other X login managers), but not with GDM. If somebody succeeded in makeing GDM to execute `/usr/bin/X` instead of `/usr/bin/Xorg`, we would love to hear about it!
- We maintain a special [repository](/en/doc/kde-dom0/) for building packages specifically for Qubes Dom0.
- We've patched the KDE's Window Manager (specifically [one of the decoration plugins](https://github.com/QubesOS/qubes-desktop-linux-kde/tree/master/plastik-for-qubes)) to draw window decorations in the color of the specific AppVM's label.
If you're interested in porting GNOME for Qubes Dom0 use, let us know -- we will most likely welcome patches in this area.
### Q: What is the recommended build environment?
Any rpm-based, 64-bit. Preferred Fedora.
### Q: How to build Qubes from sources?
See [the instruction](/en/doc/qubes-builder/)
### Q: How do I submit a patch?
See [Qubes Source Code Repositories](/en/doc/source-code/).

34
developers/QubesResearch.md
View file

@ -1,34 +0,0 @@
---
layout: doc
title: QubesResearch
permalink: /en/doc/qubes-research/
redirect_from:
- /doc/QubesResearch/
- /wiki/QubesResearch/
---
Here are some links to various papers/research projects that somehow relate to Qubes.
### Attacks on Intel TXT
- [Attacking Intel® Trusted Execution Technology](http://invisiblethingslab.com/resources/bh09dc/Attacking%20Intel%20TXT%20-%20paper.pdf) by Rafal Wojtczuk, Joanna Rutkowska
- [ACPI: Design Principles and Concerns](http://www.ssi.gouv.fr/IMG/pdf/article_acpi.pdf) by Loic Duflot, Olivier Levillain, and Benjamin Morin
- [Another Way to Circumvent Intel® Trusted Execution Technology](http://invisiblethingslab.com/resources/misc09/Another%20TXT%20Attack.pdf) by Rafal Wojtczuk, Joanna Rutkowska, Alex Tereshkin
- [Attacking Intel TXT® via SINIT code execution hijacking](http://www.invisiblethingslab.com/resources/2011/Attacking_Intel_TXT_via_SINIT_hijacking.pdf) by Rafal Wojtczuk and Joanna Rutkowska
### Software attacks coming through devices
- [Can you still trust your network card?](http://www.ssi.gouv.fr/IMG/pdf/csw-trustnetworkcard.pdf) by Loïc Duflot, Yves-Alexis Perez and others
- [Remotely Attacking Network Cards (or why we do need VT-d and TXT)](http://theinvisiblethings.blogspot.com/2010/04/remotely-attacking-network-cards-or-why.html) by Joanna Rutkowska
- [On Formally Verified Microkernels (and on attacking them)](http://theinvisiblethings.blogspot.com/2010/05/on-formally-verified-microkernels-and.html) by Joanna Rutkowska
- [Following the White Rabbit: Software Attacks against Intel® VT-d](http://www.invisiblethingslab.com/resources/2011/Software%20Attacks%20on%20Intel%20VT-d.pdf) by Rafal Wojtczuk and Joanna Rutkowska
### Application-level security
- [Virtics: A System for Privilege Separation of Legacy Desktop Applications](http://www.eecs.berkeley.edu/Pubs/TechRpts/2010/EECS-2010-70.pdf) by Matt Piotrowski
### VMM/Xen disagregation
- [[http://tjd.phlegethon.org/words/sosp11-xoar.pdf](http://tjd.phlegethon.org/words/sosp11-xoar.pdf) "Breaking Up is Hard to Do: Security and Functionality in a Commodity Hypervisor] by Patrick Colp at el.
(Also see [this thread on xen-devel](http://www.gossamer-threads.com/lists/xen/devel/230011))

47
developers/SourceCode.md
View file

@ -1,47 +0,0 @@
---
layout: doc
title: SourceCode
permalink: /en/doc/source-code/
redirect_from:
- /doc/SourceCode/
- /wiki/SourceCode/
---
Qubes Source Code Repositories
==============================
All the Qubes code is kept in GIT repositories. We divided the project into several components, each of which has its own separate repository, some of them:
- `core-admin.git` -- the core Qubes infrastructure responsible for VM management, VM templates, fs sharing, etc.
- `gui-daemon.git` -- GUI virtualization, Dom0 side.
- `gui-agent-linux.git` -- GUI virtualization, Linux VM side.
- `linux-template-builder.git` - scripts and other files used to create Qubes templates images.
You can browse the repositories [online on
GitHub](https://github.com/QubesOS/). The Qubes official repositories are on
this `QubesOS` github account.
To clone a repository:
~~~
git clone git://github.com/QubesOS/<repo_name>.git <repo_name>
~~~
e.g.:
~~~
git clone git://github.com/QubesOS/qubes-core-admin.git core-admin
~~~
## Sending a patch
If you want to contribute to the project, there are two ways:
* **Preferred**: Use github [fork & pull requests](https://guides.github.com/activities/forking/)
* Sending a patch via the project's mailing list (`git format-patch`).
1. Make all the changes in your working directory, i.e. edit files, move them around (you can use 'git mv' for this), etc.
2. Add the changes and commit them (git add, git commit). Never mix different changes into one commit! Write a good description of the commit. The first line should contain a short summary, and then, if you feel like more explanations are needed, enter an empty new line, and then start the long, detailed description (optional).
3. Test your changes NOW: check if RPMs build fine, etc.
4. Create the patch using 'git format-patch'. This has an advantage over 'git diff', because the former will also include your commit message, your name and email, so that \*your\* name will be used as a commit's author.
5. Send your patch to qubes-devel. Start the message subject with the '[PATCH]' string.

53
developers/SystemDoc.md
View file

@ -1,53 +0,0 @@
---
layout: doc
title: SystemDoc
permalink: /en/doc/system-doc/
redirect_from:
- /doc/SystemDoc/
- /wiki/SystemDoc/
---
System Documentation for Developers
===================================
Fundamentals
------------
* [Qubes OS Architecture Overview](/en/doc/qubes-architecture/)
* [Qubes OS Architecture Spec v0.3 [PDF]](/attachment/wiki/QubesArchitecture/arch-spec-0.3.pdf) (The original 2009 document that started this all...)
* [Security-critical elements of Qubes OS](/en/doc/security-critical-code/)
* Qubes RPC: [`qrexec` v2](/en/doc/qrexec/) ([R2 implementation](/en/doc/qrexec2-implementation/))
* Qubes RPC: [`qrexec` v3](/en/doc/qrexec3/) ([R3 implementation](/en/doc/qrexec3-implementation/)) (Odyssey)
* [Example for writing a `qrexec` service in Qubes OS (blog post)](http://theinvisiblethings.blogspot.com/2013/02/converting-untrusted-pdfs-into-trusted.html)
* [Qubes GUI virtualization protocol](/en/doc/gui-docs/)
* [Networking in Qubes](/en/doc/qubes-net/)
* [Implementation of template sharing and updating](/en/doc/template-implementation/)
Services
--------
* [Inter-domain file copying](/en/doc/qfilecopy/) (deprecates [`qfileexchgd`](/en/doc/qfileexchgd/))
* [Dynamic memory management in Qubes](/en/doc/qmemman/)
* [Implementation of DisposableVMs](/en/doc/dvm-impl/)
* [Article about disposable VMs](http://theinvisiblethings.blogspot.com/2010/06/disposable-vms.html)
* [Dom0 secure update mechanism](/en/doc/dom0-secure-updates/)
* VM secure update mechanism (forthcoming)
Debugging
---------
* [Profiling python code](/en/doc/profiling/)
* [Test environment in separate machine for automatic tests](/en/doc/test-bench/)
* [Automated tests](/en/doc/automated-tests/)
* [VM-dom0 internal configuration interface](/en/doc/vm-interface/)
* [Debugging Windows VMs](/en/doc/windows-debugging/)
Building
--------
* [Building Qubes](/en/doc/qubes-builder/) (["API" Details](/en/doc/qubes-builder-details/))
* [Development Workflow](/en/doc/development-workflow/)
* [KDE Dom0 packages for Qubes](/en/doc/kde-dom0/)
* [How to build Qubes installation ISO](/en/doc/installation-iso-building/)
* [Building Qubes OS 3.0 ISO](/en/doc/qubes-r3-building/)
* [Building USB passthrough support (experimental)](/en/doc/usbvm/)
* [Building a TemplateVM based on a new OS (ArchLinux example)](/en/doc/building-non-fedora-template/)
* [Building the Archlinux Template](/en/doc/building-archlinux-template/)

150
developers/VersionScheme.md
View file

@ -1,150 +0,0 @@
---
layout: doc
title: VersionScheme
permalink: /en/doc/version-scheme/
redirect_from:
- /doc/VersionScheme/
- /wiki/VersionScheme/
---
Version Scheme
==============
Beginning with R3 release, we change (and formalise) the versioning scheme.
From now on, it will be as follows.
Qubes distributions and products
--------------------------------
We intend to make it easy to make a remix of qubes, targetting another
hypervisor or isolation provider. We may also create commercial products
intended for specific circumstances. There is one distinguished distribution
called **Qubes OS**. All source code for it is available for download under GPL
licence and is openly developed on the mailing lists. The rest of this document
discusses Qubes OS. Another remix may have its own version series.
Release version
---------------
Qubes OS as a whole is released from time to time. Version scheme for all
releases is modelled after Linux kernel version numbers. When announcing new
release, we decide on the major.minor version (like `3.0`) and release
`3.0-rc1`. When we feel that enough progress has been made, we put `3.0-rc2`
and so on. All these versions are considered unstable and not ready for
production. You may ask for support on mailing lists (specifically
**qubes-devel**), but it is not guaranteed (you may for example get answer
„update to newer `-rc`”). Public ISO image may or may not be available.
When enough development has been made, we announce the first stable version,
like e.g. `3.0.0` (i.e. without `-rc`). This version is considered stable and
we support it for some period. Core components are branched at this moment and
bugfixes are backported from master branch. Questions about stable release
should be directed to the **qubes-users** mailing list. No major features and
interface incompatibilities are to be included in this release. We release
bugfixes as `3.0.1`, `3.0.2` and so on, while new features come into the next
release e.g. `3.1-rcX`.
Tickets in the tracker are sorted out by release major.minor, such as `3.0`,
`3.1` (trac calls this „milestone”).
Release schedule
----------------
There is no specific schedule for major and minor other that more general
roadmap. When time comes, Supreme Committee declares feature freeze and tags
`-rc1` and releases ISO image. From this time on, no new features are accepted.
Also a strict time schedule kicks in.
Each release candidate period is as follows. For the first two weeks we accept
and assign bugreports to be fixed before next release candidate. For the next
two weeks we generally focus on fixing assigned bugreports, so issues discovered
during this time may be postponed until later RC. Finally after that there is
one week of current-testing freeze, during which time no new packages are
released, in hope that they will be installed by wider user base and tested.
The next RC is released five weeks after the former. All packets are published
in `current` repository and the cycle starts over. There should be no less than
1 and no more than 3 release candidates before final release.
<table border>
<thead>
<tr><th>stage</th><th>time</th></tr>
</thead>
<tbody>
<tr><td>initial testing</td><td>2 weeks</td></tr>
<tr><td>bug fixing</td><td>2 weeks</td></tr>
<tr><td>`current-testing` freeze</td><td>1 week</td></tr>
</tbody>
</table>
Starting with second cycle (that is, after `-rc1`) two weeks into the cycle
(after primary bug-reporting period) the Supreme Committee decides wether there
should be another RC. If, based on remaining issues, the Committee decides to
release final, then the Committee agrees upon the release date, which should be
no later than a week after.
!["Release cycle"](/attachment/wiki/VersionScheme/release-cycle.svg)
Bug priorities
--------------
When deciding whether the current release candidate is the final one, the Committee
takes bugs priorities into consideration. The meaning of them is as follows:
* `blocker` - when any such bug is present in the current release candidate, it
can't be considered final release. Bugs with this priority must be fixed before
the next release candidate, even if that means delaying its release (which
should be considered only last resort option).
* `critical` - when any such bug is present in the current release candidate, it
can't be considered final release. But such bugs are not qualified to delay
next release candidate release.
* `major` - existence of such bugs do not strictly prevent the current release
candidate be considered final (but of course we should try hard to not have
them there). Fixing bugs of this priority can be delayed and qualified as
updates to the final stable release.
* `minor` - existence of such bugs do not prevent the current release candidate
be considered final. Fixing such bugs can be delayed to the next Qubes OS
release. Eventually such fixes might be backported as an update to the stable
release(s).
All above is about bugs, no features should be assigned to the current release
after first `-rc`. Supreme Committee is free to adjust priorities appropriately.
Component version
-----------------
Qubes release is defined as specific versions of components, which are
developed more or less separately. Their versions are composed of major and
minor version of target Qubes OS release followed by third component which is
just incremented. There is no apparent indication that given version is stable
or not.
There are some non-essential components like `qubes-apps-*` that are shared
between releases. Their versions indicate oldest qubes-release that is
supported. We try hard to support multiple releases by one branch to ease code
maintenance.
Different Qubes releases remixes may comprise of different components and
version are not guaranteed to be monotonic between releases. We may decide that
for newer release some component should be downgraded. There is no guarantee
that arbitrary combination of different versions of random components will
yield usable (or even install-able) compilation.
Git tags and branches
---------------------
We mark each component version in the repository by tag containing
`v<version>`. Likewise, each Qubes OS release is marked by `R<release>` tag.
At the release of some release we create branches named like `release2`. Only
bugfixes and compatible improvements are backported to these branches. These
branches should compile. All new development is done in `master` branch. This
branch is totally unsupported and may not even compile depending on maintainer
of repository.
All version and release tags should be made and signed by someone from ITL
staff. Public keys are included in `qubes-builder` and available at
[https://keys.qubes-os.org/keys/](https://keys.qubes-os.org/keys/).

232
developers/building/BuildingArchlinuxTemplate.md
View file

@ -1,232 +0,0 @@
---
layout: doc
title: BuildingArchlinuxTemplate
permalink: /en/doc/building-archlinux-template/
redirect_from:
- /doc/BuildingArchlinuxTemplate/
- /wiki/BuildingArchlinuxTemplate/
---
Template building
=================
The archlinux VM is now almost working as a NetVM. Based on qubes-builder code, you could find below how to build it and problem that could arise from template building to using archlinux as a netvm:
Download qubes-builder git code
-------------------------------
Prefer marmarek git repository as it is the most recent one
Change your builder.conf
------------------------
Change the following variables GIT\_SUBDIR=marmarek DISTS\_VM=archlinux
Get all required sources
------------------------
~~~
make get-sources
~~~
Note that make get-sources sometimes fails because it didn't download packages that are not used by archlinux such as xfce or kde packages.
You can ignore the repositories that are failing by adding the following line to your builder.conf:
~~~
COMPONENTS:=$(filter-out desktop-linux-kde desktop-linux-xfce,$(COMPONENTS))
~~~
Just don't forget that you need to comment this line again if you want to build the whole Qubes-OS install CD.
Make all required qubes components
----------------------------------
The first use of the builder can take several hours depending on your bandwidth as it will install an archlinux chroot:
~~~
make vmm-xen-vm
make core-vchan-xen-vm
make linux-utils-vm
make core-agent-linux-vm
make gui-common-vm
make gui-agent-linux-vm
~~~
Now build the template itself
-----------------------------
This can take again several hours, especially the first time you built and archlinux template:
~~~
make linux-template-builder
~~~
Retrieve your template
----------------------
You can now find your template in qubes-src/linux-template-builder/rpm/noarch. Install it in dom0 (just take care as it will replace your current archlinux-x64 template)
* * * * *
Known problems during building or when running the VM
=====================================================
Can't open file archlinux-2013.02.01-dual.iso
---------------------------------------------
Archlinux ISO files are sometimes removed from mirrors. Check the last version available on the archlinux mirror (eg: [http://mir.archlinux.fr/iso/](http://mir.archlinux.fr/iso/)), and update qubes-src/linux-template-builder/scripts\_archlinux/00\_prepare.sh accordingly:
~~~
ISO_VERSION=2013.06.01
~~~
You will also need to download the signature matching this ISO version inside qubes-src/linux-template-builder/scripts\_archlinux/:
~~~
wget http://mir.archlinux.fr/iso/2013.06.01/archlinux-2013.06.01-dual.iso.sig
~~~
The nm-applet (network manager icon) fails to start when archlinux is defined as a template-vm:
-----------------------------------------------------------------------------------------------
In fact /etc/dbus-1/system.d/org.freedesktop.NetworkManager.conf does not allow a standard user to run network manager clients. To allow this, one need to change inside \<policy context="default"\>:
~~~
<deny send_destination="org.freedesktop.NetworkManager"/>
~~~
to
~~~
<allow send_destination="org.freedesktop.NetworkManager"/>
~~~
DispVM, Yum proxy and most Qubes addons (thunderbird ...) have not been tested at all.
--------------------------------------------------------------------------------------
The sound does not work in AppVMs and there are messages related to pulse segfault in glibc when running dmesg (FIXED)
----------------------------------------------------------------------------------------------------------------------
This is apparently a bug in Archlinux between glibc and pulseaudio package 4.0-6. The packages pulseaudio-4.0-2 and libpulse-4.0-2 are known to work and can be downloaded and reinstalled manually.
Error when building the gui-agent-linux with pulsecore error
------------------------------------------------------------
~~~
module-vchan-sink.c:62:34: fatal error: pulsecore/core-error.h: No such file or directory
#include <pulsecore/core-error.h>
~~~
This error is because Archlinux update package too quickly. Probably, a new version of pulseaudio has been released, but the qubes team has not imported the new development headers yet.
You can create fake new headers just by copying the old headers:
~~~
cd qubes-builder/qubes-src/gui-agent-linux/pulse
ls
cp -r pulsecore-#lastversion pulsecore-#archlinuxversion
~~~
You can check the current archlinux pulseaudio version like this:
~~~
sudo chroot chroot-archlinux/ pacman -Qi pulseaudio
~~~
chroot-archlinux/dev/pts has not been unmounted
-----------------------------------------------
This is a known problem when there are errors during building. Check what is mounted using the command mount (with no parameters). Just unmount what you can (or reboot your vm if you are too lazy :) )
Known problems in Qubes R2-B2
=============================
xen-vmm-vm fail to build with a PARSETUPLE related error (FIXED):
-----------------------------------------------------------------
Commenting out "\#define HAVE\_ATTRIBUTE\_FORMAT\_PARSETUPLE" from chroot\_archlinux/usr/include/python2.7/pyconfig.h fixes the problem, but it isn't the right solution [1]...
A better fix is planned for the next python release (the bug is considered release blocking), and will be updated in archlinux chroot as soon as available.
[1] [http://bugs.python.org/issue17547](http://bugs.python.org/issue17547)
The boot process fails without visible errors in the logs, but spawn a recovery shell
-------------------------------------------------------------------------------------
The problem is a new conflict between systemd and the old sysvinit style. To fix this, you can change the master xen template in dom0 to remove sysvinit remains: Edit **INSIDE DOM0** /usr/share/qubes/vm-template.conf, and change the variable 'extra' that contains the kernel variables: from:
~~~
extra="ro nomodeset 3 console=hvc0 rd_NO_PLYMOUTH {kernelopts}"
~~~
to:
~~~
extra="ro nomodeset console=hvc0 rd_NO_PLYMOUTH {kernelopts}"
~~~
Qubes-OS is now using different xenstore variable names, which makes to archlinux VM failing to boot
----------------------------------------------------------------------------------------------------
Apply the following fix in the template to revert the variable name to the old Qubes version.
You can edit the template the following way:
~~~
sudo mkdir /mnt/vm
sudo mount /var/lib/qubes/vm-templates/archlinux-x64/root.img /mnt/vm
sudo chroot /mnt/vm
~~~
Then apply the fix:
~~~
sudo sed 's:qubes-keyboard:qubes_keyboard:g' -i /etc/X11/xinit/xinitrc.d/qubes-keymap.sh
sudo sed 's:qubes-netvm-domid:qubes_netvm_domid:g' -i /etc/NetworkManager/dispatcher.d/30-qubes-external-ip
sudo sed 's:qubes-netvm-external-ip:qubes_netvm_external_ip:g' -i /etc/NetworkManager/dispatcher.d/30-qubes-external-ip
sudo sed 's:qubes-netvm-network:qubes_netvm_network:g' -i /usr/lib/qubes/init/network-proxy-setup.sh
sudo sed 's:qubes-netvm-gateway:qubes_netvm_gateway:g' -i /usr/lib/qubes/init/network-proxy-setup.sh
sudo sed 's:qubes-netvm-netmask:qubes_netvm_netmask:g' -i /usr/lib/qubes/init/network-proxy-setup.sh
sudo sed 's:qubes-netvm-secondary-dns:qubes_netvm_secondary_dns:g' -i /usr/lib/qubes/init/network-proxy-setup.sh
sudo sed 's:qubes-vm-type:qubes_vm_type:g' -i /usr/lib/qubes/init/qubes-sysinit.sh
sudo sed 's:qubes-ip:qubes_ip:g' -i /usr/lib/qubes/setup-ip
sudo sed 's:qubes-netmask:qubes_netmask:g' -i /usr/lib/qubes/setup-ip
sudo sed 's:qubes-gateway:qubes_gateway:g' -i /usr/lib/qubes/setup-ip
sudo sed 's:qubes-secondary-dns:qubes_secondary_dns:g' -i /usr/lib/qubes/setup-ip
sudo sed 's:qubes-netvm-network:qubes_netvm_network:g' -i /usr/lib/qubes/setup-ip
sudo sed 's:qubes-netvm-gateway:qubes_netvm_gateway:g' -i /usr/lib/qubes/setup-ip
sudo sed 's:qubes-netvm-netmask:qubes_netvm_netmask:g' -i /usr/lib/qubes/setup-ip
sudo sed 's:qubes-netvm-secondary-dns:qubes_netvm_secondary_dns:g' -i /usr/lib/qubes/setup-ip
sudo sed 's:qubes-iptables-domainrules:qubes_iptables_domainrules:g' -i /usr/bin/qubes-firewall
sudo sed 's:qubes-iptables-header:qubes_iptables_header:g' -i /usr/bin/qubes-firewall
sudo sed 's:qubes-iptables-error:qubes_iptables_error:g' -i /usr/bin/qubes-firewall
sudo sed 's:qubes-iptables:qubes_iptables:g' -i /usr/bin/qubes-firewall
sudo sed 's:qubes-netvm-domid:qubes_netvm_domid:g' -i /usr/bin/qubes-netwatcher
sudo sed 's:qubes-netvm-external-ip:qubes_netvm_external_ip:g' -i /usr/bin/qubes-netwatcher
sudo sed 's:qubes-vm-updateable:qubes_vm_updateable:g' -i /usr/lib/qubes/qubes_trigger_sync_appmenus.sh
sudo sed 's:qubes-vm-type:qubes_vm_type:g' -i /usr/bin/qubes-session
sudo sed 's:qubes-vm-updateable:qubes_vm_updateable:g' -i /usr/bin/qubes-session
~~~
Do not forgot to:
~~~
umount /mnt/vm
~~~
Installing the template in dom0 fails because of a missing dependency (qubes-core-dom0-linux)
---------------------------------------------------------------------------------------------
Again you built a template based on a recent Qubes API which has not been released yet. So skip the dependency for now:
~~~
sudo rpm -U --nodeps yourpackage.rpm
~~~

159
developers/building/BuildingNonFedoraTemplate.md
View file

@ -1,159 +0,0 @@
---
layout: doc
title: BuildingNonFedoraTemplate
permalink: /en/doc/building-non-fedora-template/
redirect_from:
- /doc/BuildingNonFedoraTemplate/
- /wiki/BuildingNonFedoraTemplate/
---
Building a TemplateVM for ArchLinux (or another non fedora OS)
==============================================================
If you don't like using Fedora because of specific administration or package management / building needs, you could build a VM Template for your Distribution of choice.
This article shows how to build a template for a different OS, taking ArchLinux as an example.
Qubes builder scripts
=====================
You can start creating Qubes builder scripts for your new OS. Just note that it will probably make your testing process harder than trying to build the package directly in an HVM on which you already installed this new OS.
chroot initialization
---------------------
You need to install your OS inside a chroot that will be used to build all the required qubes agents of tools.
The scripts you will be interested in will be inside the qubes-src/linux-template-builder project:
~~~
scripts_fedora
scripts_archlinux
scripts_yourOSname
~~~
### 00\_prepare.sh
The goal of the first script 00\_prepare.sh is to download and verify the signature of the installation cd and tools. You can use the \$CACHEDIR directory variable to store files that could be reused (such as downloaded scripts or iso files)
### 01\_install\_core.sh
The goal of this script is to install a base environment of your target OS inside the \$INSTALLDIR directory variable. Generally you need to bootstrap/install your package manager inside the \$INSTALLDIR directory and install the base packages.
### Testing the installation process
Edit the builder.conf file to change the variable DISTS\_VM to your OS name (DISTS\_VM=your\_os\_name). The try to make the template to check that at least these to first scripts are working correctly:
~~~
make linux-template-builder
~~~
Qubes builder Makefiles
-----------------------
Now you need to create Makefiles specific to your OS. You will find the required scripts directly inside qubes-builder:
~~~
prepare-chroot-yourOSname
Makefile.yourOSname
~~~
### prepare-chroot-yourOSname
The goal of this file is to prepare a development environment of your target OS inside a chroot. You will reuse there the 00\_prepare.sh and 01\_install\_core.sh scripts. Additionally, the following things will be necessary to use to chroot environment:
- the \$1 variable will contain the installation directory (INSTALLDIR should contain the same value than \$1 when you run 00\_prepare or 01\_install\_core)
- after your base system is installed, you should install development tools and libraries (gcc, make, ...)
- create a user called 'user' inside your chroot, and give him enought right to run the command sudo without any password
- register all the repository that could be necessary and synchronize the package database
- register a custom repository that will be used to store qubes packages
### Makefile.yourOSname
This file will be used to define the action required when installing a package. The most important one are:
- dist-prepare-chroot: that's where you will call prepare-chroot-yourOSname if the chroot has not been initialized
- dist-package: that's where you will chroot the development environment and run the command used to build a package.
- dist-build-dep: that's where you will create the custom repository for your target OS based on already compiled packages.
These additional target need to exist once you created your first packages:
- dist-copy-out: that's where you will retrieve the package you just built and put it with all the other package you prepared.
- update-repo: that's where you will retrieve the package that have been built and that need to be installed inside the template
### Testing the development chroot
You will be able to test these script when making the first qubes packages. Don't forget that the first things that run when running 'make somcomponent-vm' will be these two scripts, and that you will need to debug it at this point.
Qubes packages
--------------
### vmm-xen-vm
### core-vchan-xen-vm
### linux-utils-vm
### core-agent-linux-vm
### gui-common-vm
### gui-agent-linux-vm
Additional Installation scripts
-------------------------------
Again you need to work on scripts inside the qubes-src/linux-template-builder project:
~~~
scripts_fedora
scripts_archlinux
scripts_yourOSname
~~~
### 02\_install\_groups.sh
The goal of this script is to install all the package that you want to use in your template (eg: firefox, thunderbird, a file manager, Xorg...)
### 04\_install\_qubes.sh
The goal of this script is to install in your template all the packages you built previously. Also you need to edit the fstab file of your template to mount qubes virtual hard drives.
### 09\_cleanup.sh
This script is use to finalize and to remove unnecessary things from your template, such as cached packages, unused development packages ...
Starting with an HVM
====================
If no Qubes packages are available for your selected OS. You could start to install an HVM with your OS. Your goals will be:
- to identify how to install the OS using command lines
- to create required Qubes packages
- to identify potential issue making all Qubes agents and scripts working correctly.
As soon as you manage to make qrexec and qubes-gui-agent working, it should be sufficient to start preparing a template VM.
### Xen libraries
Several XEN libraries are required for Qubes to work correctly. In fact, you need to make xenstore commands working before anything else. For this, Qubes git can be used as several patches have been selected by Qubes developpers that could impact the activity inside a VM. Start be retrieving a recent git and identify how you can build a package from it: `git clone git://git.qubes-os.org/marmarek/xen`
Find the .spec file in the git repository (this is the file being used to build rpm packages), and try to adapt it to your OS in order to build a package similar to the target 'xen-vm'. For example, a PKGBUILD has been created for [ArchLinux](/en/doc/templates/archlinux/) and can be found on [http://aur.archlinux.org/packages/qu/qubes-vm-xen/PKGBUILD](http://aur.archlinux.org/packages/qu/qubes-vm-xen/PKGBUILD).
Don't be afraid with the complexity of the PKGBUILD, most of the code is almost a copy/paste of required sources and patches found in the .spec file provided in the git repository.
Note once the package has been successfully compiled and installed, you need to setup XEN filesystem. Add the folowing line to your fstab (you can create this line in your package install script): `xen /proc/xen xenfs defaults 0 0`
Now install the package you built and mount /proc/xen. Verify that xenstore-read works by running: `xenstore-read qubes_vm_type` That should give you the current VM type such as HVM or AppVM.
### Qubes-OS core agents (qrexec...)
[https://aur.archlinux.org/packages/qu/qubes-vm-core/PKGBUILD](https://aur.archlinux.org/packages/qu/qubes-vm-core/PKGBUILD)
### Qubes-OS kernel modules
[https://aur.archlinux.org/packages/qu/qubes-vm-kernel-modules/PKGBUILD](https://aur.archlinux.org/packages/qu/qubes-vm-kernel-modules/PKGBUILD)
### Qubes-OS gui agents
[https://aur.archlinux.org/packages/qu/qubes-vm-gui/PKGBUILD](https://aur.archlinux.org/packages/qu/qubes-vm-gui/PKGBUILD)

204
developers/building/DevelopmentWorkflow.md
View file

@ -1,204 +0,0 @@
---
layout: doc
title: DevelopmentWorkflow
permalink: /en/doc/development-workflow/
redirect_from:
- /doc/DevelopmentWorkflow/
- /wiki/DevelopmentWorkflow/
---
Development Workflow
====================
A workflow for developing Qubes OS+
First things first, setup [QubesBuilder](/en/doc/qubes-builder/). This guide assumes you're using qubes-builder to build Qubes.
Repositories and committing Code
--------------------------------
Qubes is split into a bunch of git repos. This are all contained in the `qubes-src` directory under qubes-builder.
FIXME(ypid): Not on github?
The best way to write and contribute code is to create a git repo somewhere (e.g., github) for the repo you are interested in editing (e.g., `qubes-manager`, `core`, etc). To integrate your repo with the rest of Qubes, cd to the repo directory and add your repository as a remote in git
**Example:**
~~~
$ cd qubes-builder/qubes-src/qubes-manager
$ git remote add abel git@github.com:abeluck/qubes-manager.git
~~~
You can then proceed to easily develop in your own branches, pull in new commits from the dev branches, merge them, and eventually push to your own repo on github.
When you are ready to submit your changes to Qubes to be merged, push your changes, then create a signed git tag (using `git tag -s`). Finally, send a letter to the Qubes listserv describing the changes and including the link to your repository. Don't forget to include your public PGP key you use to sign your tags.
### Kernel-specific notes
#### Prepare fresh version of kernel sources, with Qubes-specific patches applied
In qubes-builder/qubes-src/kernel:
~~~
make prep
~~~
The resulting tree will be in kernel-\<VERSION\>/linux-\<VERSION\>:
~~~
ls -ltrd kernel*/linux*
~~~
~~~
drwxr-xr-x 23 user user 4096 Nov 5 09:50 kernel-3.4.18/linux-3.4.18
drwxr-xr-x 6 user user 4096 Nov 21 20:48 kernel-3.4.18/linux-obj
~~~
#### Go to the kernel tree and update the version
In qubes-builder/qubes-src/kernel:
~~~
cd kernel-3.4.18/linux-3.4.18
~~~
#### Changing the config
In kernel-3.4.18/linux-3.4.18:
~~~
cp ../../config-pvops .config
make oldconfig
~~~
Now change the configuration. For example, in kernel-3.4.18/linux-3.4.18:
~~~
make menuconfig
~~~
Copy the modified config back into the kernel tree:
~~~
cp .config ../../../config-pvops
~~~
#### Patching the code
TODO: describe the workflow for patching the code, below are some random notes, not working well
~~~
ln -s ../../patches.xen
export QUILT_PATCHES=patches.xen
export QUILT_REFRESH_ARGS="-p ab --no-timestamps --no-index"
export QUILT_SERIES=../../series-pvops.conf
quilt new patches.xen/pvops-3.4-0101-usb-xen-pvusb-driver-bugfix.patch
quilt add drivers/usb/host/Kconfig drivers/usb/host/Makefile \
drivers/usb/host/xen-usbback/* drivers/usb/host/xen-usbfront.c \
include/xen/interface/io/usbif.h
*edit something*
quilt refresh
cd ../..
vi series-pvops.conf
~~~
#### Building RPMS
TODO: Is this step generic for all subsystems?
Now it is a good moment to make sure you have changed kernel release name in rel-pvops file. For example, if you change it to '1debug201211116c' the resulting RPMs will be named 'kernel-3.4.18-1debug20121116c.pvops.qubes.x86\_64.rpm'. This will help distinguish between different versions of the same package.
You might want to take a moment here to review (git diff, git status), commit your changes locally.
To actually build RPMS, in qubes-src/kernel:
~~~
make rpms
~~~
RPMS will appear in qubes-src/kernel/rpm/x86\_64:
~~~
-rw-rw-r-- 1 user user 42996126 Nov 17 04:08 kernel-3.4.18-1debug20121116c.pvops.qubes.x86_64.rpm
-rw-rw-r-- 1 user user 43001450 Nov 17 05:36 kernel-3.4.18-1debug20121117a.pvops.qubes.x86_64.rpm
-rw-rw-r-- 1 user user 8940138 Nov 17 04:08 kernel-devel-3.4.18-1debug20121116c.pvops.qubes.x86_64.rpm
-rw-rw-r-- 1 user user 8937818 Nov 17 05:36 kernel-devel-3.4.18-1debug20121117a.pvops.qubes.x86_64.rpm
-rw-rw-r-- 1 user user 54490741 Nov 17 04:08 kernel-qubes-vm-3.4.18-1debug20121116c.pvops.qubes.x86_64.rpm
-rw-rw-r-- 1 user user 54502117 Nov 17 05:37 kernel-qubes-vm-3.4.18-1debug20121117a.pvops.qubes.x86_64.rpm
~~~
### Useful [QubesBuilder](/en/doc/qubes-builder/) commands
1. *make check* - will check if all the code was commited into repository and if all repository are tagged with signed tag.
2. *make show-vtags* - show version of each component (based on git tags) - mostly useful just before building ISO. **Note:** this will not show version for components containing changes since last version tag
3. *make push* - push change from **all** repositories to git server. You must set proper remotes (see above) for all repositories first.
4. *make prepare-merge* - fetch changes from remote repositories (can be specified on commandline via GIT\_SUBDIR or GIT\_REMOTE vars), (optionally) verify tags and show the changes. This do not merge the changes - there are left for review as FETCH\_HEAD ref. You can merge them using "git merge FETCH\_HEAD" (in each repo directory).
Copying Code to dom0
--------------------
When developing it is convenient to be able to rapidly test changes. Assuming you're developing Qubes on Qubes, you should be working in a special VM for Qubes and occasionally you will want to transfer code or rpms back to dom0 for testing.
Here are some handy scripts Marek has shared to facilitate this.
You may also like to run your [test environment on separate machine](/en/doc/test-bench/).
### Syncing dom0 files
TODO: edit this script to be more generic
~~~
#!/bin/sh
set -x
set -e
QUBES_PY_DIR=/usr/lib64/python2.6/site-packages/qubes
QUBES_PY=$QUBES_PY_DIR/qubes.py
QUBESUTILS_PY=$QUBES_PY_DIR/qubesutils.py
qvm-run -p qubes-devel 'cd qubes-builder/qubes-src/core/dom0; tar c qmemman/qmemman*.py qvm-core/*.py qvm-tools/* misc/vm-template-hvm.conf misc/qubes-start.desktop ../misc/block-snapshot aux-tools ../qrexec' |tar xv
cp $QUBES_PY qubes.py.bak$$
cp $QUBESUTILS_PY qubesutils.py.bak$$
cp /etc/xen/scripts/block-snapshot block-snapshot.bak$$
sudo cp qvm-core/qubes.py $QUBES_PY
sudo cp qvm-core/qubesutils.py $QUBESUTILS_PY
sudo cp qvm-core/guihelpers.py $QUBES_PY_DIR/
sudo cp qmemman/qmemman*.py $QUBES_PY_DIR/
sudo cp misc/vm-template-hvm.conf /usr/share/qubes/
sudo cp misc/qubes-start.desktop /usr/share/qubes/
sudo cp misc/block-snapshot /etc/xen/scripts/
sudo cp aux-tools/qubes-dom0-updates.cron /etc/cron.daily/
# FIXME(Abel Luck): I hope to
~~~
### Apply qvm-tools
TODO: make it more generic
~~~
#!/bin/sh
BAK=qvm-tools.bak$$
mkdir -p $BAK
cp -a /usr/bin/qvm-* /usr/bin/qubes-* $BAK/
sudo cp qvm-tools/qvm-* qvm-tools/qubes-* /usr/bin/
~~~
### Copy from dom0 to an appvm
~~~
#/bin/sh
#
# usage ./cp-domain <vm_name> <file_to_copy>
#
domain=$1
file=$2
fname=`basename $file`
qvm-run $domain 'mkdir /home/user/incoming/dom0 -p'
cat $file| qvm-run --pass-io $domain "cat > /home/user/incoming/dom0/$fname"
~~~

88
developers/building/InstallationIsoBuilding.md
View file

@ -1,88 +0,0 @@
---
layout: doc
title: InstallationIsoBuilding
permalink: /en/doc/installation-iso-building/
redirect_from:
- /doc/InstallationIsoBuilding/
- /wiki/InstallationIsoBuilding/
---
How to build Qubes installation ISO
===================================
Qubes uses [Fedora Unity Revisor](http://revisor.fedoraunity.org/) to build the installation ISO.
You may want to get familiar with [Revisor documentation](http://revisor.fedoraunity.org/documentation).
Build installer packages
------------------------
Get [Qubes Installer repository](http://git.qubes-os.org/?p=smoku/installer) and build its packages:
~~~
cd installer
make rpms
~~~
Packages will be in `rpm/noarch` and `rpm/x86_64`.
Install Revisor
---------------
Next install the freshly built revisor and anaconda:
~~~
yum install rpm/noarch/revisor*.rpm
yum install rpm/x86_64/anaconda*.rpm
~~~
Review configuration files
--------------------------
All configuration files for Qubes Revisor are kept in the ~~~conf/~~~ directory:
- `conf/qubes-install.conf` - Main Revisor configuration file. This configures Revisor to build Qubes Installation image based on Fedora 13. All other configuration files and working directories are pointed here.
- `conf/qubes-x86_64.conf` - This file describes all repositories needed to build Qubes for x86\_64 architecture.
- `conf/qubes-kickstart.cfg` - Fedora Kickstart formatted file describing which packages should land in the ISO `/Packages` repository. This describes basically what will be available for installation. The packages list built using this file will be further filtered by the comps file.
- `conf/comps-qubes.xml` - Repository Comps file for ISO `/Packages` repository, describing packages and package groups of the installer repository. Package groups are used to select which of the packages are mandatory to install, which are optional and which are to be just available on the ISO but not installed by default (not used on Qubes).
Create/Update local repository
------------------------------
Revisor fetches all RPM packages from YUM repositories. We currently use 5 repositories:
- `yum/installer` (installer-related rpms)
- `yum/qubes-dom0` (all the Qubes stuff)
- `yum/dom0-updates` (for select 3rd party packages, e.g. Xorg)
- `yum/fedora13-repo` (local fedora 13 repo, copy from DVD)
- remote fedora repo for extra packages (usually deps for qubes-dom0)
You need to manually copy the Fedora 13 installation DVD contents (`Packages/` and `repodata/` directories) into `build/fedora13-repo`.
Also, you need to copy all the qubes dom0 rpms into `build/yum/qubes-dom0/rpm` and run the `yum/update_repo.sh` script afterwards.
In order to fill the `build/yum/installer` repo one can just use `make update-repo`.
The `build/yum/dom0-updates` is to be used for select rpms that should also be used instead of those from the fedora (loacal and remote) repos.
Update your local repos:
~~~
make update-repo
~~~
Build ISO
---------
Now you're finally ready to build the ISO image:
~~~
make iso
~~~
and wait...
You may add `-d 1` (or `-d 99` if you're a masochist) in the Makefile at the end of the revisor command to get (a ton of) debugging information.

56
developers/building/KdeDom0.md
View file

@ -1,56 +0,0 @@
---
layout: doc
title: KdeDom0
permalink: /en/doc/kde-dom0/
redirect_from:
- /doc/KdeDom0/
- /wiki/KdeDom0/
---
Qubes-customized KDE packages for Dom0
======================================
The Qubes kde-dom0 project (see [Source Code](/en/doc/source-code/)) contains the source code needed for building the customized KDE packages for use in Qubes Dom0 (the user desktop). The packages are based on Fedora 12 KDE packages, but are heavily slimmed down (Qubes doesn't need lots of KDE functionality in Dom0, such as most of the KDE apps). In the near future those KDE packages will also get some Qubes specific extensions, such as coloured titlebars/frames nicely integrated into the KDE Window Manager. And, of course, custom themes, e.g. for KDM :)
Getting the sources
-------------------
~~~
git clone git://qubes-os.org/mainstream/kde-dom0.git kde-dom0
~~~
Building the packages
---------------------
It's best to use Fedora 12 or 13 as a development system.
First, you should download and verify the original KDE sources (not part of the kde-dom0 repository):
~~~
make get-sources verify-sources
~~~
Now, check if you have all the required build dependencies:
~~~
make prep
~~~
Install any required packages that `make` might have complained about. Then you're ready to build the rpms (you might want to adjust the release of each rpm package by editing the `rel` variable at the beginning of each `.spec` file):
~~~
make rpms
~~~
**Note:** The `kdebase-*` packages build process requires corresponding `kdelibs-devel` package to be installed first. If your build system is based on Fedora 12/13, and if the `kdelibs-devel` package exist in Fedora repo that is based the same KDE software version (e.g. 4.4.3) as the KDE packages you're building (see the `version` file), than you should be able to use the Fedora package:
~~~
yum install kdelibs-devel-{version}
~~~
If not, then you should build your `kdelibs-devel` first (`cd kdelibs-devel && make rpms`), then install it on your build system, and then you can build all the rest (`make rpms`).
Installing KDE packages from Qubes repository
---------------------------------------------
TODO

135
developers/building/QubesBuilder.md
View file

@ -1,135 +0,0 @@
---
layout: doc
title: QubesBuilder
permalink: /en/doc/qubes-builder/
redirect_from:
- /doc/QubesBuilder/
- /wiki/QubesBuilder/
---
Building Qubes from scratch
===========================
We have a fully automated build system for Qubes, that downloads, builds and
packages all the Qubes components, and finally should spit out a ready-to-use
installation ISO.
In order to use it one should use an rpm-based distro, like Fedora :) and should ensure the following packages are installed:
- git
- createrepo
- rpm-build
- make
- wget
- rpmdevtools
- python-sh
- dialog
- rpm-sign
Unusually one can install those packages by just issuing:
sudo yum install git createrepo rpm-build make wget rpmdevtools python-sh dialog rpm-sign
The build system creates build environments in chroots and so no other packages are needed on the host. All files created by the build system are contained within the qubes-builder directory. The full build requires some 25GB of free space, so keep that in mind when deciding where to place this directory.
The build system is configured via builder.conf file -- one should copy the attached builder.conf.default, and modify it as needed, e.g.:
cp example-configs/qubes-os-master.conf builder.conf
# edit the builder.conf file and set the following variables:
NO_SIGN=1
One additional useful requirement is that 'sudo root' work without any prompt, which is default on most distros (e.g. 'sudo bash' brings you the root shell without asking for any password). This is important as the builder needs to switch to root and then back to user several times during the build process.
Additionally, if building with signing enabled (so NO\_SIGN is not set), one must adjust \~/.rpmmacro file so that it point to the GPG key used for package signing, e.g.:
%_signature gpg
%_gpg_path /home/user/.gnupg
%_gpg_name AC1BF9B3 # <-- Key ID used for signing
It is also recommended to use an empty passphrase for the private key used for signing. Contrary to a popular belief, this doesn't affect your key or sources security -- if somebody compromised your system, then the game is over, whether you use additional passphrase for the key or not.
So, to build Qubes one would do:
# Import the Qubes master key
gpg --recv-keys 0x36879494
# Verify its fingerprint, set as 'trusted'.
# This is described here:
# https://www.qubes-os.org/doc/VerifyingSignatures
wget http://keys.qubes-os.org/keys/qubes-developers-keys.asc
gpg --import qubes-developers-keys.asc
git clone git://github.com/QubesOS/qubes-builder.git qubes-builder
cd qubes-builder
cp example-configs/qubes-os-master.conf builder.conf
# edit the builder.conf file and set the following variables:
# NO_SIGN="1"
# Download all components:
make get-sources
# And now to build all Qubes rpms (this will take a few hours):
make qubes
# ... and then to build the ISO
make iso
And this should produce a shiny new ISO.
You can also build selected component separately. Eg. to compile only gui virtualization agent/daemon:
make gui-daemon
Full list you can get from make help. For advanced use and preparing sources
for use with [QubesBuilder](/en/doc/qubes-builder/) take a look at [doc directory
in QubesBuilder](https://github.com/marmarek/qubes-builder/tree/master/doc) or
[QubesBuilderDetails](/en/doc/qubes-builder-details/) page.
Making customized build
-----------------------
### Manual source modification
If you want to somehow modify sources, you can also do it, here are some basic steps:
1. Download qubes-builder as described above (if you want to use marmarek's branches, you should also download qubes-builder from his repo - replace 'QubesOS' with 'marmarek' in above git clone command)
2. Edit builder.conf (still the same as above), some useful additions:
- You can also set GIT\_PREFIX="marmarek/qubes-" to use my repo instead of "mainstream" - it contains newer (but less tested) versions
3. Download unmodified sources
make get-sources
4. **Make your modifications here**
5. Build the Qubes
`make qubes` actually is just meta target which build all required
components in correct order. List of components is configured in
builder.conf. You can also check the current value at the end of `make
help`, or using `make build-info`.
6. `get-sources` is already done, so continue with the next one. You can skip `sign-all` if you've disabled signing
make vmm-xen core-admin linux-kernel gui-daemon template desktop-linux-kde installer-qubes-os manager linux-dom0-updates
1. build iso installation image
make iso
Code verification keys management
=================================
[QubesBuilder](/en/doc/qubes-builder/) by default verifies signed tags on every downloaded code. Public keys used for that are stored in `keyrings/git`. By default Qubes developers' keys are imported automatically, but if you need some additional keys (for example your own), you can add them using:
GNUPGHOME=$PWD/keyrings/git gpg --import /path/to/key.asc
GNUPGHOME=$PWD/keyrings/git gpg --edit-key ID_OF_JUST_IMPORTED_KEY
# here use "trust" command to set key fully or ultimately trusted - only those keys are accepted by QubesBuilder
All Qubes developers' keys are signed by the Qubes Master Signing Key (which is set as ultimately trusted key), so are trusted automatically.
If you are the owner of Master key and want to revoke such signature, use the `revsig` gpg key edit command and update the key in qubes-developers-keys.asc - now the key will be no longer trusted (unless manually set as such).

48
developers/building/QubesBuilderDetails.md
View file

@ -1,48 +0,0 @@
---
layout: doc
title: QubesBuilderDetails
permalink: /en/doc/qubes-builder-details/
redirect_from:
- /doc/QubesBuilderDetails/
- /wiki/QubesBuilderDetails/
---
[QubesBuilder](/en/doc/qubes-builder/) "API"
========================================
Components Makefile.builder file
--------------------------------
[QubesBuilder](/en/doc/qubes-builder/) expects that each component have *Makefile.builder* file in its root directory. This file specifies what should be done to build the package. As name suggests, this is normal makefile, which is included by builder as its configuration. Its main purpose is to set some variables. Generally all available variables/settings are described as comments at the beginning of Makefile.\* in [QubesBuilder](/en/doc/qubes-builder/).
Variables for Linux build:
- `RPM_SPEC_FILES` List (space separated) of spec files for RPM package build. Path should be relative to component root directory. [QubesBuilder](/en/doc/qubes-builder/) will install all BuildRequires (in chroot environment) before the build. In most Qubes components all spec files are kept in *rpm\_spec* directory. This is mainly used for Fedora packages build.
- `ARCH_BUILD_DIRS` List (space separated) of directories with PKGBUILD files for Archlinux package build. Similar to RPM build, [QubesBuilder](/en/doc/qubes-builder/) will install all makedepends, then build the package.
Most components uses *archlinux* directory for this purpose, so its good to keep this style.
Variables for Windows build:
- `WIN_COMPILER` Choose which compiler should be used for this component, thus which build scripts. Currently two options available:
- `WDK` - Windows Driver Kit (default). Command used to build: *build -cZg*.
- `mingw` - MinGW (Windows gcc port). Command used to build: *make all*
- `WIN_SOURCE_SUBDIRS` List of directories in which above command should be run. In most cases it will be only one entry: current directory (*.*).
- `WIN_PREBUILD_CMD` Command to run before build, mostly useful for WDK build (in mingw case, you can use makefile for this purpose). Can be used to set some variables, preprocess some files etc.
- `WIN_SIGN_CMD` Command used to sign resulting binaries. Note that default value is *sign.bat*. If you don't want to sign binaries, specify some placeholder here (eg. *true*). Check existing components (eg. vmm-xen-windows-pvdrivers) for example scripts. This command will be run with certain environment variables:
- `CERT_FILENAME` Path to key file (pfx format)
- `CERT_PASSWORD` Key password
- `CERT_PUBLIC_FILENAME` Certificate path in case of self-signed cert
- `CERT_CROSS_CERT_FILENAME` Certificate path in case of correct autheticode cert
- `SIGNTOOL` Path to signtool
- `WIN_PACKAGE_CMD` Command used to produce installation package (msi or msm). Default value is *wix.bat*, similar to above - use *true* if you don't want this command.
- `WIN_OUTPUT_HEADERS` Directory (relative to `WIN_SOURCE_SUBDIRS` element) with public headers of the package - for use in other components.
- `WIN_OUTPUT_LIBS` Directory (relative to `WIN_SOURCE_SUBDIRS` element) with libraries (both DLL and implib) of the package - for use in other components. Note that [QubesBuilder](/en/doc/qubes-builder/) will copy files specified as *\$(WIN\_OUTPUT\_LIBS)/\*/\** to match WDK directory layout (*\<specified directory\>/\<arch directory\>/\<actual libraries\>*), so you in mingw build you need to place libraries in some additional subdirectory.
- `WIN_BUILD_DEPS` List of components required to build this one. [QubesBuilder](/en/doc/qubes-builder/) will copy files specified with `WIN_OUTPUT_HEADERS` and `WIN_OUTPUT_LIBS` of those components to some directory and provide its path with `QUBES_INCLUDES` and `QUBES_LIBS` variables. Use those variables in your build scripts (*sources* or *Makefile* - depending on selected compiler). You can assume that the variables are always set and directories always exists, even if empty.
builder.conf settings
---------------------
Most settings are documented in *builder.conf.default* file, which can be used as template the actual configuration.
**TODO**

58
developers/building/QubesR3Building.md
View file

@ -1,58 +0,0 @@
---
layout: doc
title: QubesR3Building
permalink: /en/doc/qubes-r3-building/
redirect_from:
- /doc/QubesR3Building/
- /wiki/QubesR3Building/
---
Building Qubes OS 3.0 ISO
=========================
Ensure your system is rpm-based and that you have necessary dependencies installed (see [QubesBuilder](/en/doc/qubes-builder/) for more info):
~~~
sudo yum install git createrepo rpm-build make wget rpmdevtools pandoc
~~~
Get the necessary keys to verify the sources:
~~~
$ wget https://keys.qubes-os.org/keys/qubes-master-signing-key.asc
$ gpg --import qubes-master-signing-key.asc
$ gpg --edit-key 36879494
# Verify fingerprint!, set trust to *ultimate*
$ wget https://keys.qubes-os.org/keys/qubes-developers-keys.asc
$ gpg --import qubes-developers-keys.asc
~~~
Note we do *not* relay above on the security of our server (keys.qubes-os.org) nor the connection (ssl, cert) -- we only rely on you getting the Qubes Master Signing Key fingerprint *somehow* and ensure they match!
Now lets bootstrap the builder. Unfortunately the builder cannot verify itself (the classic Chicken and Egg problem), so we need to verify the signature manually:
~~~
$ git clone git://github.com/QubesOS/qubes-builder.git
$ cd qubes-builder
$ git describe --exact-match HEAD
<some tag>
$ git tag -v <some tag>
~~~
Assuming the verification went fine, we're good to go with all the rest without ever thinking more about verifying digital signatures on all the rest of the components, as the builder will do that for us, for each component, every time we, even for all aux files (e.g. Xen or Linux kernel sources).
Let's configure the builder first (we can use one of the example configs, either for R2 or "master", which currently means pre-released R3):
~~~
cp example-configs/qubes-os-master.conf builder.conf
~~~
You can take a loot at the `builder.conf.default` for a description of all available options. Nevertheless, the default config should be enough for start:
~~~
$ make get-sources qubes
$ make sign-all # this requires setting SIGN_KEY in the builder.conf, can be skipped for test builds.
$ make iso
~~~
Enjoy your new ISO!

37
developers/building/Usbvm.md
View file

@ -1,37 +0,0 @@
---
layout: doc
title: USBVM
permalink: /en/doc/usbvm/
redirect_from:
- /doc/USBVM/
- /wiki/USBVM/
---
USB Pass through: USBVM
-----------------------
**WARNING:** This is experimental and very broken.
Source: [https://groups.google.com/d/msg/qubes-devel/4AKulABh2Jc/\_R7SRSC4peYJ](https://groups.google.com/d/msg/qubes-devel/4AKulABh2Jc/_R7SRSC4peYJ)
You'll need the patch tagged abb\_e58b432 from [git://github.com/grwl/qubes-core.git](git://github.com/grwl/qubes-core.git).
The rest is in RPMs, yes. I roughly follow this procedure to have pvusb on Qubes 1.0:
- rebuild kernel, core, and xen from marmarek's repo (devel-3.4 branch for kernel),
- make dedicated usbvm domain, passthrough USB controller PCI devices there
- in usbvm: "rpm -Uhv qubes-core-vm-2.1.1-1.fc17.x86\_64.rpm xen-libs-2000:4.1.3-18.fc17.x86\_64.rpm xen-qubes-vm-essentials-2000:4.1.3-18.fc17.x86\_64.rpm"
- in appvm where you want to attach the device to: "rpm -Uhv qubes-core-vm-2.1.1-1.fc17.x86\_64.rpm"
- in dom0 I think the following packages need to be updated, don't have exact list now.qubes-core-dom0 is actually important, the rest are dependencies:
> kernel-qubes-vm-3.4.18-1.pvops.qubes.x86\_64.rpm
> qubes-core-dom0-2.1.1-1.fc13.x86\_64.rpm
> xen-4.1.3-18.fc13.x86\_64.rpm
> xen-hvm-4.1.3gui2.0.9-18.fc13.x86\_64.rpm
> xen-hypervisor-4.1.3-18.fc13.x86\_64.rpm
> xen-libs-4.1.3-18.fc13.x86\_64.rpm
> xen-licenses-4.1.3-18.fc13.x86\_64.rpm
> xen-runtime-4.1.3-18.fc13.x86\_64.rpm
- plug or replug some USB device -- it will be picked up by usbvm, and in dom0 it will appear in the output of qvm-usb

33
developers/contributing.md
View file

@ -1,33 +0,0 @@
---
layout: doc
title: Contributing
permalink: /en/doc/contributing/
redirect_from:
- /doc/contributing/
- "/doc/ContributingHowto/"
- "/wiki/ContributingHowto/"
---
How can I contribute to the Qubes Project?
==========================================
Ok, so you think Qubes Project is cool and you would like to contribute? You are very welcome!
First you should decide what you are interested in (and good in). The Qubes project would welcome contributions in various areas:
- Testing and [bug reporting](/en/doc/reporting-bugs/)
- Code audit (e.g. gui-daemon)
- New features
- Artwork (plymouth themes, KDM themes, installer themes, wallpapers, etc)
Perhaps the best starting point is to have a look at the [issues](https://github.com/QubesOS/qubes-issues/issues) to see what are the most urgent tasks to do.
Before you engage in some longer activity, e.g. implementing a new feature, it's always good to contact us first (preferably via the [qubes-devel](/en/doc/qubes-lists/) list), to avoid a situation when two or more independent people would work on the same feature at the same time, doubling each others work. When you contact us and devote to a particular task, we will create a ticket for this task with info who is working on this feature and what is the expected date of some early code to be posted.
When you are ready to start some work, read how to [access Qubes sources and send patches](/en/doc/source-code/).
You can also contribute in other areas than coding and testing, e.g. by providing mirrors for Qubes rpm repositories, providing feedback about what features you would like to have in Qubes, or perhaps even preparing some cool You Tube videos that would demonstrate some Qubes' features. You are always encouraged to discuss your ideas on qubes-devel.
You should be aware, however, that we will not blindly accept all the contributions! We will accept only the quality ones. Open source doesn't mean lack of quality control! If we reject your patch, please do not get discouraged, try fixing it so that it adheres to the required standards. We will only reject contributions in the good faith, to make Qubes a better OS.
Thanks, The Qubes Project.

136
developers/debugging/AutomatedTests.md
View file

@ -1,136 +0,0 @@
---
layout: doc
title: Automated tests
permalink: /en/doc/automated-tests/
redirect_from: /doc/AutomatedTests/
---
Automatic tests
===============
Starting with Qubes R3 we use [python unittest][unittest] to perform automatic
tests of Qubes OS. Regardless of the name, we use it for both [unit
tests](https://en.wikipedia.org/wiki/Unit_tests) and [integration
tests](https://en.wikipedia.org/wiki/Integration_tests). The main purpose is of
course to deliver much more stable releases.
Integration tests are written with assumption to be called on dedicated
hardware. **Do not run those test on machine where you have important data, you
can loose it**. Especially all the VMs with name starting with `test-` are
removed. All the tests are started from dom0, even when testing some VM
component. Those tests will create new VM(s), run the test, then remove the VM(s).
Most of the tests are stored in [core-admin
repository](https://github.com/QubesOS/qubes-core-admin/tree/master/tests) in
`tests` directory. To start them you can use standard python unittest runner:
python -m unittest -v qubes.tests
Or our custom one:
python -m qubes.tests.run -v
Our test runner can be used mostly the same as the standard one, with some nice
additional features like no need to prefix all the tests with "qubes.tests", or
color output. It is also the only way to run only selected template tests.
You can use `python -m qubes.tests.run -h` to get usage information:
[user@dom0 ~]$ python -m qubes.tests.run -h
usage: run.py [-h] [--verbose] [--quiet] [--list] [--failfast] [--no-failfast]
[--do-not-clean] [--do-clean] [--loglevel LEVEL]
[--logfile FILE] [--syslog] [--no-syslog] [--kmsg] [--no-kmsg]
[TESTNAME [TESTNAME ...]]
positional arguments:
TESTNAME list of tests to run named like in description
(default: run all tests)
optional arguments:
-h, --help show this help message and exit
--verbose, -v increase console verbosity level
--quiet, -q decrease console verbosity level
--list, -l list all available tests and exit
--failfast, -f stop on the first fail, error or unexpected success
--no-failfast disable --failfast
--do-not-clean, --dnc, -D
do not execute tearDown on failed tests. Implies
--failfast.
--do-clean, -C do execute tearDown even on failed tests.
--loglevel LEVEL, -L LEVEL
logging level for file and syslog forwarding (one of:
NOTSET, DEBUG, INFO, WARN, WARNING, ERROR, CRITICAL;
default: DEBUG)
--logfile FILE, -o FILE
if set, test run will be also logged to file
--syslog reenable logging to syslog
--no-syslog disable logging to syslog
--kmsg, --very-brave-or-very-stupid
log most important things to kernel ring-buffer
--no-kmsg, --i-am-smarter-than-kay-sievers
do not abuse kernel ring-buffer
When running only specific tests, write their names like in log, in format:
MODULE+"/"+CLASS+"/"+FUNCTION. MODULE should omit initial "qubes.tests.".
Example: basic/TC_00_Basic/test_000_create
For example to run only tests for fedora-21 template, you can use `-l` option, then filter the list:
[user@dom0 ~]$ python -m qubes.tests.run -l | grep fedora-21
network/VmNetworking_fedora-21/test_000_simple_networking
network/VmNetworking_fedora-21/test_010_simple_proxyvm
network/VmNetworking_fedora-21/test_020_simple_proxyvm_nm
network/VmNetworking_fedora-21/test_030_firewallvm_firewall
network/VmNetworking_fedora-21/test_040_inter_vm
vm_qrexec_gui/TC_00_AppVM_fedora-21/test_000_start_shutdown
vm_qrexec_gui/TC_00_AppVM_fedora-21/test_010_run_gui_app
vm_qrexec_gui/TC_00_AppVM_fedora-21/test_050_qrexec_simple_eof
vm_qrexec_gui/TC_00_AppVM_fedora-21/test_051_qrexec_simple_eof_reverse
vm_qrexec_gui/TC_00_AppVM_fedora-21/test_052_qrexec_vm_service_eof
vm_qrexec_gui/TC_00_AppVM_fedora-21/test_053_qrexec_vm_service_eof_reverse
vm_qrexec_gui/TC_00_AppVM_fedora-21/test_060_qrexec_exit_code_dom0
vm_qrexec_gui/TC_00_AppVM_fedora-21/test_065_qrexec_exit_code_vm
vm_qrexec_gui/TC_00_AppVM_fedora-21/test_100_qrexec_filecopy
vm_qrexec_gui/TC_00_AppVM_fedora-21/test_110_qrexec_filecopy_deny
vm_qrexec_gui/TC_00_AppVM_fedora-21/test_120_qrexec_filecopy_self
vm_qrexec_gui/TC_20_DispVM_fedora-21/test_000_prepare_dvm
vm_qrexec_gui/TC_20_DispVM_fedora-21/test_010_simple_dvm_run
vm_qrexec_gui/TC_20_DispVM_fedora-21/test_020_gui_app
vm_qrexec_gui/TC_20_DispVM_fedora-21/test_030_edit_file
[user@dom0 ~]$ python -m qubes.tests.run -v `python -m qubes.tests.run -l | grep fedora-21`
Example test run:
![snapshot-tests2.png](/attachment/wiki/developers/snapshot-tests2.png)
## Adding a new test to core-admin
After you added a new unit test to [core-admin/tests](https://github.com/QubesOS/qubes-core-admin/tree/master/tests)
you have to make sure of two things:
1. The test will be added to the RPM file created by [QubesBuilder](/en/doc/qubes-builder/)
For this you need to edit [core-admin/tests/Makefile](https://github.com/QubesOS/qubes-core-admin/tree/master/tests/Makefile)
2. The test will be loaded by [core-admin/tests/\_\_init\_\_.py](https://github.com/QubesOS/qubes-core-admin/tree/master/tests/__init__.py)
### Editing the Makefile
Add at the bottom of the file the two lines which will copy your test and its
compiled version to the right directory in the RPM file. I.e. adding `example.py`
cp example.py $(DESTDIR)$(PYTHON_TESTSPATH)
cp example.py[co] $(DESTDIR)$(PYTHON_TESTSPATH)
### Editing \_\_init\_\_.py
Add at the bottom of the file in the method `def load_tests` to the variable
`modname` your test. I.e adding `example.py`.
~~~python
for modname in (
'qubes.tests.basic',
'qubes.tests.dom0_update',
'qubes.tests.network',
'qubes.tests.vm_qrexec_gui',
'qubes.tests.backup',
'qubes.tests.backupcompatibility',
'qubes.tests.regressions',
'qubes.tests.example', # This is our newly added test
):
~~~
[unittest]: https://docs.python.org/2/library/unittest.html

97
developers/debugging/Profiling.md
View file

@ -1,97 +0,0 @@
---
layout: doc
title: Profiling
permalink: /en/doc/profiling/
redirect_from:
- /doc/Profiling/
- /wiki/Profiling/
---
Profiling
=========
This is python profiling primer.
For the purpose of this document, `qubes-dev` is name of the domain used for postprocessing profiling stats.
Requirements
------------
~~~
yum install gprof2dot graphviz
git clone http://git.woju.eu/qubes/profiling.git
~~~
If you profile something on dom0, move `Upload.sh` from repository to dom0:
~~~
mkdir -p ~/profiling
qvm-run -p qubes-dev 'cat ~/profiling/Upload.sh' > ~/profiling/Upload.sh
~~~
- WARNING: this will obviously be running third party code which is not signed by ITL nor Fedora. You have been warned.
Workflow
--------
### Identify function responsible for some slow action
You have to select area in which you suspect less than optimal performance. If you do not narrow the area, graphs may be unreadable.
### Replace suspect function with probe
Replace
def foo(self, bar):
# function content
with
def foo(self, *args, **kwargs):
profile.runctx('self.real_foo(*args, **kwargs)', globals(), locals(),
time.strftime('/home/user/profiling/foo-%Y%m%d-%H%M%S.pstats'))
def real_foo(self, bar):
# function content
### Run application
Beware that some functions may be called often. For example `qubesmanager/main.py:update_table` gets run once per second. This will produce one pstat file per second.
Remember to revert your changes to application afterwards.
### Upload statistics
If you are in dom0:
~~~
cd ~/profiling
./Upload.sh
~~~
### Analyse
~~~
make
~~~
For every `${basename}.pstats` this will produce `${basename}.txt` and `${basename}.svg`. SVG contains call graph. Text file contains list of all functions sorted by cumulative execution time. You may also try `make all-png`.
~~~
make index.html
~~~
This creates `index.html` with all SVG graphics linked to TXT files. Ready for upload.
~~~
make REMOTE=example.com:public_html/qubes/profiling/ upload
~~~
Example
-------
This example is from `qubes-manager` (`qubesmanager/main.py`).
!["update\_table-20140424-170010.svg"](//attachment/wiki/Profiling/update_table-20140424-170010.svg)
It is apparent than problem is around `get_disk_usage` which calls something via `subprocess.call`. It does it 15 times, probably once per VM.

155
developers/debugging/TestBench.md
View file

@ -1,155 +0,0 @@
---
layout: doc
title: TestBench
permalink: /en/doc/test-bench/
redirect_from:
- /doc/TestBench/
- /wiki/TestBench/
---
Test bench for Dom0
===================
This guide shows how to set up simple test bench that automatically test your code you're about to push. It is written especially for `core3` branch of `core-admin.git` repo, but some ideas are universal.
We will set up a spare machine (bare metal, not a virtual) that will be hosting our experimental Dom0. We will communicate with it via Ethernet and SSH. This tutorial assumes you are familiar with [QubesBuilder](/en/doc/qubes-builder/) and you have it set up and running flawlessly.
Setting up the machine
----------------------
First, do a clean install from ISO you built or grabbed elsewhere.
You have to fix network, because it is intentionally broken. This script should reenable your network card without depending on anything else.
#!/bin/sh
# adjust this for your NIC (run lspci)
BDF=0000:02:00.0
prog=$(basename $0)
pciunbind() {
local path
path=/sys/bus/pci/devices/${1}/driver/unbind
if ! [ -w ${path} ]; then
echo "${prog}: Device ${1} not bound"
return 1
fi
echo -n ${1} >${path}
}
pcibind() {
local path
path=/sys/bus/pci/drivers/${2}/bind
if ! [ -w ${path} ]; then
echo "${prog}: Driver ${2} not found"
return 1
fi
echo ${1} >${path}
}
pciunbind ${BDF}
pcibind ${BDF} e1000e
dhclient
TODO: describe how to run this at every startup
Now configure your DHCP server so your testbench gets static IP and connect your machine to your local network. You should ensure that your testbench can reach the Internet.
Install `openssh-server` on your testbench:
~~~
yum install openssh-server
~~~
Ensure that sudo works without password from your user account (it should by default).
Development VM
--------------
### SSH
Arrange firewall so you can reach the testbench from your `qubes-dev` VM. Generate SSH key in `qubes-dev`:
~~~
ssh-keygen -t ecdsa -b 521
~~~
Add the following section in `.ssh/config` in `qubes-dev`:
~~~
Host testbench
# substitute username in testbench
User user
# substitute address of your testbench
HostName 192.168.123.45
~~~
Then connect to your testbench and paste newly generated `id_ecdsa.pub` to `.ssh/authorized_keys` on testbench so you can log in without entering password every time.
### Scripting
This step is optional, but very helpful. Put these scripts somewhere in your `${PATH}`, like `/usr/local/bin`.
`qtb-runtests`:
#!/bin/sh
ssh testbench python -m qubes.tests.run
`qtb-install`:
#!/bin/sh
TMPDIR=/tmp/qtb-rpms
if [ $# -eq 0 ]; then
echo "usage: $(basename $0) <rpmfile> ..."
exit 2
fi
set -e
ssh testbench mkdir -p "${TMPDIR}"
scp "${@}" testbench:"${TMPDIR}"
while [ $# -gt 0 ]; do
ssh testbench sudo rpm -i --replacepkgs --replacefiles "${TMPDIR}/$(basename ${1})"
shift
done
`qtb-iterate`:
#!/bin/sh
set -e
# substitute path to your builder installation
pushd ${HOME}/builder >/dev/null
# the following are needed only if you have sources outside builder
#rm -rf qubes-src/core-admin
#make COMPONENTS=core-admin get-sources
make core-admin
qtb-install qubes-src/core-admin/rpm/x86_64/qubes-core-dom0-*.rpm
qtb-runtests
### Hooking git
I (woju) have those two git hooks. They ensure tests are passing (or are marked as expected failure) when commiting and pushing. For committing it is only possible to run tests that may be executed from git repo (even if the rest were available, I probably wouldn't want to do that). For pushing, I also install RPM and run tests on testbench.
`core-admin/.git/hooks/pre-commit`: (you may retain also the default hook, here omitted for readability)
#!/bin/sh
set -e
python -c "import sys, qubes.tests.run; sys.exit(not qubes.tests.run.main())"
`core-admin/.git/hooks/pre-push`:
#!/bin/sh
exec qtb-iterate

128
developers/debugging/VmInterface.md
View file

@ -1,128 +0,0 @@
---
layout: doc
title: VMInterface
permalink: /en/doc/vm-interface/
redirect_from:
- /doc/VMInterface/
- "/doc/SystemDoc/VMInterface/"
- "/wiki/SystemDoc/VMInterface/"
---
VM Configuration Interface
==========================
Qubes VM have some settings set by dom0 based on VM settings. There are multiple configuration channels, which includes:
- XenStore
- QubesDB - replacing most of xenstore (in R3 only)
- Qubes RPC (called at VM startup, or when configuration changed)
- GUI protocol
xenstore
--------
Keys exposed by dom0 to VM (only Qubes specific included):
- `qubes-vm-type` - VM type, the same as `type` field in `qvm-prefs`. One of `AppVM`, `ProxyVM`, `NetVM`, `TemplateVM`, `HVM`, `TemplateHVM`
- `qubes-vm-updatable` - flag whether VM is updatable (whether changes in root.img will survive VM restart). One of `True`, `False`
- `qubes-timezone - name of timezone based on dom0 timezone. For example `Europe/Warsaw`
- `qubes-keyboard` - keyboard layout based on dom0 layout. Its syntax is suitable for `xkbcomp` command (after expanding escape sequences like `\n` or `\t`). This is meant only as some default value, VM can ignore this option and choose its own keyboard layout (this is what keyboard setting from Qubes Manager does). This entry is created as part of gui-daemon initialization (so not available when gui-daemon disabled, or not started yet).
- `qubes-debug-mode` - flag whether VM have debug mode enabled (qvm-prefs setting). One of `1`, `0`
- `qubes-service/SERVICE_NAME` - subtree for VM services controlled from dom0 (using qvm-service command or Qubes Manager). One of `1`, `0`. Note that not every service will be listed here, if entry is missing, it means "use VM default". List of currently supported services is in [qvm-service man page](/wiki/Dom0Tools/QvmService)
- `qubes-netmask` - network mask (only when VM has netvm set); currently hardcoded "255.255.255.0"
- \`qubes-ip - IP address for this VM (only when VM has netvm set)
- `qubes-gateway` - default gateway IP and primary DNS address (only when VM has netvm set); VM should add host route to this address directly via eth0 (or whatever default interface name is)
- `qubes-secondary-dns` - secondary DNS address (only when VM has netvm set)
- `qubes-netvm-gateway` - same as `qubes-gateway` in connected VMs (only when VM serves as network backend - ProxyVM and NetVM); because this is also set as primary DNS in connected VMs, traffic sent to this IP on port 53 should be redirected to DNS server
- `qubes-netvm-netmask` - same as `qubes-netmask` in connected VMs (only when VM serves as network backend - ProxyVM and NetVM)
- `qubes-netvm-network` - network address (only when VM serves as network backend - ProxyVM and NetVM); can be also calculated from qubes-netvm-gateway and qubes-netvm-netmask
- `qubes-netvm-secondary-dns` - same as `qubes-secondary-dns` in connected VMs (only when VM serves as network backend - ProxyVM and NetVM); traffic sent to this IP on port 53 should be redirected to secondary DNS server
Keys set by VM for passing info to dom0:
- `memory/meminfo` - used memory (updated by qubes-meminfo-writer), input information for qmemman; Format: 6 lines (EOL encoded as `\n`), each in format "FIELD: VALUE kB"; fields: `MemTotal`, `MemFree`, `Buffers`, `Cached`, `SwapTotal`, `SwapFree`; meaning the same as in `/proc/meminfo` in Linux
- `qubes-block-devices` - list of block devices exposed by this VM, each device (subdirectory) should be named in a way that VM can attach the device based on it. Each should contain those entries:
- `desc` - device description (ASCII text)
- `size` - device size in bytes
- `mode` - default connection mode; `r` for read-only, `w` for read-write
- `qubes-usb-devices` - list of USB devices exposed by this VM, each device (subdirectory) should contain:
- `desc` - device description (ASCII text)
- `usb-ver` - USB version (1, 2 or 3)
Qubes RPC
---------
Services called by dom0 to provide some VM configuration:
- `qubes.SetMonitorLayout` - provide list of monitors, one in a line, each line contains four numbers: `width height X Y`
- `qubes.WaitForSession` - called to wait for full VM startup
- `qubes.GetAppmenus` - receive appmenus from given VM (template); TODO: describe format here
- `qubes.GetImageRGBA` - receive image/application icon. Protocol:
1. Caller sends name of requested icon. This can be one of:
* `xdgicon:NAME` - search for NAME in standard icons theme
* `-` - get icon data from stdin (the caller), can be prefixed with format name, for example `png:-`
* file name
2. The service responds with image dimensions: width and height as
decimal numbers, separated with space and with EOL marker at the and; then
image data in RGBA format (32 bits per pixel)
- `qubes.SetDateTime` - set VM time, called periodically by dom0 (can be
triggered manually from dom0 by calling `qvm-sync-clock`). The service
receives one line at stdin - time in format of `date -u -Iseconds`, for
example `2015-07-31T16:10:43+0000`.
- `qubes.SetGuiMode` - called in HVM to switch between fullscreen and seamless
GUI mode. The service receives a single word on stdin - either `FULLSCREEN`
or `SEAMLESS`
Other Qrexec services installed by default:
- `qubes.Backup` - store Qubes backup. The service receives location chosen by
the user (one line, terminated by '\n'), the backup archive ([description of
backup format](/doc/BackupEmergencyRestoreV2/))
- `qubes.DetachPciDevice` - service called in reaction to `qvm-pci -d` call on
running VM. The service receives one word - BDF of device to detach. When the
service call ends, the device will be detached
- `qubes.Filecopy` - receive some files from other VM. Files sent in [qfile format](/en/doc/qfilecopy/)
- `qubes.OpenInVM` - open a file in called VM. Service receives a single file on stdin (in
[qfile format](/en/doc/qfilecopy/). After a file viewer/editor is terminated, if
the file was modified, can be sent back (just raw content, without any
headers); otherwise service should just terminate without sending anything.
This service is used by both `qvm-open-in-vm` and `qvm-open-in-dvm` tools. When
called in DispVM, service termination will trigger DispVM cleanup.
- `qubes.Restore` - retrieve Qubes backup. The service receives backup location
entered by the user (one line, terminated by '\n'), then should output backup
archive in [qfile format](/en/doc/qfilecopy/) (core-agent-linux component contains
`tar2qfile` utility to do the conversion
- `qubes.SelectDirectory`, `qubes.SelectFile` - services which should show
file/directory selection dialog and return (to stdout) a single line
containing selected path, or nothing in case of cancellation
- `qubes.SuspendPre` - service called in every VM with PCI device attached just
before system suspend
- `qubes.SuspendPost` - service called in every VM with PCI device attached just
after system resume
- `qubes.SyncNtpClock` - service called to trigger network time synchronization.
Service should synchronize local VM time and terminate when done.
- `qubes.WindowIconUpdater` - service called by VM to send icons of individual
windows. The protocol there is simple one direction stream: VM sends window ID
followed by icon in `qubes.GetImageRGBA` format, then next window ID etc. VM
can send icon for the same window multiple times to replace previous one (for
example for animated icons)
- `qubes.VMShell` - call any command in the VM; the command(s) is passed one per line
Currently Qubes still calls few tools in VM directly, not using service
abstraction. This will change in the future. Those tools are:
- `/usr/lib/qubes/qubes-download-dom0-updates.sh` - script to download updates (or new packages to be installed) for dom0 (`qubes-dom0-update` tool)
- `date -u -Iseconds` - called directly to retrieve time after calling `qubes.SyncNtpClock` service (`qvm-sync-clock` tool)
- `nm-online -x` - called before `qubes.SyncNtpClock` service call by `qvm-sync-clock` tool
- `resize2fs` - called to resize filesystem on /rw partition by `qvm-grow-private` tool
- `gpk-update-viewer` - called by Qubes Manager to display available updates in a TemplateVM
- `systemctl start qubes-update-check.timer` (and similarly stop) - called when enabling/disabling updates checking in given VM (`qubes-update-check` [qvm-service](/en/doc/qubes-service/))
Additionally automatic tests extensively calls various commands directly in VMs. We do not plan to change that.
GUI protocol
------------
GUI initialization includes passing the whole screen dimensions from dom0 to VM. This will most likely be overwritten by qubes.SetMonitorLayout Qubes RPC call.

212
developers/debugging/WindowsDebugging.md
View file

@ -1,212 +0,0 @@
---
layout: doc
title: WindowsDebugging
permalink: /en/doc/windows-debugging/
redirect_from:
- /doc/WindowsDebugging/
- /wiki/WindowsDebugging/
---
Debugging Windows HVMs
======================
Debugging Windows code can be tricky in a virtualized environment. The guide below assumes Xen hypervisor and Windows 7 VMs.
User-mode debugging is usually straightforward if it can be done on one machine. Just duplicate your normal debugging environment in the VM.
Things get complicated if you need to perform kernel debugging or troubleshoot problems that only manifest on system boot, user logoff or similar. For that you need two Windows VMs: the *host* and the *target*. The *host* will contain [WinDbg](http://msdn.microsoft.com/en-us/library/windows/hardware/ff551063(v=vs.85).aspx) installation, your source code and private symbols. The *target* will run the code being debugged. Both will be linked by virtual serial ports.
- First, you need to prepare separate copies of both *target* and *host* VM configuration files with some changes. Copy the files from **/var/lib/qubes/appvms/vmname/vmname.conf** to some convenient location, let's call them **host.conf** and **target.conf**.
- In both copied files add the following line at the end: `serial = 'pty'`. This will make Xen connect VM's serial ports to dom0's ptys.
- From now on you need to start both VMs like this: `qvm-start --custom-config=/your/edited/host.conf host`
- To connect both VM serial ports together you will either need [socat](http://www.dest-unreach.org/socat/) or a custom utility described later.
- To determine which dom0 pty corresponds to VM's serial port you need to read xenstore, example script below:
#!/bin/sh
id1=$(xl domid "$1-dm")
tty1=$(xenstore-read /local/domain/${id1}/device/console/3/tty)
echo $tty1
Pass it a running VM name and it will output the corresponding pty name.
- To connect both ptys you can use [socat](http://www.dest-unreach.org/socat/) like that:
#!/bin/sh
id1=$(xl domid "$1-dm")
id2=$(xl domid "$2-dm")
tty1=$(xenstore-read /local/domain/${id1}/device/console/3/tty)
tty2=$(xenstore-read /local/domain/${id2}/device/console/3/tty)
socat $tty1,raw $tty2,raw
...but there is a catch. Xen seems to process the traffic that goes through serial ports and changes all **0x0a** bytes into **0x0d, 0x0a** pairs (newline conversion). I didn't find a way to turn that off (setting ptys to raw mode didn't change anything) and it's not mentioned anywhere on the Internet, so maybe it's something on my system. If the above script works for you then you don't need anything more in dom0.
- On the *target* system, run `bcdedit /set debug on` on the console to turn on kernel debugging. It defaults to the first serial port.
- On the *host* system, install [WinDbg](http://msdn.microsoft.com/en-us/library/windows/hardware/ff551063(v=vs.85).aspx) and start the kernel debug (Ctrl-K), choose **com1** as the debug port.
- Reboot the *target* VM.
- Run the above shell script in dom0.
- If everything is fine you should see the proper kernel debugging output in WinDbg. However, if you see something like that:
~~~
Opened \\.\com1
Waiting to reconnect...
Connected to Windows 7 7601 x64 target at (Wed Mar 19 20:35:43.262 2014 (UTC + 1:00)), ptr64 TRUE
Kernel Debugger connection established.
Symbol search path is: srv*c:\symbols*http://msdl.microsoft.com/download/symbols
Executable search path is:
... Retry sending the same data packet for 64 times.
The transport connection between host kernel debugger and target Windows seems lost.
please try resync with target, recycle the host debugger, or reboot the target Windows.
Unable to read KTHREAD address fffff8000281ccc0
**************************************************************************
Unable to read debugger data block header
**************************************************************************
Unable to read KTHREAD address fffff8000281ccc0
Unable to read PsLoadedModuleList
Unable to read KTHREAD address fffff8000281ccc0
**************************************************************************
Unable to read debugger data block header
**************************************************************************
~~~
...then you're most likely a victim of the CRLF issue mentioned above. To get around it I wrote a small utility that basically does what socat would do and additionally corrects those replaced bytes in the stream. It's not pretty but it works:
#include <errno.h>
#include <stdio.h>
#include <fcntl.h>
#include <termios.h>
int fd1, fd2;
char mark = ' ';
void out(unsigned char c)
{
static int count = 0;
static unsigned char buf[17] = {0};
// relay to ouptput port
write(fd2, &c, 1);
fprintf(stderr, "%c", mark);
/* dump all data going over the line
if (count == 0)
fprintf(stderr, "%c", mark);
fprintf(stderr, "%02x ", c);
if (c >= 0x20 && c < 0x80)
buf[count] = c;
else
buf[count] = '.';
count++;
if (count == 0x10)
{
count = 0;
fprintf(stderr, " %s\n", buf);
}
*/
}
int main(int argc, char* argv[])
{
unsigned char c = 0;
struct termios tio;
ssize_t size;
if (argc < 3)
{
fprintf(stderr, "Usage: %s pty1 pty2 [mark character]\n", argv[0]);
return EINVAL;
}
fd1 = open(argv[1], O_RDONLY | O_NOCTTY);
if (fd1 <= 0)
{
perror("open fd1");
return errno;
}
fd2 = open(argv[2], O_WRONLY | O_NOCTTY);
if (fd2 <= 0)
{
perror("open fd2");
return errno;
}
/*
// This doesn't make any difference which supports the theory
// that it's Xen who corrupts the byte stream.
cfmakeraw(&tio);
if (tcsetattr(fd1, TCSANOW, &tio) < 0)
{
perror("tcsetattr 1");
return errno;
}
if (tcsetattr(fd2, TCSANOW, &tio) < 0)
{
perror("tcsetattr 2");
return errno;
}
*/
if (argc == 4)
mark = argv[3][0];
while (1)
{
size = read(fd1, &c, 1);
if (size <= 0)
break;
parse:
if (c == 0x0d)
{
size = read(fd1, &c, 1);
if (size <= 0)
{
out(0x0d);
break;
}
if (c == 0x0a)
{
out(0x0a);
}
else
{
out(0x0d);
goto parse;
}
}
else
out(c);
}
close(fd1);
close(fd2);
return 0;
}
> This utility is a unidirectional relay so you need to run two instances to get duplex communication, like:
>
> #!/bin/sh
>
> id1=$(xl domid "$1-dm")
> id2=$(xl domid "$2-dm")
> tty1=$(xenstore-read /local/domain/${id1}/device/console/3/tty)
> tty2=$(xenstore-read /local/domain/${id2}/device/console/3/tty)
> ./ptycrlf ${tty1} ${tty2} - &
> ./ptycrlf ${tty2} ${tty1} + &
> With this everything should be good:
>
> ~~~
> Opened \\.\com1
> Waiting to reconnect...
> Connected to Windows 7 7601 x64 target at (Wed Mar 19 20:56:31.371 2014 (UTC + 1:00)), ptr64 TRUE
> Kernel Debugger connection established.
> Symbol search path is: srv*c:\symbols*http://msdl.microsoft.com/download/symbols
> Executable search path is:
> Windows 7 Kernel Version 7601 MP (1 procs) Free x64
> Built by: 7601.18247.amd64fre.win7sp1_gdr.130828-1532
> Machine Name:
> Kernel base = 0xfffff800`0261a000 PsLoadedModuleList = 0xfffff800`0285d6d0
> System Uptime: not available
> ~~~
Happy debugging!

58
developers/doc-guidelines.md
View file

@ -1,58 +0,0 @@
---
layout: doc
title: Documentation Guidelines
permalink: /en/doc/doc-guidelines/
redirect_from:
- /doc/doc-guidelines/
- "/wiki/DocStyle/"
- "/doc/DocStyle/"
---
Guidelines for Documentation Contributors
=========================================
All Qubes OS documentation pages are stored as plain text files in the
dedicated [qubes-doc] repository. By cloning and regularly pulling from
this repo, users can maintain their own up-to-date offline copy of all Qubes
documentation rather than relying solely on the Web. Contributions to the
documentation (both new content and edits of existing content) are welcome. To
contribute, please [fork and clone][gh-fork] this repo, make your changes,
then either [submit a pull request][gh-pull] or [send a patch][patch] to the
`qubes-devel` [mailing list][lists]. If you have a GitHub account (free), you
can simply browse the [qubes-doc] repository and edit the files there. The
GitHub interface will automatically guide you through the
[fork and pull request creation process][gh-fork].
Markdown Conventions
--------------------
All the documentation is written in Markdown for maximum accessibility. When
making contributions, please observe the following style conventions:
* Use spaces instead of tabs.
* Hard wrap Markdown lines at 80 characters.
* Hard wrap Git commit message lines at 72 characters.
* This leaves exactly four spaces on each side of the commit message when
viewed in the default `git log` format.)
* If appropriate, make numerals in numbered lists match between Markdown
source and HTML output.
* In the event that a user is required to read the Markdown source
directly, this will make it easier to follow, e.g., numbered steps in a
set of instructions.
* Use hanging indentations
where appropriate.
* Use underline headings (`=====` and `-----`) if possible. If this is not
possible, use Atx-style headings on both the left and right sides
(`### H3 ###`).
* Use `[reference-style][ref]` links.
`[ref]: http://daringfireball.net/projects/markdown/syntax#link`
[qubes-doc]: https://github.com/QubesOS/qubes-doc
[qubes]: https://github.com/QubesOS
[gh-fork]: https://guides.github.com/activities/forking/
[gh-pull]: https://help.github.com/articles/using-pull-requests/
[patch]: /doc/SourceCode/#sending-a-patch
[lists]: https://www.qubes-os.org/doc/QubesLists/

255
developers/fundamentals/GuiDocs.md
View file

@ -1,255 +0,0 @@
---
layout: doc
title: GUIdocs
permalink: /en/doc/gui-docs/
redirect_from:
- /doc/GUIdocs/
- /wiki/GUIdocs/
---
Qubes GUI protocol
==================
qubes\_gui and qubes\_guid processes
------------------------------------
All AppVM X applications connect to local (running in AppVM) Xorg server, that uses the following "hardware" drivers:
- *dummy\_drv* - video driver, that paints onto a framebuffer located in RAM, not connected to real hardware
- *qubes\_drv* - it provides a virtual keyboard and mouse (in fact, more, see below)
For each AppVM, there is a pair of *qubes\_gui* (running in AppVM) and *qubes\_guid* (running in dom0) processes, connected over vchan. Main responsibilities of *qubes\_gui* are:
- call XCompositeRedirectSubwindows on the root window, so that each window has its own composition buffer
- instruct the local Xorg server to notify it about window creation, configuration and damage events; pass information on these events to dom0
- receive information about keyboard and mouse events from dom0, tell *qubes\_drv* to fake appropriate events
- receive information about window size/position change, apply them to the local window
Main responsibilities of *qubes\_guid* are:
- create a window in dom0 whenever an information on window creation in AppVM is received from *qubes\_gui*
- whenever the local window receives XEvent, pass information on it to AppVM (particularly, mouse and keyboard data)
- whenever AppVM signals damage event, tell local Xorg server to repaint a given window fragment
- receive information about window size/position change, apply them to the local window
Note that keyboard and mouse events are passed to AppVM only if a window belonging to this AppVM has focus. AppVM has no way to get information on keystrokes fed to other AppVMs (e.g. XTEST extension will report the status of local AppVM keyboard only), nor synthetize and pass events to other AppVMs.
Window content updates implementation
-------------------------------------
Typical remote desktop applications, like *vnc*, pass the information on all changed window content in-band (say, over tcp). As the channel has limited throughput, this impacts video performance. In case of Qubes, *qubes\_gui* does not transfer all changed pixels via vchan. Instead, for each window, upon its creation or size change, *qubes\_gui*
- asks *qubes\_drv* driver for the list of physical memory frames that hold the composition buffer of a window
- pass this information via `MFNDUMP` message to *qubes\_guid* in dom0
Now, *qubes\_guid* has to tell dom0 Xorg server about the location of the buffer. There is no supported way (e.g. Xorg extension) to do this zero-copy style. The following method is used in Qubes:
- in dom0, the Xorg server is started with *LD\_PRELOAD*-ed library named *shmoverride.so*. This library hooks all function calls related to shared memory.
- *qubes\_guid* creates a shared memory segment, and then tells Xorg to attach it via *MIT-SHM* extension
- when Xorg tries to attach the segment (via glibc *shmat*) *shmoverride.so* intercepts this call and instead maps AppVM memory via *xc\_map\_foreign\_range*
- since then, we can use MIT-SHM functions, e.g. *XShmPutImage* to draw onto a dom0 window. *XShmPutImage* will paint with DRAM speed; actually, many drivers use DMA for this.
The important detail is that *xc\_map\_foreign\_range* verifies that a given mfn range actually belongs to a given domain id (and the latter is provided by trusted *qubes\_guid*). Therefore, rogue AppVM cannot gain anything by passing crafted mnfs in the `MFNDUMP` message.
To sum up, this solution has the following benefits:
- window updates at DRAM speed
- no changes to Xorg code
- minimal size of the supporting code
![gui.png](/attachment/wiki/GUIdocs/gui.png)
Security markers on dom0 windows
--------------------------------
It is important that user knows which AppVM a given window belongs to. This prevents an attack when a rogue AppVM paints a window pretending to belong to other AppVM or dom0, and tries to steal e.g. passwords.
In Qubes, the custom window decorator is used, that paints a colourful frame (the colour is determined during AppVM creation) around decorated windows. Additionally, window title always starts with **[name of the AppVM]**. If a window has a *override\_redirect* attribute, meaning that it should not be treated by a window manager (typical case is menu windows), *qubes\_guid* draws a two-pixel colourful frame around it manually.
Clipboard sharing implementation
--------------------------------
Certainly, it would be insecure to allow AppVM to read/write clipboard of other AppVMs unconditionally. Therefore, the following mechanism is used:
- there is a "qubes clipboard" in dom0 - its contents is stored in a regular file in dom0.
- if user wants to copy local AppVM clipboard to qubes clipboard, she must focus on any window belonging to this AppVM, and press **Ctrl-Shift-C**. This combination is trapped by *qubes-guid*, and `CLIPBOARD_REQ` message is sent to AppVM. *qubes-gui* responds with *CLIPBOARD\_DATA* message followed by clipboard contents.
- user focuses on other AppVM window, presses **Ctrl-Shift-V**. This combination is trapped by *qubes-guid*, and `CLIPBOARD_DATA` message followed by qubes clipboard contents is sent to AppVM; *qubes\_gui* copies data to the the local clipboard, and then user can paste its contents to local applications normally.
This way, user can quickly copy clipboards between AppVMs. This action is fully controlled by the user, it cannot be triggered/forced by any AppVM.
*qubes\_gui* and *qubes\_guid* code notes
-----------------------------------------
Both applications are structures similarly. They use *select* function to wait for any of the two event sources
- messages from the local X server
- messages from the vchan connecting to the remote party
The XEvents are handled by *handle\_xevent\_eventname* function, messages are handled by *handle\_messagename* function. One should be very careful when altering the actual *select* loop - e.g. both XEvents and vchan messages are buffered, meaning that *select* will not wake for each message.
If one changes the number/order/signature of messages, one should increase the *QUBES\_GUID\_PROTOCOL\_VERSION* constant in *messages.h* include file.
*qubes\_guid* writes debugging information to */var/log/qubes/qubes.domain\_id.log* file; *qubes\_gui* writes debugging information to */var/log/qubes/gui\_agent.log*. Include these files when reporting a bug.
AppVM -\> dom0 messages
-----------------------
Proper handling of the below messages is security-critical. Observe that beside two messages (`CLIPBOARD` and `MFNDUMP`) the rest have fixed size, so the parsing code can be small.
The *override\_redirect* window attribute is explained at [Override Redirect Flag](http://tronche.com/gui/x/xlib/window/attributes/override-redirect.html). The *transient\_for* attribute is explained at [Transient\_for attribute](http://tronche.com/gui/x/icccm/sec-4.html#WM_TRANSIENT_FOR).
Window manager hints and flags are described at [http://standards.freedesktop.org/wm-spec/latest/](http://standards.freedesktop.org/wm-spec/latest/), especially part about `_NET_WM_STATE`.
Each message starts with the following header
~~~
struct msghdr {
uint32_t type;
uint32_t window;
/* This field is intended for use by gui_agents to skip unknown
* messages from the (trusted) guid. Guid, on the other hand,
* should never rely on this field to calculate the actual len of
* message to be read, as the (untrusted) agent can put here
* whatever it wants! */
uint32_t untrusted_len;
};
~~~
The header is followed by message-specific data.
|Message name|Structure after header|Action|
|:-----------|:---------------------|:-----|
|MSG\_CLIPBOARD\_DATA|amorphic blob (length determined by the "window" field)|Store the received clipboard content (not parsing in any way)|
|MSG\_CREATE|` struct msg_create { `
` uint32_t x; `
` uint32_t y; `
` uint32_t width; `
` uint32_t height; `
` uint32_t parent; `
` uint32_t override_redirect; `
` }; `|Create a window with given parameters|
|MSG\_DESTROY|None|Destroy a window|
|MSG\_MAP|` struct msg_map_info { `
` uint32_t transient_for; `
` uint32_t override_redirect; `
` }; `|Map a window with given parameters|
|MSG\_UNMAP|None|Unmap a window|
|MSG\_CONFIGURE|` struct msg_configure { `
` uint32_t x; `
` uint32_t y; `
` uint32_t width; `
` uint32_t height; `
` uint32_t override_redirect; `
` }; `|Change window position/size/type|
|MSG\_MFNDUMP|` struct shm_cmd { `
` uint32_t shmid; `
` uint32_t width; `
` uint32_t height; `
` uint32_t bpp; `
` uint32_t off; `
` uint32_t num_mfn; `
` uint32_t domid; `
` uint32_t mfns[0]; `
` }; `|Retrieve the array of mfns that constitute the composition buffer of a remote window.
The "num\_mfn" 32bit integers follow the shm\_cmd structure; "off" is the offset of the composite buffer start in the first frame; "shmid" and "domid" parameters are just placeholders (to be filled by *qubes\_guid*), so that we can use the same structure when talking to *shmoverride.so*|
|MSG\_SHMIMAGE|` struct msg_shmimage { `
` uint32_t x; `
` uint32_t y;`
` uint32_t width;`
` uint32_t height;`
` }; `|Repaint the given window fragment|
|MSG\_WMNAME|` struct msg_wmname { `
` char data[128]; `
` } ; `|Set the window name; only printable characters are allowed|
|MSG\_DOCK|None|Dock the window in the tray|
|MSG\_WINDOW\_HINTS|` struct msg_window_hints { `
` uint32_t flags; `
` uint32_t min_width; `
` uint32_t min_height; `
` uint32_t max_width; `
` uint32_t max_height; `
` uint32_t width_inc; `
` uint32_t height_inc; `
` uint32_t base_width; `
` uint32_t base_height; `
` }; `|Size hints for window manager|
|MSG\_WINDOW\_FLAGS|` struct msg_window_flags { `
` uint32_t flags_set; `
` uint32_t flags_unset;`
` }; `|Change window state request; fields contains bitmask which flags request to be set and which unset|
Dom0 -\> AppVM messages
-----------------------
Proper handling of the below messages is NOT security-critical.
Each message starts with the following header
~~~
struct msghdr {
uint32_t type;
uint32_t window;
};
~~~
The header is followed by message-specific data.
` KEYPRESS, BUTTON, MOTION, FOCUS ` messages pass information extracted from dom0 XEvent; see appropriate event documentation.
|Message name|Structure after header|Action|
|:-----------|:---------------------|:-----|
|MSG\_KEYPRESS|` struct msg_keypress { `
` uint32_t type; `
` uint32_t x; `
` uint32_t y; `
` uint32_t state; `
` uint32_t keycode; `
` }; `|Tell *qubes\_drv* driver to generate a keypress|
|MSG\_BUTTON|` struct msg_button { `
` uint32_t type; `
` uint32_t x; `
` uint32_t y; `
` uint32_t state; `
` uint32_t button; `
` }; `|Tell *qubes\_drv* driver to generate mouseclick|
|MSG\_MOTION|` struct msg_motion { `
` uint32_t x; `
` uint32_t y; `
` uint32_t state; `
` uint32_t is_hint; `
` }; `|Tell *qubes\_drv* driver to generate motion event|
|MSG\_CONFIGURE|` struct msg_configure { `
` uint32_t x; `
` uint32_t y; `
` uint32_t width; `
` uint32_t height; `
` uint32_t override_redirect; `
` }; `|Change window position/size/type|
|MSG\_MAP|` struct msg_map_info { `
` uint32_t transient_for; `
` uint32_t override_redirect; `
` }; `|Map a window with given parameters|
|MSG\_CLOSE|None|send wmDeleteMessage to the window|
|MSG\_CROSSING|` struct msg_crossing { `
` uint32_t type; `
` uint32_t x; `
` uint32_t y; `
` uint32_t state; `
` uint32_t mode; `
` uint32_t detail; `
` uint32_t focus; `
` }; `|Notify window about enter/leave event|
|MSG\_FOCUS|` struct msg_focus { `
` uint32_t type; `
` uint32_t mode; `
` uint32_t detail; `
` }; `|Raise a window, XSetInputFocus|
|MSG\_CLIPBOARD\_REQ|None|Retrieve the local clipboard, pass contents to gui-daemon|
|MSG\_CLIPBOARD\_DATA|amorphic blob|Insert the received data into local clipboard|
|MSG\_EXECUTE|Obsolete|Obsolete, unused|
|MSG\_KEYMAP\_NOTIFY|` unsigned char remote_keys[32]; `|Synchronize the keyboard state (key pressed/released) with dom0|
|MSG\_WINDOW\_FLAGS|` struct msg_window_flags { `
` uint32_t flags_set; `
` uint32_t flags_unset;`
` }; `|Window state change confirmation|

34
developers/fundamentals/QubesArchitecture.md
View file

@ -1,34 +0,0 @@
---
layout: doc
title: QubesArchitecture
permalink: /en/doc/qubes-architecture/
redirect_from:
- /doc/QubesArchitecture/
- /wiki/QubesArchitecture/
---
Qubes Architecture Overview
===========================
Qubes implements a Security by Isolation approach. To do this, Qubes utilizes virtualization technology in order to isolate various programs from each other and even to sandbox many system-level components, such as networking and storage subsystems, so that the compromise of any of these programs or components does not affect the integrity of the rest of the system.
Qubes lets the user define many security domains, which are implemented as lightweight Virtual Machines (VMs), or “AppVMs.” For example, the user can have “personal,” “work,” “shopping,” “bank,” and “random” AppVMs and can use the applications within those VMs just as if they were executing on the local machine. At the same time, however, these applications are well isolated from each other. Qubes also supports secure copy-and-paste and file sharing between the AppVMs, of course.
[![qubes-arch-diagram-1.png](/attachment/wiki/QubesArchitecture/qubes-arch-diagram-1.png)](/attachment/wiki/QubesArchitecture/qubes-arch-diagram-1.png)
(Note: In the diagram above, "Storage domain" is actually a USB domain.)
Key Architecture features
-------------------------
- Based on a secure bare-metal hypervisor (Xen)
- Networking code sand-boxed in an unprivileged VM (using IOMMU/VT-d)
- USB stacks and drivers sand-boxed in an unprivileged VM (currently experimental feature)
- No networking code in the privileged domain (dom0)
- All user applications run in “AppVMs,” lightweight VMs based on Linux
- Centralized updates of all AppVMs based on the same template
- Qubes GUI virtualization presents applications as if they were running locally
- Qubes GUI provides isolation between apps sharing the same desktop
- Secure system boot based (optional)
[Architecture Spec v0.3 [PDF]](/attachment/wiki/QubesArchitecture/arch-spec-0.3.pdf) (The original 2009 document that started this all...)

57
developers/fundamentals/QubesNet.md
View file

@ -1,57 +0,0 @@
---
layout: doc
title: QubesNet
permalink: /en/doc/qubes-net/
redirect_from:
- /doc/QubesNet/
- /wiki/QubesNet/
---
VM network in Qubes
===================
Overall description
-------------------
In Qubes, the standard Xen networking is used, based on backend driver in the driver domain and frontend drivers in VMs. In order to eliminate layer 2 attacks originating from a compromised VM, routed networking is used instead of the default bridging of `vif` devices. The default *vif-route* script had some deficiencies (requires `eth0` device to be up, and sets some redundant iptables rules), therefore the custom *vif-route-qubes* script is used.
The IP address of `eth0` interface in AppVM, as well as two IP addresses to be used as nameservers (`DNS1` and `DNS2`), are passed via xenstore to AppVM during its boot (thus, there is no need for DHCP daemon in the network driver domain). `DNS1` and `DNS2` are private addresses; whenever an interface is brought up in the network driver domain, the */usr/lib/qubes/qubes\_setup\_dnat\_to\_ns* script sets up the DNAT iptables rules translating `DNS1` and `DNS2` to the newly learned real dns servers. This way AppVM networking configuration does not need to be changed when configuration in the network driver domain changes (e.g. user switches to a different WLAN). Moreover, in the network driver domain, there is no DNS server either, and consequently there are no ports open to the VMs.
Routing tables examples
-----------------------
VM routing table is simple:
||
|Destination|Gateway|Genmask|Flags|Metric|Ref|Use|Iface|
|0.0.0.0|0.0.0.0|0.0.0.0|U|0|0|0|eth0|
Network driver domain routing table is a bit longer:
||
|Destination|Gateway|Genmask|Flags|Metric|Ref|Use|Iface|
|10.2.0.16|0.0.0.0|255.255.255.255|UH|0|0|0|vif4.0|
|10.2.0.7|0.0.0.0|255.255.255.255|UH|0|0|0|vif10.0|
|10.2.0.9|0.0.0.0|255.255.255.255|UH|0|0|0|vif9.0|
|10.2.0.8|0.0.0.0|255.255.255.255|UH|0|0|0|vif8.0|
|10.2.0.12|0.0.0.0|255.255.255.255|UH|0|0|0|vif3.0|
|192.168.0.0|0.0.0.0|255.255.255.0|U|1|0|0|eth0|
|0.0.0.0|192.168.0.1|0.0.0.0|UG|0|0|0|eth0|
Location of the network driver domain
-------------------------------------
Traditionally, the network driver domain is dom0. This design means that a lot of code (networking stack, drivers) running in the all-powerful domain is exposed to potential attack. Although it is supported (one can execute *qvm-set-default-netvm dom0*), it is strongly discouraged.
Instead, a dedicated domain called `netvm` should be used. In order to activate it, one needs to install the `qubes-servicevm-netvm` rpm package, and enable it via command *qvm-set-default-netvm netvm*. This domain will be assigned all PCI devices that are network cards. One can interact with the *Networkmanager* daemon running in `netvm` in the same way as with any other VM GUI application (with one detail that *nm-applet* requires a system tray, thus one needs to start it via "KDEMenu-\>Applications-\>Netvm-\>Show Tray").
Note that in order to isolate `netvm` properly, the platform must support VTd and it must be activated. Otherwise, compromised `netvm` can use DMA to get control over dom0 and even the hypervisor.
When using `netvm`, there is no network connectivity in dom0. This is the desired configuration - it eliminates all network-bourne attacks. Observe that dom0 is meant to be used for administrative tasks only, and (with one exception) they do not need network. Anything not related to system administration should be done in one of AppVMs.
The above-mentioned exception is the system packages upgrade. Again, one must not install random applications in dom0, but there is a need to e.g. upgrade existing packages. While one may argue that the new packages could be downloaded on a separate machine and copied to dom0 via a pendrive, this solution has its own problems. Therefore, the advised method to temporarily grant network connectivity to dom0 is to use *qvm-dom0-network-via-netvm up* command. It will pause all running VMs (so that they can do no harm to dom0) and connect dom0 to netvm network just like another AppVM. Having completed package upgrade, execute *qvm-dom0-network-via-netvm down* to revert to the normal state.
Firewall and Proxy VMs
----------------------
TODO

57
developers/fundamentals/SecurityCriticalCode.md
View file

@ -1,57 +0,0 @@
---
layout: doc
title: SecurityCriticalCode
permalink: /en/doc/security-critical-code/
redirect_from:
- /doc/SecurityCriticalCode/
- /wiki/SecurityCriticalCode/
- /trac/wiki/SecurityCriticalCode/
---
Security-Critical Code in Qubes OS
==================================
Below is a list of security-critical (AKA trusted) code in Qubes OS. A successful attack against any of those might allow to compromise the Qubes OS security. This code can be thought of as of a Trusted Computing Base (TCB) of Qubes OS. The goal of the project has been to minimize the amount of this trusted code to an absolute minimum. The size of the current TCB is of an order of hundreds thousands of lines of C code, which is several orders of magnitude less than in other OSes, such as Windows, Linux or Mac, where it is of orders of tens of millions of lines of C code.
For more information about the security goals of Qubes OS, see [this page](/en/doc/security-goals/).
Security-Critical Qubes-Specific Components
-------------------------------------------
Below is a code produced by the Qubes project that is security-critical.
- Dom0-side of the libvchan library
- Dom0-side of the GUI virtualization code (*qubes-guid*)
- Dom0-side of the sound virtualization code (*pacat-simple-vchan*)
- Dom0-side in qrexec-related code (*qrexec\_daemon*)
- VM memory manager (*qmemman*) that runs in Dom0
- select Qubes RPC servers that run in Dom0: qubes.ReceiveUpdates and qubes.SyncAppMenus
- The qubes.Filecopy RPC server that runs in a VM -- this one is critical because it might allow one VM to compromise another one if user allows file copy operation to be performed between them
Security-Critical 3rd-Party Components
--------------------------------------
These are the components that we haven't written or designed ourselves, yet we still rely on them. At the current project stage, we cannot afford to spend time to thoroughly review and audit them, so we just more or less "blindly" trust they are secure.
- Xen hypervisor
- The Xen's xenstore backend running in Dom0
- The Xen's block backend running in Dom0 kernel
- The rpm program used in Dom0 for verifying Dom0 updates' signatures
- Somehow optional: log viewing software in dom0 that parses VM-influenced logs
Additionally when we consider attacks that originate through a compromised network domain and target the VMs connected to it. Those attacks do not apply to domains connected to other network domains (Qubes allows more than one network domains), or those with networking disabled (Dom0 is not connected to any network by default).
- Xen network PV frontends
- VM's core networking stacks (core TCP/IP code)
Buggy Code vs. Back-doored Code Distinction
-------------------------------------------
There is an important distinction between the buggy code and maliciously trojaned code. We could have the most secure architecture and the most bulletproof TCB that perfectly isolates all domains from each other, but it still would be pretty useless if all the code used within domains, e.g. the actual email clients, word processors, etc, was somehow trojaned. In that case only network-isolated domains could be somehow trusted, while all others could be not.
The above means that we must trust at least some of the vendors (not all, of course, but at least those few that provide the apps that we use in the most critical domains). In practice in Qubes OS we trust the software provided by Fedora project. This software is signed by Fedora distribution keys and so it is also critical that the tools used in domains for software updates (yum and rpm) be trusted.
Cooperative Covert Channels Between Domains
-------------------------------------------
Qubes does not attempt to eliminate all possible *cooperative* covert channels between domains, i.e. such channels that could be established between two *compromised* domains. We don't believe this is possible to achieve on x86 hardware, and we also doubt it makes any sense in practice for most users -- after all if the two domains are compromised, then it's already (almost) all lost anyway.

110
developers/fundamentals/TemplateImplementation.md
View file

@ -1,110 +0,0 @@
---
layout: doc
title: TemplateImplementation
permalink: /en/doc/template-implementation/
redirect_from:
- /doc/TemplateImplementation/
- /wiki/TemplateImplementation/
---
Overview of VM block devices
============================
Every VM has 4 block devices connected:
- **xvda** – base root device (/) – details described below
- **xvdb** – private.img – place where VM always can write.
- **xvdc** – volatile.img, discarded at each VM restart – here is placed swap and temporal "/" modifications (see below)
- **xvdd** – modules.img – kernel modules and firmware
private.img (xvdb)
------------------
This is mounted as /rw and here is placed all VM private data. This includes:
- */home* – which is bind mounted to /rw/home
- */usr/local* – which is symlink to /rw/usrlocal
- some config files (/rw/config) called by qubes core scripts (ex /rw/config/rc.local)
modules.img (xvdd)
------------------
As the kernel is chosen in dom0, there must be some way to provide matching kernel modules to VM. Qubes kernel directory consists of 3 files:
- *vmlinuz* – actual kernel
- *initramfs* – initial ramdisk containing script to setup snapshot devices (see below) and mount /lib/modules
- *modules.img* – filesystem image of /lib/modules with matching kernel modules and firmware (/lib/firmware/updates is symlinked to /lib/modules/firmware)
Normally kernel "package" is common for many VMs (can be set using qvm-prefs). One of them can be set as default (qvm-set-default-kernel) to simplify kernel updates (by default all VMs use the default kernel). All installed kernels are placed in /var/lib/qubes/vm-kernels as separate subdirs. In this case, modules.img is attached to the VM as R/O device.
There is a special case when the VM can have a custom kernel – when it is updateable (StandaloneVM or TemplateVM) and the kernel is set to "none" (by qvm-prefs). In this case the VM uses the kernel from the "kernels" VM subdir and modules.img is attached as R/W device. FIXME: "none" should be renamed to "custom".
Qubes TemplateVM implementation
===============================
TemplateVM has a shared root.img across all AppVMs that are based on it. This mechanism has some advantages over a simple common device connected to multiple VMs:
- root.img can be modified while there are AppVMs running – without corrupting the filesystem
- multiple AppVMs that are using different versions of root.img (from various points in time) can be running concurrently
There are two layers of the device-mapper snapshot device; the first one enables modifying root.img without stopping the AppVMs and the second one, which is contained in the AppVM, enables temporal modifications to its filesystem. These modifications will be discarded after a restart of the AppVM.
![TemplateSharing2.png](/attachment/wiki/TemplateImplementation/TemplateSharing2.png)
Snapshot device in Dom0
-----------------------
This device consists of:
- root.img – real template filesystem
- root-cow.img – differences between the device as seen by AppVM and the current root.img
The above is achieved through creating device-mapper snapshots for each version of root.img. When an AppVM is started, a xen hotplug script (/etc/xen/scripts/block-snapshot) reads the inode numbers of root.img and root-cow.img; these numbers are used as the snapshot device's name. When a device with the same name exists the new AppVM will use it – therefore, AppVMs based on the same version of root.img will use the same device. Of course, the device-mapper cannot use the files directly – it must be connected through /dev/loop\*. The same mechanism detects if there is a loop device associated with a file determined by the device and inode numbers – or if creating a new loop device is necessary.
When an AppVM is stopped the xen hotplug script checks whether the device is still in use – if it is not, the script removes the snapshot and frees the loop device.
### Changes to template filesystem
In order for the full potential of the snapshot device to be realized, every change in root.img must save the original version of the modified block in root-cow.img. This is achieved by a snapshot-origin device.
When TemplateVM is started, it receives the snapshot-origin device connected as a root device (in read-write mode). Therefore, every change to this device is immediately saved in root.img – but remains invisible to the AppVM, which uses the snapshot.
When TemplateVM is stopped, the xen script moves root-cow.img to root-cow.img.old and creates a new one (using the `qvm-template-commit` tool). The snapshot device will remain untouched due to the loop device, which uses an actual file on the disk (by inode, not by name). Linux kernel frees the old root-cow.img files as soon as they are unused by all snapshot devices (to be exact, loop devices). The new root-cow.img file will get a new inode number, and so new AppVMs will get new snapshot devices (with different names).
### Rollback template changes
There is possibility to rollback last template changes. Saved root-cow.img.old contains all changes made during last TemplateVM run. Rolling back changes is done by reverting this "binary patch".
This is done using snapshot-merge device-mapper target (available from 2.6.34 kernel). It requires that no other snapshot device uses underlying block devices (root.img, root-cow.img via loop device). Because of this all AppVMs based on this template must be halted during this operation.
Steps performed by **qvm-revert-template-changes**:
1. Ensure that no other VMs uses this template.
2. Prepare snapshot device with ***root-cow.img.old*** instead of *root-cow.img* (*/etc/xen/scripts/block-snapshot prepare*).
3. Replace *snapshot* device-mapper target with *snapshot-merge*, other parameters (chunk size etc) remains untouched. Now kernel starts merging changes stored in *root-cow.img.old* into *root.img*. d-m device can be used normally (if needed).
4. Waits for merge completed: *dmsetup status* shows used snapshot blocks – it should be equal to metadata size when completed.
5. Replace *snapshot-merge* d-m target back to *snapshot*.
6. Cleanup snapshot device (if nobody uses it it the moment).
7. Move *root-cow.img.old* to *root-cow.img* (overriding existing file).
Snapshot device in AppVM
------------------------
Root device is exposed to AppVM in read-only mode. AppVM can write only in:
- private.img – persistent storage (mounted in /rw) used for /home, /usr/local – in future versions, its use may be extended
- volatile.img – temporary storage, which is discarded after an AppVM restart
volatile.img is divided into two partitions:
1. changes to root device
2. swap partition
Inside of an AppVM, the root device is wrapped by the snapshot in the first partition of volatile.img. Therefore, the AppVM can write anything to its filesystem – however, such changes will be discarded after a restart.
StandaloneVM
------------
Standalone VM enables user to modify root filesystem persistently. It can be created using *--standalone* switch to *qvm-create*.
It is implemented just like TemplateVM (has own root.img connected as R/W device), but no other VMs can be based on it.

30
developers/license.md
View file

@ -1,30 +0,0 @@
---
layout: doc
title: License
permalink: /en/doc/license/
redirect_from:
- /doc/license/
- "/doc/QubesLicensing/"
- "/wiki/QubesLicensing/"
---
Qubes OS License
================
Qubes is a compilation of software packages, each under its own license. The compilation is made available under the GNU General Public License version 2.
The full text of the GPL v2 license can be found [here](http://www.gnu.org/licenses/gpl-2.0.html).
Parts of the Qubes OS under proprietary license
-----------------------------------------------
Some software produced by the Qubes Project is proprietary software of Invisible Things Lab. Those parts are not distributed as part of the Qubes OS installation ISOs, but can be downloaded separately.
The following software is licensed under a proprietary license:
- Qubes Windows Support Tools
Note on rights to double-licensing of the Qubes code
----------------------------------------------------
Invisible Things Lab (ITL), who has funded and run the Qubes project since the beginning, and who has contributed majority of Qubes-specific code (specifically: `core-*`, `gui-*`, and `qubes-*` repositories) would like to have a right to redistribute parts of this code under proprietary licenses. This is especially important for Qubes R3 and later, where the new architecture allows the creation of many editions of Qubes, using different hypervisors, some of which might not be open source. That's why we ask every developer who contributes code to Qubes project to grant ITL permission to reuse the code under a different license, and to express this consent by including the [standard signed-off line](http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/Documentation/SubmittingPatches?id=HEAD#n358) in the commit.

81
developers/reporting-bugs.md
View file

@ -1,81 +0,0 @@
---
layout: doc
title: Reporting Bugs
permalink: /en/doc/reporting-bugs/
redirect_from:
- /doc/reporting-bugs/
- "/doc/BugReportingGuide/"
- "/wiki/BugReportingGuide/"
---
Bug Reporting Guide
===================
One of the most important contribution task is reporting the bugs you have found.
Asking a Question
-----------------
Before you ask, do some searching and reading. Check [the
docs](https://www.qubes-os.org/doc/), Google, GitHub, and StackOverflow. If
your question is something that has been answered many times before, the
project maintainers might be tired of repeating themselves.
Whenever possible, ask your question on the Qubes mailing list which is
located [here](https://groups.google.com/forum/#!forum/qubes-users). This
allows anyone to answer and makes the answer available for the next person
with the same question.
Submitting a Bug Report (or "Issue")
------------------------------------
On GitHub, "Bug Reports" are called "Issues."
Issues can be submitted to the Qubes project located at
[https://github.com/QubesOS/qubes-issues](https://github.com/QubesOS/qubes-issues).
### Has This Been Asked Before?
Before you submit a bug report, you should search existing issues. Be sure
to check both currently open issues, as well as issues that are already
closed. If you find an issue that seems to be similar to yours, read
through it.
If this issue is the same as yours, you can comment with additional
information to help the maintainer debug it. Adding a comment will
subscribe you to email notifications, which can be helpful in getting
important updates regarding the issue. If you don't have anything to add
but still want to receive email updates, you can click the "watch" button
at the bottom of the comments.
### Nope, Hasn't Been Asked Before
If you can't find anything in the existing issues, don't be shy about
filing a new one.
You should be sure to include the version the project, as well as versions
of related software. For example, be sure to include the Qubes release
version (R2, R3) and specific version numbers of package causing problems
(if known).
If your issue is related to hardware, provide as many details as possible
about the hardware, which could include using commandline tools such as
`lspci`.
Project maintainers really appreciate thorough explanations. It usually
helps them address the problem more quickly, so everyone wins!
Improving the Code
------------------
The best way is to "Fork" the repo on GitHub. This will create a copy of
the repo on your GitHub account.
Before you set out to improve the code, you should have a focused idea in
mind of what you want to do.
Each commit should do one thing, and each PR should be one specific
improvement. Each PR needs to be signed.
* [How can I contribute to the Qubes Project?](https://www.qubes-os.org/doc/ContributingHowto/)
* [Developer Documentation](https://www.qubes-os.org/doc/)
* [Package Release Workflow](https://github.com/QubesOS/qubes-builder/blob/master/doc/ReleaseManagerWorkflow.md)

49
developers/services/Dom0SecureUpdates.md
View file

@ -1,49 +0,0 @@
---
layout: doc
title: Dom0SecureUpdates
permalink: /en/doc/dom0-secure-updates/
redirect_from:
- /doc/Dom0SecureUpdates/
- /wiki/Dom0SecureUpdates/
---
Qubes Dom0 secure update procedure
==================================
Reasons for Dom0 updates
------------------------
Normally there should be few reasons for updating software in Dom0. This is because there is no networking in Dom0, which means that even if some bugs will be discovered e.g. in the Dom0 Desktop Manager, this really is not a problem for Qubes, because all the 3rd party software running in Dom0 is not accessible from VMs or network in any way. Some exceptions to the above include: Qubes GUI daemon, Xen store daemon, and disk back-ends (we plan move the disk backends to untrusted domain in Qubes 2.0). Of course we believe this software is reasonably secure and we hope it will not need patching.
However, we anticipate some other situations when updating Dom0 software might be required:
- Updating drivers/libs for new hardware support
- Correcting non-security related bugs (e.g. new buttons for qubes manager)
- Adding new features (e.g. GUI backup tool)
Problems with traditional network-based update mechanisms
---------------------------------------------------------
Dom0 is the most trusted domain on Qubes OS, and for this reason we decided to design Qubes in such a way that Dom0 is not connected to any network. In fact only select domains can be connected to a network via so called network domains. There could also be more than one network domain, e.g. in case the user is connected to more than one physically or logically separated networks.
Secure update mechanism we use in Qubes (starting from Beta 2
-------------------------------------------------------------
Keeping Dom0 not connected to any network makes it hard, however, to provide updates for software in Dom0. For this reason we have come up with the following mechanism for Dom0 updates, which minimizes the amount of untrusted input processed by Dom0 software:
The update process is initiated by [qvm-dom0-update script](https://github.com/QubesOS/qubes-core-admin-linux/blob/release2/dom0-updates/qubes-dom0-update), running in Dom0.
Updates (\*.rpm files) are checked and downloaded by UpdateVM, which by default is the same as the firewall VM, but can be configured to be any other, network-connected VM. This is done by [qubes-download-dom0-updates.sh script](https://github.com/QubesOS/qubes-core-agent-linux/blob/release2/misc/qubes-download-dom0-updates.sh) (this script is executed using qrexec by the previously mentioned qvm-dom0-update). Note that we assume that this script might get compromised and might download a maliciously compromised downloads -- this is not a problem as Dom0 verifies digital signatures on updates later. The downloaded rpm files are placed in a ~~~/var/lib/qubes/dom0-updates~~~ directory on UpdateVM filesystem (again, they might get compromised while being kept there, still this isn't a problem). This directory is passed to yum using the ~~~--installroot=~~~ option.
Once updates are downloaded, the update script that runs in UpdateVM requests an RPM service [qubes.ReceiveUpdates](https://github.com/QubesOS/qubes-core-admin-linux/blob/release2/dom0-updates/qubes.ReceiveUpdates) to be executed in Dom0. This service is implemented by [qubes-receive-updates script](https://github.com/QubesOS/qubes-core-admin-linux/blob/release2/dom0-updates/qubes-receive-updates) running in Dom0. The Dom0's qvm-dom0-update script (which originally initiated the whole update process) waits until qubes-receive-updates finished.
The qubes-receive-updates script processes the untrusted input from Update VM: it first extracts the received \*.rpm files (that are sent over qrexec data connection) and then verifies digital signature on each file. The qubes-receive-updates script is a security-critical component of the Dom0 update process (as is the [qfile-dom0-unpacker.c](https://github.com/QubesOS/qubes-core-admin-linux/blob/release2/dom0-updates/qfile-dom0-unpacker.c) and the rpm utility, both used by the qubes-receive-updates for processing the untrusted input.
Once qubes-receive-updates finished unpacking and verifying the updates, the updates are placed in ``qubes-receive-updates`` directory in Dom0 filesystem. Those updates are now trusted. Dom0 is configured (see /etc/yum.conf in Dom0) to use this directory as a default (and only) [yum repository](https://github.com/QubesOS/qubes-core-admin-linux/blob/release2/dom0-updates/qubes-cached.repo).
Finally, qvm-dom0-updates runs ``yum update`` that fetches the rpms from qubes-cached repo and installs them as usual.
Security benefit of our update mechanism
----------------------------------------
The benefit of the above scheme is that one doesn't need to trust the TCP/IP stack, the HTTP stack, and wget implementation in order to safely deliver and install updates in Dom0. One only needs to trust a few hundred lines of code as discussed above, as well as the rpm utility for properly checking digital signature (but this is required in any update scheme).

55
developers/services/DvmImpl.md
View file

@ -1,55 +0,0 @@
---
layout: doc
title: DVMimpl
permalink: /en/doc/dvm-impl/
redirect_from:
- /doc/DVMimpl/
- /wiki/DVMimpl/
---
DisposableVM implementation in Qubes
====================================
DisposableVM image preparation
------------------------------
DisposableVM is not started like other VMs, by executing equivalent of *xl create* - it would be too slow. Instead, DisposableVM are started by restore from a savefile.
Preparing a savefile is done by */usr/lib/qubes/qubes\_prepare\_saved\_domain.sh* script. It takes two mandatory arguments, appvm name (APPVM) and the savefile name, and optional path to "prerun" script. The script executes the following steps:
1. APPVM is started by *qvm-start*
2. xenstore key `/local/domain/appvm_domain_id/qubes_save_request` is created
3. if prerun script was specified, copy it to `qubes_save_script` xenstore key
4. wait for the `qubes_used_mem` key to appear
5. (in APPVM) APPVM boots normally, up to the point in */etc/init.d/qubes\_core* script when the presence of `qubes_save_request` key is tested. If it exists, then
1. (in APPVM) if exists, prerun script is retrieved from the respective xenstore key and executed. This preloads filesystem cache with useful applications, so that they will start faster.
2. (in APPVM) the amount of used memory is stored to `qubes_used_mem` xenstore key
3. (in APPVM) busy-waiting for `qubes_restore_complete` xenstore key to appear
6. when `qubes_used_mem` key appears, the domain memory is reduced to this amount, to make the savefile smaller.
7. APPVM private image is detached
8. the domain is saved via *xl save*
9. the COW file volatile.img (cow for for root fs and swap) is packed to `saved_cows.tar` archive
*qubes\_prepare\_saved\_domain.sh* script is somehow lowlevel. It is usually called by *qvm-create-default-dvm* script, that takes care of creating a special AppVM (named template\_name-dvm) to be passed to *qubes\_prepare\_saved\_domain.sh*, as well as copying the savefile to /dev/shm (the latter action is not done if the `/var/lib/qubes/dvmdata/dont_use_shm` file exists).
Restoring a DisposableVM from the savefile
------------------------------------------
Normally, disposable VM is created when qubes rpc request with target *\$dispvm* is received. Then, as a part of rpc connection setup, the *qfile-daemon-dvm* program is executed; it executes */usr/lib/qubes/qubes\_restore* program. It is crucial that this program executes quickly, to make DisposableVM creation overhead bearable for the user. Its main steps are:
1. modify the savefile so that the VM name, VM UUID, MAC address and IP address are unique
2. restore the COW files from the `saved_cows.tar`
3. create the `/var/run/qubes/fast_block_attach` file, whose presence tells the */etc/xen/scripts/block* script to bypass some redundant checks and execute as fast as possible.
4. execute "xl restore" in order to restore a domain.
5. create the same xenstore keys as normally created when AppVM boots (e.g. `qubes_ip`)
6. create the `qubes_restore_complete` xenstore key. This allows the boot process in DisposableVM to continue.
The actual passing of files between AppVM and a DisposableVM is implemented via qubes rpc.
Validating the DisposableVM savefile
------------------------------------
DisposableVM savefile contains references to template rootfs and to COW files. The COW files are restored before each DisposableVM start, so they cannot change. On the other hand, if templateVM is started, the template rootfs will change, and it may not be coherent with the COW files.
Therefore, the check for template rootfs modification time being older than DisposableVM savefile modification time is required. It is done in *qfilexchgd* daemon, just before restoring DisposableVM. If necassary, an attempt is made to recreate the DisposableVM savefile, using the last template used (or default template, if run for the first time) and the default prerun script, residing at */var/lib/qubes/vm-templates/templatename/dispvm\_prerun.sh*. Unfortunately, the prerun script takes a lot of time to execute - therefore, after template rootfs modification, the next DisposableVM creation can be longer by about 2.5 minutes.

29
developers/services/Qfilecopy.md
View file

@ -1,29 +0,0 @@
---
layout: doc
title: Qfilecopy
permalink: /en/doc/qfilecopy/
redirect_from:
- /doc/Qfilecopy/
- /wiki/Qfilecopy/
---
InterVM file copy design
========================
There are two cases when we need a mechanism to copy files between VMs:
- "regular" file copy - when user instructs file manager to copy a given files/directories to a different VM
- DispVM copy - user selects "open in DispVM" on a file; this file must be copied to a Disposable VM, edited by user, and possibly a modified file copied back from DispVM to VM.
Prior to Qubes Beta1, for both cases, a block device (backed by a file in dom0 with a vfat filesystem on it) was attached to VM, file(s) copied there, and then the device was detached and attached to target VM. In the DispVM case, if a edited file has been modified, another block device is passed to requester VM in order to update the source file.
This has the following disadvantages:
- performance - dom0 has to prepare and attach/detach block devices, which is slow because of hotplug scripts involvement.
- security - VM kernel parses partition table and filesystem metadata from the block device; they are controlled by (potentially untrusted) sender VM.
In Qubes Beta1, we have reimplemented interVM file copy using qrexec, which addresses the abovementioned disadvantages. In Qubes Beta2, even more generic solution (qubes rpc) is used. See the developer docs on qrexec and qubes rpc. In a nutshell, the file sender and the file receiver just read/write from stdin/stdout, and the qubes rpc layer passes data properly - so, no block devices are used.
The rpc action for regular file copy is *qubes.Filecopy*, the rpc client is named *qfile-agent*, the rpc server is named *qfile-unpacker*. For DispVM copy, the rpc action is *qubes.OpenInVM*, the rpc client is named *qopen-in-vm*, rpc server is named *vm-file-editor*. Note that the qubes.OpenInVM action can be done on a normal AppVM, too.
Being a rpc server, *qfile-unpacker* must be coded securely, as it processes potentially untrusted data format. Particularly, we do not want to use external tar or cpio and be prone to all vulnerabilities in them; we want a simplified, small utility, that handles only directory/file/symlink file type, permissions, mtime/atime, and assume user/user ownership. In the current implementation, the code that actually parses the data from srcVM has ca 100 lines of code and executes chrooted to the destination directory. The latter is hardcoded to `~user/incoming/srcVM`; because of chroot, there is no possibility to alter files outside of this directory.

56
developers/services/Qfileexchgd.md
View file

@ -1,56 +0,0 @@
---
layout: doc
title: Qfileexchgd
permalink: /en/doc/qfileexchgd/
redirect_from:
- /doc/Qfileexchgd/
- /wiki/Qfileexchgd/
---
**This mechanism is obsolete as of Qubes Beta 1!**
==================================================
Please see this [page](/en/doc/qfilecopy/) instead.
qfilexchgd, the Qubes file exchange daemon
==========================================
Overview
--------
*qfilexchgd* is a dom0 daemon responsible for managing exchange of block devices ("virtual pendrives") between VMs. It is used for
- copying files between AppVMs
- copying a single file between an AppVM and a DisposableVM
*qfilexchgd* is started after first *qubes\_guid* has been started, so that it has access to X display in dom0 to present dialog messages.
*qfilexchgd* is event driven. The sources of events are:
- trigger of xenstore watch for the changes in `/local/domain` xenstore hierarchy - to detect start/stop of VMs, and maintain vmname-\>vm\_xid dictionary
- trigger of xenstore watch for a change in `/local/domain/domid/device/qpen` key - VMs write to this key to request service from *qfilexchgd*
Copying files between AppVMs
----------------------------
1. AppVM1 user runs *qvm-copy-to-vm* script (accessible from Dolphin file manager by right click on a "file(s)-\>Actions-\>Send to VM" menu). It calls */usr/lib/qubes/qubes\_penctl new*, and it writes "new" request to its `device/qpen` xenstore key. *qfilexchgd* creates a new 1G file, makes vfat fs on it, and does block-attach so that this file is attached as `/dev/xvdg` in AppVM1.
2. AppVM1 mounts `/dev/xvdg` on `/mnt/outgoing` and copies requested files there, then unmounts it.
3. AppVM1 writes "send DestVM" request to its `device/qpen` xenstore key (calling */usr/lib/qubes/qubes\_penctl send DestVM*). After getting confirmation by displaying a dialog box in dom0 display, *qfilexchgd* detaches `/dev/xvdg` from AppVM1, attaches it as `/dev/xvdh` to DestVM.
4. In DestVM, udev script for `/dev/xvdh` named *qubes\_add\_pendrive\_script* (see `/etc/udev/rules.d/qubes.rules`) mounts `/dev/xvdh` on `/mnt/incoming`, and then waits for `/mnt/incoming` to become unmounted. A file manager running in DestVM shows a new volume, and user in DestVM may copy files from it. When user in DestVM is done, then user unmounts `/mnt/incoming`. *qubes\_add\_pendrive*\_script then tells *qfilexchgd* to detach `/dev/xvdh` and terminates.
Copying a single file between AppVM and a DisposableVM
------------------------------------------------------
In order to minimize attack surface presented by necessity to process virtual pendrive metadata sent by (potentially compromised and malicious) DisposableVM, AppVM\<-\>DisposableVM file exchange protocol does not use any filesystem.
1. User in AppVM1 runs *qvm-open-in-dvm* (accessible from Dolphin file manager by right click on a "file-\>Actions-\>Open in DisposableVM" menu). *qvm-open-in-dvm*
1. gets a new `/dev/xvdg` (just as described in previous paragraph)
2. computes a new unique transaction seq SEQ by incrementing `/home/user/.dvm/seq` contents,
3. writes the requested file name (say, /home/user/document.txt) to `/home/user/.dvm/SEQ`
4. creates a dvm\_header (see core.git/appvm/dvm.h) on `/dev/xvdg`, followed by file contents
5. writes the "send disposable SEQ" command to its `device/qpen` xenstore key.
2. *qfilexchgd* sees that "send" argument=="disposable", and creates a new DisposableVM by calling */usr/lib/qubes/qubes\_restore*. It adds the new DisposableVM to qubesDB via qvm\_collection.add\_new\_disposablevm. Then it attaches the virtual pendrive (previously attached as `/dev/xvdg` at AppVM1) as `/dev/xvdh` in DisposableVM.
3. In DisposableVM, *qubes\_add\_pendrive\_script* sees non-zero `qubes_transaction_seq` key in xenstore, and instead processing the virtual pendrive as in the case of normal copy, treats it as DVM transaction (a request, because we run in DisposableVM). It retrieves the body of the file passed in `/dev/xvdh`, copies to /tmp, and runs *mime-open* utility to open appropriate executable to edit it. When *mime-open* returns, if the file was modified, it is sent back to AppVM1 (by writing "send AppVM1 SEQ" to `device/qpen` xenstore key). Then DisposableVM destroys itself.
4. In AppVM1, a new `/dev/xvdh` appears (because DisposableVM has sent it). *qubes\_add\_pendrive\_script* sees non-zero `qubes_transaction_seq` key, and treats it as DVM transaction (a response, because we run in AppVM, not DisposableVM). It retrieves the filename from `/home/user/.dvm/SEQ`, and copies data from `/dev/xvdh` to it.

76
developers/services/Qmemman.md
View file

@ -1,76 +0,0 @@
---
layout: doc
title: Qmemman
permalink: /en/doc/qmemman/
redirect_from:
- /doc/Qmemman/
- /wiki/Qmemman/
---
qmemman, Qubes memory manager
=============================
Rationale
---------
Traditionally, Xen VMs are assigned a fixed amount of memory. It is not the optimal solution, as some VMs may require more memory than assigned initially, while others underutilize memory. Thus, there is a need for solution capable of shifting free memory from VM to another VM.
The [tmem](http://oss.oracle.com/projects/tmem/) project provides a "pseudo-RAM" that is assigned on per-need basis. However this solution has some disadvantages:
- It does not provide real RAM, just an interface to copy memory to/from fast, RAM-based storage. It is perfect for swap, good for file cache, but not ideal for many tasks.
- It is deeply integrated with the Linux kernel. When Qubes will support Windows guests natively, we would have to port *tmem* to Windows, which may be challenging.
Therefore, in Qubes another solution is used. There is the *qmemman* dom0 daemon. All VMs report their memory usage (via xenstore) to *qmemman*, and it makes decisions on whether to balance memory across domains. The actual mechanism to add/remove memory from a domain (*xc.domain\_set\_target\_mem*) is already supported by both PV Linux guests and Windows guests (the latter via PV drivers).
Similarly, when there is need for Xen free memory (for instance, in order to create a new VM), traditionally the memory is obtained from dom0 only. When *qmemman* is running, it offers interface to obtain memory from all domains.
To sum up, *qmemman* pros and cons. Pros:
- provides automatic balancing of memory across participating PV and HVM domains, based on their memory demand
- works well in practice, with less than 1% CPU consumption in the idle case
- simple, concise implementation
Cons:
- the algorithm to calculate the memory requirement for a domain is necessarily simple, and may not closely reflect reality
- *qmemman* is notified by a VM about memory usage change not more often than 10 times per seconds (to limit CPU overhead in VM). Thus, there can be up to 0.1s delay until qmemman starts to react to the new memory requirements
- it takes more time to obtain free Xen memory, as all participating domains need to instructed to yield memory
Interface
---------
*qmemman* listens for the following events:
- writes to `/local/domain/domid/memory/meminfo` xenstore keys by *meminfo-writer* process in VM. The content of this key is taken from the VM's `/proc/meminfo` pseudofile ; *meminfo-writer* just strips some unused lines from it. Note that *meminfo-writer* writes its xenstore key only if the VM memory usage has changed significantly enough since the last update (by default 30MB), to prevent flooding with almost identical data
- commands issued over Unix socket `/var/run/qubes/qmemman.sock`. Currently, the only command recognized is to free the specified amount of memory. The QMemmanClient class implements the protocol.
- if the `/var/run/qubes/do-not-membalance` file exists, *qmemman* suspends memory balancing. It is primarily used when allocating memory for a to-be-created domain, to prevent using up the free Xen memory by the balancing algorithm before the domain creation is completed.
Algorithms basics
-----------------
The core VM property is `prefmem`. It denotes the amount of memory that should be enough for a domain to run efficiently in the nearest future. All *qmemman* algorithms will never shrink domain memory below `prefmem`. Currently, `prefmem` is simply 130% of current memory usage in a domain (without buffers and cache, but including swap). Naturally, `prefmem` is calculated by *qmemman* based on the information passed by *meminfo-writer*.
Whenever *meminfo-writer* running in domain A provides new data on memory usage to *qmemman*, the `prefmem` value for A is updated and the following balance algorithm (*qmemman\_algo.balance*) is triggered. Its output is the list of (domain\_id, new\_memory\_target\_to\_be\_set) pairs:
1. TOTAL\_PREFMEM = sum of `prefmem` of all participating domains
2. TOTAL\_MEMORY = sum of all memory assigned to participating domains plus Xen free memory
3. if TOTAL\_MEMORY \> TOTAL\_PREFMEM, then redistribute TOTAL\_MEMORY across all domains proportionally to their `prefmem`
4. if TOTAL\_MEMORY \< TOTAL\_PREFMEM, then
1. for all domains whose `prefmem` is less than actual memory, shrink them to their `prefmem`
2. redistribute memory reclaimed in the previous step between the rest of domains, proportionally to their `prefmem`
In order to avoid too frequent memory redistribution, it is actually executed only if one of the below conditions hold:
- the sum of memory size changes for all domains is more than MIN\_TOTAL\_MEMORY\_TRANSFER (150MB)
- one of the domains is below its `prefmem`, and more than MIN\_MEM\_CHANGE\_WHEN\_UNDER\_PREF (15MB) would be added to it
Additionally, the balance algorithm is tuned so that XEN\_FREE\_MEM\_LEFT (50MB) is always left as Xen free memory, to make coherent memory allocations in driver domains work.
Whenever *qmemman* is asked to return X megabytes of memory to Xen free pool, the following algorithm (*qmemman\_algo.balloon*) is executed:
1. find all domains ("donors") whose actual memory is greater than its `prefmem`
2. calculate how much memory can be reclaimed by shrinking donors to their `prefmem`. If is is less than X, return error.
3. shrink donors, proportionally to their `prefmem`, so that X MB should become free
4. wait BALOON\_DELAY (0.1s)
5. if some domain have not given back any memory, remove it from the donors list, and go to step 2, unless we already did MAX\_TRIES (20) iterations (then return error).

164
developers/services/Qrexec.md
View file

@ -1,164 +0,0 @@
---
layout: doc
title: Qrexec
permalink: /en/doc/qrexec/
redirect_from:
- /doc/Qrexec/
- /wiki/Qrexec/
---
Command execution in VM (and Qubes RPC)
=======================================
Qubes **qrexec** is a framework for implementing inter-VM (incl. Dom0-VM) services. It offers a mechanism to start programs in VMs, redirect their stdin/stdout, and a policy framework to control this all.
Basic Dom0-VM command execution
-------------------------------
During each domain creation a process named `qrexec-daemon` is started in dom0, and a process named `qrexec-agent` is started in the VM. They are connected over `vchan` channel.
Typically, the first thing that a `qrexec-client` instance does is to send a request to `qrexec-agent` to start a process in the VM. From then on, the stdin/stdout/stderr from this remote process will be passed to the `qrexec-client` process.
E.g., to start a primitive shell in a VM type the following in Dom0 console:
~~~
[user@dom0 ~]$ /usr/lib/qubes/qrexec-client -d <vm name> user:bash
~~~
The string before first semicolon specifies what user to run the command as.
Adding `-e` on the `qrexec-client` command line results in mere command execution (no data passing), and `qrexec-client` exits immediately after sending the execution request.
There is also the `-l <local program>` flag, which directs `qrexec-client` to pass stdin/stdout of the remote program not to its stdin/stdout, but to the (spawned for this purpose) `<local program>`.
The `qvm-run` command is heavily based on `qrexec-client`. It also takes care of additional activities (e.g., starting the domain, if it is not up yet, and starting the GUI daemon). Thus, it is usually more convenient to use `qvm-run`.
There can be almost arbitrary number of `qrexec-client` processes for a domain (i.e., `qrexec-client` processes connected to the same `qrexec-daemon`); their data is multiplexed independently.
There is a similar command line utility avilable inside Linux AppVMs (note the `-vm` suffix): `qrexec-client-vm` that will be described in subsequent sections.
Qubes Inter-VM Services (Qubes RPC)
-----------------------------------
Apart from simple Dom0-\>VM command executions, as discussed above, it is also useful to have more advanced infrastructure for controlled inter-VM RPC/services. This might be used for simple things like inter-VM file copy operations, as well as more complex tasks like starting a DispVM, and requesting it to do certain operations on a handed file(s).
Instead of implementing complex RPC-like mechanisms for inter-VM communication, Qubes takes a much simpler and pragmatic approach and aims to only provide simple *pipes* between the VMs, plus ability to request *pre-defined* programs (servers) to be started on the other end of such pipes, and a centralized policy (enforced by the `qrexec-policy` process running in dom0) which says which VMs can request what services from what VMs.
Thanks to the framework and automatic stdin/stdout redirection, RPC programs are very simple; both the client and server just use their stdin/stdout to pass data. The framework does all the inner work to connect these file descriptors to each other via `qrexec-daemon` and `qrexec-agent`. Additionally, DispVMs are tightly integrated; RPC to a DispVM is a simple matter of using a magic `$dispvm` keyword as the target VM name.
All services in Qubes are identified by a single string, which by convention takes a form of `qubes.ServiceName`. Each VM can provide handlers for each of the known services by providing a file in `/etc/qubes-rpc/` directory with the same name as the service it is supposed to handle. This file will then be executed by the qrexec service, if the dom0 policy allowed the service to be requested (see below). Typically, the files in `/etc/qubes-rpc/` contain just one line, which is a path to the specific binary that acts as a server for the incoming request, however they might also be the actual executable themselves. Qrexec framework is careful about connecting the stdin/stdout of the server process with the corresponding stdin/stdout of the requesting process in the requesting VM (see example Hello World service described below).
Qubes Services (RPC) policy
---------------------------
Besides each VM needing to provide explicit programs to serve each supported service, the inter-VM service RPC is also governed by a central policy in Dom0.
In dom0, there is a bunch of files in `/etc/qubes-rpc/policy/` directory, whose names describe the available RPC actions; their content is the RPC access policy database. Some example of the default services in Qubes are:
- qubes.Filecopy
- qubes.OpenInVM
- qubes.ReceiveUpdates
- qubes.SyncAppMenus
- qubes.VMShell
- qubes.ClipboardPaste
- qubes.Gpg
- qubes.NotifyUpdates
- qubes.PdfConvert
These files contain lines with the following format:
~~~
srcvm destvm (allow|deny|ask)[,user=user_to_run_as][,target=VM_to_redirect_to]
~~~
You can specify `srcvm` and `destvm` by name, or by one of `$anyvm`, `$dispvm`, `dom0` reserved keywords (note string `dom0` does not match the `$anyvm` pattern; all other names do). Only `$anyvm` keyword makes sense in the `srcvm` field (service calls from dom0 are currently always allowed, `$dispvm` means "new VM created for this particular request" - so it is never a source of request). Currently, there is no way to specify source VM by type, but this is planned for Qubes R3.
Whenever a RPC request for service named "XYZ" is received, the first line in `/etc/qubes-rpc/policy/XYZ` that matches the actual `srcvm`/`destvm` is consulted to determine whether to allow RPC, what user account the program should run in target VM under, and what VM to redirect the execution to. If the policy file does not exits, user is prompted to create one *manually*; if still there is no policy file after prompting, the action is denied.
On the target VM, the `/etc/qubes-rpc/XYZ` must exist, containing the file name of the program that will be invoked.
Requesting VM-VM (and VM-Dom0) services execution
-------------------------------------------------
In a src VM, one should invoke the qrexec client via the following command:
~~~
/usr/lib/qubes/qrexec-client-vm <target vm name> <service name> <local program path> [local program arguments]`
~~~
Note that only stdin/stdout is passed between RPC server and client - notably, no cmdline argument are passed.
The source VM name can be accessed in the server process via QREXEC\_REMOTE\_DOMAIN environment variable. (Note the source VM has *no* control over the name provided in this variable--the name of the VM is provided by dom0, and so is trusted.)
By default, stderr of client and server is logged to respective `/var/log/qubes/qrexec.XID` files, in each of the VM.
Be very careful when coding and adding a new RPC service! Any vulnerability in a RPC server can be fatal to security of the target VM!
Requesting VM-VM (and VM-Dom0) services execution (without cmdline helper)
--------------------------------------------------------------------------
Connect directly to `/var/run/qubes/qrexec-agent-fdpass` socket as described [here](/doc/Qrexec2Implementation#Allthepiecestogetheratwork).
### Revoking "Yes to All" authorization
Qubes RPC policy supports an "ask" action, that will prompt the user whether a given RPC call should be allowed. It is set as default for services such as inter-VM file copy. A prompt window launches in dom0, that gives the user option to click "Yes to All", which allows the action and adds a new entry to the policy file, which will unconditionally allow further calls for given (service, srcVM, dstVM) tuple.
In order to remove such authorization, issue this command from a Dom0 terminal (example below for qubes.Filecopy service):
~~~
sudo nano /etc/qubes-rpc/policy/qubes.Filecopy
~~~
and then remove any line(s) ending in "allow" (before the first \#\# comment) which are the "Yes to All" results.
A user might also want to set their own policies in this section. This may mostly serve to prevent the user from mistakenly copying files or text from a trusted to untrusted domain, or vice-versa.
### Qubes RPC "Hello World" service
We will show the necessary files to create a simple RPC call that adds two integers on the target VM and returns back the result to the invoking VM.
- Client code on source VM (`/usr/bin/our_test_add_client`)
~~~
#!/bin/sh
echo $1 $2 # pass data to rpc server
exec cat >&$SAVED_FD_1 # print result to the original stdout, not to the other rpc endpoint
~~~
- Server code on target VM (`/usr/bin/our_test_add_server`)
~~~
#!/bin/sh
read arg1 arg2 # read from stdin, which is received from the rpc client
echo $(($arg1+$arg2)) # print to stdout - so, pass to the rpc client
~~~
- Policy file in dom0 (`/etc/qubes-rpc/policy/test.Add`)
~~~
$anyvm $anyvm ask
~~~
- Server path definition on target VM (`/etc/qubes-rpc/test.Add`)
~~~
/usr/bin/our_test_add_server
~~~
- To test this service, run the following in the source VM:
~~~
/usr/lib/qubes/qrexec-client-vm <target VM> test.Add /usr/bin/our_test_add_client 1 2
~~~
and we should get "3" as answer, provided dom0 policy allows the call to pass through, which would happen after we click "Yes" in the popup that should appear after the invocation of this command. If we changed the policy from "ask" to "allow", then no popup should be presented, and the call will always be allowed.
More high-level RPCs?
---------------------
As previously noted, Qubes aims to provide mechanisms that are very simple and thus with very small attack surface. This is the reason why the inter-VM RPC framework is very primitive and doesn't include any serialization or other function arguments passing, etc. We should remember, however, that users/app developers are always free to run more high-level RPC protocols on top of qrexec. Care should be taken, however, to consider potential attack surfaces that are exposed to untrusted or less trusted VMs in that case.
Qubes RPC internals
-------------------
The internal implementation of qrexec in Qubes R2 is described [here](/en/doc/qrexec2-implementation/), and in Qubes R3 [here](/en/doc/qrexec3-implementation/).

74
developers/services/Qrexec2Implementation.md
View file

@ -1,74 +0,0 @@
---
layout: doc
title: Qrexec2Implementation
permalink: /en/doc/qrexec2-implementation/
redirect_from:
- /doc/Qrexec2Implementation/
- /wiki/Qrexec2Implementation/
---
Implementation of qrexec in Qubes R2
====================================
This page describes implementation of the [qrexec framework](/en/doc/qrexec/) in Qubes OS R2. Note that the implementation has changed significantly in Qubes R3 (see [Qrexec3Implementation](/en/doc/qrexec3-implementation/)), although the user API reminded backwards compatible (i.e. qrexec apps written for Qubes R2 should run without modifications on Qubes R3).
Dom0 tools implementation
-------------------------
Players:
- `/usr/lib/qubes/qrexec-daemon` \<- started by mgmt stack (qubes.py) when a VM is started.
- `/usr/lib/qubes/qrexec-policy` \<- internal program used to evaluate the policy file and making the 2nd half of the connection.
- `/usr/lib/qubes/qrexec-client` \<- raw command line tool that talks to the daemon via unix socket (`/var/run/qubes/qrexec.XID`)
Note: none of the above tools are designed to be used by users.
Linux VMs implementation
------------------------
Players:
- `/usr/lib/qubes/qrexec-agent` \<- started by VM bootup scripts, a daemon.
- `/usr/lib/qubes/qubes-rpc-multiplexer` \<- executes the actual service program, as specified in VM's `/etc/qubes-rpc/qubes.XYZ`.
- `/usr/lib/qubes/qrexec-client-vm` \<- raw command line tool that talks to the agent.
Note: none of the above tools are designed to be used by users. `qrexec-client-vm` is designed to be wrapped up by Qubes apps.
Windows VMs implemention
------------------------
%QUBES\_DIR% is the installation path (`c:\Program Files\Invisible Things Lab\Qubes OS Windows Tools` by default).
- `%QUBES_DIR%\bin\qrexec-agent.exe` \<- runs as a system service. Responsible both for raw command execution and interpreting RPC service requests.
- `%QUBES_DIR%\qubes-rpc` \<- directory with `qubes.XYZ` files that contain commands for executing RPC services. Binaries for the services are contained in `%QUBES_DIR%\qubes-rpc-services`.
- `%QUBES_DIR%\bin\qrexec-client-vm` \<- raw command line tool that talks to the agent.
Note: neither of the above tools are designed to be used by users. `qrexec-client-vm` is designed to be wrapped up by Qubes apps.
All the pieces together at work
-------------------------------
Note: this section is not needed to use qrexec for writing Qubes apps. Also note the qrexec framework implemention in Qubes R3 significantly differs from what is described in this section.
The VM-VM channels in Qubes R2 are made via "gluing" two VM-Dom0 and Dom0-VM vchan connections:
![qrexec2-internals.png](/attachment/wiki/Qrexec2Implementation/qrexec2-internals.png)
Note: Dom0 never examines the actual data flowing in neither of the two vchan connections.
When a user in a source VM executes `qrexec-client-vm` utility, the following steps are taken:
- `qrexec-client-vm` connects to `qrexec-agent`'s `/var/run/qubes/qrexec-agent-fdpass` unix socket 3 times. Reads 4 bytes from each of them, which is the fd number of the accepted socket in agent. These 3 integers, in text, concatenated, form "connection identifier" (CID)
- `qrexec-client-vm` writes to `/var/run/qubes/qrexec-agent` fifo a blob, consisting of target vmname, rpc action, and CID
- `qrexec-client-vm` executes the rpc client, passing the above mentioned unix sockets as process stdin/stdout, and optionally stderr (if the PASS\_LOCAL\_STDERR env variable is set)
- `qrexec-agent` passes the blob to `qrexec-daemon`, via MSG\_AGENT\_TO\_SERVER\_TRIGGER\_CONNECT\_EXISTING message over vchan
- `qrexec-daemon` executes `qrexec-policy`, passing source vmname, target vmname, rpc action, and CID as cmdline arguments
- `qrexec-policy` evaluates the policy file. If successful, creates a pair of `qrexec-client` processes, whose stdin/stdout are cross-connencted.
- The first `qrexec-client` connects to the src VM, using the `-c ClientID` parameter, which results in not creating a new process, but connecting to the existing process file descriptors (these are the fds of unix socket created in step 1).
- The second `qrexec-client` connects to the target VM, and executes `qubes-rpc-multiplexer` command there with the rpc action as the cmdline argument. Finally, `qubes-rpc-multiplexer` executes the correct rpc server on the target.
- In the above step, if the target VM is `$dispvm`, the DispVM is created via the `qfile-daemon-dvm` program. The latter waits for the `qrexec-client` process to exit, and then destroys the DispVM.
Protocol description ("wire-level" spec)
----------------------------------------
TODO

129
developers/services/Qrexec3.md
View file

@ -1,129 +0,0 @@
---
layout: doc
title: Qrexec3
permalink: /en/doc/qrexec3/
redirect_from:
- /doc/Qrexec3/
- /wiki/Qrexec3/
---
Command execution in VM (and Qubes RPC)
=======================================
*[Note: this documents describes Qrexec v3 (Odyssey)]*
Qrexec framework is used by core Qubes components to implement communication between domains. Qubes domains are isolated by design but there is a need for a mechanism to allow administrative domain (dom0) to force command execution in another domain (VM). For instance, when user selects an application from KDE menu, it should be started in the selected VM. Also it is often useful to be able to pass stdin/stdout/stderr from an application running in VM to dom0 (and the other way around). In specific circumstances Qubes allows VMs to be initiators of such communication (so for example a VM can notify dom0 that there are updates available for it).
Qrexec basics
-------------
Qrexec is built on top of vchan (a library providing data links between VMs). During domain creation a process named *qrexec-daemon* is started in dom0, and a process named *qrexec-agent* is started in the VM. They are connected over *vchan* channel. *qrexec-daemon* listens for connections from dom0 utility named *qrexec-client*. Typically, the first thing that a *qrexec-client* instance does is to send a request to *qrexec-daemon* to start a process (let's name it VMprocess) with a given command line in a specified VM (someVM). *qrexec-daemon* assigns unique vchan connection details and sends them both to *qrexec-client* (in dom0) and *qrexec-agent* (in someVM). *qrexec-client* starts a vchan server which *qrexec-agent* connects to. Since then, stdin/stdout/stderr from the VMprocess is passed via vchan between *qrexec-agent* and the *qrexec-client* process.
So, for example, executing in dom0
`qrexec-client -d someVM user:bash`
allows to work with the remote shell. The string before the first semicolon specifies what user to run the command as. Adding `-e` on the *qrexec-client* command line results in mere command execution (no data passing), and *qrexec-client* exits immediately after sending the execution request and receiving status code from *qrexec-agent* (whether the process creation succeeded). There is also the `-l local_program` flag -- with it, *qrexec-client* passes stdin/stdout of the remote process to the (spawned for this purpose) *local\_program*, not to its own stdin/stdout.
The `qvm-run` command is heavily based on *qrexec-client*. It also takes care of additional activities, e.g. starting the domain if it is not up yet and starting the GUI daemon. Thus, it is usually more convenient to use `qvm-run`.
There can be almost arbitrary number of *qrexec-client* processes for a domain (so, connected to the same *qrexec-daemon*, same domain) -- their data is multiplexed independently. Number of available vchan channels is the limiting factor here, it depends on the underlying hypervisor.
Qubes RPC services
------------------
Some tasks (like intervm file copy) share the same rpc-like structure: a process in one VM (say, file sender) needs to invoke and send/receive data to some process in other VM (say, file receiver). Thus, the Qubes RPC framework was created, facilitating such actions.
Obviously, inter-VM communication must be tightly controlled to prevent one VM from taking control over other, possibly more privileged, VM. Therefore the design decision was made to pass all control communication via dom0, that can enforce proper authorization. Then, it is natural to reuse the already-existing qrexec framework.
Also, note that bare qrexec provides VM\<-\>dom0 connectivity, but the command execution is always initiated by dom0. There are cases when VM needs to invoke and send data to a command in dom0 (e.g. to pass information on newly installed .desktop files). Thus, the framework allows dom0 to be the rpc target as well.
Thanks to the framework, RPC programs are very simple -- both rpc client and server just use their stdin/stdout to pass data. The framework does all the inner work to connect these processes to each other via *qrexec-daemon* and *qrexec-agent*. Additionally, disposable VMs are tightly integrated -- rpc to a disposableVM is identical to rpc to a normal domain, all one needs is to pass "\$dispvm" as the remote domain name.
Qubes RPC administration
------------------------
[TODO: fix for non-linux dom0]
In dom0, there is a bunch of files in */etc/qubes-rpc/policy* directory, whose names describe the available rpc actions; their content is the rpc access policy database. Currently defined actions are:
- qubes.Filecopy
- qubes.OpenInVM
- qubes.ReceiveUpdates
- qubes.SyncAppMenus
- qubes.VMShell
- qubes.ClipboardPaste
- qubes.Gpg
- qubes.NotifyUpdates
- qubes.PdfConvert
These files contain lines with the following format:
srcvm destvm (allow|deny|ask)[,user=user\_to\_run\_as][,target=VM\_to\_redirect\_to]
You can specify srcvm and destvm by name, or by one of "\$anyvm", "\$dispvm", "dom0" reserved keywords (note string "dom0" does not match the \$anyvm pattern; all other names do). Only "\$anyvm" keyword makes sense in srcvm field (service calls from dom0 are currently always allowed, "\$dispvm" means "new VM created for this particular request" - so it is never a source of request). Currently there is no way to specify source VM by type. Whenever a rpc request for action X is received, the first line in /etc/qubes-rpc/policy/X that match srcvm/destvm is consulted to determine whether to allow rpc, what user account the program should run in target VM under, and what VM to redirect the execution to. If the policy file does not exits, user is prompted to create one; if still there is no policy file after prompting, the action is denied.
In the target VM, the */etc/qubes-rpc/RPC\_ACTION\_NAME* must exist, containing the file name of the program that will be invoked.
In the src VM, one should invoke the client via
`/usr/lib/qubes/qrexec-client-vm target_vm_name RPC_ACTION_NAME rpc_client_path client arguments`
Note that only stdin/stdout is passed between rpc server and client -- notably, no command line argument are passed. Source VM name is specified by QREXEC\_REMOTE\_DOMAIN environment variable. By default, stderr of client and server is logged to respective /var/log/qubes/qrexec.XID files.
Be very careful when coding and adding a new rpc service. Unless the offered functionality equals full control over the target (it is the case with e.g. qubes.VMShell action), any vulnerability in a rpc server can be fatal to qubes security. On the other hand, this mechanism allows to delegate processing of untrusted input to less privileged (or throwaway) AppVMs, thus wise usage of it increases security.
### Revoking "Yes to All" authorization
Qubes RPC policy supports "ask" action. This will prompt the user whether given RPC call should be allowed. That prompt window has also "Yes to All" option, which will allow the action and add new entry to the policy file, which will unconditionally allow further calls for given service-srcVM-dstVM tuple.
In order to remove such authorization, issue this command from a Dom0 terminal (for qubes.Filecopy service):
`sudo nano /etc/qubes-rpc/policy/qubes.Filecopy`
and then remove the first line/s (before the first \#\# comment) which are the "Yes to All" results.
### Qubes RPC example
We will show the necessary files to create rpc call that adds two integers on the target and returns back the result to the invoker.
- rpc client code (*/usr/bin/our\_test\_add\_client*)
~~~
#!/bin/sh
echo $1 $2 # pass data to rpc server
exec cat >&$SAVED_FD_1 # print result to the original stdout, not to the other rpc endpoint
~~~
- rpc server code (*/usr/bin/our\_test\_add\_server*)
~~~
#!/bin/sh
read arg1 arg2 # read from stdin, which is received from the rpc client
echo $(($arg1+$arg2)) # print to stdout - so, pass to the rpc client
~~~
- policy file in dom0 (*/etc/qubes-rpc/policy/test.Add* )
~~~
$anyvm $anyvm ask
~~~
- server path definition ( */etc/qubes-rpc/test.Add*)
~~~
/usr/bin/our_test_add_server
~~~
- invoke rpc via
~~~
/usr/lib/qubes/qrexec-client-vm target_vm test.Add /usr/bin/our_test_add_client 1 2
~~~
and we should get "3" as answer, after dom0 allows it.
Qubes RPC internals
-------------------
See [QrexecProtocol?](/doc/QrexecProtocol/).

184
developers/services/Qrexec3Implementation.md
View file

@ -1,184 +0,0 @@
---
layout: doc
title: Qrexec3Implementation
permalink: /en/doc/qrexec3-implementation/
redirect_from:
- /doc/Qrexec3Implementation/
- /wiki/Qrexec3Implementation/
---
Implementation of qrexec in Qubes R3
====================================
This page describes implementation of the [qrexec framework](/en/doc/qrexec/) in Qubes OS R3.
Qrexec framework consists of a number of processes communicating with each other using common IPC protocol (described in detail below). Components residing in the same domain use pipes as the underlying transport medium, while components in separate domains use vchan link.
Dom0 tools implementation
-------------------------
- `/usr/lib/qubes/qrexec-daemon` \<- one instance is required for every active domain. Responsible for:
- Handling execution and service requests from **dom0** (source: `qrexec-client`).
- Handling service requests from the associated domain (source: `qrexec-client-vm`, then `qrexec-agent`).
> Command line: `qrexec-daemon domain-id domain-name [default user]`
> *domain-id*: numeric Qubes ID assigned to the associated domain.
> *domain-name*: associated domain name.
> *default user*: optional. If passed, `qrexec-daemon` uses this user as default for all execution requests that don't specify one.
- `/usr/lib/qubes/qrexec-policy` \<- internal program used to evaluate the RPC policy and deciding whether a RPC call should be allowed.
- `/usr/lib/qubes/qrexec-client` \<- used to pass execution and service requests to `qrexec-daemon`. Command line parameters:
> `-d target-domain-name` Specifies the target for the execution/service request.
> `-l local-program` Optional. If present, `local-program` is executed and its stdout/stdin are used when sending/receiving data to/from the remote peer.
> `-e` Optional. If present, stdout/stdin are not connected to the remote peer. Only process creation status code is received.
> `-c <request-id,src-domain-name,src-domain-id>` Used for connecting a VM-VM service request by `qrexec-policy`. Details described below in the service example.
> `cmdline` Command line to pass to `qrexec-daemon` as the execution/service request. Service request format is described below in the service example.
Note: none of the above tools are designed to be used by users directly.
VM tools implementation
-----------------------
- `qrexec-agent` \<- one instance runs in each active domain. Responsible for:
- Handling service requests from `qrexec-client-vm` and passing them to connected `qrexec-daemon` in **dom0**.
- Executing associated `qrexec-daemon` execution/service requests.
> Command line parameters: none.
- `qrexec-client-vm` \<- runs in an active domain. Used to pass service requests to `qrexec-agent`.
> Command line: `qrexec-client-vm target-domain-name service-name local-program [local program arguments]`
> `target-domain-name` Target domain for the service request. Source is the current domain.
> `service-name` Requested service name.
> `local-program` **local-program** is executed locally and its stdin/stdout are connected to the remote service endpoint.
Qrexec protocol details
-----------------------
Qrexec protocol is message-based. All messages share a common header followed by an optional data packet.
~~~
/* uniform for all peers, data type depends on message type */
struct msg_header {
uint32_t type; /* message type */
uint32_t len; /* data length */
};
~~~
When two peers establish connection, the server sends `MSG_HELLO` followed by `peer_info` struct:
~~~
struct peer_info {
uint32_t version; /* qrexec protocol version */
};
~~~
The client then should reply with its own `MSG_HELLO` and `peer_info`. If protocol versions don't match, the connection is closed. TODO: fallback for backwards compatibility, don't do handshake in the same domain?.
Details of all possible use cases and the messages involved are described below.
### dom0: request execution of some\_command in domX and pass stdin/stdout
- **dom0**: `qrexec-client` is invoked in **dom0** as follows:
> `qrexec-client -d domX [-l local_program] user:some_command`
> `user` may be substituted with the literal `DEFAULT`. In that case, default Qubes user will be used to execute `some_command`.
- **dom0**: `qrexec-client` sets `QREXEC_REMOTE_DOMAIN` environment variable to **domX**.
- **dom0**: If `local_program` is set, `qrexec-client` executes it and uses that child's stdin/stdout in place of its own when exchanging data with `qrexec-agent` later.
- **dom0**: `qrexec-client` connects to **domX**'s `qrexec-daemon`.
- **dom0**: `qrexec-daemon` sends `MSG_HELLO` header followed by `peer_info` to `qrexec-client`.
- **dom0**: `qrexec-client` replies with `MSG_HELLO` header followed by `peer_info` to `qrexec-daemon`.
- **dom0**: `qrexec-client` sends `MSG_EXEC_CMDLINE` header followed by `exec_params` to `qrexec-daemon`
~~~
/* variable size */
struct exec_params {
uint32_t connect_domain; /* target domain id */
uint32_t connect_port; /* target vchan port for i/o exchange */
char cmdline[0]; /* command line to execute, size = msg_header.len - sizeof(struct exec_params) */
};
~~~
In this case, `connect_domain` and `connect_port` are set to 0.
- **dom0**: `qrexec-daemon` replies to `qrexec-client` with `MSG_EXEC_CMDLINE` header followed by `exec_params`, but with empty `cmdline` field. `connect_domain` is set to Qubes ID of **domX** and `connect_port` is set to a vchan port allocated by `qrexec-daemon`.
- **dom0**: `qrexec-daemon` sends `MSG_EXEC_CMDLINE` header followed by `exec_params` to the associated **domX** `qrexec-agent` over vchan. `connect_domain` is set to 0 (**dom0**), `connect_port` is the same as sent to `qrexec-client`. `cmdline` is unchanged except that the literal `DEFAULT` is replaced with actual user name, if present.
- **dom0**: `qrexec-client` disconnects from `qrexec-daemon`.
- **dom0**: `qrexec-client` starts a vchan server using the details received from `qrexec-daemon` and waits for connection from **domX**'s `qrexec-agent`.
- **domX**: `qrexec-agent` receives `MSG_EXEC_CMDLINE` header followed by `exec_params` from `qrexec-daemon` over vchan.
- **domX**: `qrexec-agent` connects to `qrexec-client` over vchan using the details from `exec_params`.
- **domX**: `qrexec-agent` executes `some_command` in **domX** and connects the child's stdin/stdout to the data vchan. If the process creation fails, `qrexec-agent` sends `MSG_DATA_EXIT_CODE` to `qrexec-client` followed by the status code (**int**) and disconnects from the data vchan.
- Data read from `some_command`'s stdout is sent to the data vchan using `MSG_DATA_STDOUT` by `qrexec-agent`. `qrexec-client` passes data received as `MSG_DATA_STDOUT` to its own stdout (or to `local_program`'s stdin if used).
- `qrexec-client` sends data read from local stdin (or `local_program`'s stdout if used) to `qrexec-agent` over the data vchan using `MSG_DATA_STDIN`. `qrexec-agent` passes data received as `MSG_DATA_STDIN` to `some_command`'s stdin.
- `MSG_DATA_STDOUT` or `MSG_DATA_STDIN` with data `len` field set to 0 in `msg_header` is an EOF marker. Peer receiving such message should close the associated input/output pipe.
- When `some_command` terminates, **domX**'s `qrexec-agent` sends `MSG_DATA_EXIT_CODE` header to `qrexec-client` followed by the exit code (**int**). `qrexec-agent` then disconnects from the data vchan.
### domY: invoke execution of qubes service qubes.SomeRpc in domX and pass stdin/stdout
- **domY**: `qrexec-client-vm` is invoked as follows:
> `qrexec-client-vm domX qubes.SomeRpc local_program [params]`
- **domY**: `qrexec-client-vm` connects to `qrexec-agent` (via local socket/named pipe).
- **domY**: `qrexec-client-vm` sends `trigger_service_params` data to `qrexec-agent` (without filling the `request_id` field):
~~~
struct trigger_service_params {
char service_name[64];
char target_domain[32];
struct service_params request_id; /* service request id */
};
struct service_params {
char ident[32];
};
~~~
- **domY**: `qrexec-agent` allocates a locally-unique (for this domain) `request_id` (let's say `13`) and fills it in the `trigger_service_params` struct received from `qrexec-client-vm`.
- **domY**: `qrexec-agent` sends `MSG_TRIGGER_SERVICE` header followed by `trigger_service_params` to `qrexec-daemon` in **dom0** via vchan.
- **dom0**: **domY**'s `qrexec-daemon` executes `qrexec-policy`: `qrexec-policy domY_id domY domX qubes.SomeRpc 13`.
- **dom0**: `qrexec-policy` evaluates if the RPC should be allowed or denied. If the action is allowed it returns `0`, if the action is denied it returns `1`.
- **dom0**: **domY**'s `qrexec-daemon` checks the exit code of `qrexec-policy`.
- If `qrexec-policy` returned **not** `0`: **domY**'s `qrexec-daemon` sends `MSG_SERVICE_REFUSED` header followed by `service_params` to **domY**'s `qrexec-agent`. `service_params.ident` is identical to the one received. **domY**'s `qrexec-agent` disconnects its `qrexec-client-vm` and RPC processing is finished.
- If `qrexec-policy` returned `0`, RPC processing continues.
- **dom0**: if `qrexec-policy` allowed the RPC, it executed `qrexec-client -d domX -c 13,domY,domY_id user:QUBESRPC qubes.SomeRpc domY`.
- **dom0**: `qrexec-client` sets `QREXEC_REMOTE_DOMAIN` environment variable to **domX**.
- **dom0**: `qrexec-client` connects to **domX**'s `qrexec-daemon`.
- **dom0**: **domX**'s `qrexec-daemon` sends `MSG_HELLO` header followed by `peer_info` to `qrexec-client`.
- **dom0**: `qrexec-client` replies with `MSG_HELLO` header followed by `peer_info` to **domX**'s`qrexec-daemon`.
- **dom0**: `qrexec-client` sends `MSG_EXEC_CMDLINE` header followed by `exec_params` to **domX**'s`qrexec-daemon`
~~~
/* variable size */
struct exec_params {
uint32_t connect_domain; /* target domain id */
uint32_t connect_port; /* target vchan port for i/o exchange */
char cmdline[0]; /* command line to execute, size = msg_header.len - sizeof(struct exec_params) */
};
~~~
In this case, `connect_domain` is set to id of **domY** (from the `-c` parameter) and `connect_port` is set to 0. `cmdline` field contains the RPC to execute, in this case `user:QUBESRPC qubes.SomeRpc domY`.
- **dom0**: **domX**'s `qrexec-daemon` replies to `qrexec-client` with `MSG_EXEC_CMDLINE` header followed by `exec_params`, but with empty `cmdline` field. `connect_domain` is set to Qubes ID of **domX** and `connect_port` is set to a vchan port allocated by **domX**'s `qrexec-daemon`.
- **dom0**: **domX**'s `qrexec-daemon` sends `MSG_EXEC_CMDLINE` header followed by `exec_params` to **domX**'s `qrexec-agent`. `connect_domain` and `connect_port` fields are the same as in the step above. `cmdline` is set to the one received from `qrexec-client`, in this case `user:QUBESRPC qubes.SomeRpc domY`.
- **dom0**: `qrexec-client` disconnects from **domX**'s `qrexec-daemon` after receiving connection details.
- **dom0**: `qrexec-client` connects to **domY**'s `qrexec-daemon` and exchanges MSG\_HELLO as usual.
- **dom0**: `qrexec-client` sends `MSG_SERVICE_CONNECT` header followed by `exec_params` to **domY**'s `qrexec-daemon`. `connect_domain` is set to ID of **domX** (received from **domX**'s `qrexec-daemon`) and `connect_port` is the one received as well. `cmdline` is set to request ID (`13` in this case).
- **dom0**: **domY**'s `qrexec-daemon` sends `MSG_SERVICE_CONNECT` header followed by `exec_params` to **domY**'s `qrexec-agent`. Data fields are unchanged from the step above.
- **domY**: `qrexec-agent` starts a vchan server on the port received in the step above. It acts as a `qrexec-client` in this case because this is a VM-VM connection.
- **domX**: `qrexec-agent` connects to the vchan server of **domY**'s `qrexec-agent` (connection details were received before from **domX**'s `qrexec-daemon`).
- After that, connection follows the flow of the previous example (dom0-VM).