Git-Mirrors/qubes-doc
1
0
Fork 0
mirror of https://github.com/QubesOS/qubes-doc.git synced 2025-05-02 06:46:11 -04:00
Code Issues Projects Releases Packages Wiki Activity

Reorganize files to account for new "External" section

Browse source 
QubesOS/qubes-issues#4693
This commit is contained in:
Andrew David Wong 2019-05-26 19:32:45 -05:00
parent 5cc99a23d1
commit d31c786942
No known key found for this signature in database
GPG key ID: 8CE137352A019A17
203 changed files with 0 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

229
developer/debugging/automated-tests.md Normal file
View file

@ -0,0 +1,229 @@
---
layout: doc
title: Automated Tests
permalink: /doc/automated-tests/
redirect_from:
- /en/doc/automated-tests/
- /doc/AutomatedTests/
---
Automated Tests
===============
Unit and Integration Tests
--------------------------
Starting with Qubes R3 we use [python unittest][unittest] to perform automatic tests of Qubes OS.
Despite the name, we use it for both [unit tests](https://en.wikipedia.org/wiki/Unit_tests) and [integration tests](https://en.wikipedia.org/wiki/Integration_tests).
The main purpose is, of course, to deliver much more stable releases.
Integration tests are written with the assumption that they will be called on dedicated hardware.
**Do not run these tests on installations with important data, because you might lose it.**
Since these tests were written with this expectation, all the VMs with a name starting with `test-` on the installation are removed during the process, and all the tests are recklessly started from dom0, even when testing VM components.
Most of the tests are stored in the [core-admin repository](https://github.com/QubesOS/qubes-core-admin/tree/master/qubes/tests) in the `tests` directory.
To start them you can use standard python unittest runner:
python -m unittest -v qubes.tests
Or our custom one:
python -m qubes.tests.run -v
Our test runner runs mostly the same as the standard one, but it has some nice additional features like color output and not needing the "qubes.test" prefix.
It also has the ability to run lone selected template tests.
You can use `python -m qubes.tests.run -h` to get usage information:
[user@dom0 ~]$ python -m qubes.tests.run -h
usage: run.py [-h] [--verbose] [--quiet] [--list] [--failfast] [--no-failfast]
[--do-not-clean] [--do-clean] [--loglevel LEVEL]
[--logfile FILE] [--syslog] [--no-syslog] [--kmsg] [--no-kmsg]
[TESTNAME [TESTNAME ...]]
positional arguments:
TESTNAME list of tests to run named like in description
(default: run all tests)
optional arguments:
-h, --help show this help message and exit
--verbose, -v increase console verbosity level
--quiet, -q decrease console verbosity level
--list, -l list all available tests and exit
--failfast, -f stop on the first fail, error or unexpected success
--no-failfast disable --failfast
--do-not-clean, --dnc, -D
do not execute tearDown on failed tests. Implies
--failfast.
--do-clean, -C do execute tearDown even on failed tests.
--loglevel LEVEL, -L LEVEL
logging level for file and syslog forwarding (one of:
NOTSET, DEBUG, INFO, WARN, WARNING, ERROR, CRITICAL;
default: DEBUG)
--logfile FILE, -o FILE
if set, test run will be also logged to file
--syslog reenable logging to syslog
--no-syslog disable logging to syslog
--kmsg, --very-brave-or-very-stupid
log most important things to kernel ring-buffer
--no-kmsg, --i-am-smarter-than-kay-sievers
do not abuse kernel ring-buffer
When running only specific tests, write their names like in log, in format:
MODULE+"/"+CLASS+"/"+FUNCTION. MODULE should omit initial "qubes.tests.".
Example: basic/TC_00_Basic/test_000_create
For instance, to run only the tests for the fedora-21 template, you can use the `-l` option, then filter the list:
[user@dom0 ~]$ python -m qubes.tests.run -l | grep fedora-21
network/VmNetworking_fedora-21/test_000_simple_networking
network/VmNetworking_fedora-21/test_010_simple_proxyvm
network/VmNetworking_fedora-21/test_020_simple_proxyvm_nm
network/VmNetworking_fedora-21/test_030_firewallvm_firewall
network/VmNetworking_fedora-21/test_040_inter_vm
vm_qrexec_gui/TC_00_AppVM_fedora-21/test_000_start_shutdown
vm_qrexec_gui/TC_00_AppVM_fedora-21/test_010_run_gui_app
vm_qrexec_gui/TC_00_AppVM_fedora-21/test_050_qrexec_simple_eof
vm_qrexec_gui/TC_00_AppVM_fedora-21/test_051_qrexec_simple_eof_reverse
vm_qrexec_gui/TC_00_AppVM_fedora-21/test_052_qrexec_vm_service_eof
vm_qrexec_gui/TC_00_AppVM_fedora-21/test_053_qrexec_vm_service_eof_reverse
vm_qrexec_gui/TC_00_AppVM_fedora-21/test_060_qrexec_exit_code_dom0
vm_qrexec_gui/TC_00_AppVM_fedora-21/test_065_qrexec_exit_code_vm
vm_qrexec_gui/TC_00_AppVM_fedora-21/test_100_qrexec_filecopy
vm_qrexec_gui/TC_00_AppVM_fedora-21/test_110_qrexec_filecopy_deny
vm_qrexec_gui/TC_00_AppVM_fedora-21/test_120_qrexec_filecopy_self
vm_qrexec_gui/TC_20_DispVM_fedora-21/test_000_prepare_dvm
vm_qrexec_gui/TC_20_DispVM_fedora-21/test_010_simple_dvm_run
vm_qrexec_gui/TC_20_DispVM_fedora-21/test_020_gui_app
vm_qrexec_gui/TC_20_DispVM_fedora-21/test_030_edit_file
[user@dom0 ~]$ python -m qubes.tests.run -v `python -m qubes.tests.run -l | grep fedora-21`
Example test run:
![snapshot-tests2.png](/attachment/wiki/developers/snapshot-tests2.png)
### Qubes 4.0
Tests on Qubes 4.0 require stopping `qubesd` service first, because special instance of it is started as part of the test run.
Additionally, tests needs to be started as root. The full command to run the tests is:
sudo systemctl stop qubesd; sudo -E python3 -m qubes.tests.run -v ; sudo systemctl start qubesd
On Qubes 4.0 tests are also compatible with nose2 test runner, so you can use this instead:
sudo systemctl stop qubesd; sudo -E nose2 -v --plugin nose2.plugins.loader.loadtests qubes.tests; sudo systemctl start qubesd
This may be especially useful together with various nose2 plugins to store tests results (for example `nose2.plugins.junitxml`), to ease presenting results. This is what we use on [OpenQA].
### Tests configuration
Test run can be altered using environment variables:
- `DEFAULT_LVM_POOL` - LVM thin pool to use for tests, in `VolumeGroup/ThinPool` format
- `QUBES_TEST_PCIDEV` - PCI device to be used in PCI passthrough tests (for example sound card)
- `QUBES_TEST_TEMPLATES` - space separated list of templates to run tests on; if not set, all installed templates are tested
- `QUBES_TEST_LOAD_ALL` - load all tests (including tests for all templates) when relevant test modules are imported; this needs to be set for test runners not supporting [load_tests protocol](https://docs.python.org/3/library/unittest.html#load-tests-protocol)
### Adding a new test to core-admin
After adding a new unit test to [core-admin/tests](https://github.com/QubesOS/qubes-core-admin/tree/master/tests) you'll have to make sure of two things:
1. That the test will be added to the RPM file created by [QubesBuilder](/doc/qubes-builder/). For this you need to edit the [core-admin/tests/Makefile](https://github.com/QubesOS/qubes-core-admin/tree/master/tests/Makefile)
2. That the test will be loaded by [core-admin/tests/\_\_init\_\_.py](https://github.com/QubesOS/qubes-core-admin/tree/master/tests/__init__.py)
#### Editing the Makefile
To add your tests, you must append these two lines to the end of the makefile, which will copy your test and its compiled version to the right directory in the RPM file.
If your test is `example.py`, the appended lines would be:
cp example.py $(DESTDIR)$(PYTHON_TESTSPATH)
cp example.py[co] $(DESTDIR)$(PYTHON_TESTSPATH)
#### Editing `__init__.py`
You'll also need to add your test at the bottom of the `__init__.py` file, in the method `def load_tests`, in the for loop with `modname`.
Again, given the hypothetical `example.py` test:
~~~python
for modname in (
'qubes.tests.basic',
'qubes.tests.dom0_update',
'qubes.tests.network',
'qubes.tests.vm_qrexec_gui',
'qubes.tests.backup',
'qubes.tests.backupcompatibility',
'qubes.tests.regressions',
'qubes.tests.example', # This is our newly added test
):
~~~
### Testing PyQt applications
When testing (Py)QT application, it's useful to create separate QApplication object for each test.
But QT framework does not allow to have multiple QApplication objects in the same process at the same time.
This means it's critical to reliably cleanup previous instance before creating the new one.
This turns out to be non-trivial task, especially if _any_ test uses event loop.
Failure to perform proper cleanup in many cases results in SEGV.
Below you can find steps for the proper cleanup:
~~~python
import asyncio
import quamash
import unittest
import gc
class SomeTestCase(unittest.TestCase):
def setUp(self):
[...]
# force "cleanlooks" style, the default one on Xfce (GtkStyle) use
# static variable internally and caches pointers to later destroyed
# objects (result: SEGV)
self.qtapp = QtGui.QApplication(["test", "-style", "cleanlooks"])
# construct event loop even if this particular test doesn't use it,
# otherwise events with qtapp references will be queued there anyway and the
# first test that actually use event loop will try to dereference (already
# destroyed) objects, resulting in SEGV
self.loop = quamash.QEventLoop(self.qtapp)
def tearDown(self):
[...]
# process any pending events before destroying the object
self.qtapp.processEvents()
# queue destroying the QApplication object, do that for any other QT
# related objects here too
self.qtapp.deleteLater()
# process any pending events (other than just queued destroy), just in case
self.qtapp.processEvents()
# execute main loop, which will process all events, _including just queued destroy_
self.loop.run_until_complete(asyncio.sleep(0))
# at this point it QT objects are destroyed, cleanup all remaining references;
# del other QT object here too
self.loop.close()
del self.qtapp
del self.loop
gc.collect()
~~~
Installation Tests with openQA
------------------------------
**URL:** <https://openqa.qubes-os.org/>
**Tests:** <https://github.com/marmarek/openqa-tests-qubesos>
Manually testing the installation of Qubes OS is a time-consuming process.
We use [openQA] to automate this process.
It works by installing Qubes in KVM and interacting with it as a user would, including simulating mouse clicks and keyboard presses.
Then, it checks the output to see whether various tests were passed, e.g., by comparing the virtual screen output to screenshots of a successful installation.
Using openQA to automatically test the Qubes installation process works as of Qubes 4.0-rc4 on 2018-01-26, provided that the versions of KVM and QEMU are new enough and the hardware has VT-x and EPT.
KVM also supports nested virtualization, so HVM should theoretically work.
In practice, however, either Xen or QEMU crashes when this is attempted.
Nonetheless, PV works well, which is sufficient for automated installation testing.
Thanks to an anonymous donor, our openQA system is hosted in a datacenter on hardware that meets these requirements.
[unittest]: https://docs.python.org/2/library/unittest.html
[OpenQA]: http://open.qa/

50
developer/debugging/mount-lvm-image.md Normal file
View file

@ -0,0 +1,50 @@
---
layout: doc
title: Mount LVM image
permalink: /doc/mount-lvm-image/
---
# How to mount LVM image
You want to read your LVM image (ex: you did some errors and can't start the VM ).
1: make the image available for qubesdb.
```bash
# Example: /dev/qubes_dom0/vm-debian-9-tmp-root
[user@dom0]$ dev=$(basename $(readlink /dev/YOUR_LVM_VG/YOUR_LVM_IMAGE))
[user@dom0]$ qubesdb-write /qubes-block-devices/$dev/desc "YOUR_LVM_IMAGE"
```
2: Create a new disposable VM
```bash
[user@dom0]$ qvm-run -v --dispvm=YOUR_DVM_TEMPLATE --service qubes.StartApp+xterm &
```
3: Attach the device to your newly created disp VM
From the GUI, or from the command line:
```bash
[user@dom0]$ qvm-block attach NEWLY_CREATED_DISPVM dom0:$dev
```
4: Mount the partition you want to, and do what you want with it
```bash
[user@dispXXXX]$ mount /dev/xvdiX /mnt/
```
5: Umount and kill the VM
```
[user@dispXXXX]$ umount /mnt/
```
6: Remove the image from qubesdb
```
[user@dom0]$ qubesdb-rm /qubes-block-devices/$dev/
```
# References
https://github.com/QubesOS/qubes-issues/issues/4687#issuecomment-451626625

98
developer/debugging/profiling.md Normal file
View file

@ -0,0 +1,98 @@
---
layout: doc
title: Profiling
permalink: /doc/profiling/
redirect_from:
- /en/doc/profiling/
- /doc/Profiling/
- /wiki/Profiling/
---
Profiling
=========
This is python profiling primer.
For the purpose of this document, `qubes-dev` is name of the domain used for postprocessing profiling stats.
Requirements
------------
~~~
yum install gprof2dot graphviz
git clone http://git.woju.eu/qubes/profiling.git
~~~
If you profile something on dom0, move `Upload.sh` from repository to dom0:
~~~
mkdir -p ~/profiling
qvm-run -p qubes-dev 'cat ~/profiling/Upload.sh' > ~/profiling/Upload.sh
~~~
- WARNING: this will obviously be running third-party code which is not signed by ITL nor Fedora. You have been warned.
Workflow
--------
### Identify function responsible for some slow action
You have to select area in which you suspect less than optimal performance. If you do not narrow the area, graphs may be unreadable.
### Replace suspect function with probe
Replace
def foo(self, bar):
# function content
with
def foo(self, *args, **kwargs):
profile.runctx('self.real_foo(*args, **kwargs)', globals(), locals(),
time.strftime('/home/user/profiling/foo-%Y%m%d-%H%M%S.pstats'))
def real_foo(self, bar):
# function content
### Run application
Beware that some functions may be called often. For example `qubesmanager/main.py:update_table` gets run once per second. This will produce one pstat file per second.
Remember to revert your changes to application afterwards.
### Upload statistics
If you are in dom0:
~~~
cd ~/profiling
./Upload.sh
~~~
### Analyse
~~~
make
~~~
For every `${basename}.pstats` this will produce `${basename}.txt` and `${basename}.svg`. SVG contains call graph. Text file contains list of all functions sorted by cumulative execution time. You may also try `make all-png`.
~~~
make index.html
~~~
This creates `index.html` with all SVG graphics linked to TXT files. Ready for upload.
~~~
make REMOTE=example.com:public_html/qubes/profiling/ upload
~~~
Example
-------
This example is from `qubes-manager` (`qubesmanager/main.py`).
!["update\_table-20140424-170010.svg"](//attachment/wiki/Profiling/update_table-20140424-170010.svg)
It is apparent than problem is around `get_disk_usage` which calls something via `subprocess.call`. It does it 15 times, probably once per VM.

78
developer/debugging/safe-remote-ttys.md Normal file
View file

@ -0,0 +1,78 @@
---
layout: doc
title: Safe Remote Dom0 Terminals
permalink: /doc/safe-remote-ttys/
redirect_from:
- /en/doc/safe-remote-ttys/
- /doc/safe-remote-ttys/
---
Safe Remote Dom0 Terminals
==========================
If you do not have working graphics in Dom0, then using a terminal can be quite annoying!
This was the case for the author while trying to debug PCI-passthrough of a machine's primary (only) GPU.
Your first thought might be to just allow network access to Dom0, enable ssh, and connect in remotely.
But, this gravely violates the Qubes security model.
Instead, a better solution is to split the input and output paths of using a terminal.
Use your normal keyboard for input, but have the output go to a remote machine in a unidirectional manner.
To do this, we make use of script(1), qvm-run, and optionally your network transport of choice.
To a different VM
-----------------
As an example of forwarding terminal output to another VM on the same machine:
~~~
$ mkfifo /tmp/foo
$ qvm-run -p some-vm 'xterm -e "cat 0<&5" 5<&0' </tmp/foo >/dev/null 2>&1 &
$ script -f /tmp/foo
~~~
To a different machine
----------------------
In this case over SSH (from a network-connected VM):
~~~
$ mkfifo /tmp/foo
$ qvm-run -p some-vm \
'ssh user@host sh -c "DISPLAY=:0 xterm -e \"cat 0<&5\" 5<&0"' \
</tmp/foo >/dev/null 2>&1 &
$ script -f /tmp/foo
~~~
Note that no data received over SSH is ever treated as terminal input in Dom0.
The input path remains only from your trusted local keyboard.
Multiple terminals
------------------
For multiple terminals, you may find it easier to just use tmux than to try to blindly switch to the correct window.
Terminal size
-------------
It is up to you to ensure the sizes of the local and remote terminal are the same, otherwise things may display incorrectly (especially in interactive programs).
Depending on your shell, the size of your local (blind) terminal is likely stored in the `$LINES` and `$COLUMNS` variables.
~~~
$ echo $COLUMNS $LINES
80 24
~~~
A note on serial consoles
-------------------------
If your machine has a serial console, you may with to use that, but note that a similar split-I/O model should be used to ensure Dom0 integrity.
If you use the serial console as normal (via e.g. getty on ttyX, and logging in as normal), then the machine at the end of the serial cable could compromise your machine!
Ideally, you would take input from your trusted keyboard, and only send the output over the serial cable via e.g. disabling getty and using:
~~~
script -f /dev/ttyS0
~~~
You don't even need to connect the TX pin.

156
developer/debugging/test-bench.md Normal file
View file

@ -0,0 +1,156 @@
---
layout: doc
title: Test Bench
permalink: /doc/test-bench/
redirect_from:
- /en/doc/test-bench/
- /doc/TestBench/
- /wiki/TestBench/
---
Test bench for Dom0
===================
This guide shows how to set up simple test bench that automatically test your code you're about to push. It is written especially for `core3` branch of `core-admin.git` repo, but some ideas are universal.
We will set up a spare machine (bare metal, not a virtual) that will be hosting our experimental Dom0. We will communicate with it via Ethernet and SSH. This tutorial assumes you are familiar with [QubesBuilder](/doc/qubes-builder/) and you have it set up and running flawlessly.
Setting up the machine
----------------------
First, do a clean install from ISO you built or grabbed elsewhere.
You have to fix network, because it is intentionally broken. This script should reenable your network card without depending on anything else.
#!/bin/sh
# adjust this for your NIC (run lspci)
BDF=0000:02:00.0
prog=$(basename $0)
pciunbind() {
local path
path=/sys/bus/pci/devices/${1}/driver/unbind
if ! [ -w ${path} ]; then
echo "${prog}: Device ${1} not bound"
return 1
fi
echo -n ${1} >${path}
}
pcibind() {
local path
path=/sys/bus/pci/drivers/${2}/bind
if ! [ -w ${path} ]; then
echo "${prog}: Driver ${2} not found"
return 1
fi
echo ${1} >${path}
}
pciunbind ${BDF}
pcibind ${BDF} e1000e
dhclient
TODO: describe how to run this at every startup
Now configure your DHCP server so your testbench gets static IP and connect your machine to your local network. You should ensure that your testbench can reach the Internet.
Install `openssh-server` on your testbench:
~~~
yum install openssh-server
~~~
Ensure that sudo works without password from your user account (it should by default).
Development VM
--------------
### SSH
Arrange firewall so you can reach the testbench from your `qubes-dev` VM. Generate SSH key in `qubes-dev`:
~~~
ssh-keygen -t ecdsa -b 521
~~~
Add the following section in `.ssh/config` in `qubes-dev`:
~~~
Host testbench
# substitute username in testbench
User user
# substitute address of your testbench
HostName 192.168.123.45
~~~
Then connect to your testbench and paste newly generated `id_ecdsa.pub` to `.ssh/authorized_keys` on testbench so you can log in without entering password every time.
### Scripting
This step is optional, but very helpful. Put these scripts somewhere in your `${PATH}`, like `/usr/local/bin`.
`qtb-runtests`:
#!/bin/sh
ssh testbench python -m qubes.tests.run
`qtb-install`:
#!/bin/sh
TMPDIR=/tmp/qtb-rpms
if [ $# -eq 0 ]; then
echo "usage: $(basename $0) <rpmfile> ..."
exit 2
fi
set -e
ssh testbench mkdir -p "${TMPDIR}"
scp "${@}" testbench:"${TMPDIR}"
while [ $# -gt 0 ]; do
ssh testbench sudo rpm -i --replacepkgs --replacefiles "${TMPDIR}/$(basename ${1})"
shift
done
`qtb-iterate`:
#!/bin/sh
set -e
# substitute path to your builder installation
pushd ${HOME}/builder >/dev/null
# the following are needed only if you have sources outside builder
#rm -rf qubes-src/core-admin
#make COMPONENTS=core-admin get-sources
make core-admin
qtb-install qubes-src/core-admin/rpm/x86_64/qubes-core-dom0-*.rpm
qtb-runtests
### Hooking git
I (woju) have those two git hooks. They ensure tests are passing (or are marked as expected failure) when committing and pushing. For committing it is only possible to run tests that may be executed from git repo (even if the rest were available, I probably wouldn't want to do that). For pushing, I also install RPM and run tests on testbench.
`core-admin/.git/hooks/pre-commit`: (you may retain also the default hook, here omitted for readability)
#!/bin/sh
set -e
python -c "import sys, qubes.tests.run; sys.exit(not qubes.tests.run.main())"
`core-admin/.git/hooks/pre-push`:
#!/bin/sh
exec qtb-iterate

210
developer/debugging/vm-interface.md Normal file
View file

@ -0,0 +1,210 @@
---
layout: doc
title: VM Configuration Interface
permalink: /doc/vm-interface/
redirect_from:
- /en/doc/vm-interface/
- /doc/VMInterface/
- /doc/SystemDoc/VMInterface/
- /wiki/SystemDoc/VMInterface/
---
VM Configuration Interface
==========================
Qubes VM have some settings set by dom0 based on VM settings. There are multiple configuration channels, which includes:
- QubesDB
- XenStore (in Qubes 2, data the same as in QubesDB, keys without leading `/`)
- Qubes RPC (called at VM startup, or when configuration changed)
- GUI protocol
QubesDB
--------------------
### Keys exposed by dom0 to VM ###
- `/qubes-vm-type` - VM type, the same as `type` field in `qvm-prefs`. One of `AppVM`, `ProxyVM`, `NetVM`, `TemplateVM`, `HVM`, `TemplateHVM`
- `/qubes-vm-updatable` - flag whether VM is updatable (whether changes in root.img will survive VM restart). One of `True`, `False`
- `/qubes-vm-persistence` - what data do persist between VM restarts:
- `full` - all disks
- `rw-only` - only `/rw` disk
- `none` - none
- `/qubes-timezone - name of timezone based on dom0 timezone. For example `Europe/Warsaw`
- `/qubes-keyboard` - keyboard layout based on dom0 layout. Its syntax is suitable for `xkbcomp` command (after expanding escape sequences like `\n` or `\t`). This is meant only as some default value, VM can ignore this option and choose its own keyboard layout (this is what keyboard setting from Qubes Manager does). This entry is created as part of gui-daemon initialization (so not available when gui-daemon disabled, or not started yet).
- `/qubes-debug-mode` - flag whether VM has debug mode enabled (qvm-prefs setting). One of `1`, `0`
- `/qubes-service/SERVICE_NAME` - subtree for VM services controlled from dom0 (using the `qvm-service` command or Qubes Manager). One of `1`, `0`. Note that not every service will be listed here, if entry is missing, it means "use VM default". A list of currently supported services is in the `qvm-service` man page.
- `/qubes-netmask` - network mask (only when VM has netvm set); currently hardcoded "255.255.255.0"
- `/qubes-ip - IP address for this VM (only when VM has netvm set)
- `/qubes-gateway` - default gateway IP (only when VM has netvm set); VM should add host route to this address directly via eth0 (or whatever default interface name is)
- `/qubes-primary-dns` - primary DNS address (only when VM has netvm set)
- `/qubes-secondary-dns` - secondary DNS address (only when VM has netvm set)
- `/qubes-netvm-gateway` - same as `qubes-gateway` in connected VMs (only when VM serves as network backend - ProxyVM and NetVM)
- `/qubes-netvm-netmask` - same as `qubes-netmask` in connected VMs (only when VM serves as network backend - ProxyVM and NetVM)
- `/qubes-netvm-network` - network address (only when VM serves as network backend - ProxyVM and NetVM); can be also calculated from qubes-netvm-gateway and qubes-netvm-netmask
- `/qubes-netvm-primary-dns` - same as `qubes-primary-dns` in connected VMs (only when VM serves as network backend - ProxyVM and NetVM); traffic sent to this IP on port 53 should be redirected to primary DNS server
- `/qubes-netvm-secondary-dns` - same as `qubes-secondary-dns` in connected VMs (only when VM serves as network backend - ProxyVM and NetVM); traffic sent to this IP on port 53 should be redirected to secondary DNS server
#### Firewall rules in 3.x ####
QubesDB is also used to configure firewall in ProxyVMs. Rules are stored in
separate key for each target VM. Entries:
- `/qubes-iptables` - control entry - dom0 writing `reload` here signals `qubes-firewall` service to reload rules
- `/qubes-iptables-header` - rules not related to any particular VM, should be applied before domains rules
- `/qubes-iptables-domainrules/NNN` - rules for domain `NNN` (arbitrary number)
in `iptables-save` format. Rules are self-contained - fill `FORWARD` iptables
chain and contains all required matches (source IP address etc), as well as
final default action (`DROP`/`ACCEPT`)
VM after applying rules may signal some error, writing a message to
`/qubes-iptables-error` key. This does not exclude any other way of
communicating problems - like a popup.
#### Firewall rules in 4.x ####
QubesDB is also used to configure firewall in ProxyVMs. Each rule is stored as
a separate entry, grouped on target VM:
- `/qubes-firewall/SOURCE_IP` - base tree under which rules are placed. All
rules there should be applied to filter traffic coming from `SOURCE_IP`. This
can be either IPv4 or IPv6 address.
Dom0 will do an empty write to this top level entry after finishing rules
update, so VM can setup a watch here to trigger rules reload.
- `/qubes-firewall/SOURCE_IP/policy` - default action if no rule matches:
`drop` or `accept`.
- `/qubes-firewall/SOURCE_IP/NNNN` - rule number `NNNN` - decimal number,
padded with zeros. Se below for rule format. All the rules should be
applied in order of rules implied by those numbers. Note that QubesDB
itself does not impose any ordering (you need to sort the rules after
retrieving them). The first rule has number `0000`.
Each rule is a single QubesDB entry, consisting of pairs `key=value` separated
by space. QubesDB enforces limit on a single entry length - 3072 bytes.
Possible options for a single rule:
- `action`, values: `accept`, `drop`; this is present in every rule
- `dst4`, value: destination IPv4 address with a mask; for example: `192.168.0.0/24`
- `dst6`, value: destination IPv6 address with a mask; for example: `2000::/3`
- `dsthost`, value: DNS hostname of destination host
- `proto`, values: `tcp`, `udp`, `icmp`
- `specialtarget`, value: One of predefined target, currently defined values:
- `dns` - such option should match DNS traffic to default DNS server (but
not any DNS server), on both TCP and UDP
- `dstports`, value: destination ports range separated with `-`, valid only
together with `proto=tcp` or `proto=udp`; for example `1-1024`, `80-80`
- `icmptype`, value: numeric (decimal) icmp message type, for example `8` for
echo request, valid only together with `proto=icmp`
- `dpi`, value: Deep Packet Inspection protocol (like: HTTP, SSL, SMB, SSH, SMTP) or the default 'NO' as no DPI, only packet filtering
Options must appear in the rule in the order listed above. Duplicated options
are forbidden.
A rule matches only when all predicates match. Only one of `dst4`, `dst6` or
`dsthost` can be used in a single rule.
If tool applying firewall encounters any parse error (unknown option, invalid
value, duplicated option, etc), it should drop all the traffic coming from that `SOURCE_IP`,
regardless of properly parsed rules.
Example valid rules:
- `action=accept dst4=8.8.8.8 proto=udp dstports=53-53`
- `action=drop dst6=2a00:1450:4000::/37 proto=tcp`
- `action=accept specialtarget=dns`
- `action=drop proto=tcp specialtarget=dns` - drop DNS queries sent using TCP
- `action=drop`
### Keys set by VM for passing info to dom0 ###
- `memory/meminfo` (**xenstore**) - used memory (updated by qubes-meminfo-writer), input information for qmemman;
- Qubes 3.x format: 6 lines (EOL encoded as `\n`), each in format "FIELD: VALUE kB"; fields: `MemTotal`, `MemFree`, `Buffers`, `Cached`, `SwapTotal`, `SwapFree`; meaning the same as in `/proc/meminfo` in Linux.
- Qubes 4.0+ format: used memory size in the VM, in kbytes
- `/qubes-block-devices` - list of block devices exposed by this VM, each device (subdirectory) should be named in a way that VM can attach the device based on it. Each should contain these entries:
- `desc` - device description (ASCII text)
- `size` - device size in bytes
- `mode` - default connection mode; `r` for read-only, `w` for read-write
- `/qubes-usb-devices` - list of USB devices exposed by this VM, each device (subdirectory) should contain:
- `desc` - device description (ASCII text)
- `usb-ver` - USB version (1, 2 or 3)
Qubes RPC
---------
Services called by dom0 to provide some VM configuration:
- `qubes.SetMonitorLayout` - provide list of monitors, one per line. Each line contains four numbers: `width height X Y width_mm height_mm` (physical dimensions - `width_mm` and `height_mm` - are optional)
- `qubes.WaitForSession` - called to wait for full VM startup
- `qubes.GetAppmenus` - receive appmenus from given VM (template); TODO: describe format here
- `qubes.GetImageRGBA` - receive image/application icon. Protocol:
1. Caller sends name of requested icon. This can be one of:
* `xdgicon:NAME` - search for NAME in standard icons theme
* `-` - get icon data from stdin (the caller), can be prefixed with format name, for example `png:-`
* file name
2. The service responds with image dimensions: width and height as
decimal numbers, separated with space and with EOL marker at the and; then
image data in RGBA format (32 bits per pixel)
- `qubes.SetDateTime` - set VM time, called periodically by dom0 (can be
triggered manually from dom0 by calling `qvm-sync-clock`). The service
receives one line at stdin - time in format of `date -u -Iseconds`, for
example `2015-07-31T16:10:43+0000`.
- `qubes.SetGuiMode` - called in HVM to switch between fullscreen and seamless
GUI mode. The service receives a single word on stdin - either `FULLSCREEN`
or `SEAMLESS`
- `qubes.ResizeDisk` - called to inform that underlying disk was resized.
Name of disk image is passed on standard input (`root`, `private`, `volatile`,
or other). This is used starting with Qubes 4.0.
Other Qrexec services installed by default:
- `qubes.Backup` - store Qubes backup. The service receives location chosen by
the user (one line, terminated by '\n'), the backup archive ([description of
backup format](/doc/BackupEmergencyRestoreV2/))
- `qubes.DetachPciDevice` - service called in reaction to `qvm-pci -d` call on
running VM. The service receives one word - BDF of device to detach. When the
service call ends, the device will be detached
- `qubes.Filecopy` - receive some files from other VM. Files sent in [qfile format](/doc/qfilecopy/)
- `qubes.OpenInVM` - open a file in called VM. Service receives a single file on stdin (in
[qfile format](/doc/qfilecopy/). After a file viewer/editor is terminated, if
the file was modified, can be sent back (just raw content, without any
headers); otherwise service should just terminate without sending anything.
This service is used by both `qvm-open-in-vm` and `qvm-open-in-dvm` tools. When
called in DispVM, service termination will trigger DispVM cleanup.
- `qubes.Restore` - retrieve Qubes backup. The service receives backup location
entered by the user (one line, terminated by '\n'), then should output backup
archive in [qfile format](/doc/qfilecopy/) (core-agent-linux component contains
`tar2qfile` utility to do the conversion)
- `qubes.SelectDirectory`, `qubes.SelectFile` - services which should show
file/directory selection dialog and return (to stdout) a single line
containing selected path, or nothing in the case of cancellation
- `qubes.SuspendPre` - service called in every VM with PCI device attached just
before system suspend
- `qubes.SuspendPost` - service called in every VM with PCI device attached just
after system resume
- `qubes.SyncNtpClock` - service called to trigger network time synchronization.
Service should synchronize local VM time and terminate when done.
- `qubes.WindowIconUpdater` - service called by VM to send icons of individual
windows. The protocol there is simple one direction stream: VM sends window ID
followed by icon in `qubes.GetImageRGBA` format, then next window ID etc. VM
can send icon for the same window multiple times to replace previous one (for
example for animated icons)
- `qubes.VMShell` - call any command in the VM; the command(s) is passed one per line
Currently Qubes still calls few tools in VM directly, not using service
abstraction. This will change in the future. Those tools are:
- `/usr/lib/qubes/qubes-download-dom0-updates.sh` - script to download updates (or new packages to be installed) for dom0 (`qubes-dom0-update` tool)
- `date -u -Iseconds` - called directly to retrieve time after calling `qubes.SyncNtpClock` service (`qvm-sync-clock` tool)
- `nm-online -x` - called before `qubes.SyncNtpClock` service call by `qvm-sync-clock` tool
- `resize2fs` - called to resize filesystem on /rw partition by `qvm-grow-private` tool
- `gpk-update-viewer` - called by Qubes Manager to display available updates in a TemplateVM
- `systemctl start qubes-update-check.timer` (and similarly stop) - called when enabling/disabling updates checking in given VM (`qubes-update-check` [qvm-service](/doc/qubes-service/))
Additionally, automatic tests extensively run various commands directly in VMs. We do not plan to change that.
GUI protocol
------------
GUI initialization includes passing the whole screen dimensions from dom0 to VM. This will most likely be overwritten by qubes.SetMonitorLayout Qubes RPC call.

213
developer/debugging/windows-debugging.md Normal file
View file

@ -0,0 +1,213 @@
---
layout: doc
title: Windows Debugging
permalink: /doc/windows-debugging/
redirect_from:
- /en/doc/windows-debugging/
- /doc/WindowsDebugging/
- /wiki/WindowsDebugging/
---
Debugging Windows HVMs
======================
Debugging Windows code can be tricky in a virtualized environment. The guide below assumes Xen hypervisor and Windows 7 VMs.
User-mode debugging is usually straightforward if it can be done on one machine. Just duplicate your normal debugging environment in the VM.
Things get complicated if you need to perform kernel debugging or troubleshoot problems that only manifest on system boot, user logoff or similar. For that you need two Windows VMs: the *host* and the *target*. The *host* will contain [WinDbg](https://msdn.microsoft.com/en-us/library/windows/hardware/ff551063(v=vs.85).aspx) installation, your source code and private symbols. The *target* will run the code being debugged. Both will be linked by virtual serial ports.
- First, you need to prepare separate copies of both *target* and *host* VM configuration files with some changes. Copy the files from **/var/lib/qubes/appvms/vmname/vmname.conf** to some convenient location, let's call them **host.conf** and **target.conf**.
- In both copied files add the following line at the end: `serial = 'pty'`. This will make Xen connect VM's serial ports to dom0's ptys.
- From now on you need to start both VMs like this: `qvm-start --custom-config=/your/edited/host.conf host`
- To connect both VM serial ports together you will either need [socat](http://www.dest-unreach.org/socat/) or a custom utility described later.
- To determine which dom0 pty corresponds to VM's serial port you need to read xenstore, example script below:
#!/bin/sh
id1=$(xl domid "$1-dm")
tty1=$(xenstore-read /local/domain/${id1}/device/console/3/tty)
echo $tty1
Pass it a running VM name and it will output the corresponding pty name.
- To connect both ptys you can use [socat](http://www.dest-unreach.org/socat/) like that:
#!/bin/sh
id1=$(xl domid "$1-dm")
id2=$(xl domid "$2-dm")
tty1=$(xenstore-read /local/domain/${id1}/device/console/3/tty)
tty2=$(xenstore-read /local/domain/${id2}/device/console/3/tty)
socat $tty1,raw $tty2,raw
...but there is a catch. Xen seems to process the traffic that goes through serial ports and changes all **0x0a** bytes into **0x0d, 0x0a** pairs (newline conversion). I didn't find a way to turn that off (setting ptys to raw mode didn't change anything) and it's not mentioned anywhere on the Internet, so maybe it's something on my system. If the above script works for you then you don't need anything more in dom0.
- On the *target* system, run `bcdedit /set debug on` on the console to turn on kernel debugging. It defaults to the first serial port.
- On the *host* system, install [WinDbg](http://msdn.microsoft.com/en-us/library/windows/hardware/ff551063(v=vs.85).aspx) and start the kernel debug (Ctrl-K), choose **com1** as the debug port.
- Reboot the *target* VM.
- Run the above shell script in dom0.
- If everything is fine you should see the proper kernel debugging output in WinDbg. However, if you see something like that:
~~~
Opened \\.\com1
Waiting to reconnect...
Connected to Windows 7 7601 x64 target at (Wed Mar 19 20:35:43.262 2014 (UTC + 1:00)), ptr64 TRUE
Kernel Debugger connection established.
Symbol search path is: srv*c:\symbols*http://msdl.microsoft.com/download/symbols
Executable search path is:
... Retry sending the same data packet for 64 times.
The transport connection between host kernel debugger and target Windows seems lost.
please try resync with target, recycle the host debugger, or reboot the target Windows.
Unable to read KTHREAD address fffff8000281ccc0
**************************************************************************
Unable to read debugger data block header
**************************************************************************
Unable to read KTHREAD address fffff8000281ccc0
Unable to read PsLoadedModuleList
Unable to read KTHREAD address fffff8000281ccc0
**************************************************************************
Unable to read debugger data block header
**************************************************************************
~~~
...then you're most likely a victim of the CRLF issue mentioned above. To get around it I wrote a small utility that basically does what socat would do and additionally corrects those replaced bytes in the stream. It's not pretty but it works:
#include <errno.h>
#include <stdio.h>
#include <fcntl.h>
#include <termios.h>
int fd1, fd2;
char mark = ' ';
void out(unsigned char c)
{
static int count = 0;
static unsigned char buf[17] = {0};
// relay to ouptput port
write(fd2, &c, 1);
fprintf(stderr, "%c", mark);
/* dump all data going over the line
if (count == 0)
fprintf(stderr, "%c", mark);
fprintf(stderr, "%02x ", c);
if (c >= 0x20 && c < 0x80)
buf[count] = c;
else
buf[count] = '.';
count++;
if (count == 0x10)
{
count = 0;
fprintf(stderr, " %s\n", buf);
}
*/
}
int main(int argc, char* argv[])
{
unsigned char c = 0;
struct termios tio;
ssize_t size;
if (argc < 3)
{
fprintf(stderr, "Usage: %s pty1 pty2 [mark character]\n", argv[0]);
return EINVAL;
}
fd1 = open(argv[1], O_RDONLY | O_NOCTTY);
if (fd1 <= 0)
{
perror("open fd1");
return errno;
}
fd2 = open(argv[2], O_WRONLY | O_NOCTTY);
if (fd2 <= 0)
{
perror("open fd2");
return errno;
}
/*
// This doesn't make any difference which supports the theory
// that it's Xen who corrupts the byte stream.
cfmakeraw(&tio);
if (tcsetattr(fd1, TCSANOW, &tio) < 0)
{
perror("tcsetattr 1");
return errno;
}
if (tcsetattr(fd2, TCSANOW, &tio) < 0)
{
perror("tcsetattr 2");
return errno;
}
*/
if (argc == 4)
mark = argv[3][0];
while (1)
{
size = read(fd1, &c, 1);
if (size <= 0)
break;
parse:
if (c == 0x0d)
{
size = read(fd1, &c, 1);
if (size <= 0)
{
out(0x0d);
break;
}
if (c == 0x0a)
{
out(0x0a);
}
else
{
out(0x0d);
goto parse;
}
}
else
out(c);
}
close(fd1);
close(fd2);
return 0;
}
> This utility is a unidirectional relay so you need to run two instances to get duplex communication, like:
>
> #!/bin/sh
>
> id1=$(xl domid "$1-dm")
> id2=$(xl domid "$2-dm")
> tty1=$(xenstore-read /local/domain/${id1}/device/console/3/tty)
> tty2=$(xenstore-read /local/domain/${id2}/device/console/3/tty)
> ./ptycrlf ${tty1} ${tty2} - &
> ./ptycrlf ${tty2} ${tty1} + &
> With this everything should be good:
>
> ~~~
> Opened \\.\com1
> Waiting to reconnect...
> Connected to Windows 7 7601 x64 target at (Wed Mar 19 20:56:31.371 2014 (UTC + 1:00)), ptr64 TRUE
> Kernel Debugger connection established.
> Symbol search path is: srv*c:\symbols*http://msdl.microsoft.com/download/symbols
> Executable search path is:
> Windows 7 Kernel Version 7601 MP (1 procs) Free x64
> Built by: 7601.18247.amd64fre.win7sp1_gdr.130828-1532
> Machine Name:
> Kernel base = 0xfffff800`0261a000 PsLoadedModuleList = 0xfffff800`0285d6d0
> System Uptime: not available
> ~~~
Happy debugging!