Git-Mirrors/qubes-doc
1
0
Fork 0
mirror of https://github.com/QubesOS/qubes-doc.git synced 2025-07-28 09:14:36 -04:00
Code Issues Projects Releases Packages Wiki Activity

update

Browse source
This commit is contained in:
qubedmaiska 2025-04-16 10:37:02 -04:00
commit cf30f8bab8
No known key found for this signature in database
GPG key ID: 204BCE0FD52C0501
15 changed files with 363 additions and 229 deletions
Download patch file Download diff file Expand all files Collapse all files

81
developer/building/development-workflow.md
View file

@ -12,43 +12,41 @@ title: Development workflow
A workflow for developing Qubes OS+
First things first, setup [QubesBuilder](/doc/qubes-builder/). This guide
assumes you're using qubes-builder to build Qubes.
To begin, setup [QubesBuilder](/doc/qubes-builder-v2/). This guide
assumes you're using qubes-builder v2 to build Qubes.
## Repositories and committing Code
Qubes is split into a bunch of git repos. These are all contained in the
`qubes-src` directory under qubes-builder. Subdirectories there are separate
components, stored in separate git repositories.
Qubes source code is split into many git repos. These are all contained in the
`artifacts/sources` directory under qubes-builder. Subdirectories there are
separate components, stored in separate git repositories.
The best way to write and contribute code is to create a git repo somewhere
(e.g., github) for the repo you are interested in editing (e.g.,
(e.g., GitHub) for the repo you are interested in editing (e.g.,
`qubes-manager`, `core-agent-linux`, etc). To integrate your repo with the rest
of Qubes, cd to the repo directory and add your repository as a remote in git
**Example:**
~~~
$ cd qubes-builder/qubes-src/qubes-manager
$ git remote add abel git@github.com:abeluck/qubes-manager.git
$ cd qubes-builder/artifacts/sources/qubes-manager
$ git remote add abel git@GitHub.com:abeluck/qubes-manager.git
~~~
You can then proceed to easily develop in your own branches, pull in new
commits from the dev branches, merge them, and eventually push to your own repo
on github.
commits from the dev branches, merge them, and eventually push to your own repo.
When you are ready to submit your changes to Qubes to be merged, push your
changes, then create a signed git tag (using `git tag -s`). Finally, send a
letter to the Qubes the [qubes-devel](/support/#qubes-devel) mailing list
describing the changes and including the link to
your repository. You can also create pull request on github. Don't forget to
include your public PGP key you use to sign your tags.
letter to the Qubes listserv describing the changes, and including a link to
your repository. If you are using GitHub you can instead create a pull request.
Don't forget to include the public PGP key you use to sign your tags.
### Kernel-specific notes
#### Prepare fresh version of kernel sources, with Qubes-specific patches applied
In qubes-builder/qubes-src/linux-kernel:
In qubes-builder/artifacts/sources/linux-kernel:
~~~
make prep
@ -67,7 +65,7 @@ drwxr-xr-x 6 user user 4096 Nov 21 20:48 kernel-3.4.18/linux-obj
#### Go to the kernel tree and update the version
In qubes-builder/qubes-src/linux-kernel:
In qubes-builder/artifacts/sources/linux-kernel:
~~~
cd kernel-3.4.18/linux-3.4.18
@ -118,9 +116,7 @@ vi series.conf
#### Building RPMs
TODO: Is this step generic for all subsystems?
Now it is a good moment to make sure you have changed kernel release name in
Now is a good moment to make sure you have changed the kernel release name in
rel file. For example, if you change it to '1debug201211116c' the
resulting RPMs will be named
'kernel-3.4.18-1debug20121116c.pvops.qubes.x86\_64.rpm'. This will help
@ -132,34 +128,23 @@ your changes locally.
To actually build RPMs, in qubes-builder:
~~~
make linux-kernel
./qb -c linux-kernel package fetch prep build
~~~
RPMs will appear in qubes-src/linux-kernel/pkgs/fc20/x86\_64:
RPMs will appear in
`artifacts/repository/destination_name/package_name`
(for example `artifacts/repository/host-fc37/linux-kernel-6.6.31-1.1/`
~~~
-rw-rw-r-- 1 user user 42996126 Nov 17 04:08 kernel-3.4.18-1debug20121116c.pvops.qubes.x86_64.rpm
-rw-rw-r-- 1 user user 43001450 Nov 17 05:36 kernel-3.4.18-1debug20121117a.pvops.qubes.x86_64.rpm
-rw-rw-r-- 1 user user 8940138 Nov 17 04:08 kernel-devel-3.4.18-1debug20121116c.pvops.qubes.x86_64.rpm
-rw-rw-r-- 1 user user 8937818 Nov 17 05:36 kernel-devel-3.4.18-1debug20121117a.pvops.qubes.x86_64.rpm
-rw-rw-r-- 1 user user 54490741 Nov 17 04:08 kernel-qubes-vm-3.4.18-1debug20121116c.pvops.qubes.x86_64.rpm
-rw-rw-r-- 1 user user 54502117 Nov 17 05:37 kernel-qubes-vm-3.4.18-1debug20121117a.pvops.qubes.x86_64.rpm
~~~
### Useful [QubesBuilder](/doc/qubes-builder-v2/) commands
### Useful [QubesBuilder](/doc/qubes-builder/) commands
1. `make check` - will check if all the code was committed into repository and
if all repository are tagged with signed tag.
2. `make show-vtags` - show version of each component (based on git tags) -
mostly useful just before building ISO. **Note:** this will not show version
for components containing changes since last version tag.
3. `make push` - push change from **all** repositories to git server. You must
set proper remotes (see above) for all repositories first.
4. `make prepare-merge` - fetch changes from remote repositories (can be
specified on commandline via GIT\_SUBDIR or GIT\_REMOTE vars), (optionally)
verify tags and show the changes. This do not merge the changes - there are
left for review as FETCH\_HEAD ref. You can merge them using `git merge
FETCH_HEAD` (in each repo directory). Or `make do-merge` to merge all of them.
1. `./qb package diff` - show uncommitted changes
2. ` ./qb repository check-release-status-for-component` and
`./qb repository check-release-status-for-template`- show version of each
component/template (based on git tags)
3. `./qb package sign` - sign built packages
4. `./qb package publish` and `./qb package upload` - publish signed packages
and upload published
repository
## Copying Code to dom0
@ -298,12 +283,12 @@ if [ "$1" = "tb" ]; then
exit $?
fi
git remote add $1 git@github.com:$1/qubes-`basename $PWD`
git remote add $1 git@GitHub.com:$1/qubes-`basename $PWD`
~~~
It should be executed from component top level directory. This script takes one
argument - remote name. If it is `tb`, then it creates qrexec-based git remote
to `testbuilder` VM. Otherwise it creates remote pointing at github account of
to `testbuilder` VM. Otherwise it creates remote pointing at GitHub account of
the same name. In any case it points at repository matching current directory
name.
@ -322,7 +307,7 @@ current and current-testing).
### RPM packages - yum repo
In source VM, grab [linux-yum](https://github.com/QubesOS/qubes-linux-yum) repository (below is assumed you've made it in
In source VM, grab [linux-yum](https://GitHub.com/QubesOS/qubes-linux-yum) repository (below is assumed you've made it in
`~/repo-yum-upload` directory) and replace `update_repo.sh` script with:
~~~
@ -338,7 +323,7 @@ find -type f -name '*.rpm' -delete
qrexec-client-vm $VMNAME local.UpdateYum
~~~
In target VM, setup actual yum repository (also based on [linux-yum](https://github.com/QubesOS/qubes-linux-yum), this time
In target VM, setup actual yum repository (also based on [linux-yum](https://GitHub.com/QubesOS/qubes-linux-yum), this time
without modifications). You will also need to setup some gpg key for signing
packages (it is possible to force yum to install unsigned packages, but it
isn't possible for `qubes-dom0-update` tool). Fill `~/.rpmmacros` with
@ -418,7 +403,7 @@ Remember to also import gpg public key using `rpm --import`.
Steps are mostly the same as in the case of yum repo. The only details that differ:
- use [linux-deb](https://github.com/QubesOS/qubes-linux-deb) instead of [linux-yum](https://github.com/QubesOS/qubes-linux-yum) as a base - both in source and target VM
- use [linux-deb](https://GitHub.com/QubesOS/qubes-linux-deb) instead of [linux-yum](https://GitHub.com/QubesOS/qubes-linux-yum) as a base - both in source and target VM
- use different `update_repo.sh` script in source VM (below)
- use `local.UpdateApt` qrexec service in target VM (code below)
- in target VM additionally place `update-local-repo.sh` script in repository dir (code below)

162
developer/building/qubes-builder-v2.md Normal file
View file

@ -0,0 +1,162 @@
---
lang: en
layout: doc
permalink: /doc/qubes-builder-v2/
redirect_from:
- /en/doc/qubes-builder-v2/
- /doc/QubesBuilder2/
- /wiki/QubesBuilder2/
ref: 311
title: Qubes builder v2
---
This is a brief introduction to using Qubes Builder v2 to work with Qubes OS
sources. It will walk you through installing and configuring Builder v2, and
using it to fetch and build Qubes OS packages.
For details and customization, use [Qubes OS v2 builder documentation](https://github.com/QubesOS/qubes-builderv2/).
# Overview
In the second generation of Qubes OS builder, container or disposable qube
isolation is used to perform every stage of the build and release process.
From fetching sources to building, everything is executed inside an isolated
*cage* (either a disposable or a container) using an *executor*. For every
command that needs to perform an action on sources, like cloning and
verifying Git repos, rendering a SPEC file, generating SRPM or Debian
source packages, a new cage is used. Only the signing, publishing, and
uploading stages are executed locally outside a cage.
# Setup
This is a simple setup using a docker executor. This is a good default choice;
if you don't know which executor to use, use docker.
1. First, decide what qube you are going to use when working with Qubes
Builder v2. It can be an AppVM or a Standalone qube, with some steps
different between the two.
2. Installing dependencies
If you want to use an app qube for developing, install dependencies in the template.
If you are using a standalone, install them in the qube itself.
Dependencies are specified in `dependencies-*.
txt` files in the main builder directory, and you can install them easily
in the following ways:
1. for Fedora, use:
```shell
$ sudo dnf install $(cat dependencies-fedora.txt)
$ test -f /usr/share/qubes/marker-vm && sudo dnf install qubes-gpg-split
```
2. for Debian (note: some Debian packages require Debian version 13 or
later), use:
```shell
$ sudo apt install $(cat dependencies-debian.txt)
$ test -f /usr/share/qubes/marker-vm && sudo apt install qubes-gpg-split
```
If you have installed dependencies in the template, close it, and
(re)start the development qube.
3. Clone the qubes-builder v2 repository into a location of your choice:
```shell
git clone https://github.com/QubesOS/qubes-builderv2
cd qubes-builderv2/
```
4. If you haven't previously used docker in the current qube, you need to set up
some permissions. In particular, the user has to be added to the `docker`
group:
```shell
$ sudo usermod -aG docker user
```
Next, **restart the qube**.
5. Finally, you need to generate a docker image:
```shell
$ tools/generate-container-image.sh docker
```
In an app qube, as `/var/lib/docker` is not persistent by default, you also
need to use [bind-dirs](/doc/bind-dirs/) to avoid repeating this step after reboot, adding
the following to the `/rw/config/qubes-bind-dirs.d/docker.conf` file in
this qube:
```
binds+=( '/var/lib/docker' )
```
# Configuration
To use Qubes OS Builder v2, you need to have a `builder.yml` configuration file.
You can use one of the sample files from the `example-configs/` directory; for a
more readable `builder.yml`, you can also include one of the files from that
directory in your `builder.yml`. An example `builder.yml` is:
```yaml
# include configuration relevant for the current release
include:
- example-configs/qubes-os-r4.2.yml
# which repository to use to fetch sources
use-qubes-repo:
version: 4.2
testing: true
# each package built will have local build number appended to package release
# number. It makes it easier to update in testing environment
increment-devel-versions: true
# reduce output
debug: false
# this can be set to true if you do not want sources to be automatically
# fetched from git
skip-git-fetch: false
# executor configuration
executor:
type: docker
options:
image: "qubes-builder-fedora:latest"
```
# Using Builder v2
To fetch sources - in this example, for the `core-admin-client` package, you
can use the following command:
```shell
$ ./qb -c core-admin-client package fetch
```
This will fetch the sources for the listed package and place them in
`artifacts/sources` directory.
To build a package (from sources in the `artifacts/sources` directory), use:
```shell
$ ./qb -c core-admin-client package fetch prep build
```
or, if you want to build for a specific target (`host-fc37` is a `dom0`
using Fedora 37, `vm-fc40` would be a qube using Fedora 40 etc.), use:
```shell
$ ./qb -c core-admin-client -d host-fc37 package fetch prep build
```
If you want to fetch the entire Qubes OS source use the following:
```shell
$ ./qb package fetch
```
**caution**: some repositories might have additional requirements. You can
disable repositories that are not needed in the `example-configs/*.yml`
file you are using by commenting them out. In particular, `python-fido2`,
`lvm` and `windows`-related repositories have special requirements.

2
developer/debugging/automated-tests.md
View file

@ -132,7 +132,7 @@ Whereas integration tests are mostly stored in the [qubes-core-admin](https://gi
To for example run the `qubes-core-admin` unit tests, you currently have to clone at least [qubes-core-admin](https://github.com/QubesOS/qubes-core-admin) and
its dependency [qubes-core-qrexec](https://github.com/QubesOS/qubes-core-qrexec) repository in the branches that you want to test.
The below example however will assume that you set up a build environment as described in the [Qubes Builder documentation](/doc/qubes-builder/).
The below example however will assume that you set up a build environment as described in the [Qubes Builder documentation](/doc/qubes-builder-v2/).
Assuming you cloned the `qubes-builder` repository to your home directory inside a fedora VM, you can use the following commands to run the unit tests:

5
developer/debugging/test-bench.md
View file

@ -205,9 +205,10 @@ pushd ${HOME}/builder >/dev/null
# the following are needed only if you have sources outside builder
#rm -rf qubes-src/core-admin
#make COMPONENTS=core-admin get-sources
#qb -c core-admin package fetch
make core-admin
qb -c core-admin -d host-fc41 prep build
# update your dom0 fedora distribution as appropriate
qtb-install qubes-src/core-admin/rpm/x86_64/qubes-core-dom0-*.rpm
qtb-runtests
```

4
developer/general/gsoc.md
View file

@ -31,7 +31,7 @@ You should start learning the components that you plan on working on before the
Coming up with an interesting idea that you can realistically achieve in the time available to you (one summer) is probably the most difficult part. We strongly recommend getting involved in advance of the beginning of GSoC, and we will look favorably on applications from prospective contributors who have already started to act like free and open source developers.
Before the summer starts, there are some preparatory tasks which are highly encouraged. First, if you aren't already, definitely start using Qubes as your primary OS as soon as possible! Also, it is encouraged that you become familiar and comfortable with the Qubes development workflow sooner than later. A good way to do this (and also a great way to stand out as an awesome applicant and make us want to accept you!) might be to pick up some issues from [qubes-issues](https://github.com/QubesOS/qubes-issues/issues) (our issue-tracking repo) and submit some patches addressing them. Some suitable issues might be those with tags ["help wanted" and "P: minor"](https://github.com/QubesOS/qubes-issues/issues?q=is%3Aissue%20is%3Aopen%20label%3A%22P%3A%20minor%22%20label%3A%22help%20wanted%22) (although more significant things are also welcome, of course). Doing this will get you some practice with [qubes-builder](/doc/qubes-builder/), our code-signing policies, and some familiarity with our code base in general so you are ready to hit the ground running come summer.
Before the summer starts, there are some preparatory tasks which are highly encouraged. First, if you aren't already, definitely start using Qubes as your primary OS as soon as possible! Also, it is encouraged that you become familiar and comfortable with the Qubes development workflow sooner than later. A good way to do this (and also a great way to stand out as an awesome applicant and make us want to accept you!) might be to pick up some issues from [qubes-issues](https://github.com/QubesOS/qubes-issues/issues) (our issue-tracking repo) and submit some patches addressing them. Some suitable issues might be those with tags ["help wanted" and "P: minor"](https://github.com/QubesOS/qubes-issues/issues?q=is%3Aissue%20is%3Aopen%20label%3A%22P%3A%20minor%22%20label%3A%22help%20wanted%22) (although more significant things are also welcome, of course). Doing this will get you some practice with [qubes-builder](/doc/qubes-builder-v2/), our code-signing policies, and some familiarity with our code base in general so you are ready to hit the ground running come summer.
### Contributor proposal guidelines
@ -363,7 +363,7 @@ for more information and qubes-specific background.
**Difficulty**: medium
**Knowledge prerequisite**: qubes-builder [[1]](/doc/qubes-builder/) [[2]](/doc/qubes-builder-details/) [[3]](https://github.com/QubesOS/qubes-builder/tree/master/doc), and efficient at introspecting complex systems: comfortable with tracing and debugging tools, ability to quickly identify and locate issues within a large codebase (upstream build tools), etc.
**Knowledge prerequisite**: qubes-builder [[1]](/doc/qubes-builder-v2/) [[2]](https://github.com/QubesOS/qubes-builderv2), and efficient at introspecting complex systems: comfortable with tracing and debugging tools, ability to quickly identify and locate issues within a large codebase (upstream build tools), etc.
**Size of the project**: 350 hours