Git-Mirrors/qubes-doc
1
0
Fork 0
mirror of https://github.com/QubesOS/qubes-doc.git synced 2025-04-16 14:13:16 -04:00
Code Issues Packages Projects Releases Wiki Activity

Add introduction for Builder v2

Browse Source 
Merge branch 'pr-1409'
This commit is contained in:
unman 2025-04-13 12:47:00 +00:00
commit 7b8f4e0740
No known key found for this signature in database
GPG Key ID: BB52274595B71262
8 changed files with 192 additions and 43 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

53
developer/building/development-workflow.md
View File

@ -12,14 +12,14 @@ title: Development workflow
A workflow for developing Qubes OS+
First things first, setup [QubesBuilder](/doc/qubes-builder/). This guide
assumes you're using qubes-builder to build Qubes.
First things first, setup [QubesBuilder](/doc/qubes-builder-v2/). This guide
assumes you're using qubes-builder v2 to build Qubes.
## Repositories and committing Code
Qubes is split into a bunch of git repos. These are all contained in the
`qubes-src` directory under qubes-builder. Subdirectories there are separate
components, stored in separate git repositories.
`artifacts/sources` directory under qubes-builder. Subdirectories there are
separate components, stored in separate git repositories.
The best way to write and contribute code is to create a git repo somewhere
(e.g., github) for the repo you are interested in editing (e.g.,
@ -29,7 +29,7 @@ of Qubes, cd to the repo directory and add your repository as a remote in git
**Example:**
~~~
$ cd qubes-builder/qubes-src/qubes-manager
$ cd qubes-builder/artifacts/sources/qubes-manager
$ git remote add abel git@github.com:abeluck/qubes-manager.git
~~~
@ -47,7 +47,7 @@ include your public PGP key you use to sign your tags.
#### Prepare fresh version of kernel sources, with Qubes-specific patches applied
In qubes-builder/qubes-src/linux-kernel:
In qubes-builder/artifacts/sources/linux-kernel:
~~~
make prep
@ -66,7 +66,7 @@ drwxr-xr-x 6 user user 4096 Nov 21 20:48 kernel-3.4.18/linux-obj
#### Go to the kernel tree and update the version
In qubes-builder/qubes-src/linux-kernel:
In qubes-builder/artifacts/sources/linux-kernel:
~~~
cd kernel-3.4.18/linux-3.4.18
@ -117,8 +117,6 @@ vi series.conf
#### Building RPMs
TODO: Is this step generic for all subsystems?
Now it is a good moment to make sure you have changed kernel release name in
rel file. For example, if you change it to '1debug201211116c' the
resulting RPMs will be named
@ -131,34 +129,23 @@ your changes locally.
To actually build RPMs, in qubes-builder:
~~~
make linux-kernel
./qb -c linux-kernel package fetch prep build
~~~
RPMs will appear in qubes-src/linux-kernel/pkgs/fc20/x86\_64:
RPMs will appear in
`artifacts/repository/destination_name/package_name`
(for example `artifacts/repository/host-fc37/linux-kernel-6.6.31-1.1/`
~~~
-rw-rw-r-- 1 user user 42996126 Nov 17 04:08 kernel-3.4.18-1debug20121116c.pvops.qubes.x86_64.rpm
-rw-rw-r-- 1 user user 43001450 Nov 17 05:36 kernel-3.4.18-1debug20121117a.pvops.qubes.x86_64.rpm
-rw-rw-r-- 1 user user 8940138 Nov 17 04:08 kernel-devel-3.4.18-1debug20121116c.pvops.qubes.x86_64.rpm
-rw-rw-r-- 1 user user 8937818 Nov 17 05:36 kernel-devel-3.4.18-1debug20121117a.pvops.qubes.x86_64.rpm
-rw-rw-r-- 1 user user 54490741 Nov 17 04:08 kernel-qubes-vm-3.4.18-1debug20121116c.pvops.qubes.x86_64.rpm
-rw-rw-r-- 1 user user 54502117 Nov 17 05:37 kernel-qubes-vm-3.4.18-1debug20121117a.pvops.qubes.x86_64.rpm
~~~
### Useful [QubesBuilder](/doc/qubes-builder-v2/) commands
### Useful [QubesBuilder](/doc/qubes-builder/) commands
1. `make check` - will check if all the code was committed into repository and
if all repository are tagged with signed tag.
2. `make show-vtags` - show version of each component (based on git tags) -
mostly useful just before building ISO. **Note:** this will not show version
for components containing changes since last version tag.
3. `make push` - push change from **all** repositories to git server. You must
set proper remotes (see above) for all repositories first.
4. `make prepare-merge` - fetch changes from remote repositories (can be
specified on commandline via GIT\_SUBDIR or GIT\_REMOTE vars), (optionally)
verify tags and show the changes. This do not merge the changes - there are
left for review as FETCH\_HEAD ref. You can merge them using `git merge
FETCH_HEAD` (in each repo directory). Or `make do-merge` to merge all of them.
1. `./qb package diff` - show uncommitted changes
2. ` ./qb repository check-release-status-for-component` and
`./qb repository check-release-status-for-template`- show version of each
component/template (based on git tags)
3. `./qb package sign` - sign built packages
4. `./qb package publish` and `./qb package upload` - publish signed packages
and upload published
repository
## Copying Code to dom0

160
developer/building/qubes-builder-v2.md Normal file
View File

@ -0,0 +1,160 @@
---
lang: en
layout: doc
permalink: /doc/qubes-builder-v2/
redirect_from:
- /en/doc/qubes-builder-v2/
- /doc/QubesBuilder2/
- /wiki/QubesBuilder2/
ref: 311
title: Qubes builder v2
---
This is a brief introduction of using Qubes Builder v2 to work with Qubes OS
sources. It will walk you through installing and configuring Builder v2 and
using it to fetch and build Qubes OS packages.
For details and customization, use [Qubes OS v2 builder documentation](https://github.com/QubesOS/qubes-builderv2/).
# Overview
In the second generation of Qubes OS builder, container or disposable qube
isolation is used to perform every stage of the build and release process.
From fetching sources to building, everything is executed inside an isolated
*cage* (either a disposable or a container) using an *executor*. For every
command that needs to perform an action on sources, like cloning and
verifying Git repos, rendering a SPEC file, generating SRPM or Debian
source packages, a new cage is used. Only the signing, publishing, and
uploading stages are executed locally outside a cage.
# Setup
This is a simple setup for using a docker executor (a good default choice,
if you don't know which executor to use, use docker).
1. First, decide what qube are you going to use for working with Qubes
Builder v2. It can be an AppVM or a Standalone qube, with some steps
different between the two.
2. Then, clone the qubes-builder v2 repository into a location of your
choice:
```shell
git clone https://github.com/QubesOS/qubes-builderv2
cd qubes-builderv2/
```
3. Installing dependencies
Dependencies need to be installed for the qube you want to be developing
using Qubes Builder v2 - either in its template, or in the qube itself,
if it's a standalone.
Dependencies are specified in `dependencies-*.
txt` files in the main builder directory, and you can install them easily
in the following ways:
1. for Fedora, use:
```shell
$ sudo dnf install $(cat dependencies-fedora.txt)
$ test -f /usr/share/qubes/marker-vm && sudo dnf install qubes-gpg-split
```
2. for Debian (note: some Debian packages require Debian version 13 or
later), use:
```shell
$ sudo apt install $(cat dependencies-debian.txt)
$ test -f /usr/share/qubes/marker-vm && sudo apt install qubes-gpg-split
```
4. If you haven't used docker in the current qube, you need also to set up
some permissions. In particular, the user has to be added to the `docker`
group:
```shell
$ sudo usermod -aG docker user
```
Next, **restart the qube**.
5. Finally, you need to generate a docker image:
```shell
$ tools/generate-container-image.sh docker
```
In an AppVM, as `/var/lib/docker` is not persistent by default, you also
need to use bind-dirs to avoid repeating this step after reboot, adding
the following to the `/rw/config/qubes-bind-dirs.d/docker.conf` file in
this qube:
```
binds+=( '/var/lib/docker' )
```
# Configuration
To use Qubes OS Builder v2, you need to have a `builder.yml` configuration file.
You can use one of the sample files from the `example-configs/` directory; for a
more readable `builder.yml`, you can also include one of the files from that
directory in your `builder.yml`. An example `builder.yml` is:
```yaml
# include configuration relevant for the current release
include:
- example-configs/qubes-os-r4.2.yml
# which repository to use to fetch sources
use-qubes-repo:
version: 4.2
testing: true
# each package built will have local build number appended to package release
# number. It makes it easier to update in testing environment
increment-devel-versions: true
# reduce output
debug: false
# this can be set to true if you do not want sources to be automatically
# fetched from git
skip-git-fetch: false
# executor configuration
executor:
type: docker
options:
image: "qubes-builder-fedora:latest"
```
# Using Builder v2
To fetch sources - in this example, for the `core-admin-client` package, you
can use the following command:
```shell
$ ./qb -c core-admin-client package fetch
```
This will fetch the sources for the listed package and place them in
`artifacts/sources` directory.
To build a package (from sources in the `artifacts/sources` directory), use:
```shell
$ ./qb -c core-admin-client package fetch prep build
```
or, if you want to build for a specific target (`host-fc37` is a `dom0`
using Fedora 37, `vm-fc40` would be a qube using Fedora 40 etc.), use:
```shell
$ ./qb -c core-admin-client -d host-fc37 package fetch prep build
```
If you want to fetch entire Qubes OS sources (caution: some repositories might
have additional requirements; you can disable repositories that are not
needed in the `example-configs/*.yml` file you are using by commenting them
out. In particular, `python-fido2`, `lvm` and `windows`-related repositories
have
special requirements), use the following:
```shell
$ ./qb package fetch
```

2
developer/debugging/automated-tests.md
View File

@ -132,7 +132,7 @@ Whereas integration tests are mostly stored in the [qubes-core-admin](https://gi
To for example run the `qubes-core-admin` unit tests, you currently have to clone at least [qubes-core-admin](https://github.com/QubesOS/qubes-core-admin) and
its dependency [qubes-core-qrexec](https://github.com/QubesOS/qubes-core-qrexec) repository in the branches that you want to test.
The below example however will assume that you set up a build environment as described in the [Qubes Builder documentation](/doc/qubes-builder/).
The below example however will assume that you set up a build environment as described in the [Qubes Builder documentation](/doc/qubes-builder-v2/).
Assuming you cloned the `qubes-builder` repository to your home directory inside a fedora VM, you can use the following commands to run the unit tests:

5
developer/debugging/test-bench.md
View File

@ -205,9 +205,10 @@ pushd ${HOME}/builder >/dev/null
# the following are needed only if you have sources outside builder
#rm -rf qubes-src/core-admin
#make COMPONENTS=core-admin get-sources
#qb -c core-admin package fetch
make core-admin
qb -c core-admin -d host-fc41 prep build
# update your dom0 fedora distribution as appropriate
qtb-install qubes-src/core-admin/rpm/x86_64/qubes-core-dom0-*.rpm
qtb-runtests
```

4
developer/general/gsoc.md
View File

@ -31,7 +31,7 @@ You should start learning the components that you plan on working on before the
Coming up with an interesting idea that you can realistically achieve in the time available to you (one summer) is probably the most difficult part. We strongly recommend getting involved in advance of the beginning of GSoC, and we will look favorably on applications from prospective contributors who have already started to act like free and open source developers.
Before the summer starts, there are some preparatory tasks which are highly encouraged. First, if you aren't already, definitely start using Qubes as your primary OS as soon as possible! Also, it is encouraged that you become familiar and comfortable with the Qubes development workflow sooner than later. A good way to do this (and also a great way to stand out as an awesome applicant and make us want to accept you!) might be to pick up some issues from [qubes-issues](https://github.com/QubesOS/qubes-issues/issues) (our issue-tracking repo) and submit some patches addressing them. Some suitable issues might be those with tags ["help wanted" and "P: minor"](https://github.com/QubesOS/qubes-issues/issues?q=is%3Aissue%20is%3Aopen%20label%3A%22P%3A%20minor%22%20label%3A%22help%20wanted%22) (although more significant things are also welcome, of course). Doing this will get you some practice with [qubes-builder](/doc/qubes-builder/), our code-signing policies, and some familiarity with our code base in general so you are ready to hit the ground running come summer.
Before the summer starts, there are some preparatory tasks which are highly encouraged. First, if you aren't already, definitely start using Qubes as your primary OS as soon as possible! Also, it is encouraged that you become familiar and comfortable with the Qubes development workflow sooner than later. A good way to do this (and also a great way to stand out as an awesome applicant and make us want to accept you!) might be to pick up some issues from [qubes-issues](https://github.com/QubesOS/qubes-issues/issues) (our issue-tracking repo) and submit some patches addressing them. Some suitable issues might be those with tags ["help wanted" and "P: minor"](https://github.com/QubesOS/qubes-issues/issues?q=is%3Aissue%20is%3Aopen%20label%3A%22P%3A%20minor%22%20label%3A%22help%20wanted%22) (although more significant things are also welcome, of course). Doing this will get you some practice with [qubes-builder](/doc/qubes-builder-v2/), our code-signing policies, and some familiarity with our code base in general so you are ready to hit the ground running come summer.
### Contributor proposal guidelines
@ -422,7 +422,7 @@ for more information and qubes-specific background.
**Difficulty**: medium
**Knowledge prerequisite**: qubes-builder [[1]](/doc/qubes-builder/) [[2]](/doc/qubes-builder-details/) [[3]](https://github.com/QubesOS/qubes-builder/tree/master/doc), and efficient at introspecting complex systems: comfortable with tracing and debugging tools, ability to quickly identify and locate issues within a large codebase (upstream build tools), etc.
**Knowledge prerequisite**: qubes-builder [[1]](/doc/qubes-builder-v2/) [[2]](https://github.com/QubesOS/qubes-builderv2), and efficient at introspecting complex systems: comfortable with tracing and debugging tools, ability to quickly identify and locate issues within a large codebase (upstream build tools), etc.
**Size of the project**: 350 hours

2
introduction/faq.md
View File

@ -678,7 +678,7 @@ Any rpm-based, 64-bit environment, the preferred OS being Fedora.
### How do I build Qubes from sources?
See [these instructions](/doc/qubes-builder/).
See [these instructions](/doc/qubes-builder-v2/).
### How do I submit a patch?

2
user/downloading-installing-upgrading/testing.md
View File

@ -17,7 +17,7 @@ How to test upcoming Qubes OS releases:
- Test the latest release candidate (RC) on the [downloads](/downloads/) page, if one is currently available. (Or try an older RC from our [FTP server](https://ftp.qubes-os.org/iso/).)
- Try the [signed weekly builds](https://qubes.notset.fr/iso/). ([Learn more](https://forum.qubes-os.org/t/16929) and [track their status](https://github.com/fepitre/updates-status-iso/issues).)
- Use [qubes-builder](/doc/qubes-builder/) to build the latest release yourself.
- Use [qubes-builder](/doc/qubes-builder-v2/) to build the latest release yourself.
- (No support) Experiment with developer alpha ISOs found from time to time at [Qubes OpenQA](https://openqa.qubes-os.org/).
Please make sure to [report any bugs you encounter](/doc/issue-tracking/).

7
user/templates/templates.md
View File

@ -387,8 +387,9 @@ this context: the same as their template filesystem, of course.
* Some templates are available in ready-to-use binary form, but some of them
are available only as source code, which can be built using the [Qubes
Builder](/doc/qubes-builder/). In particular, some template "flavors" are
available in source code form only. For the technical details of the template
Builder](https://github.com/QubesOS/qubes-builderv2/). In particular, some
template "flavors" are available in source code form only. For the
technical details of the template
system, please see [Template Implementation](/doc/template-implementation/).
Take a look at the [Qubes Builder](/doc/qubes-builder/) documentation for
Take a look at the [Qubes Builder](/doc/qubes-builder-v2/) documentation for
instructions on how to compile them.