Correct code-block lexers

Changing `bash` lexer to `console` because it is appropriate most of
the time. Then after a manual review, some lexer have been changed.

I used `text` each time I was unsure, and for prompt outputs.

The page `/developer/building/qubes-iso-building.rst` still need to be
reviewed (look for lines starting with `$ #`).

I'm not sure about the Windows pages, should we use
[doscon](https://pygments.org/docs/lexers/#pygments.lexers.shell.MSDOSSessionLexer)
or `powershell`?

Is there an appropriate lexer for `guid.conf` content?

**Statistics - Before**
    870 bash
      9 python
      9 c
      2 yaml

**Statistics - After**
    684 console
    111 text
     44 bash
     16 yaml
      9 systemd
      9 c
      8 python
      4 ini
      4 doscon
      2 markdown
      2 desktop
      1 xorg.conf
      1 xml+jinja
      1 xml
      1 kconfig
      1 html

This suggests that the default lexer should be `console`.
This commit is contained in:
parulin 2025-07-30 09:43:09 -04:00
parent b53776e1eb
commit ba399ac488
No known key found for this signature in database
GPG key ID: BC3830B42F4BF1F5
98 changed files with 1022 additions and 1029 deletions

View file

@ -38,7 +38,7 @@ Prepare fresh version of kernel sources, with Qubes-specific patches applied
In ``qubes-builder/artifacts/sources/linux-kernel``:
.. code:: bash
.. code:: console
make prep
@ -46,14 +46,9 @@ In ``qubes-builder/artifacts/sources/linux-kernel``:
The resulting tree will be in kernel-<VERSION>/linux-<VERSION>:
.. code:: bash
.. code:: console
ls -ltrd kernel*/linux*
.. code:: bash
drwxr-xr-x 23 user user 4096 Nov 5 09:50 kernel-3.4.18/linux-3.4.18
drwxr-xr-x 6 user user 4096 Nov 21 20:48 kernel-3.4.18/linux-obj
@ -65,7 +60,7 @@ Go to the kernel tree and update the version
In ``qubes-builder/artifacts/sources/linux-kernel``:
.. code:: bash
.. code:: console
cd kernel-3.4.18/linux-3.4.18
@ -77,7 +72,7 @@ Changing the config
In ``kernel-3.4.18/linux-3.4.18``:
.. code:: bash
.. code:: console
cp ../../config .config
make oldconfig
@ -86,7 +81,7 @@ In ``kernel-3.4.18/linux-3.4.18``:
Now change the configuration. For example, in ``kernel-3.4.18/linux-3.4.18``:
.. code:: bash
.. code:: console
make menuconfig
@ -94,7 +89,7 @@ Now change the configuration. For example, in ``kernel-3.4.18/linux-3.4.18``:
Copy the modified config back into the kernel tree:
.. code:: bash
.. code:: console
cp .config ../../../config
@ -106,20 +101,20 @@ Patching the code
TODO: describe the workflow for patching the code, below are some random notes, not working well
.. code:: bash
.. code:: console
ln -s ../../patches.xen
export QUILT_PATCHES=patches.xen
export QUILT_REFRESH_ARGS="-p ab --no-timestamps --no-index"
export QUILT_SERIES=../../series-pvops.conf
quilt new patches.xen/pvops-3.4-0101-usb-xen-pvusb-driver-bugfix.patch
quilt add drivers/usb/host/Kconfig drivers/usb/host/Makefile \
drivers/usb/host/xen-usbback/* drivers/usb/host/xen-usbfront.c \
include/xen/interface/io/usbif.h
*edit something*
quilt refresh
cd ../..
vi series.conf
@ -136,7 +131,7 @@ You might want to take a moment here to review (git diff, git status), commit yo
To actually build RPMs, in qubes-builder:
.. code:: bash
.. code:: console
./qb -c linux-kernel package fetch prep build
@ -177,14 +172,14 @@ TODO: edit this script to be more generic
.. code:: bash
#!/bin/sh
set -x
set -e
QUBES_PY_DIR=/usr/lib64/python2.6/site-packages/qubes
QUBES_PY=$QUBES_PY_DIR/qubes.py
QUBESUTILS_PY=$QUBES_PY_DIR/qubesutils.py
qvm-run -p qubes-devel 'cd qubes-builder/qubes-src/core/dom0; tar c qmemman/qmemman*.py qvm-core/*.py qvm-tools/* misc/vm-template-hvm.conf misc/qubes-start.desktop ../misc/block-snapshot aux-tools ../qrexec' |tar xv
cp $QUBES_PY qubes.py.bak$$
cp $QUBESUTILS_PY qubesutils.py.bak$$
@ -210,7 +205,7 @@ TODO: make it more generic
.. code:: bash
#!/bin/sh
BAK=qvm-tools.bak$$
mkdir -p $BAK
cp -a /usr/bin/qvm-* /usr/bin/qubes-* $BAK/
@ -231,7 +226,7 @@ Copy from dom0 to an appvm
domain=$1
file=$2
fname=`basename $file`
qvm-run $domain 'mkdir /home/user/incoming/dom0 -p'
cat $file| qvm-run --pass-io $domain "cat > /home/user/incoming/dom0/$fname"
@ -250,9 +245,9 @@ Service file (save in ``/usr/local/etc/qubes-rpc/local.Git`` in target VM):
.. code:: bash
#!/bin/sh
exec 2>/tmp/log2
read service rel repo
echo "Params: $service $rel $repo" >&2
# Adjust regexps if needed
@ -279,9 +274,9 @@ Client script (save in ``~/bin/git-qrexec`` in source VM):
.. code:: bash
#!/bin/sh
VMNAME=$1
(echo $GIT_EXT_SERVICE $2 $3; exec cat) | qrexec-client-vm $VMNAME local.Git
@ -302,14 +297,14 @@ You can create ``~/bin/add-remote`` script to ease adding remotes:
.. code:: bash
#!/bin/sh
[ -n "$1" ] || exit 1
if [ "$1" = "tb" ]; then
git remote add $1 "ext::git-qrexec testbuilder 3 `basename $PWD`"
exit $?
fi
git remote add $1 git@GitHub.com:$1/qubes-`basename $PWD`
@ -333,9 +328,9 @@ In source VM, grab `linux-yum <https://GitHub.com/QubesOS/qubes-linux-yum>`__ re
.. code:: bash
#!/bin/sh
VMNAME=repo-vm
set -e
qvm-copy-to-vm $VMNAME $1
# remove only files, leave directory structure
@ -347,7 +342,7 @@ In source VM, grab `linux-yum <https://GitHub.com/QubesOS/qubes-linux-yum>`__ re
In target VM, setup actual yum repository (also based on `linux-yum <https://GitHub.com/QubesOS/qubes-linux-yum>`__, this time without modifications). You will also need to setup some gpg key for signing packages (it is possible to force yum to install unsigned packages, but it isnt possible for ``qubes-dom0-update`` tool). Fill ``~/.rpmmacros`` with key description:
.. code:: bash
.. code:: text
%_gpg_name Test packages signing key
@ -358,27 +353,27 @@ Then setup ``local.UpdateYum`` qrexec service (``/usr/local/etc/qubes-rpc/local.
.. code:: bash
#!/bin/sh
if [ -z "$QREXEC_REMOTE_DOMAIN" ]; then
exit 1
fi
real_repository=/home/user/linux-yum
incoming=/home/user/QubesIncoming/$QREXEC_REMOTE_DOMAIN
find $incoming -name '*.rpm' |xargs rpm -K |grep -iv pgp |cut -f1 -d: |xargs -r setsid -w rpm --addsign 2>&1
rsync -lr --remove-source-files $incoming/ $real_repository
cd $real_repository
export SKIP_REPO_CHECK=1
if [ -d $incoming/r3.1 ]; then
./update_repo-unstable.sh r3.1
fi
if [ -d $incoming/r3.0 ]; then
./update_repo-unstable.sh r3.0
fi
if [ -d $incoming/r2 ]; then
./update_repo-unstable.sh r2
fi
@ -401,7 +396,7 @@ Usage: setup ``builder.conf`` in source VM to use your dummy-uploader repository
Then use ``make update-repo-unstable`` to upload the packages. You can also specify selected components on command line, then build them and upload to the repository:
.. code:: bash
.. code:: console
make COMPONENTS="core-agent-linux gui-agent-linux linux-utils" qubes update-repo-unstable
@ -409,7 +404,7 @@ Then use ``make update-repo-unstable`` to upload the packages. You can also spec
On the test machine, add yum repository (``/etc/yum.repos.d``) pointing at just configured HTTP server. For example:
.. code:: bash
.. code:: ini
[local-test]
name=Test
@ -440,12 +435,12 @@ Steps are mostly the same as in the case of yum repo. The only details that diff
.. code:: bash
#!/bin/sh
set -e
current_release=$1
VMNAME=repo-vm
qvm-copy-to-vm $VMNAME $1
find $current_release -type f -name '*.deb' -delete
rm -f $current_release/vm/db/*
@ -458,13 +453,13 @@ Steps are mostly the same as in the case of yum repo. The only details that diff
.. code:: bash
#!/bin/sh
if [ -z "$QREXEC_REMOTE_DOMAIN" ]; then
exit 1
fi
incoming=/home/user/QubesIncoming/$QREXEC_REMOTE_DOMAIN
rsync -lr --remove-source-files $incoming/ /home/user/linux-deb/
cd /home/user/linux-deb
export SKIP_REPO_CHECK=1
@ -473,13 +468,13 @@ Steps are mostly the same as in the case of yum repo. The only details that diff
./update-local-repo.sh r3.1/vm $dist
done
fi
if [ -d $incoming/r3.0 ]; then
for dist in `ls r3.0/vm/dists`; do
./update-local-repo.sh r3.0/vm $dist
done
fi
if [ -d $incoming/r2 ]; then
for dist in `ls r2/vm/dists`; do
./update-local-repo.sh r2/vm $dist
@ -495,17 +490,17 @@ Steps are mostly the same as in the case of yum repo. The only details that diff
.. code:: bash
#!/bin/sh
set -e
# Set this to your local repository signing key
SIGN_KEY=01ABCDEF
[ -z "$1" ] && { echo "Usage: $0 <repo> <dist>"; exit 1; }
REPO_DIR=$1
DIST=$2
if [ "$DIST" = "wheezy-unstable" ]; then
DIST_TAG=deb7
elif [ "$DIST" = "jessie-unstable" ]; then
@ -513,7 +508,7 @@ Steps are mostly the same as in the case of yum repo. The only details that diff
elif [ "$DIST" = "stretch-unstable" ]; then
DIST_TAG=deb9
fi
pushd $REPO_DIR
mkdir -p dists/$DIST/main/binary-amd64
dpkg-scanpackages --multiversion --arch "*$DIST_TAG*" . > dists/$DIST/main/binary-amd64/Packages
@ -535,7 +530,7 @@ Steps are mostly the same as in the case of yum repo. The only details that diff
echo $1
}
calc_sha1 main/binary-amd64/Packages >> dists/$DIST/Release
rm -f $DIST/Release.gpg
rm -f $DIST/InRelease
gpg -abs -u "$SIGN_KEY" \
@ -543,7 +538,7 @@ Steps are mostly the same as in the case of yum repo. The only details that diff
gpg -a -s --clearsign -u "$SIGN_KEY" \
< dists/$DIST/Release > dists/$DIST/InRelease
popd
if [ `id -u` -eq 0 ]; then
chown -R --reference=$REPO_DIR $REPO_DIR
fi
@ -552,7 +547,7 @@ Steps are mostly the same as in the case of yum repo. The only details that diff
Usage: add this line to ``/etc/apt/sources.list`` on test machine (adjust host and path):
.. code:: bash
.. code:: text
deb http://local-test.lan/linux-deb/r3.1 jessie-unstable main

View file

@ -52,7 +52,7 @@ This is a simple setup using a docker executor. This is a good default choice; i
3. Clone the qubes-builder v2 repository into a location of your choice:
.. code:: bash
.. code:: console
git clone https://github.com/QubesOS/qubes-builderv2
cd qubes-builderv2/
@ -88,28 +88,28 @@ Configuration
To use Qubes OS Builder v2, you need to have a ``builder.yml`` configuration file. You can use one of the sample files from the ``example-configs/`` directory; for a more readable ``builder.yml``, you can also include one of the files from that directory in your ``builder.yml``. An example ``builder.yml`` is:
.. code:: bash
.. code:: yaml
# include configuration relevant for the current release
include:
- example-configs/qubes-os-r4.2.yml
# which repository to use to fetch sources
use-qubes-repo:
version: 4.2
testing: true
# each package built will have local build number appended to package release
# number. It makes it easier to update in testing environment
increment-devel-versions: true
# reduce output
debug: false
# this can be set to true if you do not want sources to be automatically
# fetched from git
skip-git-fetch: false
# executor configuration
executor:
type: docker

View file

@ -4,7 +4,7 @@ Qubes builder
.. warning::
**Note:** These instructions concern the older Qubes builder (v1). It supports only building Qubes 4.2 or earlier.The build process has been completely rewritten in `qubes-builder v2 <https://github.com/QubesOS/qubes-builderv2/>`__ . This can be used for building Qubes R4.2 and later, and all related components.
**Note: See** :doc:`ISO building instructions </developer/building/qubes-iso-building>` **for a streamlined overview on how to use the build system.**
@ -53,7 +53,7 @@ In order to use it, you should use an rpm-based distro, like Fedora :), and shou
Usually you can install those packages by just issuing:
.. code:: bash
.. code:: console
sudo dnf install gnupg git createrepo rpm-build make wget rpmdevtools python3-sh dialog rpm-sign dpkg-dev debootstrap python3-pyyaml devscripts perl-Digest-MD5 perl-Digest-SHA
@ -62,7 +62,7 @@ The build system creates build environments in chroots and so no other packages
The build system is configured via builder.conf file. You can use the setup.sh script to create and modify this file. Alternatively, you can copy the provided default builder.conf, and modify it as needed, e.g.:
.. code:: bash
.. code:: console
cp example-configs/qubes-os-master.conf builder.conf
# edit the builder.conf file and set the following variables:
@ -73,7 +73,7 @@ One additional useful requirement is that sudo root must work without any
Additionally, if building with signing enabled (NO_SIGN is not set), you must adjust ``\~/.rpmmacros`` file so that it points to the GPG key used for package signing, e.g.:
.. code:: bash
.. code:: text
%_signature gpg
%_gpg_path /home/user/.gnupg
@ -84,38 +84,38 @@ It is also recommended that you use an empty passphrase for the private key used
So, to build Qubes you would do:
.. code:: bash
.. code:: console
# Import the Qubes master key
gpg --recv-keys 0xDDFA1A3E36879494
# Verify its fingerprint, set as 'trusted'.
# This is described here:
# https://www.qubes-os.org/doc/VerifyingSignatures
wget https://keys.qubes-os.org/keys/qubes-developers-keys.asc
gpg --import qubes-developers-keys.asc
git clone https://github.com/QubesOS/qubes-builder.git qubes-builder
cd qubes-builder
# Verify its integrity:
git tag -v `git describe`
cp example-configs/qubes-os-master.conf builder.conf
# edit the builder.conf file and set the following variables:
# NO_SIGN="1"
# Download all components:
make get-sources
# And now to build all Qubes RPMs (this will take a few hours):
make qubes
# ... and then to build the ISO
make iso
@ -123,7 +123,7 @@ And this should produce a shiny new ISO.
You can also build selected component separately. Eg. to compile only gui virtualization agent/daemon:
.. code:: bash
.. code:: console
make gui-daemon
@ -152,7 +152,7 @@ You can also modify sources somehow if you wish. Here are some basic steps:
3. Download unmodified sources
.. code:: bash
.. code:: console
make get-sources
@ -163,14 +163,14 @@ You can also modify sources somehow if you wish. Here are some basic steps:
6. ``get-sources`` is already done, so continue with the next one. You can skip ``sign-all`` if youve disabled signing
.. code:: bash
.. code:: console
make vmm-xen core-admin linux-kernel gui-daemon template desktop-linux-kde installer-qubes-os manager linux-dom0-updates
7. build iso installation image
.. code:: bash
.. code:: console
make iso

View file

@ -4,7 +4,7 @@ Qubes ISO building
.. warning::
**Note:** These instructions concern the older Qubes builder (v1). It supports only building Qubes 4.2 or earlier.The build process has been completely rewritten in `qubes-builder v2 <https://github.com/QubesOS/qubes-builderv2/>`__ . This can be used for building Qubes R4.2 and later versions, and all related components.
Build Environment
@ -15,14 +15,14 @@ Fedora 36 (and 37) has been successfully used to build Qubes R4.1 with the below
**Notes:** On modern Fedora system (like Fedora 37) SeLinux is enforced by default and is blocking the build system. You would get error like “cant create transaction lock on /…/rpm/.rpm.lock (Permission denied)”. You can set SeLinux to permissive mode with
.. code:: bash
.. code:: console
sudo setenforce 0
In ``dom0``, install the Fedora 36 (or 37) template if you dont already have it.
.. code:: bash
.. code:: console
sudo qubes-dom0-update qubes-template-fedora-36
@ -40,17 +40,17 @@ Once youve built the development AppVM, open a Terminal window to it and inst
Get the necessary keys to verify the sources (run these and other commands below as a regular user, not root):
.. code:: bash
.. code:: console
wget https://keys.qubes-os.org/keys/qubes-master-signing-key.asc
gpg --import qubes-master-signing-key.asc
gpg --edit-key 36879494
fpr
# Verify fingerprint! See Note below!
# Once verified, set trust to *ultimate*
# (Typical sequence is trust, 5, Y, q)
wget https://keys.qubes-os.org/keys/qubes-developers-keys.asc
gpg --import qubes-developers-keys.asc
$ wget https://keys.qubes-os.org/keys/qubes-master-signing-key.asc
$ gpg --import qubes-master-signing-key.asc
$ gpg --edit-key 36879494
$ fpr
$ # Verify fingerprint! See Note below!
$ # Once verified, set trust to *ultimate*
$ # (Typical sequence is trust, 5, Y, q)
$ wget https://keys.qubes-os.org/keys/qubes-developers-keys.asc
$ gpg --import qubes-developers-keys.asc
@ -58,7 +58,7 @@ Get the necessary keys to verify the sources (run these and other commands below
Now lets bootstrap the builder. Unfortunately, the builder cannot verify itself (the classic Chicken and Egg problem), so we need to verify the signature manually:
.. code:: bash
.. code:: console
git clone https://github.com/QubesOS/qubes-builder.git
cd qubes-builder
@ -76,37 +76,37 @@ Build using setup script
Lets configure the builder first (see :ref:`procedure <developer/building/qubes-iso-building:build using manual steps>` at bottom if you would prefer to manually configure):
.. code:: bash
.. code:: console
cd ~/qubes-builder
./setup
# Select Yes to add Qubes Master Signing Key
# Select Yes to add Qubes OS Signing Key
# Select 4.1 for version
# Stable
# Select Yes for fast Git cloning
# Select Current (if you want the option to use pre-built packages)
# Select No (we want a full build)
# Select fc36 and bullseye (for the currently shipping templates)
# Select builder-rpm, builder-debian, template-whonix, mgmt-salt
# Select Yes to add adrelanos's third party key
# Select Yes (to download)
$ cd ~/qubes-builder
$ ./setup
$ # Select Yes to add Qubes Master Signing Key
$ # Select Yes to add Qubes OS Signing Key
$ # Select 4.1 for version
$ # Stable
$ # Select Yes for fast Git cloning
$ # Select Current (if you want the option to use pre-built packages)
$ # Select No (we want a full build)
$ # Select fc36 and bullseye (for the currently shipping templates)
$ # Select builder-rpm, builder-debian, template-whonix, mgmt-salt
$ # Select Yes to add adrelanos's third party key
$ # Select Yes (to download)
Once it completes downloading, re-run ``setup`` to add the Whonix templates:
.. code:: bash
.. code:: console
./setup
# Choose the same options as above, except at templates select:
# fc36, bullseye, whonix-gateway-16, whonix-workstation-16
$ ./setup
$ # Choose the same options as above, except at templates select:
$ # fc36, bullseye, whonix-gateway-16, whonix-workstation-16
Continue the build process with:
.. code:: bash
.. code:: console
make install-deps
make get-sources
@ -119,7 +119,7 @@ You may also want to add ``COMPONENTS := $(filter-out gcc,$(COMPONENTS))`` to by
Finally, if you are making a test build, use:
.. code:: bash
.. code:: console
make qubes
make iso
@ -128,7 +128,7 @@ Finally, if you are making a test build, use:
Or for a fully signed build (this requires setting ``SIGN_KEY`` in ``builder.conf``):
.. code:: bash
.. code:: console
make qubes
make sign-all
@ -146,7 +146,7 @@ Instead of using ``./setup``, you can manually configure the build. The script t
If you will be building Whonix templates:
.. code:: bash
.. code:: console
cd ~
gpg --keyserver pgp.mit.edu --recv-keys 916B8D99C38EAF5E8ADC7A2A8D66066A2EEACCDA
@ -156,7 +156,7 @@ If you will be building Whonix templates:
**Note:** Its very important to check the fingerprint displayed against multiple sources such as the `Whonix web site <https://www.whonix.org/wiki/Whonix_Signing_Key>`__, etc. It should look something like this:
.. code:: bash
.. code:: text
pub rsa4096 2014-01-16 [SC] [expires: 2026-01-23]
Key fingerprint = 916B 8D99 C38E AF5E 8ADC 7A2A 8D66 066A 2EEA CCDA
@ -171,7 +171,7 @@ If you will be building Whonix templates:
Next, prepare the Git keyring directory and copy them in:
.. code:: bash
.. code:: console
export GNUPGHOME=~/qubes-builder/keyrings/git
mkdir --parents "$GNUPGHOME"
@ -183,7 +183,7 @@ Next, prepare the Git keyring directory and copy them in:
Copy one of the example configurations:
.. code:: bash
.. code:: console
cd ~/qubes-builder
cp example-configs/qubes-os-master.conf builder.conf
@ -194,7 +194,7 @@ Edit ``builder.conf``, referring to ``doc/Configuration.md`` for a description o
Continue the build process with:
.. code:: bash
.. code:: console
make install-deps
make get-sources
@ -206,7 +206,7 @@ When building the Whonix templates, you will often need to add/update the ``WHON
Finally, if you are making a test build, use:
.. code:: bash
.. code:: console
make qubes
make iso
@ -215,7 +215,7 @@ Finally, if you are making a test build, use:
Or for a fully signed build (this requires setting ``SIGN_KEY`` in ``builder.conf``):
.. code:: bash
.. code:: console
make qubes
make sign-all