mirror of
https://github.com/Qubes-Community/Contents.git
synced 2024-12-22 13:55:08 -05:00
gaming hvm doc update
This commit is contained in:
Neowutran
parent
10be0e29df
commit
602565906e
@ -42,7 +42,7 @@ requests](https://github.com/Qubes-Community/Contents/pulls).
|
|||||||
`customization`
|
`customization`
|
||||||
- [change DPI scaling in dom0 and VMs](/docs/customization/dpi-scaling.md)
|
- [change DPI scaling in dom0 and VMs](/docs/customization/dpi-scaling.md)
|
||||||
- [setup mirage firewall](/docs/customization/mirage-firewall.md)
|
- [setup mirage firewall](/docs/customization/mirage-firewall.md)
|
||||||
- [windows gaming HVM with GPU passthrough](/docs/customization/windows-gaming-hvm.md)
|
- [gaming HVM with GPU passthrough](/docs/customization/gaming-hvm.md)
|
||||||
- [Choose deafult terminal settings for a TemplateVM](/docs/customization/terminal-defaults.md)
|
- [Choose deafult terminal settings for a TemplateVM](/docs/customization/terminal-defaults.md)
|
||||||
- [Screenlockers](/docs/customization/screenlockers.md)
|
- [Screenlockers](/docs/customization/screenlockers.md)
|
||||||
|
|
||||||
|
|||||||
349
docs/customization/gaming-hvm.md
Normal file
349
docs/customization/gaming-hvm.md
Normal file
@ -0,0 +1,349 @@
|
|||||||
|
# Create a Gaming HVM
|
||||||
|
|
||||||
|
## Hardware
|
||||||
|
|
||||||
|
To have an 'HVM' for gaming, you must have
|
||||||
|
|
||||||
|
- A dedicated GPU. By dedicated, it means: it is a secondary GPU, not
|
||||||
|
the GPU used to display dom0. In 2023, 'Nvidia' and 'Amd' GPU work.
|
||||||
|
Not tested with Intel GPUs.
|
||||||
|
|
||||||
|
- A screen available for the gaming 'HVM'. (It can be a physical
|
||||||
|
monitor or just to have multiple cables connected to the screen and
|
||||||
|
switching between input source)
|
||||||
|
|
||||||
|
- Dedicated gaming mouse and keyboard.
|
||||||
|
|
||||||
|
- A lot of patience. GPU passthrough is not trivial, and you will need
|
||||||
|
to spend time debugging.
|
||||||
|
|
||||||
|
## IOMMU Group
|
||||||
|
|
||||||
|
You need to check what are the things/devices that are in the same IOMMU
|
||||||
|
group as the GPU you want to passthrough. You can't see your IOMMU Group
|
||||||
|
when you are using Xen (the information is hidden from dom0). So, start
|
||||||
|
a live linux distribution, enable iommu in the grub options (iommu=1
|
||||||
|
iommu_amd=on), and then displayed the folder structure of
|
||||||
|
/sys/kernel/iommu_group
|
||||||
|
|
||||||
|
``` bash
|
||||||
|
#!/bin/bash
|
||||||
|
shopt -s nullglob
|
||||||
|
for g in /sys/kernel/iommu_groups/*; do
|
||||||
|
echo "IOMMU Group ${g##*/}:"
|
||||||
|
for d in $g/devices/*; do
|
||||||
|
echo -e "\t$(lspci -nns ${d##*/})"
|
||||||
|
done
|
||||||
|
done
|
||||||
|
```
|
||||||
|
|
||||||
|
## GRUB modification
|
||||||
|
|
||||||
|
You must hide your secondary GPU from dom0. To do that, you have to
|
||||||
|
modify the GRUB. In a dom0 Terminal, type:
|
||||||
|
|
||||||
|
``` bash
|
||||||
|
qvm-pci
|
||||||
|
```
|
||||||
|
|
||||||
|
Then find the devices id for your secondary GPU. In my case, it is
|
||||||
|
`dom0:0a_00.0`{.text} and `dom0:0a_00.1`{.text}. Edit /etc/default/grub,
|
||||||
|
and add the PCI hiding.
|
||||||
|
|
||||||
|
``` text
|
||||||
|
GRUB_CMDLINE_LINUX="... rd.qubes.hide_pci=0a:00.0,0a:00.1 "
|
||||||
|
```
|
||||||
|
|
||||||
|
then regenerate the grub
|
||||||
|
|
||||||
|
``` bash
|
||||||
|
grub2-mkconfig -o /boot/grub2/grub.cfg
|
||||||
|
```
|
||||||
|
|
||||||
|
If you are using UEFI, the file to override with `grub2-mkconfig`{.text}
|
||||||
|
is `/boot/efi/EFI/qubes/grub.cfg`{.text}.
|
||||||
|
|
||||||
|
Note: if after this step when you reboot the computer you get stuck in
|
||||||
|
the QubesOS startup that means you are trying to use the GPU you just
|
||||||
|
hide. Check your BIOS options. Also check the cables, BIOS have some GPU
|
||||||
|
priority based on the type of cable. For example, DisplayPort can be
|
||||||
|
favoured over HDMI.
|
||||||
|
|
||||||
|
Once you have rebooted, in dom0, type `sudo lspci -vvn`{.bash}, you
|
||||||
|
should see "Kernel driver in use: pciback" for the GPU you just hide.
|
||||||
|
|
||||||
|
## Patching stubdom-linux-rootfs.gz
|
||||||
|
|
||||||
|
[github.com/QubesOS/qubes-issues/issues/4321](https://github.com/QubesOS/qubes-issues/issues/4321#issuecomment-423011787)
|
||||||
|
|
||||||
|
Copy-paste of the comment:
|
||||||
|
|
||||||
|
This is caused by the default TOLUD (Top of Low Usable DRAM) of 3.75G
|
||||||
|
provided by qemu not being large enough to accommodate the larger BARs
|
||||||
|
that a graphics card typically has. The code to pass a custom
|
||||||
|
max-ram-below-4g value to the qemu command line does exist in the
|
||||||
|
libxl_dm.c file of xen, but there is no functionality in libvirt to add
|
||||||
|
this parameter. It is possible to manually add this parameter to the
|
||||||
|
qemu commandline by doing the following in a dom0 terminal. (I modified
|
||||||
|
the code so it works with 4.1 and remove one of the original limitations
|
||||||
|
by restricting the modification to VM with a name starting with
|
||||||
|
"gpu\_\")
|
||||||
|
|
||||||
|
``` bash
|
||||||
|
mkdir stubroot
|
||||||
|
cp /usr/libexec/xen/boot/qemu-stubdom-linux-rootfs stubroot/qemu-stubdom-linux-rootfs.gz
|
||||||
|
cd stubroot
|
||||||
|
gunzip qemu-stubdom-linux-rootfs.gz
|
||||||
|
cpio -i -d -H newc --no-absolute-filenames < qemu-stubdom-linux-rootfs
|
||||||
|
rm qemu-stubdom-linux-rootfs
|
||||||
|
nano init
|
||||||
|
```
|
||||||
|
|
||||||
|
Before the line
|
||||||
|
|
||||||
|
``` text
|
||||||
|
# $dm_args and $kernel are separated with \n to allow for spaces in arguments
|
||||||
|
```
|
||||||
|
|
||||||
|
add:
|
||||||
|
|
||||||
|
``` bash
|
||||||
|
# Patch 3.5 GB limit
|
||||||
|
vm_name=$(xenstore-read "/local/domain/$domid/name")
|
||||||
|
# Apply the patch only if the qube name start by "gpu_"
|
||||||
|
if [ $(echo "$vm_name" | grep -iEc '^gpu_' ) -eq 1 ]; then
|
||||||
|
dm_args=$(echo "$dm_args" | sed -n '1h;2,$H;${g;s/\(-machine\nxenfv\)/\1,max-ram-below-4g=3.5G/g;p}')
|
||||||
|
fi
|
||||||
|
```
|
||||||
|
|
||||||
|
Then execute:
|
||||||
|
|
||||||
|
``` bash
|
||||||
|
find . -print0 | cpio --null -ov \
|
||||||
|
--format=newc | gzip -9 > ../qemu-stubdom-linux-rootfs
|
||||||
|
sudo mv ../qemu-stubdom-linux-rootfs /usr/libexec/xen/boot/
|
||||||
|
```
|
||||||
|
|
||||||
|
Note that this will apply the change to the HVM with a name starting
|
||||||
|
with \"gpu\_\". So you need to name your gaming HVM \"gpu_SOMETHING\".
|
||||||
|
|
||||||
|
## Preparing the guest
|
||||||
|
|
||||||
|
As of 2023, I recommend using a Linux guest instead of a window guest.
|
||||||
|
|
||||||
|
### Windows
|
||||||
|
|
||||||
|
Install a window VM, you can use this
|
||||||
|
[qvm-create-windows-qube](https://github.com/elliotkillick/qvm-create-windows-qube)
|
||||||
|
|
||||||
|
### Linux
|
||||||
|
|
||||||
|
Create a new standalone Qube based on the template of your choice.
|
||||||
|
|
||||||
|
You must run the kernel provided by the guest distribution, because we
|
||||||
|
will use some non-default kernel module for the GPU driver. Just follow
|
||||||
|
the doc:
|
||||||
|
[managing-vm-kernel](https://www.qubes-os.org/doc/managing-vm-kernel/#distribution-kernel).
|
||||||
|
|
||||||
|
Install the GPU drivers you need.
|
||||||
|
|
||||||
|
## Pass the GPU
|
||||||
|
|
||||||
|
In qubes settings for the HVM, go to the 'devices' tab, pass the ID
|
||||||
|
corresponding to your GPU.
|
||||||
|
|
||||||
|
You may or may not need to add the option \"permissive\" or
|
||||||
|
\"no-strict-reset\".
|
||||||
|
|
||||||
|
[Some word about the security implication of thoses
|
||||||
|
parameters.](https://www.qubes-os.org/doc/device-handling-security/#pci-security)
|
||||||
|
|
||||||
|
``` bash
|
||||||
|
qvm-pci attach gpu_gaming_archlinux dom0:0a_00.0 -o permissive=True -o no-strict-reset=True
|
||||||
|
qvm-pci attach gpu_gaming_archlinux dom0:0a_00.1 -o permissive=True -o no-strict-reset=True
|
||||||
|
```
|
||||||
|
|
||||||
|
## Starting the guest
|
||||||
|
|
||||||
|
This is where you will have a lot of issues to debug.
|
||||||
|
|
||||||
|
For Linux guests, run 'sudo dmesg' to have all the kernel log indicating
|
||||||
|
you if there is a issue with your GPU driver. For some hardware, the MSI
|
||||||
|
calls won't work. You can work around that using for example
|
||||||
|
`pci=nomsi`{.text} or `NVreg_EnableMSI=0`{.text} or something else.
|
||||||
|
Check your drivers options. Check if alternative drivers exist (amdgpu,
|
||||||
|
nvidia, nouveau, nvidia-open, using drivers from the official website,
|
||||||
|
...). Check multiple kernel version.
|
||||||
|
|
||||||
|
Some links that could help you to debug the issues you will have
|
||||||
|
|
||||||
|
- https://forum.qubes-os.org/t/ryzen-7000-serie/
|
||||||
|
|
||||||
|
- https://dri.freedesktop.org/docs/drm/gpu/amdgpu.html
|
||||||
|
|
||||||
|
For windows guests you will probably have the same issues but it will be
|
||||||
|
harder to debug. I recommend using the drivers from Windows Update
|
||||||
|
instead of the official drivers from the website of the constructor.
|
||||||
|
|
||||||
|
Some things that may be useful for debugging:
|
||||||
|
|
||||||
|
- Virsh (start, define, \...)
|
||||||
|
|
||||||
|
- /etc/libvirt/libxl/
|
||||||
|
|
||||||
|
- xl
|
||||||
|
|
||||||
|
- /etc/qubes/templates/libvirt/xen/by-name/
|
||||||
|
|
||||||
|
- /usr/lib/xen/boot/
|
||||||
|
|
||||||
|
- virsh -c xen:/// domxml-to-native xen-xm /etc/libvirt/libxl/\...
|
||||||
|
|
||||||
|
Issues with the drivers could be related to
|
||||||
|
'qubes-vmm-xen-stubdom-linux', 'qubes-vmm-xen', and the Linux kernel you
|
||||||
|
will be using.
|
||||||
|
|
||||||
|
## Linux guest --- Integration with QubesOS
|
||||||
|
|
||||||
|
### Xorg
|
||||||
|
|
||||||
|
Now Xorg and Pulseaudio. From XKCD:
|
||||||
|
|
||||||
|
[{width="\\linewidth"}](https://xkcd.com/963/)
|
||||||
|
|
||||||
|
Things you need to install:
|
||||||
|
|
||||||
|
- The Xorg input driver to support your mouse and keyboard
|
||||||
|
|
||||||
|
- A pulseaudio gui client
|
||||||
|
|
||||||
|
- Your favorite Windows Manager
|
||||||
|
|
||||||
|
In my case, it is:
|
||||||
|
|
||||||
|
``` bash
|
||||||
|
apt install xserver-xorg-input-kbd xserver-xorg-input-libinput xserver-xorg-input-mouse pavucontrol i3
|
||||||
|
```
|
||||||
|
|
||||||
|
Then create a XORG configuration file for your GPU and screen. My file
|
||||||
|
named 'AOC.conf':
|
||||||
|
|
||||||
|
``` xorg.conf
|
||||||
|
Section "ServerLayout"
|
||||||
|
Identifier "Gaming"
|
||||||
|
Screen 0 "AMD AOC" Absolute 0 0
|
||||||
|
EndSection
|
||||||
|
|
||||||
|
Section "Device"
|
||||||
|
Identifier "AMD"
|
||||||
|
|
||||||
|
# name of the driver to use. Can be "amdgpu", "nvidia", or something else
|
||||||
|
Driver "amdgpu"
|
||||||
|
|
||||||
|
# The BusID value will change after each qube reboot.
|
||||||
|
BusID "PCI:0:8:0"
|
||||||
|
EndSection
|
||||||
|
|
||||||
|
Section "Monitor"
|
||||||
|
Identifier "AOC"
|
||||||
|
VertRefresh 60
|
||||||
|
# https://arachnoid.com/modelines/ . IMPORTANT TO GET RIGHT. MUST ADJUST WITH EACH SCREEN.
|
||||||
|
Modeline "1920x1080" 172.80 1920 2040 2248 2576 1080 1081 1084 1118
|
||||||
|
EndSection
|
||||||
|
|
||||||
|
Section "Screen"
|
||||||
|
Identifier "AMD AOC"
|
||||||
|
Device "AMD"
|
||||||
|
Monitor "AOC"
|
||||||
|
EndSection
|
||||||
|
```
|
||||||
|
|
||||||
|
We can't know what is the correct BusID before the qube is started. And
|
||||||
|
it change after each reboot. So let's write a script --- named
|
||||||
|
\"xorgX1.sh\" --- that update this configuration file with the correct
|
||||||
|
value, then start a binary on the Xorg X screen n°1.
|
||||||
|
|
||||||
|
``` bash
|
||||||
|
#!/bin/bash
|
||||||
|
|
||||||
|
binary=${1:?binary required}
|
||||||
|
|
||||||
|
# Find the correct BusID of the AMD GPU, then set it in the Xorg configuration file
|
||||||
|
pci=$(lspci | grep "VGA" | grep "NVIDIA|AMD/ATI" | cut -d " " -f 1 | cut -d ":" -f 2 | cut -d "." -f 1 | cut -d "0" -f 2)
|
||||||
|
sed -i "s/PCI:0:[0-9]:0/PCI:0:$pci:0/g" /home/user/AOC.conf
|
||||||
|
|
||||||
|
# Pulseaudio setup
|
||||||
|
sudo killall pulseaudio
|
||||||
|
sudo sed -i "s/load-module module-vchan-sink.*/load-module module-vchan-sink domid=$(qubesdb-read -w /qubes-audio-domain-xid)/" /etc/pulse/qubes-default.pa
|
||||||
|
sudo rm /home/user/.pulse/client.conf
|
||||||
|
start-pulseaudio-with-vchan
|
||||||
|
sleep 5 && sudo chmod -R 777 /root/ &
|
||||||
|
sleep 5 && sudo chmod -R 777 /root/* &
|
||||||
|
sleep 5 && sudo cp /root/.pulse/client.conf /home/user/.pulse/client.conf && sudo chown -R user:user /home/user/.pulse/client.conf &
|
||||||
|
|
||||||
|
setxkbmap fr
|
||||||
|
sudo setxkbmap fr
|
||||||
|
|
||||||
|
# Start the Xorg server for the X screen number 1.
|
||||||
|
# The X screen n°0 is already used for QubesOS integration
|
||||||
|
sudo startx "$binary" -- :1 -config /home/user/AOC.conf
|
||||||
|
```
|
||||||
|
|
||||||
|
### Pulseaudio
|
||||||
|
|
||||||
|
So you need to configure pulseaudio for Xorg multiseat. The archlinux
|
||||||
|
documentation explain that very well: [Xorg
|
||||||
|
multiseat](https://wiki.archlinux.org/index.php/Xorg_multiseat#Multiple_users_on_single_sound_card:_PulseAudio)
|
||||||
|
Use the option without system-mode deamon and adapt it to qube: Add the
|
||||||
|
following line to /etc/pulse/qubes-default.pa
|
||||||
|
|
||||||
|
``` bash
|
||||||
|
load-module module-native-protocol-tcp auth-ip-acl=127.0.0.1
|
||||||
|
```
|
||||||
|
|
||||||
|
Then add this config for root:
|
||||||
|
|
||||||
|
``` bash
|
||||||
|
mkdir /root/.pulse
|
||||||
|
echo "default-server = 127.0.0.1" > /root/.pulse/client.conf
|
||||||
|
```
|
||||||
|
|
||||||
|
The sound was buggy/laggy on my computer. So tried to find a workaround
|
||||||
|
by playing with pulseaudio settings. It was more or less random tries,
|
||||||
|
so I can't really explain it: In `/etc/pulse/daemon.conf`{.text} add the
|
||||||
|
following lines:
|
||||||
|
|
||||||
|
``` bash
|
||||||
|
default-fragments = 60
|
||||||
|
default-fragment-size-msec = 1
|
||||||
|
high-priority = no
|
||||||
|
realtime-scheduling = no
|
||||||
|
nice-level = 18
|
||||||
|
```
|
||||||
|
|
||||||
|
In `/etc/pulse/qubes-default.pa`{.text} change
|
||||||
|
|
||||||
|
``` bash
|
||||||
|
load-module module-udev-detect
|
||||||
|
```
|
||||||
|
|
||||||
|
to
|
||||||
|
|
||||||
|
``` bash
|
||||||
|
load-module module-udev-detect tsched=0
|
||||||
|
```
|
||||||
|
|
||||||
|
You can launch you favorite Windows Manager like that
|
||||||
|
|
||||||
|
``` bash
|
||||||
|
sudo ./xorgX1.sh /usr/bin/i3
|
||||||
|
```
|
||||||
|
|
||||||
|
### References
|
||||||
|
|
||||||
|
- [Archlinux:
|
||||||
|
PulseAudio](https://wiki.archlinux.org/index.php/PulseAudio)
|
||||||
|
|
||||||
|
- [Archlinux:
|
||||||
|
PulseAudio/Troubleshooting](https://wiki.archlinux.org/index.php/PulseAudio/Troubleshooting)
|
||||||
|
|
||||||
@ -1,253 +0,0 @@
|
|||||||
# Create a Gaming HVM
|
|
||||||
|
|
||||||
Some information to configure a windows HVM for gaming.
|
|
||||||
This is not officially supported, just some community trial & errors.
|
|
||||||
This doc is also hosted on https://neowutran.ovh/qubes/articles/gaming_windows_hvm.html
|
|
||||||
|
|
||||||
## References
|
|
||||||
|
|
||||||
Everything needed is referenced here
|
|
||||||
|
|
||||||
- [Useful technical details](https://paste.debian.net/1043341/)
|
|
||||||
|
|
||||||
- [Reddit thread of what is needed for GPU
|
|
||||||
passthrough](https://www.reddit.com/r/Qubes/comments/9hp3e7/gpu_passthrough_howto/)
|
|
||||||
|
|
||||||
- [Solution to have more than 3 GB of RAM in the Windows
|
|
||||||
HVM](https://github.com/QubesOS/qubes-issues/issues/4321#issuecomment-423011787)
|
|
||||||
|
|
||||||
- [Some old
|
|
||||||
references](https://www.reddit.com/r/Qubes/comments/66wk4q/gpu_passthrough/)
|
|
||||||
|
|
||||||
## Prerequisite
|
|
||||||
|
|
||||||
You have a functional Windows HVM (Windows 7 or Windows 10). The \"how
|
|
||||||
to\" for this part can be found on the Qubes OS documentation and here:
|
|
||||||
[Usefull github
|
|
||||||
comment](https://github.com/QubesOS/qubes-issues/issues/3585#issuecomment-453200971).
|
|
||||||
However, few tips:
|
|
||||||
|
|
||||||
- Do a backup (clone VM) of the Windows HVM BEFORE starting to install
|
|
||||||
QWT (installing QWT is not required)
|
|
||||||
|
|
||||||
## Hardware
|
|
||||||
|
|
||||||
To have a Windows HVM for gaming, you must have:
|
|
||||||
|
|
||||||
- A dedicated AMD GPU. By dedicated, it means: it is a secondary GPU,
|
|
||||||
not the GPU used to display dom0. Nvidia GPU are not supported (or
|
|
||||||
maybe with a lot of tricks).
|
|
||||||
|
|
||||||
- A really fast disk (M.2 disk)
|
|
||||||
|
|
||||||
- A lot of RAM
|
|
||||||
|
|
||||||
- A dedicated screen
|
|
||||||
|
|
||||||
- Dedicated gaming mouse and keyboard
|
|
||||||
|
|
||||||
In my case:
|
|
||||||
|
|
||||||
- Secondary GPU: AMD RX580
|
|
||||||
|
|
||||||
- Primary GPU: Some Nvidia trash, used for dom0
|
|
||||||
|
|
||||||
- 32 GB of RAM. 12 GB of RAM will be dedicated to the Windows HVM
|
|
||||||
|
|
||||||
- A fast M.2 disk
|
|
||||||
|
|
||||||
## Checklist
|
|
||||||
|
|
||||||
Short list of things to do to make the GPU passthrough work:
|
|
||||||
|
|
||||||
- You verified and confirmed that the secondary GPU is alone in its
|
|
||||||
IOMMU Group
|
|
||||||
|
|
||||||
- In dom0, you edited the file `/etc/default/grub` or
|
|
||||||
`/boot/efi/EFI/qubes/xen.cfg` or
|
|
||||||
`/boot/efi/EFI/qubes/grub.cfg` to allow PCI hiding for your
|
|
||||||
secondary GPU, and regenerated the grub if needed
|
|
||||||
|
|
||||||
- You have patched stubdom-linux-rootfs.gz to allow to have more than
|
|
||||||
3 GB of RAM for your HVM
|
|
||||||
|
|
||||||
## IOMMU Group
|
|
||||||
|
|
||||||
Warning: I am far from understanding the IOMMU group. Check online
|
|
||||||
references on that subject. It seems that you can only do a successfull
|
|
||||||
GPU passthough if you can passthrough everything that is in the IOMMU
|
|
||||||
Group of the GPU. Also, you can't see your IOMMU Group when you are
|
|
||||||
using Xen (the information is hidden from dom0). So, what I did: I
|
|
||||||
booted from a Linux Mint Live USB. In the grub I enabled the IOMMU
|
|
||||||
(iommu=1 iommu_amd=on), and then displayed the folder structure of
|
|
||||||
/sys/kernel/iommu_group
|
|
||||||
|
|
||||||
``` bash
|
|
||||||
tree /sys/kernel/iommu_group
|
|
||||||
```
|
|
||||||
|
|
||||||
My secondary GPU was alone in its IOMMU group.
|
|
||||||
|
|
||||||
## GRUB modification
|
|
||||||
|
|
||||||
You must hide your secondary GPU from dom0. To do that, you have to edit
|
|
||||||
the GRUB. In a dom0 Terminal, type:
|
|
||||||
|
|
||||||
``` bash
|
|
||||||
qvm-pci
|
|
||||||
```
|
|
||||||
|
|
||||||
Then find the devices id for your secondary GPU. In my case, it is
|
|
||||||
`dom0:0a_00.0` and `dom0:0a_00.1`. Edit /etc/default/grub,
|
|
||||||
and add the PCI hiding
|
|
||||||
|
|
||||||
``` text
|
|
||||||
GRUB_CMDLINE_LINUX="... rd.qubes.hide_pci=0a:00.0,0a:00.1 "
|
|
||||||
```
|
|
||||||
|
|
||||||
then regenerate the grub
|
|
||||||
|
|
||||||
``` bash
|
|
||||||
grub2-mkconfig -o /boot/grub2/grub.cfg
|
|
||||||
```
|
|
||||||
|
|
||||||
Or if using UEFI boot, edit `/boot/efi/EFI/qubes/xen.cfg` or
|
|
||||||
`/boot/efi/EFI/qubes/grub.cfg` and add the
|
|
||||||
`rd.qubes.hide_pci=` option to the `kernel=` line.
|
|
||||||
|
|
||||||
## Patching stubdom-linux-rootfs.gz
|
|
||||||
|
|
||||||
### Qubes R4.0
|
|
||||||
|
|
||||||
Follow the instructions here:\
|
|
||||||
[github.com/QubesOS/qubes-issues/issues/4321](https://github.com/QubesOS/qubes-issues/issues/4321#issuecomment-423011787)
|
|
||||||
|
|
||||||
Copy-paste of the comment:
|
|
||||||
|
|
||||||
This is caused by the default TOLUD (Top of Low Usable DRAM) of 3.75G
|
|
||||||
provided by qemu not being large enough to accommodate the larger BARs
|
|
||||||
that a graphics card typically has. The code to pass a custom
|
|
||||||
max-ram-below-4g value to the qemu command line does exist in the
|
|
||||||
libxl_dm.c file of xen, but there is no functionality in libvirt to add
|
|
||||||
this parameter. It is possible to manually add this parameter to the
|
|
||||||
qemu commandline by doing the following in a dom0 terminal:
|
|
||||||
|
|
||||||
``` bash
|
|
||||||
mkdir stubroot
|
|
||||||
cp /usr/lib/xen/boot/stubdom-linux-rootfs stubroot/stubdom-linux-rootfs.gz
|
|
||||||
cd stubroot
|
|
||||||
gunzip stubdom-linux-rootfs.gz
|
|
||||||
cpio -i -d -H newc --no-absolute-filenames < stubdom-linux-rootfs
|
|
||||||
rm stubdom-linux-rootfs
|
|
||||||
nano init
|
|
||||||
```
|
|
||||||
|
|
||||||
Before the line
|
|
||||||
|
|
||||||
``` text
|
|
||||||
#$dm_args and $kernel are separated with \x1b to allow for spaces in arguments.
|
|
||||||
```
|
|
||||||
|
|
||||||
add:
|
|
||||||
|
|
||||||
``` bash
|
|
||||||
SP=$'\x1b'
|
|
||||||
dm_args=$(echo "$dm_args" \
|
|
||||||
| sed "s/-machine\\${SP}xenfv/-machine\
|
|
||||||
\\${SP}xenfv,max-ram-below-4g=3.5G/g")
|
|
||||||
```
|
|
||||||
|
|
||||||
Then execute:
|
|
||||||
|
|
||||||
``` bash
|
|
||||||
find . -print0 | cpio --null -ov \
|
|
||||||
--format=newc | gzip -9 > ../stubdom-linux-rootfs
|
|
||||||
sudo mv ../stubdom-linux-rootfs /usr/lib/xen/boot/
|
|
||||||
```
|
|
||||||
|
|
||||||
Note that this will apply the change to all HVMs, so if you have any
|
|
||||||
other HVM with more than 3.5GB ram assigned, they will not start without
|
|
||||||
the adapter being passed through. Ideally to fix this libvirt should be
|
|
||||||
extended to pass the max-ram-below-4g parameter through to xen, and then
|
|
||||||
a calculation added to determine the correct TOLUD based on the total
|
|
||||||
BAR size of the PCI devices are being passed through to the vm.
|
|
||||||
|
|
||||||
### Qubes R4.1
|
|
||||||
|
|
||||||
For Qubes R4.1 follow the R4.0 section, except for two things.
|
|
||||||
|
|
||||||
- The file that need to be patched is now
|
|
||||||
\"/usr/libexec/xen/boot/qemu-stubdom-linux-rootfs\"
|
|
||||||
|
|
||||||
- The content of the \"init\" file to modify.
|
|
||||||
|
|
||||||
#### The content of the \"init\" file
|
|
||||||
|
|
||||||
Before the line
|
|
||||||
|
|
||||||
``` text
|
|
||||||
# $dm_args and $kernel are separated with \n to allow for spaces in arguments
|
|
||||||
```
|
|
||||||
|
|
||||||
add:
|
|
||||||
|
|
||||||
``` bash
|
|
||||||
# Patch 3.5 GB limit
|
|
||||||
vm_name=$(xenstore-read "/local/domain/$domid/name")
|
|
||||||
# Apply the patch only if the qube name start by "gpu_"
|
|
||||||
if [ $(echo "$vm_name" | grep -iEc '^gpu_' ) -eq 1 ]; then
|
|
||||||
dm_args=$(echo "$dm_args" | sed -n '1h;2,$H;${g;s/\(-machine\nxenfv\)/\1,max-ram-below-4g=3.5G/g;p}')
|
|
||||||
fi
|
|
||||||
```
|
|
||||||
|
|
||||||
## Pass the GPU
|
|
||||||
|
|
||||||
In qubes settings for the windows HVM, go to the \"devices\" tab, pass
|
|
||||||
the ID corresponding to your AMD GPU. (in my case, it was 0a:00.0 and
|
|
||||||
0a:00.1) And check the option for \"nostrict reset\" for those 2. In
|
|
||||||
some case, you might also need to set the \"permissive\" flag to true
|
|
||||||
(But I didn't need that with the RX 580):
|
|
||||||
|
|
||||||
``` bash
|
|
||||||
qvm-pci attach windows-hvm dom0:0a_00.0 -o permissive=True -o no-strict-reset=True
|
|
||||||
qvm-pci attach windows-hvm dom0:0a_00.1 -o permissive=True -o no-strict-reset=True
|
|
||||||
```
|
|
||||||
|
|
||||||
## Conclusion
|
|
||||||
|
|
||||||
Don't forget to install the GPU drivers, you can install the official
|
|
||||||
one from AMD website, no modification or trick to do. Nothing else is
|
|
||||||
required to make it work (in my case at least, once I finish to fight to
|
|
||||||
find those informations). If you have issues, you can refer to the links
|
|
||||||
in the first sections. If it doesn't work and you need to debug more
|
|
||||||
things, you can go deeper.
|
|
||||||
|
|
||||||
- Virsh (start, define, \...)
|
|
||||||
|
|
||||||
- /etc/libvirt/libxl/
|
|
||||||
|
|
||||||
- xl
|
|
||||||
|
|
||||||
- /etc/qubes/templates/libvirt/xen/by-name/
|
|
||||||
|
|
||||||
- /usr/lib/xen/boot/
|
|
||||||
|
|
||||||
- virsh -c xen:/// domxml-to-native xen-xm /etc/libvirt/libxl/\...
|
|
||||||
|
|
||||||
I am able to play games on my windows HVM with very good performances.
|
|
||||||
And safely.
|
|
||||||
|
|
||||||
## Bugs
|
|
||||||
|
|
||||||
The AMD GPUs have a bug when used in HVM: each time you will reboot your
|
|
||||||
windows HVM, it will get slower and slower. It is because the AMD GPUs
|
|
||||||
is not correctly reset when you restart your Windows HVM. Two
|
|
||||||
solutions for that:
|
|
||||||
|
|
||||||
- Reboot your computer
|
|
||||||
|
|
||||||
- In the windows HVM, use to windows option in the system tray to
|
|
||||||
\"safely remove devices\", remove your GPU. Restart the HVM.
|
|
||||||
|
|
||||||
This bug is referenced somewhere, but lost the link and too lazy to
|
|
||||||
search for it.
|
|
||||||
@ -11,5 +11,5 @@ For more information about Windows VMs in Qubes OS, please see the following ext
|
|||||||
* [Installing and Using Qubes Windows Tools in Qubes R4.0](https://github.com/Qubes-Community/Contents/blob/master/docs/os/windows/windows-tools.md)
|
* [Installing and Using Qubes Windows Tools in Qubes R4.0](https://github.com/Qubes-Community/Contents/blob/master/docs/os/windows/windows-tools.md)
|
||||||
* [Installing and Using Windows-based VMs in Qubes R4.1](https://github.com/Qubes-Community/Contents/blob/master/docs/os/windows/windows-vm41.md)
|
* [Installing and Using Windows-based VMs in Qubes R4.1](https://github.com/Qubes-Community/Contents/blob/master/docs/os/windows/windows-vm41.md)
|
||||||
* [Installing and Using Qubes Windows Tools in Qubes R4.1](https://github.com/Qubes-Community/Contents/blob/master/docs/os/windows/windows-tools41.md)
|
* [Installing and Using Qubes Windows Tools in Qubes R4.1](https://github.com/Qubes-Community/Contents/blob/master/docs/os/windows/windows-tools41.md)
|
||||||
* [Create a Gaming HVM in Qubes 4.1](https://github.com/Qubes-Community/Contents/blob/master/docs/customization/windows-gaming-hvm.md)
|
* [Create a Gaming HVM in Qubes 4.1](https://github.com/Qubes-Community/Contents/blob/master/docs/customization/gaming-hvm.md)
|
||||||
* [Migrate backups of Windows VMs created under Qubes R4.0 to R4.1](https://github.com/Qubes-Community/Contents/blob/master/docs/os/windows/windows-migrate41.md)
|
* [Migrate backups of Windows VMs created under Qubes R4.0 to R4.1](https://github.com/Qubes-Community/Contents/blob/master/docs/os/windows/windows-migrate41.md)
|
||||||
|
|||||||
Loading…
Reference in New Issue
Block a user