Create a Gaming HVM revisions

Go back to topic: Create a Gaming HVM

  1. v42 anchor; v42 full version
  2. v41 anchor; v41 full version
  3. v40 anchor; v40 full version
  4. v39 anchor; v39 full version
  5. v38 anchor; v38 full version
  6. v37 anchor; v37 full version
  7. v36 anchor; v36 full version
  8. v35 anchor; v35 full version
  9. v34 anchor; v34 full version
  10. v33 anchor; v33 full version
  11. v32 anchor; v32 full version
  12. v31 anchor; v31 full version
  13. v30 anchor; v30 full version
  14. v29 anchor; v29 full version
  15. v28 anchor; v28 full version
  16. v27 anchor; v27 full version
  17. v26 anchor; v26 full version
  18. v25 anchor; v25 full version
  19. v24 anchor; v24 full version
  20. v23 anchor; v23 full version
  21. v22 anchor; v22 full version
  22. v21 anchor; v21 full version
  23. v20 anchor; v20 full version
  24. v19 anchor; v19 full version
  25. v18 anchor; v18 full version
  26. v17 anchor; v17 full version
  27. v16 anchor; v16 full version
  28. v15 anchor; v15 full version
  29. v14 anchor; v14 full version
  30. v13 anchor; v13 full version
  31. v12 anchor; v12 full version
  32. v11 anchor; v11 full version
  33. v10 anchor; v10 full version
  34. v9 anchor; v9 full version
  35. v8 anchor; v8 full version
  36. downloaded local copies of images v7 anchor; v7 full version
  37. v6 anchor; v6 full version
  38. v5 anchor; v5 full version
  39. v4 anchor; v4 full version
  40. v3 anchor; v3 full version
  41. v2 anchor; v2 full version

Revision #42

Edited on
2025-01-17
Edited by user
neowutran
- A screen available for the gaming 'HVM'. (It can be a physical monitor or just to have multiple cables connected to the screen and switching between input source) - Dedicated gaming mouse and keyboard.
- A screen available for the gaming 'HVM'. (It can be a physical monitor or just to have multiple cables connected to the screen and switching between input source) **[Optional]** - Dedicated gaming mouse and keyboard **[Optional]** .
# Linux guest --- Integration with QubesOS # [Case: No dedicated screen, mouse or keyboard] VirtualGL and vulkan In some cases, it will just work out of the box (application will automatically detect the GPU and will be able to use it without modification. In some cases, an additionnal layer is required: using VirtualGL, mentionned here: https://forum.qubes-os.org/t/seamless-gpu-passthrough-on-qubes-os-with-virtualgl/20265 In that case you may need to specify a new variable ``LD_LIBRARY_PATH=/usr/lib`` You may have issue with how the mouse is handled by the game. AFAIK, no one tried to solve this issue. # [Case: Dedicated Screen, mouse and keyboard] Linux guest --- Integration with QubesOS
## Issues and fixes In one case in a setup with Intel IGPU + Nvidia DGPU, dom0 xorg crashed. Solved the case by adding a Xorg configuration to explicitly use Intel: https://forum.qubes-os.org/t/gpu-passthrough-lightdm-service-wont-start/22220/20

Revision #41

Edited on
2025-01-17
Edited by user
neowutran

Revision #40

Edited on
2024-11-08
Edited by user
neowutran
And automated way to do that is proposed here: https://forum.qubes-os.org/t/create-a-gaming-hvm/19000#p-86347-automating-most-of-that-24
You may or may not need to add the option "permissive" or "no-strict-reset". You may or may not need to add the option "permissive" or "no-strict-reset". You may or may not need to passthrough additional devices depending on the result of the IOMMU script https://forum.qubes-os.org/t/create-a-gaming-hvm/19000#p-86347-iommu-group-2
Then create a XORG configuration file for your GPU and screen. My file named 'AOC.conf': Then create a XORG configuration file for your GPU and screen. My file named 'xorg.conf':
Identifier "Gaming" Screen 0 "AMD AOC" Absolute 0 0 Identifier "Passthrough Layout" Screen 0 "Passthrough Screen" Absolute 0 0
Identifier "AMD" Identifier "Passthrough GPU"
Driver "amdgpu" Driver "driver" Option "Coolbits" "4"
BusID "PCI:0:8:0" BusID "PCI:0::0"
Identifier "AOC" VertRefresh 60 # https://arachnoid.com/modelines/ . IMPORTANT TO GET RIGHT. MUST ADJUST WITH EACH SCREEN. Modeline "1920x1080" 172.80 1920 2040 2248 2576 1080 1081 1084 1118 Identifier "Passthrough monitor"
Identifier "AMD AOC" Device "AMD" Monitor "AOC" Identifier "Passthrough Screen" Device "Passthrough GPU" Monitor "Passthrough Monitor"
#!/bin/bash #!/bin/bash
lspci | grep "VGA" | grep -E "NVIDIA" && sed -i 's/^Driver .*/Driver "nvidia"/g' /opt/xorg.conf lspci | grep "VGA" | grep -E "AMD/ATI" && sed -i 's/^Driver .*/Driver "amdgpu"/g' /opt/xorg.conf
sed -i 's/"PCI:[^"]*"/"PCI:0:'$pci':0"/g' /home/user/AOC.conf sed -i 's/"PCI:[^"]*"/"PCI:0:'$pci':0"/g' /opt/xorg.conf
# The X screen n°0 is already used for QubesOS integration sudo startx "$binary" -- :1 -config /home/user/AOC.conf # The X screen n__0 is already used for QubesOS integration sudo startx "$binary" -- :1 -config /opt/xorg.conf

Revision #39

Edited on
2024-09-21
Edited by user
neowutran
[Some word about the security implication of thoses parameters.](https://www.qubes-os.org/doc/device-handling-security/#pci-security) [Some word about the security implication of thoses parameters.](https://www.qubes-os.org/doc/device-handling-security/#pci-security)
# Automating most of that I trying to write a script to automate all the previous steps. It is available here: https://git.sr.ht/~yukikoo/gpu_template Please be carefull and read the code, not much tests have been done

Revision #38

Edited on
2024-07-05
Edited by user
apparatus
If you are using UEFI, the file to override with `grub2-mkconfig` is `/boot/efi/EFI/qubes/grub.cfg`. If you are using UEFI and Qubes OS 4.1 or earlier, the file to override with `grub2-mkconfig` is `/boot/efi/EFI/qubes/grub.cfg`.

Revision #37

Edited on
2024-05-03
Edited by user
neowutran
## Audio - Delete any packages related to pulseaudio and install pipewire ``` sudo pacman -Rdd qubes-vm-pulseaudio pulseaudio sudo pacman -S pipewire-{jack,alsa,pulse} pipewire-qubes ``` - Enable the "pipewire" service for this qube using Qubes Manager ![pipewire|690x168](upload://rBE1PzenJrSoiAB3hcvCgSapeCv.png) ## Audio
``` [details="Deprecated: old way of doing it"] - Delete any packages related to pulseaudio and install pipewire
[details="Deprecated: old way of doing it"] sudo pacman -Rdd qubes-vm-pulseaudio pulseaudio sudo pacman -S pipewire-{jack,alsa,pulse} pipewire-qubes ``` - Enable the "pipewire" service for this qube using Qubes Manager ![pipewire|690x168](upload://rBE1PzenJrSoiAB3hcvCgSapeCv.png) [/details] [details="Deprecated: old way of doing it2"]

Revision #36

Edited on
2024-03-12
Edited by user
LinuxUser
Note: if after this step when you reboot the computer you get stuck in the QubesOS startup that means you are trying to use the GPU you just hide. Check your BIOS options. Also check the cables, BIOS have some GPU priority based on the type of cable. For example, DisplayPort can be favoured over HDMI. Once you have rebooted, in dom0, type `sudo lspci -vvn`, you should see "Kernel driver in use: pciback" for the GPU you just hide. Note: if after this step when you reboot the computer you get stuck in the QubesOS startup that means you are trying to use the GPU you just hid. Check your BIOS options. Also check the cables, BIOS have some GPU priority based on the type of cable. For example, DisplayPort can be favoured over HDMI. Once you have rebooted, in dom0, type `sudo lspci -vvn`, you should see "Kernel driver in use: pciback" for the GPU you just hid.

Revision #35

Edited on
2024-02-12
Edited by user
neowutran
``` bash qvm-pci attach gpu_gaming_archlinux dom0:0a_00.0 --persistent ```
qvm-pci attach gpu_gaming_archlinux dom0:0a_00.0 -o permissive=True -o no-strict-reset=True qvm-pci attach gpu_gaming_archlinux dom0:0a_00.0 -o permissive=True -o no-strict-reset=True --persistent

Revision #34

Edited on
2024-02-12
Edited by user
neowutran
You need to check what are the things/devices that are in the same IOMMU group as the GPU you want to passthrough. You can't see your IOMMU Group when you are using Xen (the information is hidden from dom0). So, start a live linux distribution, enable iommu in the grub options (iommu=1 iommu_amd=on), and then displayed the folder structure of /sys/kernel/iommu_group ## Goal #### What The goal of this step if to retrieve the default IOMMU Group (https://www.kernel.org/doc/html/latest/driver-api/vfio.html#groups-devices-and-iommus) of your hardware. #### Why It can help understanding potential issue with your setup (what devices live in the same IOMMU group as your GPU) / finding potential workaround. If you feel lucky, skip this step. #### How You can't see your IOMMU Group when you are using Xen (the information is hidden from dom0). - Boot a live linux distribution - In the grub, enable iommu: Add the parameters ``iommu=1 iommu_amd=on`` to the linux commandline - Once you logged in your live linux distribution, you need to retrieve the folder structure of ``/sys/kernel/iommu_group``. You can use the following script to do that:

Revision #33

Edited on
2024-01-22
Edited by user
neowutran
This is where you will have a lot of issues to debug. **This is where you will have a lot of issues to debug. Prepare for intense suffering**

Revision #32

Edited on
2024-01-21
Edited by user
neowutran
Remove any existing "max-ram-below-4g" workaround

Revision #31

Edited on
2024-01-07
Edited by user
neowutran
Since the release of vmm-xen version [4.17.2-8](https://github.com/QubesOS/qubes-vmm-xen/commit/b567bd91d0cda0b65c6dd7f28468f97cbb85f0c7) (R4.2, 2024-01-03), no additional configuration is required. Since the release of this qubes version of xen: [4.17.2-8](https://github.com/QubesOS/qubes-vmm-xen/commit/b567bd91d0cda0b65c6dd7f28468f97cbb85f0c7) (R4.2, 2024-01-03), no additional configuration is required.

Revision #30

Edited on
2024-01-04
Edited by user
neowutran
# Configuring the parameter "max-ram-below-4g" # Configuring the parameter "max-ram-below-4g" Since the release of vmm-xen version [4.17.2-8](https://github.com/QubesOS/qubes-vmm-xen/commit/b567bd91d0cda0b65c6dd7f28468f97cbb85f0c7) (R4.2, 2024-01-03), no additional configuration is required. [details="If you are using an older version"]
``` ```
[/details]

Revision #29

Edited on
2023-12-08
Edited by user
neowutran
if the max value of ``[size=XXXX]`` is 256MB, try 3.5G for max-ram-below-4g. If the max value is bigger, try 2G for max-ram-below-4g. if the max value of ``[size=XXXX]`` is 256MB, try 3.5G for max-ram-below-4g. If the max value is bigger, try 2G for max-ram-below-4g. **Update**: I think I discovered the reason (https://github.com/QubesOS/qubes-issues/issues/4321#issuecomment-1828226780 https://lists.xenproject.org/archives/html/xen-devel/2023-12/msg00133.html ). If you want and have the skills required to compile the xen package, try to apply this patch [Fix guest memory corruption caused by hvmloader by neowutran · Pull Request #172 · QubesOS/qubes-vmm-xen · GitHub](https://github.com/QubesOS/qubes-vmm-xen/pull/172/files) and confirm it if work as expected. With this patch, the part about "patching stubdom-linux-rootfs.gz" is not needed.
## Things to do Try to find why "max-ram-below-4g" doesn't work as expected https://github.com/QubesOS/qubes-issues/issues/4321#issuecomment-1206000974 https://github.com/qemu/qemu/blob/master/hw/i386/pc_piix.c#L131 Edit: I think I found the issue, if you want and have the skills required to compile the xen package, try to apply this patch https://github.com/QubesOS/qubes-vmm-xen/pull/172/files and confirm it if work as expected

Revision #28

Edited on
2023-12-08
Edited by user
neowutran

Revision #27

Edited on
2023-12-05
Edited by user
neowutran
Edit: I think I found the issue, if you want and have the skills required to compile the xen package, try to apply this patch https://github.com/QubesOS/qubes-vmm-xen/pull/172/files and confirm it if work as expected

Revision #26

Edited on
2023-11-17
Edited by user
neowutran
**Important note: Below, we set the "max-ram-below-4g" parameter to "3.5G".** **For some GPU this value need to be "2G" (discovered here: https://forum.qubes-os.org/t/quick-howto-gpu-passthrough-with-lots-of-ram/21156). It is not currently well understood why the value need to be exactly "2G" or exactly "3.5G", or maybe some other values for other GPU/configuration we never saw yet.** More investigation is required to understand what is going on with this parameter.** ## Why do we need to do that ? [github.com/QubesOS/qubes-issues/issues/4321](https://github.com/QubesOS/qubes-issues/issues/4321#issuecomment-423011787) Copy-paste of the comment: This is caused by the default TOLUD (Top of Low Usable DRAM) of 3.75G provided by qemu not being large enough to accommodate the larger BARs that a graphics card typically has. The code to pass a custom max-ram-below-4g value to the qemu command line does exist in the libxl_dm.c file of xen, but there is no functionality in libvirt to add this parameter. It is possible to manually add this parameter to the qemu commandline by doing the following in a dom0 terminal. ------- ( "max-ram-below-4g" is not related to the amount of ram you can pass to the VM. It is not related to VRAM either. It is related to WHAT is available in the memory space for 32 bits system and WHAT is stored in the memory space for 64 bits system (usable ram is only a part of what need to be mapped in the memory space) ) ## Finding the correct value for this parameter Below, we set the "max-ram-below-4g" parameter to "3.5G". For some GPU this value need to be "2G" (discovered here: https://forum.qubes-os.org/t/quick-howto-gpu-passthrough-with-lots-of-ram/21156). It is not currently well understood why the value need to be exactly "2G" or exactly "3.5G", or maybe some other values for other GPU/configuration we never saw yet. ( https://github.com/QubesOS/qubes-issues/issues/4321#issuecomment-1206000974 ) More investigation is required to understand what is going on with this parameter.
if the max value of ``[size=XXXX]`` is 256MB, try 3.5G for max-ram-below-4g. If the max value is bigger, try 2G for max-ram-below-4g. ------- "max-ram-below-4g" is not related to the amount of ram you can pass to the VM. It is not related to VRAM either. It is related to WHAT is available in the memory space for 32 bits system and WHAT is stored in the memory space for 64 bits system (usable ram is only a part of what need to be mapped in the memory space) ------- if the max value of ``[size=XXXX]`` is 256MB, try 3.5G for max-ram-below-4g. If the max value is bigger, try 2G for max-ram-below-4g.
### Why do we need to do that ? [github.com/QubesOS/qubes-issues/issues/4321](https://github.com/QubesOS/qubes-issues/issues/4321#issuecomment-423011787) Copy-paste of the comment: This is caused by the default TOLUD (Top of Low Usable DRAM) of 3.75G provided by qemu not being large enough to accommodate the larger BARs that a graphics card typically has. The code to pass a custom max-ram-below-4g value to the qemu command line does exist in the libxl_dm.c file of xen, but there is no functionality in libvirt to add this parameter. It is possible to manually add this parameter to the qemu commandline by doing the following in a dom0 terminal. ----- ### Patching
[details="OUTDATED - DO NOT USE - Other method: Patching xen.xml"] [details="OUTDATED - DO NOT USE - Other method: Patching xen.xml instead of stubdom"]
A better patch for xen.xml is available here: https://github.com/QubesOS/qubes-issues/issues/4321#issuecomment-1551203532
https://github.com/QubesOS/qubes-issues/issues/4321#issuecomment-1551203532 https://github.com/QubesOS/qubes-issues/issues/4321#issuecomment-1206000974

Revision #25

Edited on
2023-11-17
Edited by user
neowutran
**For some GPU this value need to be "2G" (discovered here: https://forum.qubes-os.org/t/quick-howto-gpu-passthrough-with-lots-of-ram/21156). It is not currently well understood why the value need to be exactly "2G" or exactly "3.5G", or maybe some other values for other GPU/configuration we never saw yet. More investigation is required to understand what is going on with this parameter.** **For some GPU this value need to be "2G" (discovered here: https://forum.qubes-os.org/t/quick-howto-gpu-passthrough-with-lots-of-ram/21156). It is not currently well understood why the value need to be exactly "2G" or exactly "3.5G", or maybe some other values for other GPU/configuration we never saw yet.** More investigation is required to understand what is going on with this parameter.**
[details="Method 1: Patching stubdom-linux-rootfs.gz"] ------- "max-ram-below-4g" is not related to the amount of ram you can pass to the VM. It is not related to VRAM either. It is related to WHAT is available in the memory space for 32 bits system and WHAT is stored in the memory space for 64 bits system (usable ram is only a part of what need to be mapped in the memory space) ------- ## Patching stubdom-linux-rootfs.gz ### Why do we need to do that ?
Copy-paste of the comment: This is caused by the default TOLUD (Top of Low Usable DRAM) of 3.75G provided by qemu not being large enough to accommodate the larger BARs that a graphics card typically has. The code to pass a custom max-ram-below-4g value to the qemu command line does exist in the libxl_dm.c file of xen, but there is no functionality in libvirt to add this parameter. It is possible to manually add this parameter to the qemu commandline by doing the following in a dom0 terminal. (I modified the code so it works with 4.1 and remove one of the original limitations by restricting the modification to VM with a name starting with "gpu\_") Copy-paste of the comment: This is caused by the default TOLUD (Top of Low Usable DRAM) of 3.75G provided by qemu not being large enough to accommodate the larger BARs that a graphics card typically has. The code to pass a custom max-ram-below-4g value to the qemu command line does exist in the libxl_dm.c file of xen, but there is no functionality in libvirt to add this parameter. It is possible to manually add this parameter to the qemu commandline by doing the following in a dom0 terminal. ----- ### Patching I modified the original code to: - make it works with Qubes R4.1/R4.2 - removed one of the original limitations by restricting the modification to VM with a name starting with "gpu_" - Added a way to modify per vm the value for "max-ram-below-4g". Ex, if you want specifically to use 2G for "max-ram-below-4g", name the vm "gpu_2G_YOURNAME", if you want specifically to use 3.5G for "max-ram-below-4g", name the vm "gpu_3n5G_YOURNAME"
# Patch 3.5 GB limit
# Apply the patch only if the qube name start by "gpu_"
dm_args=$(echo "$dm_args" | sed -n '1h;2,$H;${g;s/\(-machine\nxenfv\)/\1,max-ram-below-4g=3.5G/g;p}') max_ram_below_4g=$(echo "$vm_name" | grep -iEo "_([0-9]*(n[0-9]*)?)._" | cut -d '_' -f 2) if [[ -z "$max_ram_below_4g" ]]; then max_ram_below_4g="3.5G" fi max_ram_below_4g=$(echo "$max_ram_below_4g" | sed 's/n/./g') dm_args=$(echo "$dm_args" | sed -n '1h;2,$H;${g;s/\(-machine\nxenfv\)/\1,max-ram-below-4g='"$max_ram_below_4g"'/g;p}')
Note that this will apply the change to the HVM with a name starting with "gpu\_". So you need to name your gaming HVM "gpu_SOMETHING". [/details] [details="OUTDATED - DO NOT USE - Method 2: Patching xen.xml"] Note that this will apply the change to the HVM with a name starting with "gpu\_". So you need to name your gaming HVM "gpu_SOMETHING". Alternatively, the following dom0 script "patch_stubdom.sh" does all the previous steps: ``` #!/bin/bash patch_rootfs(){ filename=${1?Filename is required} cd ~/ sudo rm -R "patched_$filename" mkdir "patched_$filename" cp /usr/libexec/xen/boot/$filename "patched_$filename/$filename.gz" cp /usr/libexec/xen/boot/$filename "$filename.original" cd patched_$filename gunzip $filename.gz cpio -i -d -H newc --no-absolute-filenames < "$filename" sudo rm $filename grep -i "max-ram-below-4g" init && echo "!!ERROR!! The thing is already patched ! EXITING ! " >&2 && exit patch_string=$(cat <<'EOF' vm_name=$(xenstore-read "/local/domain/$domid/name") if [ $(echo "$vm_name" | grep -iEc '^gpu_' ) -eq 1 ]; then max_ram_below_4g=$(echo "$vm_name" | grep -iEo "_([0-9]*(n[0-9]*)?)._" | cut -d '_' -f 2) if [[ -z "$max_ram_below_4g" ]]; then max_ram_below_4g="3.5G" fi max_ram_below_4g=$(echo "$max_ram_below_4g" | sed 's/n/./g') dm_args=$(echo "$dm_args" | sed -n '1h;2,$H;${g;s/\\(-machine\\nxenfv\\)/\\1,max-ram-below-4g='"$max_ram_below_4g"'/g;p}') fi \# $dm_args and $kernel EOF ) awk -v r="$patch_string" '{gsub(/^# \$dm_args and \$kernel/,r)}1' init > init2 cp init /tmp/init_$filename mv init2 init chmod +x init find . -print0 | cpio --null -ov \ --format=newc | gzip -9 > ../$filename.patched sudo cp ../$filename.patched /usr/libexec/xen/boot/$filename cd ~/ } grep -i "max-ram-below-4g" /usr/share/qubes/templates/libvirt/xen.xml && "!!ERROR!! xen.xml is patched ! EXITING ! " && exit patch_rootfs "qemu-stubdom-linux-rootfs" patch_rootfs "qemu-stubdom-linux-full-rootfs" echo "stubdom have been patched." ``` [details="OUTDATED - DO NOT USE - Other method: Patching xen.xml"]
## Things to do Try to find why "max-ram-below-4g" doesn't work as expected https://github.com/QubesOS/qubes-issues/issues/4321#issuecomment-1551203532 https://github.com/qemu/qemu/blob/master/hw/i386/pc_piix.c#L131

Revision #24

Edited on
2023-11-17
Edited by user
neowutran

Revision #23

Edited on
2023-11-17
Edited by user
neowutran
[details="Method 2: Patching xen.xml"] [details="OUTDATED - DO NOT USE - Method 2: Patching xen.xml"]

Revision #22

Edited on
2023-10-26
Edited by user
neowutran
For nvidia GPU, I recommand using "nvidia-open" drivers instead of "nvidia"

Revision #21

Edited on
2023-10-26
Edited by user
neowutran
Current best guest is to run this command in dom0: Current best guess is to run this command in dom0:

Revision #20

Edited on
2023-10-26
Edited by user
neowutran
Current best guest is to run this command in dom0: ``lspci -vvs GPU_IDENTIFIER | grep Region`` , for example: ``lspci -vvs 0a:00.0 | grep Region``. if the max value of ``[size=XXXX]`` is 256MB, try 3.5G for max-ram-below-4g. If the max value is bigger, try 2G for max-ram-below-4g.

Revision #19

Edited on
2023-10-21
Edited by user
neowutran
Then find the devices id for your secondary GPU. In my case, it is `dom0:0a_00.0` and `dom0:0a_00.1`. Edit /etc/default/grub, and add the PCI hiding. Then find the devices id for your secondary GPU. In my case, it is `dom0:0a_00.0`. Edit /etc/default/grub, and add the PCI hiding.
GRUB_CMDLINE_LINUX="... rd.qubes.hide_pci=0a:00.0,0a:00.1 " GRUB_CMDLINE_LINUX="... rd.qubes.hide_pci=0a:00.0 "
qvm-pci attach gpu_gaming_archlinux dom0:0a_00.1 -o permissive=True -o no-strict-reset=True
Note: You do not need to pass the audio part of the GPU
pci=$(lspci | grep "VGA" | grep "NVIDIA|AMD/ATI" | cut -d " " -f 1 | cut -d ":" -f 2 | cut -d "." -f 1 | cut -d "0" -f 2) sed -i &quot;s/PCI:0:[0-9]:0/PCI:0:$pci:0/g&quot; /home/user/AOC.conf pci=$(lspci | grep "VGA" | grep -E "NVIDIA|AMD/ATI" | cut -d " " -f 1 | cut -d ":" -f 2 | cut -d "." -f 1 | cut -d "0" -f 2) sed -i &#39;s/"PCI:[^"]*"/"PCI:0:'$pci':0"/g&#39; /home/user/AOC.conf
setxkbmap fr /bin/sudo -u user PULSE_SERVER=unix:/run/user/1000/pulse/native /usr/bin/i3 sleep 5 && sudo setxkbmap -display :1 fr & /bin/sudo -u user PULSE_SERVER=unix:/run/user/1000/pulse/native bash -c 'sudo xhost + local:;/usr/bin/i3'

Revision #18

Edited on
2023-10-19
Edited by user
neowutran
pci=$(lspci | grep "VGA" | grep "NVIDIA|AMD/ATI" | cut -d " " -f 1 | cut -d ":" -f 2 | cut -d "." -f 1 | cut -d "0" -f 2) pci=$(lspci | grep "VGA" | grep -E "NVIDIA|AMD/ATI" | cut -d " " -f 1 | cut -d ":" -f 2 | cut -d "." -f 1 | cut -d "0" -f 2)

Revision #17

Edited on
2023-10-06
Edited by user
neowutran
Note: You do not need to pass the audio part of the GPU

Revision #16

Edited on
2023-10-01
Edited by user
neowutran
# Patching stubdom-linux-rootfs.gz # Configuring the parameter "max-ram-below-4g"
------------------- [details="Method 1: Patching stubdom-linux-rootfs.gz"]
Note that this will apply the change to the HVM with a name starting with "gpu\_". So you need to name your gaming HVM "gpu_SOMETHING". # Alternative without patching stubdom-linux-rootfs Note that this will apply the change to the HVM with a name starting with "gpu\_". So you need to name your gaming HVM "gpu_SOMETHING". [/details] [details="Method 2: Patching xen.xml"]
I haven’t personnally tested this alternative but it should work and some users reported that it work. This method is less tested than patching stubdom-linux-rootfs, so I recommend patching stubdom-linux-rootfs. I haven’t personnally tested this alternative but it should work and some users reported that it work. This method is less tested than patching stubdom-linux-rootfs, so I recommend patching stubdom-linux-rootfs. [/details]

Revision #15

Edited on
2023-09-28
Edited by user
neowutran
**For some GPU this value need to be "2G". It is not currently well understood why the value need to be exactly "2G" or exactly "3.5G", or maybe some other values for other GPU/configuration we never saw yet. More investigation is required to understand what is going on with this parameter.** **For some GPU this value need to be "2G" (discovered here: https://forum.qubes-os.org/t/quick-howto-gpu-passthrough-with-lots-of-ram/21156). It is not currently well understood why the value need to be exactly "2G" or exactly "3.5G", or maybe some other values for other GPU/configuration we never saw yet. More investigation is required to understand what is going on with this parameter.**

Revision #14

Edited on
2023-09-28
Edited by user
neowutran
**Important note: Below, we set the "max-ram-below-4g" parameter to "3.5G".** **For some GPU this value need to be "2G". It is not currently well understood why the value need to be exactly "2G" or exactly "3.5G", or maybe some other values for other GPU/configuration we never saw yet. More investigation is required to understand what is going on with this parameter.** -------------------
- @xvrhthxn

Revision #13

Edited on
2023-09-22
Edited by user
neowutran
- A dedicated GPU. By dedicated, it means: it is a secondary GPU, not the GPU used to display dom0. In 2023, 'Nvidia' and 'Amd' GPU work. Not tested with Intel GPUs. - A dedicated GPU. By dedicated, it means: it is a secondary GPU, not the GPU used to display dom0. In 2023, 'Nvidia' and 'Amd' GPU work. Not tested with Intel GPUs. External GPU using thunderbolt work (https://forum.qubes-os.org/t/create-a-gaming-hvm/19000/8)

Revision #12

Edited on
2023-08-31
Edited by user
neowutran
![pipewire|690x168](upload://rBE1PzenJrSoiAB3hcvCgSapeCv.png)

Revision #11

Edited on
2023-08-31
Edited by user
neowutran
# Alternative without patching stubdom-linux-rootfs Instead of patching stubdom-linux-rootfs, you could inject the command directly inside the configuration template. It is the file "core-admin/ "templates/libvirt/xen.xml" in the "qubes-core-admin" repository. In dom0 this file is in "/usr/share/qubes/templates/libvirt/xen.xml" See below the part that have been modified to add the needed "max-ram-below-4g" option. ``` <!-- server_ip is the address of stubdomain. It hosts it's own DNS server. --> <emulator {% if vm.features.check_with_template('linux-stubdom', True) %} type="stubdom-linux" {% else %} type="stubdom" {% endif %} {% if vm.netvm %} {% if vm.features.check_with_template('linux-stubdom', True) %} cmdline="-qubes-net:client_ip={{ vm.ip -}} ,dns_0={{ vm.dns[0] -}} ,dns_1={{ vm.dns[1] -}} ,gw={{ vm.netvm.gateway -}} ,netmask={{ vm.netmask }} -machine xenfv,max-ram-below-4g=3.5G" {% else %} cmdline="-net lwip,client_ip={{ vm.ip -}} ,server_ip={{ vm.dns[1] -}} ,dns={{ vm.dns[0] -}} ,gw={{ vm.netvm.gateway -}} ,netmask={{ vm.netmask }} -machine xenfv,max-ram-below-4g=3.5G" {% endif %} {% else %} cmdline="-machine xenfv,max-ram-below-4g=3.5G" {% endif %} ``` I haven’t personnally tested this alternative but it should work and some users reported that it work. This method is less tested than patching stubdom-linux-rootfs, so I recommend patching stubdom-linux-rootfs.

Revision #10

Edited on
2023-08-31
Edited by user
neowutran

Revision #9

Edited on
2023-08-30
Edited by user
neowutran
Then find the devices id for your secondary GPU. In my case, it is `dom0:0a_00.0`{.text} and `dom0:0a_00.1`{.text}. Edit /etc/default/grub, and add the PCI hiding. Then find the devices id for your secondary GPU. In my case, it is `dom0:0a_00.0` and `dom0:0a_00.1`. Edit /etc/default/grub, and add the PCI hiding.
If you are using UEFI, the file to override with `grub2-mkconfig`{.text} is `/boot/efi/EFI/qubes/grub.cfg`{.text}. If you are using UEFI, the file to override with `grub2-mkconfig` is `/boot/efi/EFI/qubes/grub.cfg`.
Once you have rebooted, in dom0, type `sudo lspci -vvn`{.bash}, you should see "Kernel driver in use: pciback" for the GPU you just hide. Once you have rebooted, in dom0, type `sudo lspci -vvn`, you should see "Kernel driver in use: pciback" for the GPU you just hide.
For Linux guests, run 'sudo dmesg' to have all the kernel log indicating you if there is a issue with your GPU driver. For some hardware, the MSI calls won't work. You can work around that using for example `pci=nomsi`{.text} or `NVreg_EnableMSI=0`{.text} or something else. Check your drivers options. Check if alternative drivers exist (amdgpu, nvidia, nouveau, nvidia-open, using drivers from the official website, ...). Check multiple kernel version. For Linux guests, run 'sudo dmesg' to have all the kernel log indicating you if there is a issue with your GPU driver. For some hardware, the MSI calls won't work. You can work around that using for example `pci=nomsi` or `NVreg_EnableMSI=0` or something else. Check your drivers options. Check if alternative drivers exist (amdgpu, nvidia, nouveau, nvidia-open, using drivers from the official website, ...). Check multiple kernel version.
Now Xorg and Pulseaudio. From XKCD: Now Xorg. From XKCD:
- A pulseaudio gui client
**archlinux version:**
apt install xserver-xorg-input-kbd xserver-xorg-input-libinput xserver-xorg-input-mouse pavucontrol i3 pacman -S xorg i3
**debian version:** ``` bash apt install xserver-xorg-input-kbd xserver-xorg-input-libinput xserver-xorg-input-mouse i3 ```
# Start the Xorg server for the X screen number 1. # The X screen n°0 is already used for QubesOS integration sudo startx "$binary" -- :1 -config /home/user/AOC.conf ``` [details="Deprecated: old way of doing it"] #!/bin/bash binary=${1:?binary required} # Find the correct BusID of the AMD GPU, then set it in the Xorg configuration file pci=$(lspci | grep "VGA" | grep "NVIDIA|AMD/ATI" | cut -d " " -f 1 | cut -d ":" -f 2 | cut -d "." -f 1 | cut -d "0" -f 2) sed -i "s/PCI:0:[0-9]:0/PCI:0:$pci:0/g" /home/user/AOC.conf
setxkbmap fr sudo setxkbmap fr
[/details] ## Audio - Delete any packages related to pulseaudio and install pipewire ``` sudo pacman -Rdd qubes-vm-pulseaudio pulseaudio sudo pacman -S pipewire-{jack,alsa,pulse} pipewire-qubes
## Pulseaudio - Enable the "pipewire" service for this qube using Qubes Manager - Create a script to launch your favorite Windows Manager and force it to use a specific pulse server. Example "i3.sh": ``` #!/bin/bash setxkbmap fr /bin/sudo -u user PULSE_SERVER=unix:/run/user/1000/pulse/native /usr/bin/i3 ``` And launch it: ``` bash sudo ./xorgX1.sh ./i3.sh ``` [details="Deprecated: old way of doing it"]
The sound was buggy/laggy on my computer. So tried to find a workaround by playing with pulseaudio settings. It was more or less random tries, so I can't really explain it: In `/etc/pulse/daemon.conf`{.text} add the following lines: The sound was buggy/laggy on my computer. So tried to find a workaround by playing with pulseaudio settings. It was more or less random tries, so I can't really explain it: In `/etc/pulse/daemon.conf` add the following lines:
``` ``` [/details]
[/details] [details="Contributors"] - @neowutran - @deeplow

Revision #8

Edited on
2023-08-30
Edited by user
neowutran

Revision #7

Edited on
2023-08-30
Edited by user
system
Edit reason
downloaded local copies of images
[![image](https://imgs.xkcd.com/comics/x11.png "X11 Satisfaction increases as time since dealing with X11 passes.")](https://xkcd.com/963/) [![image](upload://cIbsmjKaevmY02rfuIRAzaSBghO.png "X11 Satisfaction increases as time since dealing with X11 passes.")](https://xkcd.com/963/)

Revision #6

Edited on
2023-08-30
Edited by user
gonzalo-bulnes
[![image](x11){width="\\linewidth"}](https://xkcd.com/963/) [![image](https://imgs.xkcd.com/comics/x11.png "X11 Satisfaction increases as time since dealing with X11 passes.")](https://xkcd.com/963/)

Revision #5

Edited on
2023-08-14
Edited by user
deeplow

Revision #4

Edited on
2023-06-03
Edited by user
deeplow

Revision #3

Edited on
2023-06-02
Edited by user
taradiddles

Revision #2

Edited on
2023-06-02
Edited by user
taradiddles