Qubes on a Legion 5 Pro with Windows and Linux Gaming VMs with GPU Passthrough

Original forum link: https://forum.qubes-os.org/t/31963
Original poster: car
Created at: 2025-02-03 18:23:16
Posts count: 1
Likes count: 1
Tags: configuration

I’m just documenting how I got Qubes working on my Legion 5 Pro for work, personal and when I can some game time. This laptop is plugged in 24x7, and basically acts as a desktop these days. Some other considerations for my particular needs: First, I’m willing to compromise on some things, ex installing windows tools for performance. Second, I’m not trying to squeeze every ms, or 1000 FPS from this machine. Third, my daily driver will be Qubes used in its traditional way, with standalone VMs only spun up for specific use cases, ex. Gaming.

I tried Qubes on this machine in the past but it just wasn’t stable and easy enough at the time. Fast forward a couple of years and I thought I’d give it another try. Fortunately, this time, it was smooth and easy.

I play a large variety of games, including FPS (COD/Fortnite), some driving and flight simulators and if I have more time, I'll play something like Civ 6 or BG3. These days, however, most of my gaming has moved on to GeForce Now (Ultimate) for its convenience, and its pick up/play anywhere. My goal for the Gaming VMs using GeForce Now is: Linux - 60 FPS with no stutter, highest resolution(GeForce limitations), Windows - Depending on the type of game, I will either prioritize FPS or Quality and max resolution on the monitor. When running local installs our game server latency in general is virtually none (Fortnite shows “0” - 5ms ping in games on local installs….seeing 20ms+ would be very rare.) Our GeForce Now latency is also very low.

This entire setup and testing was run over a couple of days, with 2-3 reinstalls of Qubes and VMs to test the documentation (I think I captured it all).

Setup

Qubes for personal, unauthenticated web, vpn and work.
VM(s) for gaming on Geforce Now, XCloud and Steam. VMS must work with GPU/USB passthrough (should be obvious, but only one VM can be running at a time)
Bonus: VMs for my kids to use with GPU passthrough so that they can game on them with different ids or different games and don’t break my stuff.

Qubes and VM must support

Qubes:

Must work on the laptop screen and second monitor and can switch to it at any time with primary keyboard/mouse support even when VM is running and switch back.
Qubes can still be accessed through keyboard/trackpad and monitor display on the laptop when VM takes over primary mouse/keyboard and monitor.

VM:

GeForce Now - COD (Controller), Fortnite (KBM), Baldur’s Gate 3 (KBM)
XCloud - COD (Controller), Fortnite (Controller) (sometimes use it for backup or games that are not on GeForce Now). Now also supports KBM.
Local Steam install - Civilization 6 - Used for Benchmarking
Youtube - 4K, full screen, no stutters
Switching keyboard/mouse and monitor between Qubes and VM works without breaking anything.

Hardware specs

Legion 5 Pro

CPU: R7-6800H 32 GB DDR5 Dual Boot with 1x NVME (Samsung from Lenovo. Similar/same specs to 980 Pro) 1 TB Drive 100% dedicated to Qubes and Grub 1x NVME dedicated to Windows and Windows Boot Manager for Bare Metal installs. RTX 3070Ti Laptop Using Ethernet for network USB C ports: USB C 1 -> iGPU USB C 2 -> dGPU USB C 3 -> dGPU

Monitor

3440x1440 max 144Hz 2xHDMI (100Hz) 1x Display Port (144Hz) 1x USB C DP (144Hz) KVM Switch built in

I also tested with a second monitor, 240Hz, with external KVM Switch since most won’t have a monitor with one built in.

KVM Switch has the following connected: * 2x Lightspeed Adapters (keyboard/mouse) * 1x Gaming Controller * 1x Camera

Monitor with KVM switch built in has: USB C 1 (iGPU) -> Monitor DP for Qubes Any USB A port for Qubes -> Monitor USB A connection USB C 2 (dGPU) -> Monitor USB C for VMs

Monitor with no KVM switch built in USB C 1 (iGPU) -> Monitor DP for Qubes (same as above) Any USB A port for Qubes -> KVM 1 USB USB C (dGPU) - > hub with USB A -> KVM 2 USB HDMI (dGPU) -> Monitor HDMI for VMs

Installing Qubes:

Install Qubes normally ensuring that you use a separate sys-usb. First install I tried to install KDE, and I think this broke a number of things so I have now kept it default and everything is smooth.

Hiding Nvidia from Qubes

To use Nvidia, open up Qubes Manager, select a qube and look at the devices. You’ll see all the devices you have. Can also use lspci in dom0 terminal.

Hopefully the Nvidia card and Nvidia Audio are isolated from everything else.

Assuming they are, you need to hide them and blacklist nouveau in grub, rebuild, restart. A little messing around and this is what I have.

Edit /etc/default/grub

GRUB_CMDLINE_LINUX="rd.luks.uuid=luks-5d98748c-6c22-4f7f-806e-b99f0bbd77d5 rd.lvm.lv=qubes_dom0/root rd.lvm.lv=qubes_dom0/swap plymouth.ignore-serial-consoles 6.10.10-1.qubes.fc37.x86_64 x86_64 rhgb quiet rd.qubes.hide_pci=01:00.0 01:00.1 nouveau.modeset=0 rd.driver.blacklist=nouveau video=vesa:off"

grub2-mkconfig -o /boot/grub2/grub.cfg reboot

Next, I needed to find the USB that is tied to the Nvidia card and not to anything else. This took a little trial and error. First, I plugged something into all the ports. I then removed each USB one at a time from sys-usb (stop/start) using the Qubes Manager to see what would disappear. On my laptop, removing some of the USBs made the keyboard or some other peripheral inactive that I needed. It turned out that I could remove the USB C on the back, and that would keep the trackpad and keyboard on the laptop functional. Fortunately that was all I needed as it’s tied directly to the Nvidia card and my monitor supports USB C DP (in my case, it was USB 6). Remove it from sys-usb permanently.

I then went ahead and installed the following:

Windows 11 Ubuntu (Debian distro) (24.04) Debian Bookworm Endeavouros (Arch distro) (latest release) Fedora and Nobara (41 based) Tumbleweed (latest release)

Each VM had more or less identical specs 100GB (120 for Windows) storage drive, 12GB private space 10 VCPUs 16GB Mem (initial/max. I unselect “Include in memory balancing” Nvidia and USB devices passed through to the VM. Firewall always default For Linux distros, Always switched to use ext4 For Windows, tried both with and without tooling. All were installed with latest downloads from their respective sites Used the default WM for the Linux distros. Latest Nvidia drivers installed from their repos or for Windows, directly from Nvidia Youtube 4K + sound Steam installed from repos or website (Windows) XBox Elite Controller PS 5 Dualsense Controller

Windows 11

Installed with ISO .
Follow parts of https://www.qubes-os.org/doc/templates/windows/windows-qubes-4-1/
At a minimum, this must be done
Shift F10-> regedit -> KEY_LOCAL_MACHINE\SYSTEM\Setup.
create the key LabConfig.
Under LabConfig create 3 DWORD values called BypassTPMCheck, BypassSecureBootCheck and BypassRAMCheck(can probably skip) and set each value to 1
Finish installing Windows
Nvidia default driver is already installed with Windows so monitor works but it’s choppy. Had multiple installs with update Game Ready driver that quickly failed with Nvidia error code 43 so tried Studio Driver which seemed to work. The error code may be unrelated. After installing Qubes again and not trying to install KDE, or messing with anything not documented in this setup, have not seen it again. I do clone the Windows Qube before every major step just in case.
Install Studio Driver
Disable the basic display adapter to stop Windows using the qube monitor. Don’t need it.
Reboot
Everything should work with Windows on the external monitor. The Qube monitor display will have a Windows logo and spinner that’s stopped but the external monitor should be fine.
Display should feel more responsive than with the default driver
If everything worked, recommend to clone the Qube in Qube manager.
Install GeForce Now, everything works can change to 120 FPS, Reflex, 1 etc.
Network is very slow, 20-50Mbs on 1Gb connection
Crystal Disk Mark speed will be very slow (For testing, assigned second drive controller in the qube manager devices to get hold of partition. Drive Speed was basically 100% when tested through CDM on second drive. 6000MB/s)
Gaming on GeForce Now is poor
XBox Streaming is poor
Internet speed test is poor
XBox Controller connected to Hub passthrough working
PS5 Controller connected to Hub passthrough working
Roblox (extra game my son wanted me to try again) did not work. Detects VM. Solved later in Ubuntu…..

Improve Performance:

Install Windows Tools if comfortable. Include Disk and Network drivers
Network will have to be reconfigured with IP etc from Qubes settings. change to private network
4K Video no stuttering, sound works though controller
GeForce Now is excellent
XBox Streaming is excellent
Internet speed is excellent
Disk speed is better

Windows VM Issues:

Sound Issue over Nvidia. Pointing Windows to laptop or controller sound works fine.
Some mouse/app stuttering, slowness but nothing serious and goes away
To slow for in VM gaming
Have to install tools to improve performance (do at your own risk)
Clone and keep backups of the Qube as some would eventually break the Nvidia driver (Error 43). Have not reproduced on latest installs with Studio driver.

Network and Disk Tests

Without Windows Tools CDM: Read: 214.47 MB/s, Write: 195.56 MB/s Speedtest: DL: 21.17Mbs, UL: 25.61Mbs, Ping: 11ms

With Windows Tools CDM: Read: 1242.61 MB/s, Write: 1203.00 MB/s Speedtest: DL: 785.98Mbs, UL: 935.77Mbs, Ping: 3ms

Linux

All distros that successfully installed were generally out of the box with not a lot of digging if something wasn't working. Many issues are probably a quick google to fix.
All were tested with: All needed manual network entries per the Qubes settings Wayland works on both monitors in most cases out of the box but is choppy and slow Switched to X11. Latest Nvidia drivers installed from the distro repos Ran nvidia-xconfig Tested with Youtube 4K + sound Geforce Now. Used latest Chrome restricted to 1920x1200 Max and 60FPS (Chromium had some issues on some of the distros).
Steam installed from repos Civ 6 with Proton 9. Graphics Benchmark tests (Linux Runtime was not reliable) XBox Elite Controller tested PS 5 Dualsense Controller tested

Ubuntu 24.04 LTS

Have to setup network manually during install
Detects Nvidia and Monitor when starting, black screen during install
Install proprietary network and third party etc
During reboot, when unmounting cdrom, hit enter if it’s waiting
After install/restart, the monitor will be off
Ubuntu can take a minute to actually start and I had to restart it a couple of times and make sure that the monitor was connected to the Ubuntu VM
Nvidia should be detected at the terminal using nvidia-smi or the Nvidia App
In terminal Run sudo nvidia-xconfig
Reboot
Should now see Ubuntu on monitor and not on the Qube monitor. (will just freeze)
4K Youtube + Sound works perfectly.
Install Steam from App Center
Steam needs/recommends
- sudo dpkg --add-architecture i386
- sudo apt update
- sudo apt install libnvidia-gl-550:i386
Civ 6 works out of the box.
Install Chrome
GeForce Now over Chrome is good.
XBox Streaming over Chrome is good
PS5 Controller working, has sound
Xbox Controller working but no headphone
Roblox works using Sober

Issues.

Once in a while it would lose the sound output of the Nvidia (monitor has built in speakers) but reboot would fix. Usually use the output from the controller.

Network and Disk Tests:

KDiskMark: Read:3108 MB/s, Write:2756 MB/s Speedtest: DL: 885Mbs, UL:939Mbs, Ping: 3ms

Debian Bookworm

Install with KDE Nvidia/USB passthrough
Wayland works out of the box. Monitor and usb works. Nvidia shows up
Choppy 4K under Wayland.
Install Nvidia drivers
Trying to disable Wayland broke KDE.

Reinstall with XFCE Mirrors stopped working for 3 retried installs…restart firewall/sys-net and change the hostname seemed to fix it.

Reinstall again: Could not install nvidia drivers. Kept giving error Install gnome to see if that fixes it Does not…and since I have other working distros with no work, moving on.

EndeavourOS

Select option Install with Nvidia
Works out of the box with dual (Wayland) monitors but slow and choppy on actual monitor. 4K video is choppy
Run nvidia-xconfig
Reboot
On Login screen, Switch to X11
Had to add panels back at top/bottom (Right Click, edit -> default panels)
4K Video on Youtube runs perfectly.
Sound works through controller output, no Nvidia output at monitor
Install Chrome
GeForce Now over Chrome is good.
XBox Streaming over Chrome is good.
Xbox Controller working
PS5 Controller working

Network and Disk Tests:

KDiskMark: Read: 2524 MB/s, Write: 3261 MB/s Speedtest: DL: 889Mbs, UL: 938Mbs, Ping: 2ms

Fedora Variants

Fedora forced Wayland after 41 which gave me trouble. Could not get 4K Youtube streaming to work well, and Gaming was really not happening. I’m typically not a Fedora user, so someone can probably do a better job and actually get these working.

Fedora

See Fedora spinner but Boots to black screen
Reboot
Boots to black screen
Moving on

Nobara

Slow to boot but eventually gets there, unlike Fedora
during install -
Configure Network
Filesystem = No Swap + ext4
Both screens will work…this is wayland
Let it update
Reboot
Stuttering video playback, bad sound.
Install latest Nvidia drivers, black screen flashing cursor. Eventually loads
Still slow/stutters
Install Cinnamon
Run nvidia-xconfig
Reboot
4K in FF and Chromium (better) still stutters
Moving on

Tumbleweed

Install

Setup network
Disks - Start with suggested layout
Delete swap
Delete root partition
New partition for /
Change disk to ext4
Disable automatic login
update
Install proprietary drivers
Sudo zypper install openSUSE-repos-NVIDIA
Sudo zypper install nvidia-compute-utils-G06
Reboot
Run nvidia-xconfig
Reboot
Install Chrome,
Geforce Now good
XBox Good
Xbox Controller working
PS5 Controller working

Steam: Tried with Software Center. Steam not known in the Tumbleweed default. *Do not install Steam from Flatpack per some recommendations...** <start> Install Steam from Flatpack. Install Civ 6 use Proton 9.x Play Civ DX11, Let Shaders compile Laggy graphics and quickly Crashed Installed Flatseal Set GPU etc….hung up the VM Removed flatseal, steam and flatpack <end>

Sudo zypper install steam < Works and then install Civ 6

Network and Disk Tests:

KDiskMark: Read: 2584 MB/s, Write: 3163 MB/s Speedtest: DL: 883Mbs, UL: 841Mbs, Ping: 2ms

General Laptop Issues: Sleep/Hibernate not working. Fans spin and battery eventually dies. Laptop screen never fully turns off. Fans are constantly spinning at least on low speed.

Gaming Outcomes

Using the Windows VM, in GeForce Now (KBM), my son and I had five 2nd place finishes in Fortnite Duos Ranked - Champion level out of 10 games. Unfortunately, we could not pull off a win.

Civ 6 Benchmarks

Only tested Graphics and Gathering Storm Graphics benchmarks Resolution - Full Screen 3440x1440 Settings High (I changed nothing under the menus) DX11 For Linux: Proton 9.x Runtime

Windows - Bare Metal for reference Graphics: 120+FPS, Avg FrameTime: 7.852ms Gathering Storm Graphics: 70+FPS, AFT: 12.226

Windows Qube with Tools installed (not playable) Graphics: 20-40FPS, Avg FrameTime: 34.64 Gathering Storm Graphics: 10-12 FPS, AFT: 90.384

Ubuntu: Graphics: 60-110+FPS, Avg FrameTime: 14.146 Gathering Storm Graphics: 40-60FPS, AFT: 23.048

Endeavour: Graphics: 50-110FPS, Avg FrameTime: 15.568 Gathering Storm Graphics: 30-45 FPS, AFT: 25.309

Tumbleweed: Graphics: 50-120+FPS, Avg FrameTime: 12.146 Gathering Storm Graphics: 50-70FPS, AFT: 17.486