I have long bothered by the issue [https://github.com/QubesOS/qubes-issues/issues/7340] . marmarek recently mentioned that I can debug xen using a USB 3.0 debug cable. I have a hard time trying to make it work, but now I have succeeded. I have finally reach the point where I can debug the xen on another machine - using USB 3.0 console. Now I record the successful setup here. Debugee is Thinkpad L15 Gen 2, debug host is Thinkpad T440p. Here is the difficulties and notes that the earlier post of marmarek [https://github.com/QubesOS/qubes-issues/issues/6834#issuecomment-1296221396] may not mention: 1. A USB 3.0 debug cable (for example the cable for WinDBG) is required and DIY will mostly fail as most USB A-A cable is 123456789 <-> 123456789 whereas a USB debug cable should be 123456789 <-> zzz489756. 2. In order to make debug environment controllable and stable, one can use LiveCD Ubuntu 22.04 as debug host. 3. Append usbcore.autosuspend=-1 to the debug host kernel command line (I am not sure whether this is needed) 4. Make sure that your debug host has at least one USB3.0 plug. 5. If USB 3.0 debug does not work, it is a good idea to debug on a LiveCD Ubuntu 22.04 as it has every software and kernel module to let it work definitely. Make sure your machine has at least 2 USB 3.0 port, connect your debug cable on one end, "look for /sys/bus/pci/devices/*/dbc file" (you should go to the correct device and change the asterisk to that device number, for example 00:14.0), and then write "enable" to the dbc file (echo enable &gt; dbc). Then you plug the other end onto a different USB 3.0 port. The debug info should appear in dmesg and /dev/ttyUSB0 /dev/ttyDBC0 should appear at your /dev directory. If this does not work, most likely your hardware has some problem. 6. In ubuntu 22.04, load usb_debug module and install picocom 

sudo modprobe usb_debug
sudo apt update
sudo apt install picocom
7. The usb_debug seems not to work when host plugs first and debugee pauses later - When debugee pauses, plug the usb debug cable to host. sudo dmesg -w can be useful in both debug host side and debug guest side. If you manages to plug a debugee to debug host, on debug host side, the debugee shows up as a USB device: 
[12107.070400] usb 4-2: new SuperSpeed USB device number 2 using xhci_hcd
[12107.090923] usb 4-2: LPM exit latency is zeroed, disabling LPM.
[12107.091160] usb 4-2: New USB device found, idVendor=1d6b, idProduct=0010, bcdDevice= 0.00
[12107.091162] usb 4-2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[12107.091164] usb 4-2: Product: Debug console
[12107.091166] usb 4-2: Manufacturer: Xen
[12107.091167] usb 4-2: SerialNumber: 0
so whenever the debug host side has any problem about usb, you can catch it immediately. 8. xen boot command: remove console=none and add dbgp=xhci@pci00:14.0,share=yes console=vga,xhci loglvl=all guest_loglvl=all (if your debugee USB 3.0 is at 00:14.0). You can edit the command at grub2 boot menu.

  1. When you edited the command and attempt booting, xen will pause until debug host is ready. This command is recommended in debug host, so that tthe console will pop out when the debugee is ready: 
    while true; do if [ -e /dev/ttyUSB0 ]; then sudo picocom -b 115200 /dev/ttyUSB0; fi; sleep 1;done
    Connect the two computers with the USB 3.0 debug cable. You can see that (1) a Debug console device appears in sudo dmesg -w; (2) xen debug info appears in picocom; (3) debugee boots up.
  2. You should be very careful when xen is accepting the console input - type 'h' to see why (it is help). If you typed 'r' accidentally, the computer will hard reset and your hard disk will hurt.
  3. When Qubes OS fully boots, dom0 log in prompt will appear in the console. You can log in and run sudo dmesg -w. It will write dom0 log persistently to the console.
  4. Triple ctrl-a will switch input between xen and dom0. When you are using picocom it is typing ctrl-a 6 times since ctrl-a is also escape command for picocom by default.
  5. When you are doing suspend-resume experiments, the USB 3.0 debug device will frequently power off and power on. When you resume, the device may not work. Unplug and plug each time will work. If that works, here is another trick that also works and does not hurt your USB 3.0 plug: 
    root@ubuntu:/sys/bus/pci/drivers/xhci_hcd# echo 0000:00:14.0 &gt; unbind
root@ubuntu:/sys/bus/pci/drivers/xhci_hcd# echo 0000:00:14.0 &gt; bind
root@ubuntu:/sys/bus/pci/drivers/xhci_hcd#