DKVM has been my only hypervisor for the last few weeks. My normal workstation has been moved into a VM (xubuntu), and when I feel like gaming - I can fire up my Windows 10 VM.
The boot-up time isn't that bad from a USB stick. As the DKVM OS is using a run-from-ram method, the performance once booted is excellent.
While several improvements has been pushed to my github repository for DKVM, the majority of the work has gone into a new menu-system
I decided to add BASH to my Alpine setup, and build a simple menu-structure using ncurses dialog for BASH.
The menu is far from complete, but currently is a usable state for me.
The add/edit menu options still needs a lot more work, and the internal structure could be a lot better.
The menu simply searches for "dkvm_vmconfig.*" files, and displays a simple menu with the found VMs.
Kernel 4.18.5
As I needed to rebuild the kernel anyway to get HUGETLBFS support, I decided to use the latest stable kernel. The easiest solution for me, was to clone the official AlpineLinux aport repository, and add the modifications to my own fork.
This enables me to easily keep up with the official fixes, while still preserving my own updated kernel with an updated kernel-config optimized for virtualization.
The DKVM buildsystem now has a "stage03" - which is compiling my updated kernel in a chrooted environment.
Build demo
As this is still mainly for my own use, and should be considered to be somewhere in the early alpha/development stages - making sense of the collection of shell-scripts can be troublesome.
So, I decided to record a video of the build-process - in case anyone wants to follow in the same footsteps.
In my DKVM setup, I always use my 2nd graphicscard for my guests. Sadly my monitor does not have automatic input detection, so a little trickery is needed to switch input.
ddccontrol -p is used for probing the I2C bus for displays.
To find the information needed, first find the I2C bus for the display - it should look similar to this
In my case, my monitor is in dev:/dev/i2c-17
Finding the needed i2c address and values.
Still looking at the output from "ddccontrol -p", find a section like "Input settings"
In my case, the i2c address for setting inputsource is 0x60, and the accepted values are
15: DisplayPort
3: DVI
4: HDMI
1: VGA
Writing to the i2c address
To set another display input, use the information gathered, and use "-w" to write
example : ddccontrol -r 0x60 -w 15 dev:/dev/i2c-17
This will set the input source to DisplayPort
Wrapping it up
In my case, I need to set the input to DisplayPort when my Linux guest is starting, and set it back to DVI when I am done.
While there are plenty of ways to run scripts in startup/shutdown, I opted for the easy solution - and just went with an old-fashioned OpenRC init style script, and placed symlinks to the different rc levels.
/etc/init.d/ddccontrol
#!/bin/sh
case "$1" in
start)
/usr/bin/ddccontrol -r 0x60 -w 15 dev:/dev/i2c-17
;;
stop)
/usr/bin/ddccontrol -r 0x60 -w 3 dev:/dev/i2c-17
esac
I have never really liked the idea of having Windows running native on my machines - so for about the last year QEMU/KVM + VFIO has been my solution to running Windows applications.
While many others are currently using QEMU/KVM + VFIO Passthrough to run Windows at near native speed - I was unable to find a dedicated "KVM OS" for my needs.
My "daily-driver" is Linux - and, I wanted an option to allocate more resources to the Windows VM, without scarifying resources to my main Linux OS.
A way to accomplish this, is to have both the "daily driver" and the "gaming os" being virtualized, and not running at the same time.
So, my journey began to find a KVM hypervisor OS focused on running desktop VM's.
My KVM Desktop OS whishes
VFIO support
Be able to use my one "good" graphics-card for both Linux and Windows
Run-From-RAM style OS.
Boot from USB stick
Minimal memory foot-print
Support for software raid (preferable mdadm)
Support for bcache
The failed attempts
I gave oVirt a go, which honestly is quite nice. But, making this a Run-From-RAM style OS gave me quite some issues. A lot of tweaking was required - and maintaining all my tweaks has too much of a hazzle.
VMware ESXi recently has good pass-through support, and qorks really well booting from a USB stick (And it is already a Run-From-RAM style OS). Sadly ESXi has no support for software raid, and the SSD cache functionality is limited compared to bcache.
LinuxFromScratch was next in line, and I scriptet most of the build process - and got a fairly good working system. Though, I lacked the "easy maintenance" - and put the code on the shelf for now.
DKVM - My Desktop KVM OS
Lastly, I have AlpineLinux a go - as this has builtin support for Run-From-RAM style booting - including booting from a USB stick.
In short, I use QEMU to build a AlpineLinux bootable image, which can be flashed to a USB stick.
As of 3.8.0 release of AlpineLinux, both UEFI and Legacy booting now works out-of-the-box.
The build process
The setup.sh script is a wrapper for starting the build process - and is also creating a new empty disk-image file.
Here we will also be creating two addition ISO's used for stage01 and stage02.
stage01
Here we setup the base of the operating system.
I have chosen to include an "answer.txt" file for the alpine-setup program. This allowed the setup to run non-interactive.
The result of stage01 is a bootable disk image in a Run-From-RAM style
stage02
Here we setup the more custom part of the OS. Basic configuration like support for saving changes to USB disk is done here - among tweaks to varies aspects of the OS
We also do things like setting up both UEFI and Legacy bootloaders with vfio-pci.ids, and other KVM optimization arguments.
In my case, I have chosen to go for a setup with CPU isolation to minimize latency inside the running guests.
This is done using the isolcpus, nohz_full and rcu_nocbs arguments to the kernel - in combination with taskset to make affinity-rules for the vCPU threads.
vm_start
When the DKVM OS is loaded, i am presented with a simple menu to start my configured VMs.
At the time of writing, I only have two VM's configured - but I would imagine that will change.
This is done fairly simple with a wrapper-script around QEMU.
vCPU affinity and isolation
Isolation is done using isolcpus - this will ensure no other threads are running on those cores.
The affinity is done using QEMU's builtin QMP system. This allowed the user to query for the threads used by QEMU to serve the vCPUs.
NOTE: In my case, my hyperthreaded cores is interleaved with the physical cores when looking at the processor ID. This can be checked using something similar to:
# cat /proc/cpuinfo | grep "core id\|processor"
The CORELIST and isolcpu will need to reflect this.
All of the sources are available at my github - though, keep in mind this is written for personal use only!
Any recommendations/patches are welcome though :)
Other useful information / links
Linux and Wendell teamed up to test gaming on Linux. The different options are all quite interesting.
Having the need to run a few Windows applications in my Lenovo Carbon X1 - but unwillingly to dual-boot or in other ways run Windows natively, I decided to give VirtualBox a go.
My laptop came with Windows pre-installed, and is using the SLIC table as identification for activation of Windows.
Host OS: Ubuntu (Usually following the LTS releases)
Guest OS: Windows 10.
Getting SLIC information
In any recent Linux distribution, it is fairly easy to retrieve such information from the DMI tables.
The SLIC table is used for identification when activation Windows in combination with a a product-key and a SLIC certificate. As with my Lenovo laptop, this product-key was embedded in the MSDM table - and can easily be extracted under Linux.
# Get product key from MSDM table ###########################
In VirtualBox, this can be done using "vboxmanage setextradata" to configure "VBoxInternal/Devices/pcbios/0/Config"
Activating Windows
To use the SLIC and productkey table for activation, we need a SLIC certificate. If you already have Windows installed, there is ways to retrieve the same information - including the certificate using a SLIC_ToolKit.
