cloudymax / scrap-metal

Notes for creating GPU accelerated VMs using only FOSS tools

Scrap Metal

My notes on setting up GPU accelerated virtual machines using only FOSS tools.

Subjects:

• Free and Open-Source
• Seamless provisioning via cloud-init
• Compatible with Azure/AWS/GCP and other clouds
• VM creation from LiveUSB/ISO images
• Static IP address assignment via Tap/Tun networking
• PCI-e/iommu pass-through
• GPU acceleration
• VNC and RDP support

Host OS Support

Built to run on X86 AMD64 Ubuntu/Debian Server host machines that have been pre-provisioned with a tools like Pxeless, Cloud-Init, Ansible etc...

Guest OS support

• Ubuntu and Debian Linux guests created from cloud images

• Other Linux distros supported via ISO/LiveUSB install.

• Windows guests installed from ISO

• MacOS guest support is enabled via MacOS-KVM.

Why QEMU?

QEMU is an open source machine emulator and virtualizer. It can be used for system emulation, where it provides a virtual model of an entire machine to run a guest OS or it may work with a another hypervisor like KVM or Xen. QEMU can also provide user mode emulation, where QEMU can launch processes compiled for one CPU on another CPU via emulation.

QEMU is special amongst its counterparts for a couple important reasons:

• Like ESXi, its capable of PCI passthrough for GPUs (VirtualBox cant help us here)
• Unlike ESXi, it's free
• It's multi-platform
• It's fast - not as fast as LXD, FireCracker, or Cloud-Hypervisor (formerly NEMU), but its far more mature and thoroughly documented.
• Unlike a system container or Multipass it can create windows hosts
• Unlike Firecracker it supports pinning memmory addresses where firecracker cannot because it would break their core feature of over-subscription.

These qualities make QEMU well-suited for those seeking a general-purpose hypervisor running the first layer of virtualization. For maximum speed or density though, you should consider if the lighter, but less generalized LXD, Firecracker, or Cloud-Hypervisor better suits your needs.

Disclaimers and Warnings

• There are system-specific kernel modules that must be in-place for features like IOMMU/VirtIO passthrough to work properly. While non-accelerated Linux/Windows guests will work without these steps, they are a hard requirement for MacOS and GPU-enabled guests.

• Support for other Debian-Based distros on the host is a W.I.P and blocked by pre-seed support.

• GPU passthrough is supported for Intel CPU's and Nvidia GPU's ONLY. This is because I don't have any AMD hardware, not because it isnt possible.

• GPU Acceleration relies on X11vnc or Nvidia Container Runtime. This means you need a screen, or monitor stub attached to the host machine. Laptops that use Nvidia Optimus or Prime don't need to worry about this as theres a monitor hard-wired into your GPU anyway.

• Support for the process for preparing a Host for GPU-passthrough is best-effort only. There are garunteed to be issues across hardware models and vendors. To minimize the chances of misconfiguration follow the full-process of re-imaging your host with the supported ISO.

Helper Scripts

The TUI and CLI are being created to wrap the multitude of small helper scripts and functions needed to make your Virtual machines work.

latest-kernel.sh: downloads the latest ubuntu mainline kernel to the /tmp/new_kernel directory

bridge.sh: documents the full process for creating a bridged network and tap interface and the needed IPtables rules.

bridge.conf: file to allow netwok traffic over the bridge

ip-tables.sh: the required IPtables rules to allow bridged traffic

tap.sh: script to create a tap interface

netplan config: bridge host netplan config

netplan config: dynamic IP guest config

netplan config: static ip guest config

governor.sh: script to control CPU power states

vmhost.sh: get the PCI IDs of the GPU and alter grub and other config files to enable pass- through

Other projects to check out:

• Quickemu After using scrap-metal to get your host configured, use quick-emu to launch all kinds of virtual machines, not just ubuntu and debian server.
• cannoli Use Cannoli to profile your QEMU virtual machines to identity performance issues in code.
• multipass Multipass is a cross-platform VMM that can get you to a linux environment from anywhere. Especially useful since it has great support for ARM64 and supports cloud-init. No GPU passthrough possible though.
• Cloud Hypervisor Intel's spin off of QEMU, this porject (formerly called NEMU) is based on the Rust VMM just like Amazon's Firecracker but it supports PCI passthrough and other useful features that firecracker cannot accomodate. Cloud Hypervisor also powers the Kubevirt project.
• Metal3 The Metal³ project (pronounced: “Metal Kubed”) provides components for bare metal host management with Kubernetes. You can enrol your bare metal machines, provision operating system images, and then, if you like, deploy Kubernetes clusters to them.

Tunnels

rdp tunnel over ssh:

 ssh -L 3389:10.0.2.15:3389 176.9.44.19 -p23 -l max -N
 ssh -L 3389:<vm-private-ip>:3389 <host-ip> -p<vm-ssh-port> -l max -N

Roadmap

• Support for LookingGlass
• GPU Sharding via LibVF.IO integration
• Intel iGPU acceleration for non-GPU enabled hosts
• VNC security options
• Wireguard networking (WiP)

scrap-metal's Issues

Windows VDI image

Docs site

GPU splitting w/ LibVF.io

https://github.com/Arc-Compute/LibVF.IO

MacOS VDI image

expose host forwarding

user,id=network$NETWORK_NUMBER,hostfwd=tcp::"${VM_SSH_PORT}"-:"${HOST_SSH_PORT}",hostfwd=tcp::80-:8080 \\"

Core Pinning

Linux VDI Image