GPU-Accelerated Containers on Windows

A while ago I have already written a side note on external GPUs, but what could be a real game changer is Nvidia Container Toolkit support in the Windows Subsystem for Linux 2 which available both on Windows 10 (Insider Preview) and 11. I recently got a Thinkpad P16 with an Nvidia Quadro RTX A4500 and Windows. While I am a fan of Linux, I also need proprietary software like Office, Visio, Adobe, etc., so if I could run GPU containers on Windows that would be ideal. I have been following GPU (para)virtualization for about a decade now and it seems this finally becomes viable without major type 1 hypervisor setups. So without further ado, let’s glance over what I did. I didn’t keep an exact history, but it must have been quite similar to this:

My Windows 10 is part of the insider preview. I installed an up-to-date driver from Nvidia and then the Windows Subsystem for Linux 2 (WSL2):

wsl.exe --install
wsl.exe -l -o
wsl.exe --update

Then in WSL Ubuntu I followed the user guide instructions:

sudo apt-key del 7fa2af80
wget https://developer.download.nvidia.com/compute/cuda/repos/wsl-ubuntu/x86_64/cuda-wsl-ubuntu.pin
sudo mv cuda-wsl-ubuntu.pin /etc/apt/preferences.d/cuda-repository-pin-600
wget https://developer.download.nvidia.com/compute/cuda/12.0.0/local_installers/cuda-repo-wsl-ubuntu-12-0-local_12.0.0-1_amd64.deb
sudo dpkg -i cuda-repo-wsl-ubuntu-12-0-local_12.0.0-1_amd64.deb
sudo cp /var/cuda-repo-wsl-ubuntu-12-0-local/cuda-*-keyring.gpg /usr/share/keyrings/
sudo apt-get update
sudo apt-get -y install cuda

Afterwards, I installed Docker Desktop selecting WSL 2 as the backend and I was able to run docker run -it --rm --gpus all ubuntu nvidia-smi from Windows Terminal which yielded the expected result:

And if we launch an Ubuntu terminal and try the same, it works just as well including seamlessly using the cached image:

If this approach holds water (I now have to try remote interpreters etc.), it might become my default way of working, since it allows me to use develop with Linux and Windows in parallel without the dual boot trick I oftentimes have to resort to, i.e., having an external SSD with Linux and an internal one with Windows. Exciting.