Install Nvidia Container Toolkit

The NVIDIA Container Toolkit allows users to build and run GPU accelerated Docker containers. The toolkit includes a container runtime library and utilities to automatically configure containers to leverage NVIDIA GPUs.

I was only really interested in this because it was a requirement for me using my Nvidia graphics card for a Stable Diffussion Docker container.

Steps

Run the commands below to install the Nvidia container toolkit on Ubuntu or Debian systems. I have tested and run this on Ubuntu 20.04, and I'm pretty certain it works on Ubuntu 22.04.

#!/bin/bash

# Add the nvidia repository so we can fetch packages signed by nvidia.
distribution=$(. /etc/os-release;echo $ID$VERSION_ID) \
      && curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
      && curl -s -L https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.list | \
            sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
            sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

# Now install the nvidia container toolkit
sudo apt-get update
sudo apt-get install -y nvidia-container-toolkit nvidia-docker2

# Run the nvidia command from the toolkit to configure the docker runtime and restart docker.
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

Testing

You will want to test that it is working. You can do this by running:

sudo docker run \
  --rm \
  --runtime=nvidia \
  --gpus all \
  nvidia/cuda:11.6.2-base-ubuntu20.04 nvidia-smi

You should get some output similar to below:

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.199.02   Driver Version: 470.199.02   CUDA Version: 11.6     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce ...  Off  | 00000000:05:00.0  On |                  N/A |
|  0%   49C    P8    16W / 170W |     58MiB / 12050MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
+-----------------------------------------------------------------------------+

If you got an error message instead, please try rebooting! I had understood that restarting the Docker daemon earlier should have done the job, but when I first ran this I got an error message that "resolved itself" by simply rebooting the computer.