Programster's Blog

Tutorials focusing on Linux, programming, and open-source

Install Nvidia Container Toolkit

The NVIDIA Container Toolkit allows users to build and run GPU accelerated Docker containers. The toolkit includes a container runtime library and utilities to automatically configure containers to leverage NVIDIA GPUs.

I was only really interested in this because it was a requirement for me using my Nvidia graphics card for a Stable Diffussion Docker container.

Steps

Run the commands below to install the Nvidia container toolkit on Ubuntu or Debian systems. I have tested and run this on Ubuntu 20.04, and I'm pretty certain it works on Ubuntu 22.04.

#!/bin/bash

# Add the nvidia repository so we can fetch packages signed by nvidia.
distribution=$(. /etc/os-release;echo $ID$VERSION_ID) \
      && curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
      && curl -s -L https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.list | \
            sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
            sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

# Now install the nvidia container toolkit
sudo apt-get update
sudo apt-get install -y nvidia-container-toolkit nvidia-docker2

# Run the nvidia command from the toolkit to configure the docker runtime and restart docker.
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

Testing

You will want to test that it is working. You can do this by running:

sudo docker run \
  --rm \
  --runtime=nvidia \
  --gpus all \
  nvidia/cuda:11.6.2-base-ubuntu20.04 nvidia-smi

You should get some output similar to below:

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.199.02   Driver Version: 470.199.02   CUDA Version: 11.6     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce ...  Off  | 00000000:05:00.0  On |                  N/A |
|  0%   49C    P8    16W / 170W |     58MiB / 12050MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
+-----------------------------------------------------------------------------+

If you got an error message instead, please try rebooting! I had understood that restarting the Docker daemon earlier should have done the job, but when I first ran this I got an error message that "resolved itself" by simply rebooting the computer.

Last updated: 29th August 2023
First published: 28th August 2023

This blog is created by Stuart Page

I'm a freelance web developer and technology consultant based in Surrey, UK, with over 10 years experience in web development, DevOps, Linux Administration, and IT solutions.

Need support with your infrastructure or web services?

Get in touch