NVIDIA Docker Install

Background

Docker is a virtualization technology. I first truly used it when deploying OpenWebUI, and found it remarkably elegant for rapid deployment. Later, when doing machine learning, I found configuring the environment quite tedious. This time, as it happened, there was a somewhat engineering-oriented plan, so I asked for root access while setting up the newly arrived GPU server in the configuration group.

Internet

When you find that a server cannot access the internet, don't panic—I have a method inherited from my senior fellow student.

1. On your local machine, use proxy software to open a port (for example, Clash's default 7892) and make sure it can access the internet through the proxy.

2. Run on your local machine

ssh -R  2333:localhost:7892 <user>@<host.adress>

3. Run on the server

export ALL_PROXY="http://127.0.0.1:2333"
export https_proxy="http://127.0.0.1:2333"
export http_proxy="http://127.0.0.1:2333"

Then remember that not all traffic will go through the proxy. But now you can browse the internet happily!

Install

Two parts need to be installed: Docker and the NVIDIA Container Toolki.

I'll use rocky_8 as an example

Docker

Once you're online, it's easy enough, but you can't install it without root (

Using an administrative account, follow the tutorial as shown

dnf config-manager --add-repo=https://download.docker.com/linux/centos/docker-ce.repo

dnf update

dnf install docker-ce

Then set the usage permissions for a regular user (skip rootless)

usermod -aG docker $USER

newgrp docker

NVIDIA Container Toolki

Following the tutorial provided by Nvidia will most likely run into problems (that's right! it's the internet access problem) So I still followed the Rocky community tutorial

wget -O /etc/yum.repos.d/nvidia-container-toolkit.repo https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo # 配置仓库信息

dnf install -y nvidia-container-toolkit # 安装容器工具包 

nvidia-ctk runtime configure --runtime=docker # 该 nvidia-ctk 命令会修改 /etc/docker/daemon.json 主机上的文件，以便 Docker 可以使用 NVIDIA 容器运行

systemctl restart docker # 重启 docker 服务

Then remember to give Docker internet access; by default it does not use the proxy (

Vscode

This is just a record, so I won't explain it. Building a VSCode development container is too troublesome (when you can't access the internet)

#.devcontainer/devcontainer.json

{
  "name": "GPU Development",
  "build": {
      "dockerfile": "Dockerfile"
  },
  "runArgs": [
      "--gpus=all",
      "--network=host"
  ],
  "remoteUser": "root", // 使用 root 用户，避免用户不存在的问题
  "postCreateCommand": "nvidia-smi"
}

#.devcontainer/Dockerfile

# 使用 NVIDIA CUDA 12.6.1 开发镜像
FROM nvidia/cuda:12.6.1-runtime-rockylinux8

USER root

# 配置国内镜像源
RUN sed -i 's|^mirrorlist=|#mirrorlist=|g' /etc/yum.repos.d/Rocky-*.repo && \
    sed -i 's|^#baseurl=http://dl.rockylinux.org/$contentdir|baseurl=https://mirrors.aliyun.com/rockylinux|g' /etc/yum.repos.d/Rocky-*.repo

# 禁用CUDA存储库（因为基础镜像已经包含CUDA）
RUN if [ -f /etc/yum.repos.d/cuda.repo ]; then \
      sed -i 's|enabled=1|enabled=0|g' /etc/yum.repos.d/cuda.repo; \
    fi

RUN yum install -y kmod

# 设置容器的工作目录
WORKDIR /workspace