关于Proxmox VE上运行AI的折腾记录

[4cld62ca]

详细的折腾记录不再展开，有点心塞
该方法适用于
1 显卡出品时间较新
2 虚拟机配置直通完成，虚拟机关闭或者重启不释放显卡资源，再次启动时报错,比如

 error writing '1' to '/sys/bus/pci/devices/0000:04:00.0/reset': Inappropriate ioctl for device

宿主机环境

CPU(s) 72 x Intel(R) Xeon(R) Gold 6154 CPU @ 3.00GHz (2 CPU 插槽数)
内核版本 Linux 7.0.6-2-pve (2026-05-29T11:08Z)
显卡 AMD 9070XT

配置直通:

/etc/default/grub:
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt pcie_acs_override=downstream,multifunction split_lock_detect=off usbcore.autosuspend=-1"

/etc/modules
vfio
vfio_iommu_type1
vfio_pci

/etc/modprobe.d/ 的部分我没有做

宿主机

宿主机配置文件举例

root@pve:~#  cat  /etc/pve/qemu-server/129.conf 
args: -cpu host,hv_passthrough,level=35,-hypervisor,+svm          ###关键配置
bios: ovmf
boot: order=scsi0;ide0;net0
cores: 32
cpu: host
efidisk0: data1:vm-129-disk-0,efitype=4m,pre-enrolled-keys=1,size=1M
hookscript: local:snippets/ryzen_igpu_reset.sh         ###关键配置
hostpci0: 0000:da:00.0,pcie=1         <<<关键配置
hostpci1: 0000:da:00.1,pcie=1,romfile=9070xt.rom         ###关键配置
machine: pc-q35-10.0+pve1
memory: 65536
meta: creation-qemu=10.0.2,ctime=1761193467
name: win-ai1
net0: e1000e=BC:24:11:E8:29:C5,bridge=vmbr0,firewall=1
numa: 0
ostype: win11
scsi0: data1:vm-129-disk-1,iothread=1,size=64G
scsi1: data1:vm-129-disk-2,iothread=1,size=64G
scsi2: data1:vm-129-disk-3,iothread=1,size=128G

钩子脚本ryzen_igpu_reset.sh

root@pve:~# cat /var/lib/vz/snippets/ryzen_igpu_reset.sh 
#!/bin/bash
phase="$2"
echo "Phase is $phase"
if [ "$phase" == "pre-start" ]; then
    # Unbind gpu from amdgpu
    echo "0000:da:00.0" > /sys/bus/pci/drivers/amdgpu/unbind 2>/dev/null     ###修改路径
    echo "0000:da:00.1" > /sys/bus/pci/drivers/snd_hda_intel/unbind 2>/dev/null   ###修改路径
    sleep 5
elif [ "$phase" == "post-stop" ]; then
    # Unbind gpu from vfio-pci
    sleep 5
    echo "0000:da:00.0" > /sys/bus/pci/drivers/vfio-pci/unbind 2>/dev/null  
    sleep 2
    echo "0000:da:00.1" > /sys/bus/pci/drivers/vfio-pci/unbind 2>/dev/null
    sleep 5
    # Bind amdgpu
    echo "0000:da:00.0" > /sys/bus/pci/drivers/amdgpu/bind 2>/dev/null
    # Bind snd_hda_intel
    echo "0000:da:00.1" > /sys/bus/pci/drivers/snd_hda_intel/bind 2>/dev/null
    sleep 2
fi

###记得授权
root@pve:~# chmod +x /var/lib/vz/snippets/ryzen_igpu_reset.sh   

上传rom文件到以下路径

/usr/share/kvm/9070xt.rom

PS

1 gpu路径获取

root@pve:~# lspci -nnk
输出类似
da:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 48 [Radeon RX 9070/9070 XT/9070 GRE] [1002:7550] (rev c0)
        Subsystem: Sapphire Technology Limited Navi 48 XTX [Sapphire Pulse Radeon RX 9070 XT] [1da2:e490]
        Kernel driver in use: vfio-pci
        Kernel modules: amdgpu
da:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 48 HDMI/DP Audio Controller [1002:ab40]
        Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] Navi 48 HDMI/DP Audio Controller [1002:ab40]
        Kernel driver in use: vfio-pci
        Kernel modules: snd_hda_intel
da:00就是路径了，后面的.0和.1区分了声卡，不过配置文件中使用，要在前面加0000，如0000:da:00.0

虚拟机

操作系统

[root@pod1 ~]# cat /etc/os-release 
NAME="Oracle Linux Server"
VERSION="9.7"
ID="ol"
ID_LIKE="fedora"
VARIANT="Server"
VARIANT_ID="server"
VERSION_ID="9.7"
PLATFORM_ID="platform:el9"
PRETTY_NAME="Oracle Linux Server 9.7"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:oracle:linux:9:7:server"
HOME_URL="https://linux.oracle.com/"
BUG_REPORT_URL="https://github.com/oracle/oracle-linux"

ORACLE_BUGZILLA_PRODUCT="Oracle Linux 9"
ORACLE_BUGZILLA_PRODUCT_VERSION=9.7
ORACLE_SUPPORT_PRODUCT="Oracle Linux"
ORACLE_SUPPORT_PRODUCT_VERSION=9.7

软件依赖

这里要注意两个事项

1. 先打个快照，避免安装失败删除文件的繁琐，直接回滚

2. 空间大一些

[root@pod2 ~]# df -Th
Filesystem                        Type      Size  Used Avail Use% Mounted on
devtmpfs                          devtmpfs  4.0M     0  4.0M   0% /dev
tmpfs                             tmpfs      32G     0   32G   0% /dev/shm
tmpfs                             tmpfs      13G  9.1M   13G   1% /run
/dev/mapper/ol-root               xfs        47G   29G   18G  62% /
/dev/sda2                         xfs      1014M  605M  410M  60% /boot
/dev/sda1                         vfat      599M  7.6M  592M   2% /boot/efi
/dev/mapper/podmanvg-podmandatalv xfs        96G   51G   46G  53% /podmandata
/dev/mapper/podmanvg-podmanisolv  xfs        64G   20G   45G  31% /var/lib/containers
tmpfs                             tmpfs     6.3G     0  6.3G   0% /run/user/0
[root@pod2 ~]# ls /opt/
amdgpu  rocm  rocm-7.2.3
[root@pod2 ~]# du -sh /opt/
24G     /opt/

/podmandata 是容器持久化的路径
/var/lib/containers 是各类容器使用的路径

安装方式：

参考AMD官方文档安装基于Linux的驱动，并成功让 rocminfo 命令有完整的回显

AI的部署

这里我选择了容器化运行，大不了我删了容器再来一次，少折腾

[root@pod2 ~]# cat ollama.yaml
services:
  ollama:
    image: docker.io/ollama/ollama:0.30.0-rc23-rocm
    container_name: ollama
    devices:
      - /dev/kfd:/dev/kfd
      - /dev/dri:/dev/dri
    ports:
      - "0.0.0.0:11434:11434"
    env_file:
      - /root/.ollama.env
    volumes:
      - /podmandata/ollama:/root/.ollama:Z
      - /opt/rocm/lib/hipblaslt/library:/usr/lib/ollama/rocm/hipblaslt/library:ro
      - /etc/localtime:/etc/localtime:ro
    networks:
      - podman-net
    restart: unless-stopped

  think-filter:
    image: docker.io/library/python:3.13-alpine
    container_name: ollama-think-filter
    ports:
      - "0.0.0.0:8099:8099"
    command: sh -c "pip install -q httpx && python3 /app/think-filter.py"
    volumes:
      - /podmandata/think-filter:/app:Z
      - /etc/localtime:/etc/localtime:ro
    networks:
      - podman-net
    restart: unless-stopped

  proxy:
    image: docker.io/library/caddy:alpine
    container_name: ollama-proxy
    env_file:
      - /root/.ollama.env
    ports:
      - "0.0.0.0:8080:8080"
    volumes:
      - /podmandata/caddy/Caddyfile:/etc/caddy/Caddyfile:ro,Z
      - /podmandata/caddy/data:/data:Z
      - /etc/localtime:/etc/localtime:ro
    networks:
      - podman-net
    restart: unless-stopped

networks:
  podman-net:
    external: true
    name: podman

podman pull ollama/ollama:0.30.0-rc23-rocm

#启动
podman-compose -f ollama.yaml up -d 
#停止
podman-compose -f ollama.yaml down  

PS

1. ollama的proxy

   这个是因为ollama看起来没有鉴权，我之前配置hermes 时又需要填写api_key，所以用AI写了一个代理，这个可以完全可以用AI自己写的。不再赘述

[terry]

搭配下实际运行截图，否则容易被判断为AI内容。

[Grayson Ren]

pve 显卡直通只要主板设置没问题 基本不会有太大烦恼