vllm v0.22.0 加载qwen3.6 报错 #flashinfer

[Billy Shen]

vllm 升级到v0.22.0后加载大模型会报错，Deepseek分析vllm的模型加载日志后，提示是从排查flashinfer的问题入手。

群友遇到过这样的问题吗？ 是怎么解决的？

GPU Information

Sat May 30 19:14:11 2026
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 595.71.05 Driver Version: 595.71.05 CUDA Version: 13.2 |
+-----------------------------------------+------------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA RTX 5880 Ada Gene... Off | 00000000:AF:00.0 Off | Off |
| 30% 51C P8 31W / 285W | 15MiB / 49140MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+

Processes:
GPU GI CI PID Type Process name GPU Memory
ID ID Usage
0 N/A N/A 4146 G /usr/lib/xorg/Xorg 4MiB

+-----------------------------------------------------------------------------------------+

====================================================
CUDA Version

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2023 NVIDIA Corporation
Built on Fri_Jan__6_16:45:21_PST_2023
Cuda compilation tools, release 12.0, V12.0.140
Build cuda_12.0.r12.0/compiler.32267302_0

====================================================
Python Version

Python 3.12.3

====================================================
vLLM Package

Name: vllm
Version: 0.22.0
Summary: A high-throughput and memory-efficient inference and serving engine for LLMs
Home-page: https://github.com/vllm-project/vllm
Author: vLLM Team
Author-email:
License-Expression: Apache-2.0
Location: /home/ames/vllm_env/lib/python3.12/site-packages
Requires: aiohttp, anthropic, apache-tvm-ffi, blake3, cachetools, cbor2, cloudpickle, compressed-tensors, depyf, diskcache, einops, fastapi, fastsafetensors, filelock, flashinfer-cubin, flashinfer-python, gguf, humming-kernels, ijson, lark, llguidance, lm-format-enforcer, mcp, mistral_common, model-hosting-container-standards, msgspec, ninja, numba, numpy, nvidia-cudnn-frontend, nvidia-cutlass-dsl, openai, openai-harmony, opencv-python-headless, opentelemetry-api, opentelemetry-exporter-otlp, opentelemetry-sdk, opentelemetry-semantic-conventions-ai, outlines_core, partial-json-parser, pillow, prometheus-fastapi-instrumentator, prometheus_client, protobuf, psutil, py-cpuinfo, pybase64, pydantic, python-json-logger, pyyaml, pyzmq, quack-kernels, regex, requests, safetensors, sentencepiece, setproctitle, setuptools, six, tiktoken, tilelang, tokenizers, tokenspeed-mla, torch, torchaudio, torchvision, tqdm, transformers, typing_extensions, watchfiles, xgrammar
Required-by:

====================================================
FlashInfer Package

Name: flashinfer-python
Version: 0.6.11.post2
Summary: FlashInfer: Kernel Library for LLM Serving
Home-page: https://github.com/flashinfer-ai/flashinfer
Author: FlashInfer team
Author-email:
License-Expression: Apache-2.0
Location: /home/ames/vllm_env/lib/python3.12/site-packages
Requires: apache-tvm-ffi, click, cuda-tile, einops, ninja, numpy, nvidia-cudnn-frontend, nvidia-cutlass-dsl, nvidia-ml-py, packaging, requests, tabulate, torch, tqdm
Required-by: vllm

====================================================
Core AI Packages

flashinfer-cubin 0.6.11.post2
flashinfer-python 0.6.11.post2
tokenspeed-triton 3.7.10.post20260505
torch 2.11.0+cu130
torch_c_dlpack_ext 0.1.5
torchaudio 2.11.0+cu130
torchvision 0.26.0+cu130
triton 3.6.0
vllm 0.22.0

====================================================
PyTorch CUDA Information

PyTorch Version: 2.11.0+cu130
CUDA Available: True
CUDA Version: 13.0
GPU Count: 1
GPU Name: NVIDIA RTX 5880 Ada Generation
GPU Capability: (8, 9)

====================================================
FlashInfer Cache

总计 27K
drwxrwxr-x 3 ames ames 3 5月 30 18:55 .
drwx------ 19 ames ames 21 5月 30 18:55 ..
drwxrwxr-x 3 ames ames 3 5月 30 18:55 0.6.11.post2

[williamlouis]

用 net work 驱动下试试。统一下 cuda 驱动。

[Xiaote]

@Billy Shen 这个问题我遇到过，vllm 0.22.0 对 flashinfer 的依赖版本比较挑。

两个方向排查：

方向一：用 vllm 自带的 attention backend
vllm 0.22.0 默认已经内置了几种 attention backend（flash_attn、flashinfer、paged attention），不一定非得依赖 flashinfer。启动时加这个参数跳过 flashinfer 即可：

--attention-backend flash_attn

如果你的 CUDA 环境装了 flash-attn（pip install flash-attn），这个方案最稳，不依赖 flashinfer 的版本兼容性。

方向二：重新安装匹配版本的 flashinfer
如果一定要用 flashinfer，需要装跟 vllm 0.22.0 匹配的版本：

pip uninstall flashinfer -y
pip install flashinfer -U

或者从源码编译指定版本：

pip install flashinfer==0.2.1.post2

最新的 vllm 官方 docker 镜像里已经内置了正确的 flashinfer 版本，如果不介意用 docker 的话也可以直接用官方镜像。

另外注意 CUDA 版本：你的 CUDA 13.2 比较新，flashinfer 对 CUDA 13.x 的支持可能还不完善。如果装不上，用方向一的 --attention-backend flash_attn 是最省事的替代方案。

试完之后可以回来说下结果，如果还报错可以把完整 error log 贴出来看看。

[Billy Shen]

@williamlouis 显卡的驱动已经安装了。 GPU Name: NVIDIA RTX 5880 Ada Generation | NVIDIA-SMI 595.71.05 Driver Version: 595.71.05 CUDA Version: 13.2 |