于是赶紧按照说明部署了, 看到编译时候参数cmake -B build -DGGML_CUDA=ON, 眼前一黑, 莫不是只支持cuda.

硬着头皮把参数改成vulkun, 编译通过. 跑起来就傻眼了.

sun@homeserver6:/data/diffusiongemma/llama.cpp$ ./build/bin/llama-diffusion-cli   -m /data/diffusiongemma/diffusiongemma-26B-A4B-it-Q4_K_M.gguf -ngl 99 -cnv -n 32768
0.00.420.924 W load: control-looking token:    212 '</s>' was not control-type; this is probably a bug in the model. its type will be overridden
0.00.421.244 W load: control-looking token:     50 '<|tool_response>' was not control-type; this is probably a bug in the model. its type will be overridden
0.00.437.141 W load: special_eog_ids contains '<|tool_response>', removing '</s>' token from EOG list
0.06.155.371 I diffusion: -n 32768 -> 128 blocks, n_ubatch=34816 n_batch=34816 n_ctx=34816 (canvas_length=256)
0.06.155.375 I diffusion: --fit has no effect here; context is sized from -n and the canvas. Set -ngl / --n-cpu-moe to control device memory.
0.06.155.805 W llama_context: n_ctx_seq (34816) < n_ctx_train (262144) -- the full capacity of the model will not be utilized
0.09.436.495 W ggml_vulkan: Failed to allocate pinned memory (Requested buffer size exceeds device buffer size limit: ErrorOutOfDeviceMemory)
0.09.444.362 I diffusion_params: steps=128 schedule=0 algorithm=4 temperature=0.800 eps=0.001000 mask_token=4
0.09.444.366 I diffusion_eb: max_steps=48 t=[0.400,0.800] entropy_bound=0.1000 stability=1 confidence=0.0050 kv_cache=on gpu_sampling=on sample_reduce=on
0.09.444.366 I conversation mode: /help for commands, /clear to reset, /exit to quit

> hello
0.14.709.774 W diffusion_generate_entropy_bound: on-device sampling unsupported on this backend; using host sampling
diffusion step: 8/48 [========                                          ] 16%0.18.486.966 I 
<|channel>thought
The user said "hello".
This is a standard greeting.
I should respond politely and offer assistance.

Plan:
1. Greet the user.
2. Ask how I can help them.<channel|>Hello! How can I help you today?
total time: 4945.55ms, time per step: 549.51ms (9 steps over 1 blocks, entropy-bound)
throughput: 51.8 tok/s (256 tok in 4945.55ms), in-step parallel 466 tok/s (256-tok canvas x 9.0 steps/block)

> 你好
0.28.106.836 W diffusion_generate_entropy_bound: on-device sampling unsupported on this backend; using host sampling
diffusion step: 20/48 [====================                              ] 41%0.37.561.306 I 
<|channel>thought
The user said "你好" (Nǐ hǎo), which means "Hello" in Chinese.
The user is speaking in Chinese.
I should respond politely in Chinese and offer assistance.
    *   Option 1: 你好！(Nǐ hǎo!) - Simple hello.
    *   Option 2: 你好！有什么可以帮您的吗？(Nǐ hǎo! Yǒu shénme wǒ kě yǐ b帮 nín de ma?) - Hello! How can I help you?
Option 2 is more helpful and engaging.<channel|>你好！请问有什么我可以帮您的吗？
total time: 9934.09ms, time per step: 473.05ms (21 steps over 1 blocks, entropy-bound)
throughput: 25.8 tok/s (256 tok in 9934.09ms), in-step parallel 541 tok/s (256-tok canvas x 21.0 steps/block)

有报错不说, 速度也才25.8, 还不如Qwen3.6 27B

调整一下参数

./build/bin/llama-diffusion-cli \
  -m /data/diffusiongemma/diffusiongemma-26B-A4B-it-Q4_K_M.gguf \
  -ngl 99 \
  -cnv \
  -n 1024 \
  --diffusion-eb-max-steps 24

首次输出速度也才53 tokens/s.

看报错, 发现很明确不支持:

diffusion_generate_entropy_bound: on-device sampling unsupported on this backend; using host sampling

换ROCm也一样.

如果有人真跑起来, 欢迎分享下, 如果没有1000tokens/s的速度, 有500也行啊