抡锤者

@learningai
可以給原始連結嗎? 抄作業卡到一半了

目前R9700在機器上的bench mark test
OS: ubuntu 24.04

llama-bench -m Models/Qwen3.6-27B-GGUF/Qwen3.6-27B-Q4_K_M.gguf
ggml_cuda_init: found 1 ROCm devices (Total VRAM: 32624 MiB):
  Device 0: AMD Radeon AI PRO R9700, gfx1201 (0x1201), VMM: no, Wave Size: 32, VRAM: 32624 MiB
| model                          |       size |     params | backend    | ngl |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | --------------: | -------------------: |
| qwen35 27B Q4_K - Medium       |  15.65 GiB |    26.90 B | ROCm       |  99 |           pp512 |      1008.59 ± 25.13 |
| qwen35 27B Q4_K - Medium       |  15.65 GiB |    26.90 B | ROCm       |  99 |           tg128 |         26.38 ± 0.03 |

build: 838374375 (9103)

Hi @高乐天 Good point!
是屬於TP1

能否有個測試數據看看?
我看國外有人用cyankiwi/Qwen3.6-27B-AWQ-INT4可以測到287.38 tok/s (ROCM)
https://kyuz0.github.io/amd-r9700-vllm-toolboxes/