30 t/s、700+ pp 这个量级是否可信，取决于三件事：

1. 模型是 MoE 且激活参数小（总参数大、每次只过少量专家），24G 显存放得下量化后的权重与 KV；
2. prefill 是算力型，7900XTX 的 fp16/INT8 峰值够，700 pp 不离谱；decode 是带宽型，30 t/s 要看权重加 KV 的等效带宽在 960GB/s 量级内是否站得住；
3. 视频里的设置有没有开 n-gram / 投机解码、KV 是什么 dtype、上下文多长、batch 多少。

单开发者引擎加新后端，建议先用自己固定的 prompt 和上下文长度复现，再把 pp/tg 分开和 llama.cpp 对比，之后决定要不要迁。

@franklee006 说:
https://github.com/Headmaster218/MoE4All

这个仓库好像有点搞头啊. 让 AI 看了下仓库, 让它回答了几个关键问题:

MoE4All Summary

1. What Inference Engine Does It Use?

MoE4All is a custom Rust-based inference engine forked from infr (kryptic-sh/infr). It's not based on llama.cpp, vLLM, or SGLang. It's a native Rust implementation.

From Cargo.toml:

• infr-core — Core engine
• infr-vulkan — AMD Vulkan backend (main focus)
• infr-metal — Apple Metal backend
• infr-cpu — CPU fallback
• infr-gguf — GGUF model loading

2. Does It Support TP=2?

Yes! It supports tensor parallelism.

From infr.example.toml:

# Device lists: [0, 1] or ["Vulkan0", "Vulkan1"]. pipeline and tensor_parallel
# tensor_parallel = [0, 1]
tp_p2p = true

So you can configure multiple GPUs for tensor parallelism.

3. How Difficult to Convert to Linux?

Medium-High difficulty:

The code is already cross-platform Rust, but:

• Windows-specific GPU driver APIs may need Linux equivalents
• Memory management differs between Windows and Linux
• The Vulkan layer is already cross-platform

4. Does It Only Use Vulkan?

No! It supports multiple backends:

Backend	Status
Vulkan	Main (AMD GPU focus)
Metal	For Apple GPUs
CPU	Fallback

The project is designed with swappable backends (infr-vulkan, infr-metal, infr-cpu).

Summary

Question	Answer
Inference engine	Custom Rust (fork of infr)
TP=2 support	Yes (tensor_parallel config)
Linux port difficulty	Medium-High (Rust is cross-platform, but GPU APIs need adaptation)
Vulkan only?	No — also Metal and CPU

ReportID: d7eb9374-e784-4ef9-af49-1621406bfd74
ConversationID: a7dbb0f5-74ca-48e0-b9fd-2cfc3d04848a