FP8：

GPU KV cache size: 1,006,391 tokens

TurboQuant：

GPU KV cache size: 1,900,544 tokens

官方声称 TurboQuant 吞吐量比 FP8 低很多，我还不以为意；接入 harness 实战后，降速凸显：

FP8，多并发：

Avg generation throughput: 443.6 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 18.0%

TurboQuant，多并发：

Avg generation throughput: 264.0 tokens/s, Running: 12 reqs, Waiting: 0 reqs, GPU KV cache usage: 9.2%

FP8，单并发，约 72k 时：

Avg generation throughput: 56.0 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 7.3%

TurboQuant，单并发，约 72k 时：

Avg generation throughput: 22.0 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 3.8%

可见 TurboQuant 比 FP8 至少慢了 40%，而且越长越慢。真的是没有白捡的 KVCache