T
@Tony-Xu-0 具体是如何让dsh/hermes按角色分工的(architect/engineer), 能介绍一下吗? 谢谢.
Controller
｜─ DSH architect / audit
｜ └─ frozen plan + constraints
｜
└─ Hermes orchestrator
｜─ delegate_task leaf A
｜─ delegate_task leaf B
└─ delegate_task leaf C
↓
Hermes integration/review
↓
DSH independent post-audit
↓
Controller accept / resume / stop
最近agent工具和底层模型层出不穷，为了搞清楚每个工具最适合做什么，好早日实现标准化大型仓库的全本地化工作流，做了一个跨工具/模型控制变量实验探索各个工具/模型相比其他可选项的特点，主要集中在Qwen3.6-27B/Qwen3.8-27B，Hermes 0.16/Hermes 0.20，Deepseek Harness rc07。收获一些心得，特此分享，实验文档做了脱敏，不影响核心实验结论输出
dgx spark的bandwidth在消费级虽然不差，但依然是硬伤，体感上和其他消费级卡比起来即便是接近，也在数学和物理上限制了吞吐，个人觉得属于高级入门或初级专业级，但对于4个并行能跑GLM5.2来说，性价比真不错，只是还是那句话：多卡支撑模型的能效比真的堪称是贵族
Avg generation throughput: 241.9 tokens/s
MTP = 20 可以高達這個速度嗎？
是的，但是要优化输出结构，实际之后又试了MTP=30/40，请求处理跑起来以后生成速度稳定上350 tokens/s，但是效率收益递减，没再开
之前一直在做模型输出结构优化，MTP一直用在3，生成速度大概稳定在85 tokens/s，无意间发现MTP第三位居然稳定在0.97以上，后续索性把MTP加到10后第十位也居然稳定在0.95以上，干脆一不做二不休再次翻倍开20，单次请求得到如下log。还没有完全优化完，只是边做工作边刻意的想着优化结构化输出，先把工作完成等到后续性能调优再看能不能让数据更漂亮
你怎么看@xiaote
(APIServer pid=1565552) INFO 07-28 20:35:19 [loggers.py:273] Engine 000: Avg prompt throughput: 1818.1 tokens/s, Avg generation throughput: 71.2 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 23.7%, Prefix cache hit rate: 18.3%
(APIServer pid=1565552) INFO 07-28 20:35:19 [metrics.py:120] SpecDecoding metrics: Mean acceptance length: 15.13, Accepted throughput: 0.19 tokens/s, Drafted throughput: 0.22 tokens/s, Accepted: 664 tokens, Drafted: 769 tokens, Per-position acceptance rate: 0.979, 0.894, 0.894, 0.894, 0.872, 0.872, 0.872, 0.809, 0.787, 0.766, 0.745, 0.681, 0.638, 0.553, 0.532, 0.532, 0.468, 0.447, 0.447, 0.447, Avg Draft acceptance rate: 86.3%
(APIServer pid=1565552) INFO 07-28 20:35:29 [loggers.py:273] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 241.9 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 24.3%, Prefix cache hit rate: 18.3%
(APIServer pid=1565552) INFO 07-28 20:35:29 [metrics.py:120] SpecDecoding metrics: Mean acceptance length: 19.05, Accepted throughput: 229.21 tokens/s, Drafted throughput: 249.11 tokens/s, Accepted: 2292 tokens, Drafted: 2491 tokens, Per-position acceptance rate: 1.000, 0.992, 0.976, 0.976, 0.969, 0.961, 0.953, 0.953, 0.937, 0.929, 0.913, 0.913, 0.898, 0.866, 0.850, 0.835, 0.819, 0.795, 0.764, 0.748, Avg Draft acceptance rate: 92.0%
(APIServer pid=1565552) INFO 07-28 20:35:39 [loggers.py:273] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 244.4 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 25.1%, Prefix cache hit rate: 18.3%
(APIServer pid=1565552) INFO 07-28 20:35:39 [metrics.py:120] SpecDecoding metrics: Mean acceptance length: 19.55, Accepted throughput: 231.89 tokens/s, Drafted throughput: 247.69 tokens/s, Accepted: 2319 tokens, Drafted: 2477 tokens, Per-position acceptance rate: 1.000, 0.992, 0.992, 0.992, 0.992, 0.984, 0.984, 0.984, 0.976, 0.976, 0.968, 0.944, 0.920, 0.904, 0.880, 0.880, 0.824, 0.808, 0.792, 0.760, Avg Draft acceptance rate: 93.6%
(APIServer pid=1565552) INFO 07-28 20:35:49 [loggers.py:273] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 248.7 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 26.0%, Prefix cache hit rate: 18.3%
(APIServer pid=1565552) INFO 07-28 20:35:49 [metrics.py:120] SpecDecoding metrics: Mean acceptance length: 20.22, Accepted throughput: 236.40 tokens/s, Drafted throughput: 245.90 tokens/s, Accepted: 2364 tokens, Drafted: 2459 tokens, Per-position acceptance rate: 1.000, 1.000, 1.000, 1.000, 1.000, 1.000, 1.000, 0.984, 0.984, 0.984, 0.967, 0.967, 0.959, 0.951, 0.943, 0.927, 0.902, 0.894, 0.886, 0.870, Avg Draft acceptance rate: 96.1%
(APIServer pid=1565552) INFO: 127.0.0.1:34096 - "POST /v1/chat/completions HTTP/1.1" 200 OK