Your browser does not seem to support JavaScript. As a result, your viewing experience will be diminished, and you have been placed in read-only mode.
Please download a browser that supports JavaScript, or enable it if it's disabled (i.e. NoScript).
感谢分享。在rtx 6000d上，同样模型参考设置，mem fraction 0.88，chunked prefill 4096，mamba dtype bf16，kv cache fp8，draft tokens 16，做到了c1输出244-390/s。c8 694/s。效率还不错
B