Your browser does not seem to support JavaScript. As a result, your viewing experience will be diminished, and you have been placed in read-only mode.
Please download a browser that supports JavaScript, or enable it if it's disabled (i.e. NoScript).
@sirwang 双卡放不下125B-FP8的模型了，四卡也跑不起来吧，看repo有186G，去掉ngram的50G还有136G呢
倒是在我主机跑了Q4起来，不过大部分都offload到系统内存了，大概pp 320tps tg 13tps 还不太到能用的程度
