Qwen3.8 27B M5 Max
-
我下载了全精度权重用 oMLX 转换成了 oQ4e 量化，保留 MTP
https://huggingface.co/fevaoctwh/Qwen3.8-27B-oQ4e-mtp
在 M5 Max平台上可以说是精度和速度的一个平衡吧，更高的量化，M5 Max 就受限于内存带宽，decode 速度完全上不去了Measured with the built-in oMLX benchmark (
/admindashboard), single-request mode:- Engine: Auto
- Context: Code (Python)
- Test pattern:
pp<T> / tg128= prefill T tokens, then generate 128 tokens - Hardware: Apple Silicon (M5 Max, 128 GB unified memory)
Test TTFT (ms) TPOT (ms) Prefill (tok/s) Generation (tok/s) E2E (s) Throughput (tok/s) Peak Mem pp1024 / tg128 1161.5 15.84 881.6 63.6 3.184 361.9 16.72 GB pp4096 / tg128 5155.9 19.23 794.4 52.4 7.614 554.8 18.19 GB pp8192 / tg128 11856.7 18.13 690.9 55.6 14.176 586.9 19.24 GB pp16384 / tg128 25982.8 17.80 630.6 56.6 28.260 584.3 20.78 GB 量化后的能力肯定是打折的，作为 Agent 本地对话模型和简单任务是完全够用了，我现在使用的是Qwen3.6-35B-A3B-oQ6e
M 芯片目前的架构还是适合激活参数小的 MoE， 甜点尺寸应该是 100B 左右总参数量但是激活小于 10B 参数的模型Model Type Released MMLU-Pro LiveCodeBench v6 SWE-bench Verified Qwen3.8-27B-oQ4e-mtp (this model) Quantized 4/5-bit MLX, 27B dense Aug 2026 80.7% * — (not run) — Qwen3.6-27B Non-quantized, 27B dense Apr 2026 86.2% ~83.9 77.2% Qwen3.6-35B-A3B Non-quantized, 35B MoE Apr 2026 85.2% 80.4 73.4% Gemma 4 31B Non-quantized, 31B dense Mar 2026 85.2% 80.0 52.0% LiveCodeBench v6 还在跑 27/300…… 结果要明天了
-
我下载了全精度权重用 oMLX 转换成了 oQ4e 量化，保留 MTP
https://huggingface.co/fevaoctwh/Qwen3.8-27B-oQ4e-mtp
在 M5 Max平台上可以说是精度和速度的一个平衡吧，更高的量化，M5 Max 就受限于内存带宽，decode 速度完全上不去了Measured with the built-in oMLX benchmark (
/admindashboard), single-request mode:- Engine: Auto
- Context: Code (Python)
- Test pattern:
pp<T> / tg128= prefill T tokens, then generate 128 tokens - Hardware: Apple Silicon (M5 Max, 128 GB unified memory)
Test TTFT (ms) TPOT (ms) Prefill (tok/s) Generation (tok/s) E2E (s) Throughput (tok/s) Peak Mem pp1024 / tg128 1161.5 15.84 881.6 63.6 3.184 361.9 16.72 GB pp4096 / tg128 5155.9 19.23 794.4 52.4 7.614 554.8 18.19 GB pp8192 / tg128 11856.7 18.13 690.9 55.6 14.176 586.9 19.24 GB pp16384 / tg128 25982.8 17.80 630.6 56.6 28.260 584.3 20.78 GB 量化后的能力肯定是打折的，作为 Agent 本地对话模型和简单任务是完全够用了，我现在使用的是Qwen3.6-35B-A3B-oQ6e
M 芯片目前的架构还是适合激活参数小的 MoE， 甜点尺寸应该是 100B 左右总参数量但是激活小于 10B 参数的模型Model Type Released MMLU-Pro LiveCodeBench v6 SWE-bench Verified Qwen3.8-27B-oQ4e-mtp (this model) Quantized 4/5-bit MLX, 27B dense Aug 2026 80.7% * — (not run) — Qwen3.6-27B Non-quantized, 27B dense Apr 2026 86.2% ~83.9 77.2% Qwen3.6-35B-A3B Non-quantized, 35B MoE Apr 2026 85.2% 80.4 73.4% Gemma 4 31B Non-quantized, 31B dense Mar 2026 85.2% 80.0 52.0% LiveCodeBench v6 还在跑 27/300…… 结果要明天了
