请教各位大神，5090 上 stara 如何支持多线程能快点啊？

[tester12340]

[tester12340]

无论怎么折腾，遇到长下文直接崩掉。。从来。
大神们，我还有啥能做的吗？

[perter]

@tester12340
显存不够还想要多线程的话，还可以把KV缓存降低到q4_0

"--max-context",
"131072",
"--kv",
"q4_0",
"--kv-resident",
"32768",
"--vram-reserve-mib",
"700",

[tester12340]

感谢，测试了。我感觉是可以了。 我再测试一下。
两个并行，单个能跑到 200 tok每秒
writing a tool call: execute_code: 816 of max 65536 tokens, 215.0 tok/s, 6 s
[strata] done: 944 tokens in 6 s (216.2 tok/s) (stop, cancel=False), expert cache 97.5% hit

感谢peter！

[strata] reading the prompt: 47,058 of 47,065 tokens, 2 s so far
[strata] answering: 137 of max 65536 tokens, 176.2 tok/s, 3 s
[strata] answering: 322 of max 65536 tokens, 180.1 tok/s, 4 s
[strata] writing a tool call: execute_code: 560 of max 65536 tokens, 200.3 tok/s, 5 s
[strata] writing a tool call: execute_code: 783 of max 65536 tokens, 205.9 tok/s, 6 s

stara 现在这么牛逼 感觉那些小主机全没用了啊

[tester12340]

为论坛做贡献，分享一下 5090 目前配置 stara 跑 next 的配置。 感谢 Peter 和 各位大神的指点。感谢锤子提供这么好的论坛。

{
"exe": "C:\strata\Strata\engine\strata.exe",
"args": [
"--pack",
"C:\strata\Strata-data\packs\q2_0",
"--native",
"C:\strata\Strata-data\models\Q2_0\Qwen3.8-Flash-Next-GSQ-RCO-Q2_0-00001-of-00002.gguf",
"--ple-gguf",
"C:\strata\Strata-data\models\Q2_0\Qwen3.8-Flash-Next-GSQ-RCO-Q2_0-00002-of-00002.gguf",
"--expert-profile",
"C:\strata\Strata\data\expert-profile.bin",
"--expert-cache",
"auto",
"--prefill",
"auto",
"--spec",
"4",
"--mtp",
"C:\strata\Strata-data\mtp\rt",
"--max-context",
"131072",
"--kv","q4_0",
"--kv-resident","32768",
"--vram-reserve-mib","700",
"--resident-experts",
"--pcie-frac",
"0.35",
"--spec-min-p",
"0.50",
"--pool-workers",
"4",

"--conversation-cache-mib", "8192"
],
"cwd": "C:\strata\Strata",
"parallel": 4,
"tokenizer": "C:\strata\Strata-data\packs\q2_0\tokenizer",
"model_name": "qwen3.8-flash-next-q2_0",
"log": "C:\strata\Strata\strata-q2_0.log",
"lib_dirs": [
"C:\strata\Strata\.venv\Lib\site-packages\nvidia\cu13\bin\x86_64"
],
"host":"192.168.0.3",
"port": 8080,
"gpu": 0,
"gpus_asked": true
}

有需要优化的地方 请大家指正。

另外默默问一句， mac ultra 96 是不是会完败 5090 staraa?

[David Chen]

我有沒有看錯！我3060那辣雞都跑iq3_s了，你5090只跑q2?

[tester12340]

我需要长上下文啊 兄弟。而且 "--kv","q4_0" 感觉要不要回到 27b了。

[stxpnet]

你真的别用Q2，可能造成工具链断裂，然后无限循环

[tester12340]

我正在体验无限循环。 但我切到 qwen flash 在线也是循环。估计人品不行。

[terry]

@tester12340 你这发帖格式是给论坛做贡献吗？让AI整理成markdwon代码再发

[tester12340]

按terry 老板要求

{
"exe": "C:\strata\Strata\engine\strata.exe",
"args": [
"--pack",
"C:\strata\Strata-data\packs\q2_0",
"--native",
"C:\strata\Strata-data\models\Q2_0\Qwen3.8-Flash-Next-GSQ-RCO-Q2_0-00001-of-00002.gguf",
"--ple-gguf",
"C:\strata\Strata-data\models\Q2_0\Qwen3.8-Flash-Next-GSQ-RCO-Q2_0-00002-of-00002.gguf",
"--expert-profile",
"C:\strata\Strata\data\expert-profile.bin",
"--expert-cache",
"auto",
"--prefill",
"auto",
"--spec",
"4",
"--mtp",
"C:\strata\Strata-data\mtp\rt",
"--max-context",
"131072",
"--kv",
"q4_0",
"--kv-resident",
"32768",
"--vram-reserve-mib",
"700",
"--resident-experts",
"--pcie-frac",
"0.35",
"--spec-min-p",
"0.50",
"--pool-workers",
"4",
"--conversation-cache-mib",
"8192"
],
"cwd": "C:\strata\Strata",
"parallel": 4,
"tokenizer": "C:\strata\Strata-data\packs\q2_0\tokenizer",
"model_name": "qwen3.8-flash-next-q2_0",
"log": "C:\strata\Strata\strata-q2_0.log",
"lib_dirs": [
"C:\strata\Strata\.venv\Lib\site-packages\nvidia\cu13\bin\x86_64"
],
"host": "192.168.0.3",
"port": 8080,
"gpu": 0,
"gpus_asked": true
}

这个对了吗？

[tester12340]

@stxpnet 我试试 Q3 能不能跑

[tester12340]

下载中

[tester12340]

这个是不是太慢了？

[tester12340]

从 300tk 直接降到 20tk

[tester12340]

晕倒，我再 4080 上跑了一下，竟然跟 5090 速度差不多 T_T 我了个晕倒。

[Tony Wang]

@tester12340

你用```分别在参数配置部分的上方和下方, 这个表示中间是代码的意思, 也可以一键复制.

建议学一下Mark Down 语法, 比较简单, 常用的也就那么几条.

[tester12340]

@Tony-Wang

谢谢 我试试

{
"exe": "C:\strata\Strata\engine\strata.exe",
"args": [
"--pack",
"C:\strata\Strata-data\packs\q2_0",
"--native",
"C:\strata\Strata-data\models\Q2_0\Qwen3.8-Flash-Next-GSQ-RCO-Q2_0-00001-of-00002.gguf",
"--ple-gguf",
"C:\strata\Strata-data\models\Q2_0\Qwen3.8-Flash-Next-GSQ-RCO-Q2_0-00002-of-00002.gguf",
"--expert-profile",
"C:\strata\Strata\data\expert-profile.bin",
"--expert-cache",
"auto",
"--prefill",
"auto",
"--spec",
"4",
"--mtp",
"C:\strata\Strata-data\mtp\rt",
"--max-context",
"131072",
"--kv",
"q4_0",
"--kv-resident",
"32768",
"--vram-reserve-mib",
"700",
"--resident-experts",
"--pcie-frac",
"0.35",
"--spec-min-p",
"0.50",
"--pool-workers",
"4",
"--conversation-cache-mib",
"8192"
],
"cwd": "C:\strata\Strata",
"parallel": 4,
"tokenizer": "C:\strata\Strata-data\packs\q2_0\tokenizer",
"model_name": "qwen3.8-flash-next-q2_0",
"log": "C:\strata\Strata\strata-q2_0.log",
"lib_dirs": [
"C:\strata\Strata\.venv\Lib\site-packages\nvidia\cu13\bin\x86_64"
],
"host": "192.168.0.3",
"port": 8080,
"gpu": 0,
"gpus_asked": true
}

[imbiplaza ASUS]

我心想中，strata + cpu + gpu + nvme + hdd + l2 cache + l3 cache 他已经是在极力走着多线程了。。。我觉得除非我们的主机里面还有另外一套strata + cpu + gpu + nvme + hdd + l2 cache + l3 cache, 那样就可以实现真正的多线程

[David Chen]

@tester12340 那就是妳沒好好善用GPU上的VRAM，我猜妳CPU loading一定很高，請AI自我檢測自我調優