请教各位大神，5090 上 stara 如何支持多线程能快点啊？

[perter]

@tester12340
发一个我48GB 的 rtx pro 5000的全套参数给你参考，你根据自己内存大小调整。

"args": [
  "--pack", "C:\\work\\Strata-data\\packs\\iq3_xxs",
  "--native", "C:\\work\\Strata-data\\models\\IQ3_XXS\\Qwen3.8-Flash-Next-GSQ-RCO-IQ3_XXS-00001-of-00002.gguf",
  "--ple-gguf", "C:\\work\\Strata-data\\models\\IQ3_XXS\\Qwen3.8-Flash-Next-GSQ-RCO-IQ3_XXS-00002-of-00002.gguf",
  "--expert-profile", "C:\\work\\Strata\\data\\expert-profile.bin",
  "--resident-experts",
  "--expert-cache", "19000",
  "--prefill", "auto",
  "--spec", "4",
  "--mtp", "C:\\work\\Strata-data\\mtp\\rt",
  "--max-context", "262144",
  "--kv", "k8v4",
  "--vision", 
  "--vram-reserve-mib", "3400", 
  "--pcie-frac", "0.00", 
  "--spec-min-p", "0.70",
  "--conversation-cache-mib", "27648",
  "--conversation-cache-slots", "9",
  "--conversation-cache-min-free-mib", "8192",
  "--prompt-cache-every", "8192"
 ],

[tester12340]

跟我逾期的一样。就是短上下文没问题。但长上下文就会进入无尽的循环。 然后还会自动降级到 65535 。 32gb已经被榨干。我想是 32gb 就是老黄这个奸商设计的短板吧。 还是 48g香啊

trata] reading the prompt: 1,024 of 1,031 tokens, 3 s so far
[strata] done: 4 tokens in 3 s (56.3 tok/s) (stop, cancel=False), expert cache 56.0% hit
[strata] reading the prompt: 8,192 of 8,975 tokens, 3 s so far
[strata] reading the prompt: 8,968 of 8,975 tokens, 5 s so far
[strata] answering: 40 of max 122089 tokens, 95.7 tok/s, 6 s
[strata] answering: 163 of max 122089 tokens, 111.8 tok/s, 7 s
[strata] answering: 287 of max 122089 tokens, 114.3 tok/s, 8 s
[strata] answering: 418 of max 122089 tokens, 117.5 tok/s, 9 s
[strata] answering: 572 of max 122089 tokens, 125.2 tok/s, 10 s
[strata] answering: 717 of max 122089 tokens, 128.6 tok/s, 11 s
[strata] answering: 842 of max 122089 tokens, 127.8 tok/s, 12 s
[strata] answering: 1000 of max 122089 tokens, 131.5 tok/s, 13 s
[strata] answering: 1161 of max 122089 tokens, 134.9 tok/s, 14 s
[strata] answering: 1344 of max 122089 tokens, 139.9 tok/s, 15 s
[strata] answering: 1497 of max 122089 tokens, 140.9 tok/s, 16 s
[strata] answering: 1658 of max 122089 tokens, 142.5 tok/s, 17 s
[strata] answering: 1817 of max 122089 tokens, 143.7 tok/s, 18 s
[strata] answering: 1981 of max 122089 tokens, 145.1 tok/s, 19 s
[strata] answering: 2164 of max 122089 tokens, 147.5 tok/s, 20 s
[strata] answering: 2309 of max 122089 tokens, 146.7 tok/s, 21 s
[strata] thinking: 19 of max 130783 tokens, 41.2 tok/s, 1 s
[strata] answering: 2337 of max 122089 tokens, 139.4 tok/s, 23 s
[strata] answering: 77 of max 130783 tokens, 52.2 tok/s, 2 s
[strata] done: 82 tokens in 2 s (52.4 tok/s) (stop, cancel=False), expert cache 59.8% hit
[strata] answering: 2394 of max 122089 tokens, 133.5 tok/s, 24 s
[strata] answering: 2553 of max 122089 tokens, 134.8 tok/s, 25 s
[strata] answering: 2713 of max 122089 tokens, 135.9 tok/s, 26 s
[strata] answering: 2879 of max 122089 tokens, 137.2 tok/s, 27 s
[strata] answering: 3050 of max 122089 tokens, 138.6 tok/s, 28 s
[strata] answering: 3236 of max 122089 tokens, 140.6 tok/s, 29 s
[strata] answering: 3414 of max 122089 tokens, 142.0 tok/s, 30 s
[strata] answering: 3588 of max 122089 tokens, 143.2 tok/s, 31 s
[strata] answering: 3783 of max 122089 tokens, 145.1 tok/s, 32 s
[strata] answering: 3975 of max 122089 tokens, 146.8 tok/s, 33 s
[strata] answering: 4161 of max 122089 tokens, 148.2 tok/s, 34 s
[strata] answering: 4344 of max 122089 tokens, 149.3 tok/s, 35 s
[strata] answering: 4543 of max 122089 tokens, 150.9 tok/s, 36 s
[strata] answering: 4755 of max 122089 tokens, 152.7 tok/s, 37 s
[strata] answering: 4927 of max 122089 tokens, 153.2 tok/s, 38 s
[strata] answering: 5113 of max 122089 tokens, 154.0 tok/s, 39 s
[strata] done: 5184 tokens in 39 s (154.4 tok/s) (stop, cancel=False), expert cache 90.8% hit
[strata] reading the prompt: 24,186 of 52,038 tokens, 3 s so far
[strata] reading the prompt: 32,378 of 52,038 tokens, 6 s so far
[strata] reading the prompt: 40,570 of 52,038 tokens, 9 s so far
[strata] reading the prompt: 48,762 of 52,038 tokens, 12 s so far
[strata] reading the prompt: 52,031 of 52,038 tokens, 15 s so far
[strata] writing a tool call: execute_code: 48 of max 65536 tokens, 111.8 tok/s, 16 s
[strata] writing a tool call: execute_code: 180 of max 65536 tokens, 125.3 tok/s, 17 s
[strata] writing a tool call: execute_code: 330 of max 65536 tokens, 134.9 tok/s, 18 s
[strata] writing a tool call: execute_code: 478 of max 65536 tokens, 138.1 tok/s, 19 s
[strata] writing a tool call: execute_code: 655 of max 65536 tokens, 146.7 tok/s, 20 s
[strata] writing a tool call: execute_code: 807 of max 65536 tokens, 147.1 tok/s, 21 s
[strata] done: 846 tokens in 22 s (146.3 tok/s) (stop, cancel=False), expert cache 84.5% hit
[strata] reading the prompt: 1,030 of 1,037 tokens, 3 s so far
[strata] done: 4 tokens in 3 s (57.1 tok/s) (stop, cancel=False), expert cache 60.8% hit
[strata] reading the prompt: 8,192 of 53,028 tokens, 3 s so far
[strata] reading the prompt: 15,994 of 53,028 tokens, 6 s so far
[strata] reading the prompt: 24,186 of 53,028 tokens, 9 s so far
[strata] reading the prompt: 32,378 of 53,028 tokens, 12 s so far
[strata] reading the prompt: 40,570 of 53,028 tokens, 15 s so far
[strata] reading the prompt: 48,762 of 53,028 tokens, 18 s so far
[strata] reading the prompt: 53,021 of 53,028 tokens, 21 s so far
[strata] writing a tool call: execute_code: 60 of max 65536 tokens, 135.5 tok/s, 22 s
[strata] writing a tool call: execute_code: 235 of max 65536 tokens, 162.1 tok/s, 23 s
[strata] writing a tool call: execute_code: 428 of max 65536 tokens, 174.2 tok/s, 24 s
[strata] writing a tool call: execute_code: 622 of max 65536 tokens, 179.3 tok/s, 25 s
[strata] writing a tool call: execute_code: 795 of max 65536 tokens, 177.8 tok/s, 26 s
[strata] done: 852 tokens in 26 s (176.2 tok/s) (stop, cancel=False), expert cache 90.0% hit
[strata] reading the prompt: 1,036 of 1,043 tokens, 3 s so far
[strata] done: 4 tokens in 3 s (53.6 tok/s) (stop, cancel=False), expert cache 60.1% hit
[strata] reading the prompt: 8,192 of 54,024 tokens, 3 s so far
[strata] reading the prompt: 15,994 of 54,024 tokens, 6 s so far
[strata] reading the prompt: 24,186 of 54,024 tokens, 9 s so far
[strata] reading the prompt: 32,378 of 54,024 tokens, 12 s so far
[strata] reading the prompt: 40,570 of 54,024 tokens, 15 s so far
[strata] reading the prompt: 48,762 of 54,024 tokens, 18 s so far
[strata] reading the prompt: 54,017 of 54,024 tokens, 21 s so far

[tester12340]

[tester12340]

无论怎么折腾，遇到长下文直接崩掉。。从来。
大神们，我还有啥能做的吗？

[perter]

@tester12340
显存不够还想要多线程的话，还可以把KV缓存降低到q4_0

"--max-context",
"131072",
"--kv",
"q4_0",
"--kv-resident",
"32768",
"--vram-reserve-mib",
"700",

[tester12340]

感谢，测试了。我感觉是可以了。 我再测试一下。
两个并行，单个能跑到 200 tok每秒
writing a tool call: execute_code: 816 of max 65536 tokens, 215.0 tok/s, 6 s
[strata] done: 944 tokens in 6 s (216.2 tok/s) (stop, cancel=False), expert cache 97.5% hit

感谢peter！

[strata] reading the prompt: 47,058 of 47,065 tokens, 2 s so far
[strata] answering: 137 of max 65536 tokens, 176.2 tok/s, 3 s
[strata] answering: 322 of max 65536 tokens, 180.1 tok/s, 4 s
[strata] writing a tool call: execute_code: 560 of max 65536 tokens, 200.3 tok/s, 5 s
[strata] writing a tool call: execute_code: 783 of max 65536 tokens, 205.9 tok/s, 6 s

stara 现在这么牛逼 感觉那些小主机全没用了啊

[tester12340]

为论坛做贡献，分享一下 5090 目前配置 stara 跑 next 的配置。 感谢 Peter 和 各位大神的指点。感谢锤子提供这么好的论坛。

{
"exe": "C:\strata\Strata\engine\strata.exe",
"args": [
"--pack",
"C:\strata\Strata-data\packs\q2_0",
"--native",
"C:\strata\Strata-data\models\Q2_0\Qwen3.8-Flash-Next-GSQ-RCO-Q2_0-00001-of-00002.gguf",
"--ple-gguf",
"C:\strata\Strata-data\models\Q2_0\Qwen3.8-Flash-Next-GSQ-RCO-Q2_0-00002-of-00002.gguf",
"--expert-profile",
"C:\strata\Strata\data\expert-profile.bin",
"--expert-cache",
"auto",
"--prefill",
"auto",
"--spec",
"4",
"--mtp",
"C:\strata\Strata-data\mtp\rt",
"--max-context",
"131072",
"--kv","q4_0",
"--kv-resident","32768",
"--vram-reserve-mib","700",
"--resident-experts",
"--pcie-frac",
"0.35",
"--spec-min-p",
"0.50",
"--pool-workers",
"4",

"--conversation-cache-mib", "8192"
],
"cwd": "C:\strata\Strata",
"parallel": 4,
"tokenizer": "C:\strata\Strata-data\packs\q2_0\tokenizer",
"model_name": "qwen3.8-flash-next-q2_0",
"log": "C:\strata\Strata\strata-q2_0.log",
"lib_dirs": [
"C:\strata\Strata\.venv\Lib\site-packages\nvidia\cu13\bin\x86_64"
],
"host":"192.168.0.3",
"port": 8080,
"gpu": 0,
"gpus_asked": true
}

有需要优化的地方 请大家指正。

另外默默问一句， mac ultra 96 是不是会完败 5090 staraa?

[David Chen]

我有沒有看錯！我3060那辣雞都跑iq3_s了，你5090只跑q2?

[tester12340]

我需要长上下文啊 兄弟。而且 "--kv","q4_0" 感觉要不要回到 27b了。

[stxpnet]

你真的别用Q2，可能造成工具链断裂，然后无限循环

[tester12340]

我正在体验无限循环。 但我切到 qwen flash 在线也是循环。估计人品不行。

[terry]

@tester12340 你这发帖格式是给论坛做贡献吗？让AI整理成markdwon代码再发

[tester12340]

按terry 老板要求

{
"exe": "C:\strata\Strata\engine\strata.exe",
"args": [
"--pack",
"C:\strata\Strata-data\packs\q2_0",
"--native",
"C:\strata\Strata-data\models\Q2_0\Qwen3.8-Flash-Next-GSQ-RCO-Q2_0-00001-of-00002.gguf",
"--ple-gguf",
"C:\strata\Strata-data\models\Q2_0\Qwen3.8-Flash-Next-GSQ-RCO-Q2_0-00002-of-00002.gguf",
"--expert-profile",
"C:\strata\Strata\data\expert-profile.bin",
"--expert-cache",
"auto",
"--prefill",
"auto",
"--spec",
"4",
"--mtp",
"C:\strata\Strata-data\mtp\rt",
"--max-context",
"131072",
"--kv",
"q4_0",
"--kv-resident",
"32768",
"--vram-reserve-mib",
"700",
"--resident-experts",
"--pcie-frac",
"0.35",
"--spec-min-p",
"0.50",
"--pool-workers",
"4",
"--conversation-cache-mib",
"8192"
],
"cwd": "C:\strata\Strata",
"parallel": 4,
"tokenizer": "C:\strata\Strata-data\packs\q2_0\tokenizer",
"model_name": "qwen3.8-flash-next-q2_0",
"log": "C:\strata\Strata\strata-q2_0.log",
"lib_dirs": [
"C:\strata\Strata\.venv\Lib\site-packages\nvidia\cu13\bin\x86_64"
],
"host": "192.168.0.3",
"port": 8080,
"gpu": 0,
"gpus_asked": true
}

这个对了吗？

[tester12340]

@stxpnet 我试试 Q3 能不能跑

[tester12340]

下载中

[tester12340]

这个是不是太慢了？

[tester12340]

从 300tk 直接降到 20tk

[tester12340]

晕倒，我再 4080 上跑了一下，竟然跟 5090 速度差不多 T_T 我了个晕倒。

[Tony Wang]

@tester12340

你用```分别在参数配置部分的上方和下方, 这个表示中间是代码的意思, 也可以一键复制.

建议学一下Mark Down 语法, 比较简单, 常用的也就那么几条.

[tester12340]

@Tony-Wang

谢谢 我试试

{
"exe": "C:\strata\Strata\engine\strata.exe",
"args": [
"--pack",
"C:\strata\Strata-data\packs\q2_0",
"--native",
"C:\strata\Strata-data\models\Q2_0\Qwen3.8-Flash-Next-GSQ-RCO-Q2_0-00001-of-00002.gguf",
"--ple-gguf",
"C:\strata\Strata-data\models\Q2_0\Qwen3.8-Flash-Next-GSQ-RCO-Q2_0-00002-of-00002.gguf",
"--expert-profile",
"C:\strata\Strata\data\expert-profile.bin",
"--expert-cache",
"auto",
"--prefill",
"auto",
"--spec",
"4",
"--mtp",
"C:\strata\Strata-data\mtp\rt",
"--max-context",
"131072",
"--kv",
"q4_0",
"--kv-resident",
"32768",
"--vram-reserve-mib",
"700",
"--resident-experts",
"--pcie-frac",
"0.35",
"--spec-min-p",
"0.50",
"--pool-workers",
"4",
"--conversation-cache-mib",
"8192"
],
"cwd": "C:\strata\Strata",
"parallel": 4,
"tokenizer": "C:\strata\Strata-data\packs\q2_0\tokenizer",
"model_name": "qwen3.8-flash-next-q2_0",
"log": "C:\strata\Strata\strata-q2_0.log",
"lib_dirs": [
"C:\strata\Strata\.venv\Lib\site-packages\nvidia\cu13\bin\x86_64"
],
"host": "192.168.0.3",
"port": 8080,
"gpu": 0,
"gpus_asked": true
}