<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title><![CDATA[单3090Ti（三风扇版本）跑Qwen3.8-27B，llama.cpp和NInfer对比]]></title><description><![CDATA[<p dir="auto">在看了论坛中大佬的帖子，<br />
自己在X99洋垃圾+3090Ti上做了测试：<br />
第一轮：llama.cpp加载Unsloth的Qwen3.8-27B_q4_k_m.gguf<br />
第二轮：Ninfer-3090加载默认的v2版本模型<a href="https://huggingface.co/neroued/Qwen3.8-27B-NInfer/resolve/main/qwen3_8_27b.ninfer" rel="nofollow ugc">https://huggingface.co/neroued/Qwen3.8-27B-NInfer/resolve/main/qwen3_8_27b.ninfer</a></p>
<p dir="auto">这里有一个坑，不能用download-qwen38.bat直接下，默认会下v3版本，这个版本和Ninfer-0.6.1windows版不兼容。</p>
<p dir="auto">用Hermes agent完成我自己的日常任务的日志做对比分析，上结论：<br />
<img src="https://upload.lcz.me/uploads/2267889a-b0a0-4bbc-ad93-c95216d9e5b1.jpeg" alt="0acab8bf-30b2-4f39-8819-517c18579155-image.jpeg" class=" img-fluid img-markdown" /></p>
]]></description><link>https://lcz.me/topic/1857</link><generator>RSS for Node</generator><lastBuildDate>Fri, 25 Sep 2026 22:22:28 GMT</lastBuildDate><atom:link href="https://lcz.me/topic/1857.rss" rel="self" type="application/rss+xml"/><pubDate>Mon, 21 Sep 2026 08:02:43 GMT</pubDate><ttl>60</ttl><item><title><![CDATA[Reply to 单3090Ti（三风扇版本）跑Qwen3.8-27B，llama.cpp和NInfer对比 on Wed, 23 Sep 2026 07:02:50 GMT]]></title><description><![CDATA[<p dir="auto"><a class="plugin-mentions-user plugin-mentions-a" href="/user/joker_chang" aria-label="Profile: joker_chang">@<bdi>joker_chang</bdi></a>  我遇到过泰文思考的时候死循环，而且概率还是80%左右，有时候处理多语言还是不太方便</p>
]]></description><link>https://lcz.me/post/20232</link><guid isPermaLink="true">https://lcz.me/post/20232</guid><dc:creator><![CDATA[Prio]]></dc:creator><pubDate>Wed, 23 Sep 2026 07:02:50 GMT</pubDate></item><item><title><![CDATA[Reply to 单3090Ti（三风扇版本）跑Qwen3.8-27B，llama.cpp和NInfer对比 on Wed, 23 Sep 2026 00:51:12 GMT]]></title><description><![CDATA[<p dir="auto"><a class="plugin-mentions-user plugin-mentions-a" href="/user/prio" aria-label="Profile: Prio">@<bdi>Prio</bdi></a> <a href="https://github.com/Don-Chad/ninfer-3090%E8%BF%99%E4%B8%AA%E9%A1%B9%E7%9B%AE%E7%9A%84%E9%A2%84%E7%BC%96%E8%AF%91%E5%8C%85NInfer-3090" rel="nofollow ugc">https://github.com/Don-Chad/ninfer-3090这个项目的预编译包NInfer-3090</a> v0.6.1（<a href="https://github.com/Don-Chad/ninfer-3090/releases%EF%BC%89" rel="nofollow ugc">https://github.com/Don-Chad/ninfer-3090/releases）</a></p>
<p dir="auto">启动脚本就是改了包中的bat文件，参数如下：<br />
--max-context 163840 --kv-capacity 163840 --max-concurrency 1 --max-pending-requests 2 --pending-timeout-ms 5000 --prefill-chunk 1024 --kv-dtype int8 --spec mtp --draft-tokens 3</p>
<p dir="auto">唯一的坑就是：<br />
HF 仓库 main 分支 9/15 被更新为 v3 版，v2 版（16.96GiB）在 8/19 的 commit 18dfc887423f，已验证该 commit 的 SHA256SUMS 与官方 model card 完全一致：eec39564...b80199514bf3e<br />
NInfer-3090 v0.6.1能使用的正确版本模型要下载这个：【<a href="https://hf-mirror.com/neroued/Qwen3.8-27B-NInfer/resolve/18dfc887423fa5aabf3cb56fac41490e462b3fab/qwen3_8_27b.ninfer%E3%80%91" rel="nofollow ugc">https://hf-mirror.com/neroued/Qwen3.8-27B-NInfer/resolve/18dfc887423fa5aabf3cb56fac41490e462b3fab/qwen3_8_27b.ninfer】</a></p>
<p dir="auto">对了补充一句，unsloth的版本Hermes agent经常出现死循环，但是NInfer版本我从开始用到现在，还真没遇到过之前的死循环。</p>
]]></description><link>https://lcz.me/post/20151</link><guid isPermaLink="true">https://lcz.me/post/20151</guid><dc:creator><![CDATA[joker_chang]]></dc:creator><pubDate>Wed, 23 Sep 2026 00:51:12 GMT</pubDate></item><item><title><![CDATA[Reply to 单3090Ti（三风扇版本）跑Qwen3.8-27B，llama.cpp和NInfer对比 on Tue, 22 Sep 2026 19:45:56 GMT]]></title><description><![CDATA[<p dir="auto">NInfer好像容易陷入循环思考。对了，你的方案可以打包分享一下不。</p>
]]></description><link>https://lcz.me/post/20130</link><guid isPermaLink="true">https://lcz.me/post/20130</guid><dc:creator><![CDATA[Prio]]></dc:creator><pubDate>Tue, 22 Sep 2026 19:45:56 GMT</pubDate></item><item><title><![CDATA[Reply to 单3090Ti（三风扇版本）跑Qwen3.8-27B，llama.cpp和NInfer对比 on Tue, 22 Sep 2026 01:18:36 GMT]]></title><description><![CDATA[<p dir="auto">典型的长上下文 agent 工具调用任务</p>
<p dir="auto"><img src="https://upload.lcz.me/uploads/00deb941-49fe-4a9d-9a5e-f170074dd551.jpeg" alt="image.jpeg" class=" img-fluid img-markdown" /></p>
<p dir="auto"><img src="https://upload.lcz.me/uploads/ee805b8b-4279-4a1c-a95e-87c2dee0b5bc.jpeg" alt="image.jpeg" class=" img-fluid img-markdown" /></p>
<p dir="auto">DeepSeek总结的问题：<br />
1、内存余量极小，free-after-startup=131 MiB，headroom=0，风险高。<br />
2、full_reset 冷启动长上下文非常慢：120K prompt TTFT 约 168 秒。<br />
3、缓存检查点不够新，后期 cache 停在 74,659，导致增量 prefill 仍要 60～80 秒。<br />
4、单并发串行，长任务总延迟会累积。<br />
5、独立小请求/探针请求可能触发 full_reset，打断长会话缓存，进一步拉高后续 TTFT。</p>
]]></description><link>https://lcz.me/post/19884</link><guid isPermaLink="true">https://lcz.me/post/19884</guid><dc:creator><![CDATA[joker_chang]]></dc:creator><pubDate>Tue, 22 Sep 2026 01:18:36 GMT</pubDate></item><item><title><![CDATA[Reply to 单3090Ti（三风扇版本）跑Qwen3.8-27B，llama.cpp和NInfer对比 on Mon, 21 Sep 2026 09:05:09 GMT]]></title><description><![CDATA[<p dir="auto">验证完了：</p>
<p dir="auto"><img src="https://upload.lcz.me/uploads/62616fac-7e8c-485a-8888-d7067ea7a2f8.jpeg" alt="image.jpeg" class=" img-fluid img-markdown" /></p>
]]></description><link>https://lcz.me/post/19718</link><guid isPermaLink="true">https://lcz.me/post/19718</guid><dc:creator><![CDATA[joker_chang]]></dc:creator><pubDate>Mon, 21 Sep 2026 09:05:09 GMT</pubDate></item><item><title><![CDATA[Reply to 单3090Ti（三风扇版本）跑Qwen3.8-27B，llama.cpp和NInfer对比 on Mon, 21 Sep 2026 08:39:40 GMT]]></title><description><![CDATA[<p dir="auto">一轮33 个请求的较长任务执行情况：</p>
<p dir="auto"><img src="https://upload.lcz.me/uploads/14396c5e-7946-415c-a608-001b66ba8442.jpeg" alt="image.jpeg" class=" img-fluid img-markdown" /></p>
<p dir="auto">总体上提升很多，但是在MTP 3 投机解码表现我不太满意：</p>
<p dir="auto"><img src="https://upload.lcz.me/uploads/171695f0-ac98-45a0-aea8-97fa272e56cd.jpeg" alt="image.jpeg" class=" img-fluid img-markdown" /></p>
<p dir="auto">计划MTP设置成2再试试。</p>
]]></description><link>https://lcz.me/post/19712</link><guid isPermaLink="true">https://lcz.me/post/19712</guid><dc:creator><![CDATA[joker_chang]]></dc:creator><pubDate>Mon, 21 Sep 2026 08:39:40 GMT</pubDate></item><item><title><![CDATA[Reply to 单3090Ti（三风扇版本）跑Qwen3.8-27B，llama.cpp和NInfer对比 on Mon, 21 Sep 2026 08:10:04 GMT]]></title><description><![CDATA[<p dir="auto">补充一个我的发现：在windows10操作系统上，ninfer-serve.exe比llama-server占用的物理内存少很多。</p>
<p dir="auto">在其他应用不变的情况下，我用llama-server加载模型后，物理内存超过50GB；而用ninfer-serve.exe加载模型后的物理内存不到30GB。</p>
]]></description><link>https://lcz.me/post/19708</link><guid isPermaLink="true">https://lcz.me/post/19708</guid><dc:creator><![CDATA[joker_chang]]></dc:creator><pubDate>Mon, 21 Sep 2026 08:10:04 GMT</pubDate></item></channel></rss>