<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title><![CDATA[Proxmox VE 9.2+ Intel 核显 SR-IOV + AMD Radeon AI PRO R9700 直通+Ubuntu24.04部署实测（四）]]></title><description><![CDATA[<h1>Proxmox VE 9.2+ Intel 核显 SR-IOV + AMD Radeon AI PRO R9700 直通+Ubuntu24.04部署实测（四）</h1>
<hr />
<h2>一、本机验证过的调优</h2>
<p dir="auto">读取工作区实体机三份文档，核心调优配方如下，已对齐应用到本机：</p>
<table class="table table-bordered table-striped">
<thead>
<tr>
<th>参数</th>
<th>作用</th>
<th>实体机实测依据</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>--spec-type draft-mtp --spec-draft-n-max 2</code></td>
<td>MTP 投机解码</td>
<td>26 → 42~48 t/s（≈1.6 倍）</td>
</tr>
<tr>
<td><code>-ctk q8_0 -ctv q8_0</code></td>
<td>KV 缓存量化</td>
<td>省约一半 KV 显存并提速</td>
</tr>
<tr>
<td><code>--reasoning-effort medium</code></td>
<td>控制思考长度</td>
<td>治好默认 xhigh 的"雷霆大思考"</td>
</tr>
<tr>
<td><code>--temp 0.3 --top-p 0.9</code></td>
<td>低温采样</td>
<td>准确度↑ + MTP 接受率↑（94.2%）</td>
</tr>
<tr>
<td><code>--load-mode none</code></td>
<td>权重不经 mmap</td>
<td>模型 4 秒加载</td>
</tr>
<tr>
<td><code>-ngl all -np 1</code></td>
<td>全层进显存 + 单槽位</td>
<td>单用户 KV 池全归当前会话</td>
</tr>
<tr>
<td><code>--override-kv qwen35.context_length=int:131072</code></td>
<td>前端如实显示 128K</td>
<td>防客户端按 256K 发送超限</td>
</tr>
<tr>
<td><code>--alias qwen3.8</code></td>
<td>API 模型别名</td>
<td>客户端用 <code>qwen3.8</code> 调用</td>
</tr>
</tbody>
</table>
<hr />
<h2>二、最终生效的启动命令</h2>
<p dir="auto">由 systemd 单元 <code>/etc/systemd/system/qwen38.service</code> 执行（开机自启 + 崩溃自愈）：</p>
<pre><code class="language-bash">/opt/llama.cpp/bin/llama-server \
  -m /opt/llama.cpp/models/Qwen3.8-27B-Uncensored-Q5_K_M.gguf \
  --mmproj /opt/llama.cpp/models/mmproj-Qwen3.8-27B-Uncensored-F16.gguf \
  --alias qwen3.8 \
  -c 131072 --override-kv qwen35.context_length=int:131072 \
  -ctk q8_0 -ctv q8_0 -fa on --load-mode none \
  --reasoning-effort medium -ngl 99 -np 1 \
  --temp 0.3 --top-p 0.9 \
  --spec-type draft-mtp --spec-draft-n-max 2 \
  --host 0.0.0.0 --port 8080
</code></pre>
<blockquote>
<p dir="auto">环境变量：<code>LD_LIBRARY_PATH=/opt/llama.cpp/bin:/opt/rocm/lib</code>、<code>HIP_VISIBLE_DEVICES=0</code></p>
</blockquote>
<hr />
<h2>三、调优结果（前后对比）</h2>
<table class="table table-bordered table-striped">
<thead>
<tr>
<th>指标</th>
<th>调优前</th>
<th>调优后</th>
<th>提升</th>
</tr>
</thead>
<tbody>
<tr>
<td>生成速度</td>
<td>23 t/s</td>
<td><strong>40–41 t/s</strong></td>
<td><strong>↑ 1.75 倍</strong></td>
</tr>
<tr>
<td>MTP 投机解码</td>
<td>未启用</td>
<td>生效（接受率 67–78%）</td>
<td>—</td>
</tr>
<tr>
<td>思考长度</td>
<td>失控（曾跑 7900+ token）</td>
<td><strong>~126 字受控</strong></td>
<td>修复</td>
</tr>
<tr>
<td>显存占用</td>
<td>28.2GB（92%）</td>
<td><strong>26.8GB（88%）</strong></td>
<td>省 1.4GB</td>
</tr>
<tr>
<td>采样退化</td>
<td>有风险（temp 0.6）</td>
<td>无退化（temp 0.3）</td>
<td>修复</td>
</tr>
<tr>
<td>KV 缓存</td>
<td>f16</td>
<td>q8_0（省一半）</td>
<td>—</td>
</tr>
<tr>
<td>日志告警</td>
<td>有 unused tensor 警告</td>
<td>零告警零错误</td>
<td>修复</td>
</tr>
</tbody>
</table>
<hr />
<h2>四、调稳措施</h2>
<ol>
<li><strong>systemd 常驻</strong>：<code>/etc/systemd/system/qwen38.service</code>，状态 <code>enabled + active</code>，开机自启 + 崩溃 5 秒自动拉起（对齐实体机方案）。</li>
<li><strong>单槽位 <code>-np 1</code></strong>：单用户场景 KV 池全归当前会话，避免多槽抢占显存。</li>
<li><strong>低温采样 <code>temp 0.3</code></strong>：从根源杜绝"你你我我"式退化重复。</li>
</ol>
<h3>systemd 单元文件内容</h3>
<pre><code class="language-ini">[Unit]
Description=Qwen3.8-27B llama.cpp server (R9700 HIP, MTP, 128K)
After=network.target

[Service]
User=magicz890
Group=magicz890
Environment=LD_LIBRARY_PATH=/opt/llama.cpp/bin:/opt/rocm/lib
Environment=HIP_VISIBLE_DEVICES=0
ExecStart=/opt/llama.cpp/bin/llama-server \
  -m /opt/llama.cpp/models/Qwen3.8-27B-Uncensored-Q5_K_M.gguf \
  --mmproj /opt/llama.cpp/models/mmproj-Qwen3.8-27B-Uncensored-F16.gguf \
  --alias qwen3.8 \
  -c 131072 --override-kv qwen35.context_length=int:131072 \
  -ctk q8_0 -ctv q8_0 -fa on --load-mode none \
  --reasoning-effort medium -ngl 99 -np 1 \
  --temp 0.3 --top-p 0.9 \
  --spec-type draft-mtp --spec-draft-n-max 2 \
  --host 0.0.0.0 --port 8080
Restart=on-failure
RestartSec=5
TimeoutStopSec=60
KillSignal=SIGTERM

[Install]
WantedBy=multi-user.target
</code></pre>
<hr />
<h2>五、最终测试结果（systemd 运行中实测）</h2>
<table class="table table-bordered table-striped">
<thead>
<tr>
<th>测试项</th>
<th>结果</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>/health</code></td>
<td><code>{"status":"ok"}</code></td>
</tr>
<tr>
<td><code>/v1/models</code></td>
<td>别名 <code>qwen3.8</code>，上下文 131072（override-kv 生效）</td>
</tr>
<tr>
<td>文本生成</td>
<td>"我是一个由阿里巴巴通义实验室研发的大语言模型…"（40.17 t/s）</td>
</tr>
<tr>
<td>MTP 接受率</td>
<td>67–78%（随内容波动，mean len ≈ 2.2–2.6）</td>
</tr>
<tr>
<td>视觉识图</td>
<td>正确识别 "LLaMA C++" 封面及副标题</td>
</tr>
<tr>
<td>思考控制</td>
<td>思考长度 126 字（reasoning-effort medium 生效）</td>
</tr>
<tr>
<td>日志告警</td>
<td>零告警零错误（MTP 张量已正确加载，无 unused）</td>
</tr>
<tr>
<td>GPU</td>
<td>40°C、空闲 3%、无硬件错误（UEC=0）</td>
</tr>
</tbody>
</table>
<hr />
<p dir="auto"><img src="https://upload.lcz.me/uploads/0dc264b8-ee2d-441b-b971-5697042eef35.jpeg" alt="84b37c92-748d-4c7e-bd68-d928937297d4-image.jpeg" class=" img-fluid img-markdown" /></p>
]]></description><link>https://lcz.me/topic/1637</link><generator>RSS for Node</generator><lastBuildDate>Wed, 23 Sep 2026 03:26:29 GMT</lastBuildDate><atom:link href="https://lcz.me/topic/1637.rss" rel="self" type="application/rss+xml"/><pubDate>Fri, 11 Sep 2026 20:47:36 GMT</pubDate><ttl>60</ttl><item><title><![CDATA[Reply to Proxmox VE 9.2+ Intel 核显 SR-IOV + AMD Radeon AI PRO R9700 直通+Ubuntu24.04部署实测（四） on Sat, 12 Sep 2026 10:00:25 GMT]]></title><description><![CDATA[<p dir="auto"><a class="plugin-mentions-user plugin-mentions-a" href="/user/dajian-zhang" aria-label="Profile: Dajian-Zhang">@<bdi>Dajian-Zhang</bdi></a> 我给了48G，如果多开就给40G应该可以够用，其他分给别的虚拟机用。</p>
]]></description><link>https://lcz.me/post/17542</link><guid isPermaLink="true">https://lcz.me/post/17542</guid><dc:creator><![CDATA[Magic629]]></dc:creator><pubDate>Sat, 12 Sep 2026 10:00:25 GMT</pubDate></item><item><title><![CDATA[Reply to Proxmox VE 9.2+ Intel 核显 SR-IOV + AMD Radeon AI PRO R9700 直通+Ubuntu24.04部署实测（四） on Sat, 12 Sep 2026 00:33:05 GMT]]></title><description><![CDATA[<p dir="auto">好文， Host内存多大？</p>
]]></description><link>https://lcz.me/post/17414</link><guid isPermaLink="true">https://lcz.me/post/17414</guid><dc:creator><![CDATA[Dajian Zhang]]></dc:creator><pubDate>Sat, 12 Sep 2026 00:33:05 GMT</pubDate></item></channel></rss>