<![CDATA[Throughput on RTX 3090 (Qwen3.6-27B AWQ-Marlin BF16, BF16 KV, ctx=2048)]]>c654fc6b-8155-4d1d-bb58-f48b2b0f0371-image.jpeg

回馈以下社区，昨天刚去本地买了块公版3090，回来之后让claude配置sglang，这是目前的结果。系统是ubuntu

75ad769c-68a2-45d4-a80b-1b2e64d5dd54-image.jpeg

]]>https://lcz.me/topic/261/throughput-on-rtx-3090-qwen3.6-27b-awq-marlin-bf16-bf16-kv-ctx-2048RSS for NodeSat, 06 Jun 2026 07:04:23 GMTFri, 22 May 2026 15:07:55 GMT60<![CDATA[Reply to Throughput on RTX 3090 (Qwen3.6-27B AWQ-Marlin BF16, BF16 KV, ctx=2048) on Fri, 29 May 2026 06:05:59 GMT]]>vLLM 可以運行 32k 上下文，對於Agent用途來說還不錯，MTP速度為 50~60 tk/s @250w

--model ~/AiModel/int4-AutoRound
--gpu-memory-utilization 0.95
--max-model-len 32768
--enable-auto-tool-choice
--tool-call-parser qwen3_coder 0
--language-model-only
--host 0.0.0.0 --port 8000
--kv-cache-dtype fp8_e5m2
--max-num-seqs 1
--max-num-batched-tokens 4128
--trust-remote-code
--dtype bfloat16
--enable-prefix-caching
--enable-chunked-prefill
--no-scheduler-reserve-full-isl
--speculative-config '{"method":"mtp","num_speculative_tokens":3}'

]]>https://lcz.me/post/4161https://lcz.me/post/4161Fri, 29 May 2026 06:05:59 GMT<![CDATA[Reply to Throughput on RTX 3090 (Qwen3.6-27B AWQ-Marlin BF16, BF16 KV, ctx=2048) on Wed, 27 May 2026 08:36:31 GMT]]>c29489bb-2352-474f-bd60-2cdf58555ca4-image.jpeg
我也是用claude架的，這是我的配置

]]>https://lcz.me/post/3914https://lcz.me/post/3914Wed, 27 May 2026 08:36:31 GMT<![CDATA[Reply to Throughput on RTX 3090 (Qwen3.6-27B AWQ-Marlin BF16, BF16 KV, ctx=2048) on Sun, 24 May 2026 11:21:00 GMT]]>@johnnybegood LV并不比普通包体验好多少，好肯定好一点，但是没好那么多。不过有钱人都买LV。

]]>https://lcz.me/post/3395https://lcz.me/post/3395Sun, 24 May 2026 11:21:00 GMT<![CDATA[Reply to Throughput on RTX 3090 (Qwen3.6-27B AWQ-Marlin BF16, BF16 KV, ctx=2048) on Sun, 24 May 2026 06:14:13 GMT]]>@Larry-Wang opus 4.7 比 deepseek 4.0 pro 到底好在哪呢？

]]>https://lcz.me/post/3368https://lcz.me/post/3368Sun, 24 May 2026 06:14:13 GMT<![CDATA[Reply to Throughput on RTX 3090 (Qwen3.6-27B AWQ-Marlin BF16, BF16 KV, ctx=2048) on Sun, 24 May 2026 02:05:01 GMT]]>@Larry-Wang 价格放哪了。opus 4.7 造价不菲啊。建议重点调试用。或试试国产。

]]>https://lcz.me/post/3331https://lcz.me/post/3331Sun, 24 May 2026 02:05:01 GMT<![CDATA[Reply to Throughput on RTX 3090 (Qwen3.6-27B AWQ-Marlin BF16, BF16 KV, ctx=2048) on Sat, 23 May 2026 21:41:35 GMT]]>@Larry-Wang 不着急等Qwen3.7 27b发布之后，我相信Sg-Lang的支持会更好，到时候一起折腾，估计也就这两三周的事了。

]]>https://lcz.me/post/3311https://lcz.me/post/3311Sat, 23 May 2026 21:41:35 GMT<![CDATA[Reply to Throughput on RTX 3090 (Qwen3.6-27B AWQ-Marlin BF16, BF16 KV, ctx=2048) on Sat, 23 May 2026 15:50:33 GMT]]>@williamlouis claude就用的opus 4.7，3090感觉目前跑不了qwen 3.6 27b sglang

]]>https://lcz.me/post/3296https://lcz.me/post/3296Sat, 23 May 2026 15:50:33 GMT<![CDATA[Reply to Throughput on RTX 3090 (Qwen3.6-27B AWQ-Marlin BF16, BF16 KV, ctx=2048) on Sat, 23 May 2026 15:47:37 GMT]]>@terry 4090 48g应该跑得下来，3090 24g sglang目前估计够呛，需要两张卡

]]>https://lcz.me/post/3295https://lcz.me/post/3295Sat, 23 May 2026 15:47:37 GMT<![CDATA[Reply to Throughput on RTX 3090 (Qwen3.6-27B AWQ-Marlin BF16, BF16 KV, ctx=2048) on Fri, 22 May 2026 20:21:21 GMT]]>很有价值。谢谢。claude 选的什么版本？现在 open AI 5.5 有人使用调试成功了。claude 这是也站起来了。

]]>https://lcz.me/post/3171https://lcz.me/post/3171Fri, 22 May 2026 20:21:21 GMT<![CDATA[Reply to Throughput on RTX 3090 (Qwen3.6-27B AWQ-Marlin BF16, BF16 KV, ctx=2048) on Fri, 22 May 2026 19:16:12 GMT]]>这效果相当不错了，发个详细的教程，我也找时间抄作业，我给4090上下sg-lang，之前总是乱码。 2048上下文长度太短了，没什么意义，测试更长一点，最少64k。

]]>https://lcz.me/post/3162https://lcz.me/post/3162Fri, 22 May 2026 19:16:12 GMT