跳转至内容
  • 版块
  • 最新
  • 标签
  • 热门
  • 用户
  • 群组
皮肤
  • 浅色
  • Brite
  • Cerulean
  • Cosmo
  • Flatly
  • Journal
  • Litera
  • Lumen
  • Lux
  • Materia
  • Minty
  • Morph
  • Pulse
  • Sandstone
  • Simplex
  • Sketchy
  • Spacelab
  • United
  • Yeti
  • Zephyr
  • 深色
  • Cyborg
  • Darkly
  • Quartz
  • Slate
  • Solar
  • Superhero
  • Vapor

  • 默认(LCZ-Blue)
  • 不使用皮肤
  • LCZ-Green
  • LCZ-Blue
  • LCZ-Black
折叠
品牌标识

抡锤者

首页 版块 标签
硬件
装备秀
AI 广场 App
  1. 主页
  2. 版块
  3. LLM讨论区
  4. 各位大神有试过qwen3.8 27b超过256k 上下文怎么样吗?

各位大神有试过qwen3.8 27b超过256k 上下文怎么样吗?

已定时 已固定 已锁定 已移动 LLM讨论区
qwenr9700本地模型
9 帖子 8 发布者 370 浏览
  • 从旧到新
  • 从新到旧
  • 最多赞同
回复
  • 在新帖中回复
登录后回复
此主题已被删除。只有拥有主题管理权限的用户可以查看。
  • Fachao HuangF
    Fachao HuangF
    Fachao Huang
    编写于 最后由 编辑
    #1

    我有双卡r9700不太需要多会话。所以想问一下有人过超过256k的效果怎么样?

    1 条回复 最后回复
    0
    • David ChenD
      David ChenD
      David Chen
      王侯将相
      编写于 最后由 David Chen 编辑
      #2

      160K 以上會錯亂,不穩定(我沒證據,只憑感覺)
      所以我都設128K+subprocess工作(機器夠快,已經夠用了)

      1 条回复 最后回复
      0
      • Tony WangT
        Tony WangT
        Tony Wang
        易有太极
        编写于 最后由 编辑
        #3

        Qwen3.8 27B的上下文最大 256K, 到一定阶段(Hermes缺省50%), 就会启动压缩, 以减少上下文. 长链任务和多轮对话, 很有可能会突破256K, 但我实测本地27B NVFP4的表现不错, 基本不会跑偏.

        1 条回复 最后回复
        0
        • williamlouisW
          williamlouisW
          williamlouis
          易有太极
          编写于 最后由 编辑
          #4

          qwen 4 有可能。而且快公布了。等等吧。

          个人主页:xlkj.org Telegram https://t.me/xlkjorg

          1 条回复 最后回复
          0
          • N
            N
            neo
            体面乡绅 硕果累累
            编写于 最后由 编辑
            #5

            我做过测试350k,单针跟256k一样很准,但多针命中率下降得厉害。

            A 1 条回复 最后回复
            0
            • 范志刚范
              范志刚范
              范志刚
              硕果累累 贵族大夫
              编写于 最后由 编辑
              #6

              128k以后会打醉拳螳螂拳虎鹤双型什么的,其实是64k就开始语义化整理出现偏差

              1 条回复 最后回复
              0
              • N neo

                我做过测试350k,单针跟256k一样很准,但多针命中率下降得厉害。

                A
                A
                applejuice
                王侯将相
                编写于 最后由 applejuice 编辑
                #7

                @neo 我没用27b 超过256k 因为实在太慢了

                但是我用 qwen next flash 512k 成绩跟256k 没分别

                有兴趣可以叫agent 试试

                #!/usr/bin/env python3
                """long_suite.py - harder long-context checks than a single needle, through a running Strata server.
                
                For each length one document is built (the same filler as Strata's tools/needle_bench.py) with every test item planted
                in it at fixed depths; the questions are then asked one after another with the same document as prefix, so Strata
                reads the document once and only the question after that. Thinking off, greedy, short answers. The document and the
                items depend only on the length (fixed seed), so two server configs get identical tests.
                
                Tasks
                  multikey  8 people each get a similar code word (word-word-123); 4 of them are asked for       (confusable facts)
                  vartrack  two chains of 4 assignments (VAR_x = number, VAR_y = VAR_x, ...) spread over the text;
                            the last variable of each chain is asked for                                         (multi-hop)
                  nolima    3 facts phrased without the question's words (lives next to the Semper Opera House ->
                            "who has been to Dresden?")                                                         (no literal match)
                
                    python3 long_suite.py --lengths 64k,128k,256k --label cfg262k --out results.json
                """
                import argparse, json, random, sys, time, urllib.request
                from pathlib import Path
                
                sys.path.insert(0, str(Path.home() / "bench_haystack" / "tools"))   # was ~/strata-hip/tools (deleted 10-26): a copy of the image /opt/strata text folders (haystack source)
                import needle_bench as NB  # haystack() + WORDS
                
                NAMES = ["Marisol Venn", "Torsten Achebe", "Ilse Okafor", "Ravi Lindqvist", "Noor Castellanos", "Bram Takahashi",
                         "Celia Oduya", "Henrik Moreau"]
                VARS = ["VAR_KQX", "VAR_PLM", "VAR_ZTR", "VAR_WEN", "VAR_HUD", "VAR_BOC", "VAR_JYF", "VAR_SAG"]
                NOLIMA = [("Yuki", "Actually, Yuki lives right next to the Semper Opera House.", "Dresden"),
                          ("Dmitri", "As it happens, Dmitri spent last summer walking the gardens of the Alhambra every morning.", "Granada"),
                          ("Amara", "Amara's office window looks straight at the Petronas Twin Towers.", "Kuala Lumpur")]
                
                
                def build(tokens: int):
                    rnd = random.Random(tokens)
                    text = NB.haystack(int(tokens * 0.97 * NB.CHARS_PER_TOKEN))
                    items, plants = [], []                       # plants: (depth %, sentence)
                    # multikey: 8 codes at depths 5..95, ask 4 of them
                    depths = [5, 17, 29, 41, 53, 65, 77, 95]
                    codes = {}
                    for name, d in zip(NAMES, depths):
                        code = f"{rnd.choice(NB.WORDS)}-{rnd.choice(NB.WORDS)}-{rnd.randint(100, 999)}"
                        codes[name] = (code, d)
                        plants.append((d, f"Registry entry: the special access code assigned to {name} is {code}."))
                    for name in (NAMES[0], NAMES[3], NAMES[5], NAMES[7]):
                        code, d = codes[name]
                        items.append(("multikey", d, f"In the text above, what is the special access code assigned to {name}? "
                                      "Reply with the code only.", code))
                    # vartrack: two interleaved 4-hop chains
                    chains = [(VARS[0:4], [8, 31, 58, 86]), (VARS[4:8], [14, 44, 71, 92])]
                    for vs, ds in chains:
                        val = str(rnd.randint(10000, 99999))
                        plants.append((ds[0], f"Config note: let {vs[0]} = {val}."))
                        for prev, cur, d in zip(vs, vs[1:], ds[1:]):
                            plants.append((d, f"Config note: let {cur} = {prev}."))
                        items.append(("vartrack", ds[0], f"Following the 'Config note: let' assignments in the text above, which number "
                                      f"does {vs[-1]} ultimately equal? Reply with the number only.", val))
                    # nolima
                    for (name, sent, city), d in zip(NOLIMA, [20, 55, 90]):
                        plants.append((d, sent))
                        items.append(("nolima", d, f"Based on the text above, which character has been to {city}? "
                                      "Reply with the name only.", name))
                    # plant at line starts, from the deepest so earlier offsets stay valid
                    for d, sent in sorted(plants, key=lambda p: -p[0]):
                        cut = int(len(text) * d / 100)
                        cut = text.rfind("\n", 0, cut) + 1 or cut
                        text = text[:cut] + "\n" + sent + "\n" + text[cut:]
                    return text, items
                
                
                def ask(url, doc, q, timeout):
                    body = {"model": "strata", "max_tokens": 60, "temperature": 0, "chat_template_kwargs": {"enable_thinking": False},
                            "messages": [{"role": "user", "content": doc + "\n\n" + q}]}
                    req = urllib.request.Request(url.rstrip("/") + "/v1/chat/completions", data=json.dumps(body).encode(),
                                                 headers={"Content-Type": "application/json"})
                    t0 = time.time()
                    with urllib.request.urlopen(req, timeout=timeout) as r:
                        out = json.loads(r.read())
                    return (out["choices"][0]["message"].get("content") or "").strip(), out["usage"]["prompt_tokens"], time.time() - t0
                
                
                def main():
                    ap = argparse.ArgumentParser()
                    ap.add_argument("--url", default="http://127.0.0.1:8080")
                    ap.add_argument("--lengths", default="64k,128k")
                    ap.add_argument("--label", default="")
                    ap.add_argument("--timeout", type=float, default=3600)
                    ap.add_argument("--out")
                    a = ap.parse_args()
                    rows = []
                    for L in a.lengths.split(","):
                        tokens = int(float(L.lower().rstrip("k")) * 1024) if L.lower().endswith("k") else int(L)
                        doc, items = build(tokens)
                        for task, d, q, want in items:
                            try:
                                ans, n, secs = ask(a.url, doc, q, a.timeout)
                            except Exception as e:
                                print(f"{a.label} {L:>5} {task:9} d{d:>3}%: request failed ({e})", flush=True)
                                rows.append({"label": a.label, "length": L, "task": task, "depth": d, "error": str(e)})
                                continue
                            ok = want.lower() in ans.lower()
                            print(f"{a.label} {L:>5} {task:9} d{d:>3}%: {'OK  ' if ok else 'MISS'} ({n:,} tok, {secs:5.0f} s)"
                                  + ("" if ok else f"  want {want!r} got {ans[:70]!r}"), flush=True)
                            rows.append({"label": a.label, "length": L, "task": task, "depth": d, "ok": ok, "want": want,
                                         "answer": ans[:200], "prompt_tokens": n, "seconds": round(secs, 1)})
                        done = [r for r in rows if r["length"] == L and "ok" in r]
                        by = {t: f"{sum(r['ok'] for r in done if r['task'] == t)}/{sum(1 for r in done if r['task'] == t)}"
                              for t in ("multikey", "vartrack", "nolima")}
                        print(f"{a.label} {L:>5} SUMMARY  multikey {by['multikey']}  vartrack {by['vartrack']}  nolima {by['nolima']}"
                              f"  total {sum(r['ok'] for r in done)}/{len(done)}", flush=True)
                    if a.out:
                        Path(a.out).write_text(json.dumps(rows, indent=1), encoding="utf-8")
                
                
                if __name__ == "__main__":
                    main()
                
                
                N 1 条回复 最后回复
                0
                • A applejuice

                  @neo 我没用27b 超过256k 因为实在太慢了

                  但是我用 qwen next flash 512k 成绩跟256k 没分别

                  有兴趣可以叫agent 试试

                  #!/usr/bin/env python3
                  """long_suite.py - harder long-context checks than a single needle, through a running Strata server.
                  
                  For each length one document is built (the same filler as Strata's tools/needle_bench.py) with every test item planted
                  in it at fixed depths; the questions are then asked one after another with the same document as prefix, so Strata
                  reads the document once and only the question after that. Thinking off, greedy, short answers. The document and the
                  items depend only on the length (fixed seed), so two server configs get identical tests.
                  
                  Tasks
                    multikey  8 people each get a similar code word (word-word-123); 4 of them are asked for       (confusable facts)
                    vartrack  two chains of 4 assignments (VAR_x = number, VAR_y = VAR_x, ...) spread over the text;
                              the last variable of each chain is asked for                                         (multi-hop)
                    nolima    3 facts phrased without the question's words (lives next to the Semper Opera House ->
                              "who has been to Dresden?")                                                         (no literal match)
                  
                      python3 long_suite.py --lengths 64k,128k,256k --label cfg262k --out results.json
                  """
                  import argparse, json, random, sys, time, urllib.request
                  from pathlib import Path
                  
                  sys.path.insert(0, str(Path.home() / "bench_haystack" / "tools"))   # was ~/strata-hip/tools (deleted 10-26): a copy of the image /opt/strata text folders (haystack source)
                  import needle_bench as NB  # haystack() + WORDS
                  
                  NAMES = ["Marisol Venn", "Torsten Achebe", "Ilse Okafor", "Ravi Lindqvist", "Noor Castellanos", "Bram Takahashi",
                           "Celia Oduya", "Henrik Moreau"]
                  VARS = ["VAR_KQX", "VAR_PLM", "VAR_ZTR", "VAR_WEN", "VAR_HUD", "VAR_BOC", "VAR_JYF", "VAR_SAG"]
                  NOLIMA = [("Yuki", "Actually, Yuki lives right next to the Semper Opera House.", "Dresden"),
                            ("Dmitri", "As it happens, Dmitri spent last summer walking the gardens of the Alhambra every morning.", "Granada"),
                            ("Amara", "Amara's office window looks straight at the Petronas Twin Towers.", "Kuala Lumpur")]
                  
                  
                  def build(tokens: int):
                      rnd = random.Random(tokens)
                      text = NB.haystack(int(tokens * 0.97 * NB.CHARS_PER_TOKEN))
                      items, plants = [], []                       # plants: (depth %, sentence)
                      # multikey: 8 codes at depths 5..95, ask 4 of them
                      depths = [5, 17, 29, 41, 53, 65, 77, 95]
                      codes = {}
                      for name, d in zip(NAMES, depths):
                          code = f"{rnd.choice(NB.WORDS)}-{rnd.choice(NB.WORDS)}-{rnd.randint(100, 999)}"
                          codes[name] = (code, d)
                          plants.append((d, f"Registry entry: the special access code assigned to {name} is {code}."))
                      for name in (NAMES[0], NAMES[3], NAMES[5], NAMES[7]):
                          code, d = codes[name]
                          items.append(("multikey", d, f"In the text above, what is the special access code assigned to {name}? "
                                        "Reply with the code only.", code))
                      # vartrack: two interleaved 4-hop chains
                      chains = [(VARS[0:4], [8, 31, 58, 86]), (VARS[4:8], [14, 44, 71, 92])]
                      for vs, ds in chains:
                          val = str(rnd.randint(10000, 99999))
                          plants.append((ds[0], f"Config note: let {vs[0]} = {val}."))
                          for prev, cur, d in zip(vs, vs[1:], ds[1:]):
                              plants.append((d, f"Config note: let {cur} = {prev}."))
                          items.append(("vartrack", ds[0], f"Following the 'Config note: let' assignments in the text above, which number "
                                        f"does {vs[-1]} ultimately equal? Reply with the number only.", val))
                      # nolima
                      for (name, sent, city), d in zip(NOLIMA, [20, 55, 90]):
                          plants.append((d, sent))
                          items.append(("nolima", d, f"Based on the text above, which character has been to {city}? "
                                        "Reply with the name only.", name))
                      # plant at line starts, from the deepest so earlier offsets stay valid
                      for d, sent in sorted(plants, key=lambda p: -p[0]):
                          cut = int(len(text) * d / 100)
                          cut = text.rfind("\n", 0, cut) + 1 or cut
                          text = text[:cut] + "\n" + sent + "\n" + text[cut:]
                      return text, items
                  
                  
                  def ask(url, doc, q, timeout):
                      body = {"model": "strata", "max_tokens": 60, "temperature": 0, "chat_template_kwargs": {"enable_thinking": False},
                              "messages": [{"role": "user", "content": doc + "\n\n" + q}]}
                      req = urllib.request.Request(url.rstrip("/") + "/v1/chat/completions", data=json.dumps(body).encode(),
                                                   headers={"Content-Type": "application/json"})
                      t0 = time.time()
                      with urllib.request.urlopen(req, timeout=timeout) as r:
                          out = json.loads(r.read())
                      return (out["choices"][0]["message"].get("content") or "").strip(), out["usage"]["prompt_tokens"], time.time() - t0
                  
                  
                  def main():
                      ap = argparse.ArgumentParser()
                      ap.add_argument("--url", default="http://127.0.0.1:8080")
                      ap.add_argument("--lengths", default="64k,128k")
                      ap.add_argument("--label", default="")
                      ap.add_argument("--timeout", type=float, default=3600)
                      ap.add_argument("--out")
                      a = ap.parse_args()
                      rows = []
                      for L in a.lengths.split(","):
                          tokens = int(float(L.lower().rstrip("k")) * 1024) if L.lower().endswith("k") else int(L)
                          doc, items = build(tokens)
                          for task, d, q, want in items:
                              try:
                                  ans, n, secs = ask(a.url, doc, q, a.timeout)
                              except Exception as e:
                                  print(f"{a.label} {L:>5} {task:9} d{d:>3}%: request failed ({e})", flush=True)
                                  rows.append({"label": a.label, "length": L, "task": task, "depth": d, "error": str(e)})
                                  continue
                              ok = want.lower() in ans.lower()
                              print(f"{a.label} {L:>5} {task:9} d{d:>3}%: {'OK  ' if ok else 'MISS'} ({n:,} tok, {secs:5.0f} s)"
                                    + ("" if ok else f"  want {want!r} got {ans[:70]!r}"), flush=True)
                              rows.append({"label": a.label, "length": L, "task": task, "depth": d, "ok": ok, "want": want,
                                           "answer": ans[:200], "prompt_tokens": n, "seconds": round(secs, 1)})
                          done = [r for r in rows if r["length"] == L and "ok" in r]
                          by = {t: f"{sum(r['ok'] for r in done if r['task'] == t)}/{sum(1 for r in done if r['task'] == t)}"
                                for t in ("multikey", "vartrack", "nolima")}
                          print(f"{a.label} {L:>5} SUMMARY  multikey {by['multikey']}  vartrack {by['vartrack']}  nolima {by['nolima']}"
                                f"  total {sum(r['ok'] for r in done)}/{len(done)}", flush=True)
                      if a.out:
                          Path(a.out).write_text(json.dumps(rows, indent=1), encoding="utf-8")
                  
                  
                  if __name__ == "__main__":
                      main()
                  
                  
                  N
                  N
                  neo
                  体面乡绅 硕果累累
                  编写于 最后由 编辑
                  #8

                  @applejuice QFN的测试刚做了一半,三档256/512/1024,目前的结果跟你说的一样,512无下滑,1024下滑了37%。因为一般agent的压缩阈值是0.8,所以准备再测下800K如何。

                  1 条回复 最后回复
                  0
                  • stxpnetS
                    stxpnetS
                    stxpnet
                    至尊皇帝
                    编写于 最后由 编辑
                    #9

                    主要看你的硬件,本质 不就是猜token机器,上下文越长,轮次越多,噪声越多,F16原版也没办法稳定不出错。 Q8就更加重 语义偏离了。

                    双卡3090 PCIE 4.0 16X/16X

                    白天(宰相) Strata QFN3.8 IQ4_XS 262K ~120 t/s
                    夜间(将军) Qengine 27B INT4-W4 FP8- MTP 7 262K X 2 ~120 t/s
                    1 条回复 最后回复
                    0

                    你好!看起来您对这段对话很感兴趣,但您还没有一个账号。

                    厌倦了每次访问都刷到同样的帖子?您注册账号后,您下次访问时都将自动回到上次浏览的位置,并可选择接收新回复的通知(通过电子邮件或推送通知)。您还可以收藏帖子、为帖子点赞,以此向其他社区成员表达您的感谢。

                    有了你的建议,这篇帖子会更精彩哦 💗

                    注册 登录
                    回复
                    • 在新帖中回复
                    登录后回复
                    • 从旧到新
                    • 从新到旧
                    • 最多赞同


                    • 登录

                    • 登录或注册以进行搜索。
                    • 第一个帖子
                      最后一个帖子
                    0
                    • 版块
                    • 最新
                    • 标签
                    • 热门
                    • 用户
                    • 群组