Strata/单卡7900xtx 配置一例 (IQ3_XXS, pp:~1400,tp:~65)

[laobenxiong]

最近也折腾了一下 Strata. 报告一下配置情况:

• 主机: 铭凡 um880pro;
• 内存: 96G DDR5
• 显卡: 7900xtx, oculink, gen4x4

配置 (venv 中 rocm 版本 v10.1):

 $ cat strata-swift-iq3_xxs.json
{
  "exe": "/tank/github/Strata/engine/strata",
  "args": [
    "--pack",
    "/tank/models/Strata-data/packs/swift-iq3_xxs",
    "--native",
    "/tank/models/Strata-data/models/swift-IQ3_XXS/Swift-Qwen3.8-Flash-Next-GSQ-RCO-IQ3_XXS-00001-of-00002.gguf",
    "--ple-gguf",
    "/tank/models/Strata-data/models/swift-IQ3_XXS/Swift-Qwen3.8-Flash-Next-GSQ-RCO-IQ3_XXS-00001-of-00002.gguf",
    "--expert-profile",
    "/tank/github/Strata/data/expert-profile.bin",
    "--expert-cache",
    "auto",
    "--prefill",
    "32768",
    "--spec",
    "4",
    "--spec-min-p",
    "0.5",
    "--mtp",
    "/tank/models/Strata-data/mtp/rt",
    "--max-context",
    "262144",
    "--kv",
    "int8",
    "--kv-resident",
    "32768"
  ],
  "cwd": "/tank/github/Strata",
  "tokenizer": "/tank/models/Strata-data/packs/swift-iq3_xxs/tokenizer",
  "model_name": "swift-1.5-iq3_xxs",
  "log": "/tank/github/Strata/strata-swift-iq3_xxs.log",
  "lib_dirs": [
    "/tank/github/Strata/.venv/lib/python3.13/site-packages/_rocm_sdk_devel/lib",
    "/tank/github/Strata/.venv/lib/python3.13/site-packages/_rocm_sdk_libraries_gfx110X_dgpu/lib"
  ],
  "port": 8080,
  "backend": "hip",
  "env": {
    "STRATA_HIPBLASLT_TUNING": "/home/bruin/strata-setup/100401-retuned.txt",
    "STRATA_HIPBLASLT_VERBOSE": "1"
  },
  "gpu": 0,
  "gpus_asked": true,
  "host": "0.0.0.0",
  "api_key": "strata",
  "open_browser": false

btw, 双卡的情况下 (peer-device), 速度下降, 应该是我的pcie带宽不够 ,一个gen4x4, 一个gen3x4(雷电3).

[laobenxiong]

不过长上下文实测有 loop: