Qwen3.8-Flash-Next on 2x3090 + DDR4: 17 → 25-29 t/s decode with the expert cache PR

Excerpt

Sharing some numbers because most posts on this model are either using a single 3090 or unified systems from what I've seen. My current setup: 2x RTX 3090 (PCIe 3.0), dual Xeon E5-2696 v4, 188 GB usable (192GB) DDR4-2133 LRDIMM, llama.cpp, unsloth UD-Q6_K_XL. All 48 expert layers pinned in host RAM, everything else on the GPUs. Full 261k context, f16 KV. Before: ~17 t/s decode, ~350 t/s prefill on a 26k prompt, 12 t/s decode at 131k depth. Now: 25-29 t/s decode short and mid context, ~17 at 131k