Running DeepSeek V4 Flash Q4_K_XL at ~100 tok/s prompt processing on 4× RTX 3060 12GB

Excerpt

I managed to run the 143–144 GiB DeepSeek-V4-Flash-0731 UD-Q4_K_XL GGUF on four RTX 3060 12GB cards while keeping a 360k–376k context window. Hardware: CPU: Intel Core i9-10920X, 12C/24T RAM: 128 GB DDR4-3200, quad-channel GPU: 4× NVIDIA RTX 3060 12GB Total VRAM: 48 GB Storage: NVMe SSD Engine: llama.cpp, build b10181 Model: unsloth/DeepSeek-V4-Flash-0731-GGUF Quant: UD-Q4_K_XL, approximately 144 GiB KV cache: Q8_0 The best high-speed configuration so far: llama-server \ -m DeepSeek-V4-Flash-073