exllamav3 comfortably beats llama.cpp running CPU-offloaded Qwen-3.8-Flash-Next on my setup!

Excerpt

I got 2x 20GB RTX 3080s + 128GB of DDR4 2666hz RAM (only 4 of 6 channels populated) + a Xeon 6148 I've always been a llama.cpp person and I've been running Unsloth's Q4_K_XL quant of Qwen 3.8 Flash Next at ~270tps prefill and ~13tps decode (starts off close to 20 and falls down to 13 with growing context, never recovers) I've been recently seeing this buzz about exllamav3: its new "CPU-offloaded inference" support, how its quants are more efficient in terms of size-to-performance compared to GGU