Confirmed bolting Q8 NGram into IQ4 Qwen no speed degradation

Excerpt

This came from another thread or comment. I forgot exactly where, but the basic idea was to replace the 51B N-gram layer in Qwen 3.8 Next with a much higher precision version. Someone running a 5090 replaced the N-gram portion of their Qwen 3.8 UD Q4 model with BF16. Since I'm already running IQ4_XS, I wanted to try something similar. I don't have enough storage for the BF16 N-gram weights right now, so I replaced the lower-precision N-gram portion with Q8 instead. I'm still testing whether it i