Reddit r/LocalLLaMAAugust 22, 2026
Need support for llama.cpp with multi GPU
Excerpt
Using llama.cpp I seem to be unable to get my to GPUs working tougether correclty, so I need help somehow. Setup: 96GB RAM, one Blackwell 5000 (48GB) and one 3090 (24GB). I am trying to run the UD-Q3_K_XL quant of Deepseek4 flash which has about 120GB size. Using just the Blackwell I would put most of the experts on the system RAM. I would have thought that putting some experts on the 3090 I would have thought to be faster (or at least not slower) than putting the experts just on the RAM. What I