GGUFs in transformers natively!

Excerpt

Hey there folks! Aritra here from Hugging Face. I wanted to update you all about the latest changes in `transformers`. We now natively support GGUFs (llama cpp quants). You can use it like so: from transformers import AutoModelForCausalLM, AutoTokenizer model_id = "unsloth/Qwen3.5-4B-GGUF" filename = "Qwen3.5-4B-Q4_K_M.gguf" model = AutoModelForCausalLM.from_pretrained( model_id, gguf_file=filename, ) After loading, you're using the normal Transformers APIs. Why did we want to do this? Quantized