Exploring Quantization Backends in Diffusers

Source: HuggingFace Date: 2025-05-21 URL: https://huggingface.co/blog/diffusers-quantization

Summary

Library update and tutorial covering five quantization backends now integrated in Diffusers: bitsandbytes (4/8-bit), torchao, Quanto, GGUF, and FP8 layerwise casting. Benchmarked on Flux.1-dev on H100: BnB 4-bit cuts memory from 31.4GB to 12.6GB at same 12s inference time; FP8 + group offloading reaches 9.3GB loaded at 58s. torchao int4 achieves lowest memory (10.6GB) but 109s inference. Includes a PipelineQuantizationConfig API for per-component quantization configuration.

Implications

Thread: transformers library trajectory / open-weights ecosystem health. Diffusers bringing quantization backends in-line with Transformers (which has had this for longer) levels the playing field for image generation deployment. The BnB 4-bit result — same quality, same speed, half the memory — is the practically important finding: Flux.1-dev becomes runnable on 24GB consumer cards. The GGUF support is notable: llama.cpp’s format is now first-class in Diffusers, which means the ecosystem for community-quantized diffusion models (currently fragmented across ComfyUI, A1111, etc.) gets a unified API path.