Remote VAEs for decoding with Inference Endpoints 🤗

Source: HuggingFace Date: 2025-02-24 URL: https://huggingface.co/blog/remote_vae

Summary

Feature announcement: Diffusers experimental remote_decode() offloads VAE decoding to HF Inference Endpoints, reducing local VRAM requirements without the quality penalty of tiled decoding. Benchmarks: local VAE decoding is 2x faster than tiled on RTX 3070 at 1024x1024. Supports SD v1.5, SDXL, FLUX, and HunyuanVideo. ComfyUI integration already community-built.

Implications

Thread: HF as open-source ML hub / open-weights ecosystem health. Remote VAE decoding is an elegant solution to the memory bottleneck in high-resolution diffusion: the VAE is the memory spike at decode time, and offloading it to an endpoint eliminates both the VRAM requirement and the quality/latency cost of tiling. The HF Inference Endpoints as the backend ties this feature directly to Hub infrastructure — it’s a usage driver for Inference Endpoints rather than a standalone optimization. ComfyUI community adoption before official release is a signal the demand is real; watch for this to become a standard feature in Diffusers and downstream tools.