DiffusionGemma: 4x faster text generation

Source: DeepMind Date: 2026-06-10 URL: https://deepmind.google/blog/diffusiongemma-4x-faster-text-generation/

Summary

Google DeepMind released DiffusionGemma, an open-source experimental 26B Mixture-of-Experts model (3.8B parameters active at inference) that uses text diffusion rather than autoregressive token generation. Instead of generating tokens sequentially, it drafts 256 tokens in parallel then refines them iteratively, shifting the decode bottleneck from memory bandwidth to compute and achieving fuller GPU utilisation. On an H100 it exceeds 1,000 tokens/second; on an RTX 5090 it reaches 700+. The article explicitly notes that output quality is lower than standard Gemma 4, positioning this as experimental and speed-optimised rather than quality-competitive.

Implications

• Diffusion as a serious inference architecture, not just a research curiosity. 4x throughput at reduced quality is a real trade-off with practical applications: streaming UI, latency-sensitive interactive agents, and local on-device inference where memory bandwidth is the binding constraint. The quality gap is real but bounded.
• Local/open inference gets a new option. RTX 5090 hitting 700+ tok/s is an on-device capable speed for interactive applications. Combined with the open-source release, this adds a concrete alternative decode strategy for practitioners who currently only have autoregressive options.
• MoE + diffusion architecture signal. The 26B/3.8B active parameter ratio is aggressive. If diffusion models can match autoregressive quality on more tasks over time, this architectural direction has compounding implications for inference cost across the entire local/open model ecosystem.
• Feeds: local/open models, dev substrate (inference architecture), agentic coding tooling (latency-sensitive agent loops).