Models · · 2 min read

DiffusionGemma: Google's open text-diffusion model hits 700+ tokens/sec on an RTX 5090

Google DeepMind released DiffusionGemma on June 10, 2026 — an experimental, Apache 2.0 open model that generates text by diffusion instead of one token at a time. It's a 26B Mixture-of-Experts that activates 3.8B parameters, runs up to 4x faster than autoregressive decoding, and hits 1,000+ tokens/sec on an H100 and 700+ on a consumer RTX 5090. The catch: Google says output quality is below standard Gemma 4.

Google DeepMind released DiffusionGemma on June 10, 2026 — an experimental open model that generates text by diffusion rather than one token at a time. (Source: Google, 2026-06-10) Instead of predicting the next token autoregressively, it denoises 256 tokens in parallel on each forward pass, which is where the speed comes from. The headline for builders: up to 4x faster text generation, under an Apache 2.0 license you can ship commercially.

Key facts:

  • It is a 26B Mixture-of-Experts model that activates only 3.8B parameters at inference. The weights ship as google/diffusiongemma-26B-A4B-it. (Source: Google, 2026-06-10)
  • License is Apache 2.0 — commercial use is permitted, weights are on Hugging Face.
  • It generates 256 tokens in parallel per forward pass instead of one token at a time. (Source: Google, 2026-06-10)
  • Google’s published throughput: 1,000+ tokens/sec on a single NVIDIA H100 and 700+ tokens/sec on a consumer GeForce RTX 5090. (Source: Google, 2026-06-10)
  • NVIDIA’s own numbers add up to 2,000 tokens/sec on a DGX Station and 150 tokens/sec on a DGX Spark, and confirm it runs on an RTX 5090 out of the box via Hugging Face Transformers. (Source: NVIDIA, 2026-06-10)
  • Day-zero support: Hugging Face Transformers, vLLM, Unsloth, MLX, and NVIDIA NeMo. llama.cpp support is “coming soon” — so Ollama/LM Studio users are waiting on that. (Source: Google, 2026-06-10)
DiffusionGemma brand visual — text diffusion generating 256 tokens in parallel, 700+ tokens per second on an RTX 5090
DiffusionGemma denoises a block of tokens at once instead of decoding left to right, which is what makes it up to 4x faster on local hardware. (Source: Google)

Why diffusion is faster — and where it isn’t

Autoregressive models like standard Gemma 4 generate one token, feed it back in, and generate the next — the loop is inherently sequential. DiffusionGemma starts from noise and refines a whole block of 256 tokens simultaneously, so a single GPU forward pass produces many tokens at once. That parallelism is why the speedup shows up on local and low-concurrency inference — a single user, a single GPU — and fades in high-QPS cloud serving, where batching already keeps an autoregressive GPU saturated. (Source: Google, 2026-06-10)

Independent confirmation is early but consistent: Simon Willison ran the released model through NVIDIA’s free NIM API and clocked 2,409 tokens in 4.4 seconds — at least 500 tokens/sec — versus the 857 tokens/sec he measured on Google’s closed Gemini Diffusion preview back in May 2025. (Source: Simon Willison, 2026-06-10) The difference now: this one ships with open weights.

The catch you have to read correctly

Google is unusually direct about the trade-off: DiffusionGemma’s overall output quality is lower than standard Gemma 4, and Google recommends the autoregressive Gemma 4 for production use. (Source: Google, 2026-06-10) This is an experimental model that trades some quality for a different speed profile — not a drop-in upgrade to your current Gemma stack.

So the honest framing is: DiffusionGemma is worth a look if you’re latency-bound on a single local GPU and can tolerate slightly weaker outputs — interactive drafting, autocomplete, local agents where speed beats polish. If you’re optimizing quality, or serving many concurrent users, standard Gemma 4 is still the call.

What this means if you run models locally

If you already run Gemma locally, the toolchain overlaps: the same Hugging Face Transformers and vLLM setup from our guide to running Gemma 4 12B locally is what loads DiffusionGemma today, and the Gemma 4 MTP speed-up guide covers the other half of the “make local Gemma faster” story — MTP and diffusion are two different bets on the same goal. The one thing to wait for, if you live in Ollama or LM Studio, is llama.cpp support — until that lands, local runs go through Transformers, vLLM, or MLX. For a local-coding baseline to compare against, see running Qwen 3.6 locally for coding.

The bigger signal: after a year of diffusion being mostly an image-generation technique, Google just put a fast, open-weight, text diffusion model on hardware you already own. It’s experimental — but it’s the first one you can actually download and run.

Sources

Source: Google (The Keyword)