Google 《DiffusionGemma》 Converts an Existing Language Model into a Text Diffusion Model
DiffusionGemma converts Gemma 4 into an experimental text diffusion model that refines 256-token blocks in parallel, trading some reasoning quality and concurrent throughput for very fast single-user
Developer Google DeepMind
Base model Gemma 4 26B-A4B
Release 2026-06-10
License Apache 2.0
A diffusion model that refines text in parallel
Google DeepMind’s 《DiffusionGemma》 places 256-token blocks all at once and refines them over multiple steps, unlike conventional autoregressive language models that predict the next token one at a time. It is an experimental text-generation model that handles an entire sentence simultaneously, much as image diffusion models restore images from noise.
Rather than training a completely new model, Google converted Gemma 4 26B-A4B to a diffusion architecture. According to the technical report, it used less than 10% of the original training-token budget, and only about 3.8B parameters are activated during inference within the 26B MoE architecture.
Bidirectional attention allows it to correct errors in the beginning while generating by looking at the context that follows. It recorded about 1,500 tokens per second on a single H100 GPU, while Google’s official guidance cites output of more than 1,000 tokens per second depending on the hardware and settings, and up to four times the speed of the conventional approach.
However, it is not better than Gemma 4 in every situation. Its general reasoning quality does not match that of autoregressive models, repeated output can occur, and its speed advantage in a single-user environment diminishes when there are many simultaneous requests. Google also describes it as an experimental public model for research and specialized model development.