ELT: Elastic Looped Transformers for Visual Generation

Sahil Goyal$^{}$, Swayam Agrawal$^{}$, Gautham Govind Anil, Prateek Jain, Sujoy Paul$^{\dagger}$, Aditya Kusupati$^{\dagger}$
Google DeepMind

$^{*}$ Equal contributions, $^{\dagger}$ Equal advising

Abstract

We introduce Elastic Looped Transformers (ELT), a highly parameter-efficient class of visual generative models based on a recurrent transformer architecture. While conventional generative models rely on deep stacks of unique transformer layers, our approach employs iterative, weight-shared transformer blocks to drastically reduce parameter counts while maintaining high synthesis quality. To effectively train these models for image and video generation, we propose the idea of Intra-Loop Self Distillation (ILSD), where student configurations (intermediate loops) are distilled from the teacher configuration (maximum training loops) to ensure consistency across the model's depth in a single training step. Our framework yields a family of elastic models from a single training run, enabling Any-Time inference capability with dynamic trade-offs between computational cost and generation quality, with the same parameter count. ELT significantly shifts the efficiency frontier for visual synthesis. With $4\times$ reduction in parameter count under iso-inference-compute settings, ELT achieves a competitive FID of 2.0 on class-conditional ImageNet $256 \times 256$ and FVD of 72.8 on class-conditional UCF-101.

Executive Summary: ELT introduces a new class of visual generative models that reuse the same transformer layers repeatedly rather than stacking unique ones. This addresses the core problem that modern image and video generators demand large parameter counts and memory, making them costly to train and deploy across devices with different compute budgets.

The work set out to build looped transformers that remain effective at any number of iterations, delivering high-quality synthesis while cutting parameters dramatically and enabling flexible inference. Researchers trained models on class-conditional ImageNet images and UCF-101 videos using both masked generative and diffusion frameworks, then compared them against strong baselines under matched compute.

The key results show that a single trained ELT model achieves competitive FID of 2.0 on ImageNet 256×256 and FVD of 72.8 on UCF-101 with roughly four times fewer parameters than MaskGIT and MAGVIT baselines. The same model supports Any-Time inference: raising or lowering the loop count at test time trades quality for speed without retraining. Throughput improves up to 3.5 times on TPU hardware because the compact shared weights reduce memory transfers. Training also converges faster in iso-compute settings, and the approach scales across model sizes.

These gains matter because they lower memory and hardware costs while letting one model serve both latency-sensitive and high-fidelity use cases. The architecture decouples parameter count from effective depth, shifting the efficiency frontier for visual synthesis.

Practitioners should evaluate ELT configurations on target hardware to select the best loop count for each deployment tier. Further work could test adaptive loop allocation across denoising steps, combine ELT with few-step sampling methods, and examine behavior at larger scales or on additional tasks.

The main limitations are that a minimum number of unique layers is required for good performance and that quality can degrade when loop counts greatly exceed the maximum used in training. Results are consistent across the reported experiments, but readers should verify calibration on new datasets and accelerators.

1. Introduction

Section Summary: Traditional deep learning models grow more powerful by adding layers or width, but this also inflates memory use. Recurrence reuses the same parameters repeatedly to deliver far more computation with a much smaller memory footprint, an approach that mirrors how biological vision works. The paper introduces Elastic Looped Transformers trained with intra-loop self-distillation, which lets a single compact model refine its output across repeated passes so that even early stopping produces high-quality images or video while allowing users to trade compute for quality at inference time.

Traditional techniques to increase compute capacity in deep learning models, such as stacking deeper layers or increasing network width, inevitably lead to a proportionally larger memory footprint. Recurrence offers a powerful alternative paradigm, enabling models to utilize large amounts of compute while maintaining a minimal memory footprint by leveraging the same set of parameters repeatedly. This architectural efficiency parallels the biological visual systems ([1, 2]), where recurrent processing, rather than strictly feedforward pathways, is essential for resolving complex visual inputs.

While looping of transformers was popularized by Universal Transformers ([3]) and has recently empowered language models with stronger reasoning capabilities ([4, 5]), its potential for high-fidelity visual generation remains largely untapped. From a practical standpoint, compared to the traditional models, Looped Transformers (a) are extremely parameter efficient and can perform significantly more compute (FLOPs) per parameter, (b) can have higher throughput by minimizing the "memory wall" bottleneck. By reusing a compact set of shared parameters across $L$ iterations, looped transformers achieve up to $L\times$ higher arithmetic intensity (FLOPs per byte transferred from memory) compared to standard transformers of equivalent depth. This avoids the cost of repeated transfers between different units of the accelerator (GPUs/TPUs) required in large transformers, and (c) can exhibit robustness against overfitting in data-constrained settings.

Despite the parameter efficiency of looped architectures, their training remains challenging because intermediate representations often remain uninterpretable until the final loop (see Figure 2). We address this by introducing Elastic Looped Transformers (ELT), a class of generative models designed for progressive refinement. Unlike traditional recurrent transformers, ELT is optimized to provide meaningful, high-quality synthesis even at intermediate repeats. This enables Any-Time (elastic) inference capability - where a single model can scale its compute based on available resources without sacrificing much on generation quality. We present a pictorial representation of our proposed method in Figure 3 and generation results for Any-Time inference capability in Figure 1.

To achieve this functional flexibility across loops, we propose an Intra-Loop Self Distillation (ILSD) algorithm for training looped transformers. Rather than treating the loop as a fixed-depth process, our framework operates as a dual-path system: a teacher path executes the full loop count to provide a high-fidelity target, while a student path, defined strictly as a subset of the teacher’s trajectory, learns to produce comparable results with fewer iterations. Note that both the paths have the same parameter count. By framing the student process as a literal subset of the teacher’s forward propagation, we ensure there is no additional overhead during training. This approach forces the shared parameters to compress complex transformations into earlier loops. Consequently, the model does not just fix the output at the end. It learns an efficient, progressive refinement process that maintains the generation quality regardless of the exit point as motivated in Figure 2. Our contributions and findings are summarized as follows:

flushleft State-of-the-Art Parameter Efficiency: Through the reuse of block of transformer layers across loops, ELT achieves a competitive FID of 2.0 on class-conditional ImageNet $256 \times 256$ and FVD of 72.8 on class-conditional UCF-101. This represents a 4 $\times$ reduction in parameters compared to baselines MaskGIT ([6]) (image generation) and MAGVIT ([7]) (video generation), while matching or improving their performance under iso-inference-compute settings.

flushleft Elastic/Any-Time Inference: By treating the looped block as an iterative refiner, our models enable Any-Time inference ([8]), allowing to traverse the pareto frontier of quality versus compute at test-time without retraining. This allows to serve diverse deployment tiers: from latency-critical on-device generation (few loops) to high-fidelity cloud rendering (more loops).

flushleft Scalability: While model size remains a primary driver of quality, recursive looping provides a unique test-time compute lever that scales predictably across both Masked Generative Transformers ([6, 7, 9]) and Diffusion Transformers ([10]).

2. Related Work

Section Summary: Researchers have explored recursive or "looped" transformer designs that reuse the same parameters across multiple iterations to improve efficiency, as seen in Universal Transformers and more recent work on dynamic depth, reasoning, and latent computation; this differs from Deep Equilibrium Models, which solve for a fixed point analytically rather than unrolling explicit steps. In visual generation, efforts toward parameter-efficient models include pruning and lightweight architectures like MobileDiffusion, while elastic or any-time methods such as E-DiT and LoopViT enable variable computation through skipping or early exits. The surveyed approach is positioned as orthogonal to few-step consistency models, since it reduces computation inside each sampling step via intra-loop distillation rather than cutting the number of steps, allowing the two strategies to be combined.

flushleft Recursive Architectures: The principle of recursively applying a shared block of parameters to enhance model efficiency is a well-established concept. This approach, often referred to as looping has been shaped by Universal Transformers ([3]), which introduced the idea of iterating over a single transformer layer. Notably, there has been a resurgence of interest in this area. [4] demonstrated the power of looping for sophisticated reasoning tasks. [11] showed that Looped Transformers can learn to implement multi-step gradient descent for in-context learning, providing a deeper understanding of their capabilities. Mixture-of-Recursions ([12]) further explored input-dependent dynamic and adaptive depth in looped transformers. Mixture-of-Recursions-VIT ([13]) extends it for image understanding. [14] utilized looping for length generalization. Geiping et al. [15] demonstrated that scaling test-time compute via recurrent depth allows language models to perform complex latent reasoning.

Deep Equilibrium Models (DEQs) ([16, 17, 18, 19, 20, 21]), instead of unrolling a weight-tied layer for a fixed number of iterations, define the output as the fixed point of a non-linear transformation. Unlike DEQs that rely on black-box solvers for an analytical fixed point, our ELT framework explicitly optimizes unrolled intermediate states via Intra-Loop Self Distillation (ILSD), retaining the flexibility of Any-Time inference without requiring the network to reach a strict analytical fixed point.

flushleft

Parameter-Efficient Visual Generation: Standard efficiency techniques ([22]) that work across deep learning models also help in speeding up visual generation models. MobileDiffusion ([23]) prunes redundant residual blocks and replaces standard layers with separable convolutions in UNet to get an optimized architecture ($\sim$ 400M) for on-device visual generation. EdgeFusion ([24]) employs BK-SDM, a lightweight Stable Diffusion variant, and refines the step distillation process of Latent Consistency Model. MaGNeTS ([25]) trains a family of nested transformers ([26, 27]) with schedules of model sizes over the generation process, without increasing the parameter count.

flushleft

Elastic Visual Generation: The paradigm of Any-Time or elastic generation focuses on decoupling model's parameter count from its computational depth. E-DiT ([28]) introduces adaptive block skipping and MLP width reduction, allowing a single model to traverse varying computational budgets without retraining. In the context of visual reasoning, LoopViT ([29]) uses a weight-tied recursive architecture, employing a parameter-free dynamic exit mechanism to halt inference based on uncertainty of prediction. EvoSearch ([30]) proposes a search-based strategy that optimizes sampling trajectories at inference time. Unlike these methods that rely on architectural skipping or external search, our ELT framework equipped with Intra-Loop Self Distillation (ILSD) algorithm directly regularizes the recursive process, ensuring stability across the entire loop spectrum. Methods like ALIT ([31]), FlexTok ([32]), One-D-Piece ([33]), ElasticTok ([34]), CAT ([35]), and DC-DiT ([36]) use tail dropping to allow elasticity in token sequence length.

flushleft Relation to Few-Step and Consistency Models: Recent approaches such as Consistency Models ([37]) and progressive distillation ([38]) address inference efficiency by reducing the number of sampling steps (inter-step acceleration). ELT is fundamentally orthogonal: it reduces compute within each sampling step by varying the loop count $L$ (intra-step acceleration). These two axes are complementary, we can combine ELT with few-step methods to achieve further efficiency gains. Moreover, unlike consistency models which require specialized training objectives and architectural constraints to enable variable-step inference, ELT's elastic capability emerges naturally from ILSD applied to standard training objectives. We note that ELT is particularly compelling for one-step generative paradigms, where the loop count $L$ becomes the sole lever for controlling the compute-quality trade-off at inference time.

3. Preliminaries

Section Summary: Masked generative models such as MaskGIT create images by producing all tokens in parallel and then iteratively refining them through repeated cycles of masking and sampling, which is faster than generating tokens one by one in sequence. Diffusion models work differently by learning to reverse a gradual noising process, starting from random noise and using iterative denoising steps to recover a clean image or video. Both approaches rely on repeated refinement with shared model parameters, which the section connects to a new transformer design that performs extra internal refinement at each step.

flushleft Masked Generative Models: Masked Generative Image Transformer (MaskGIT) ([6]) introduced a novel approach to image generation that significantly differs from traditional autoregressive models. In autoregressive decoding, images are generated sequentially, one pixel/token at a time, following a raster scan order ([39, 40, 41, 42]). This sequential approach is computationally inefficient, as each token is conditioned only on the previously generated tokens, leading to a bottleneck in processing time. MaskGIT generates all tokens of an image simultaneously, while iteratively refining them. This method enables significant acceleration in the decoding process. The tokens are discrete and obtained using Vector Quantized (VQ) autoencoders, learned with self-reconstruction and photo-realism losses ([7]). The iterative parallel decoding process is represented as:

$ \mathbf{X}{k} \leftarrow \mathrm{Mask \circ Sample}(M({\mathbf{X}}{k-1}, c), k)\tag{1} $

where $\mathbf{X} \in \mathbb{Z}^N_{\geq 0}$, are the input tokens, $N$ is the number of tokens, $k \in [1, K]$ denote the iteration number, with $K$ being the total number of iterations, $\mathbf{X}_0$ is either completely masked for full generation, and partially masked for conditional generation tasks like frame prediction, $c$ is the category of image/video under generation. The $\mathrm{Sample}$ function utilizes logits predicted by the model ${M(.)}$, introduces certain randomness, and sorts them by confidence, unmasking only top-p tokens while masking the rest.

flushleft Diffusion Models: Diffusion models ([43, 44]) generate data by learning to reverse a process that gradually corrupts a signal $\mathbf{X}_0$ into Gaussian noise $\mathbf{X}_T$ through a predefined noise schedule. At its core, diffusion denoising/sampling proceeds iteratively as follows:

$ \mathbf{X}{t-1} \leftarrow \alpha_1(t) \cdot \mathbf{X}{t} + \alpha_2(t) \cdot M(\mathbf{X}_{t}, t, c) + \alpha_3(t) \cdot \mathbf{z} $

where $\mathbf{X}{t}$ is the (partially) denoised vector at time $t$, $\alpha_1(t), \alpha_2(t), \alpha_3(t) $ are time-dependent scalars defined by the noise schedule and $\mathbf{z}$ is a sample drawn from standard Normal distribution. The model $M$ takes as input $\mathbf{X}{t}$ along with time $t$ and class label $c$. From the perspective of the forward diffusion process, the generative task can be thought of as predicting the noise added to a latent representation $\mathbf{X}_t$ at timestep $t$, conditioned on a class label $c$. While traditionally architectures based on U-Nets ([45]) have been used, the Diffusion Transformer (DiT) ([10]) architecture shifts away from this design by treating image latents as sequences of tokens, and using transformer blocks for processing these tokens. Similar to MaskGIT, we explore replacing the typical DiT transformers with Elastic Looped Transformers for denoising at time $t$.

In summary, both masked generative models and diffusion models involve recursive refinement over multiple sampling steps, sharing parameters across sampling steps. Unlike standard transformers, our ELT framework aligns the architecture of model $M$ with the progressive refinement process by implementing $M$ as a recurrent, weight-shared transformer blocks. This allows the model to perform recursive refinement within each sampling step, providing a test-time compute lever to trade-off inference speed and generation quality.

4. Method

Section Summary: The method introduces a looping mechanism that reuses a small block of N transformer layers L times during each sampling step, achieving greater effective depth and representational power without increasing the total number of unique parameters. To make intermediate loop counts useful rather than suboptimal, the approach adds intra-loop self-distillation, in which a full-depth teacher path supervises randomly chosen shallower student paths through a combination of ground-truth loss and distillation loss. A stochastic sampling schedule and a decaying weighting factor between the two losses further encourage the shared parameters to produce progressively refined outputs at any depth, enabling early exit during inference.

flushleft Looping Mechanism: Let the number of transformer layers to be looped be $N$ and number of loops per sampling step be $L$. The total effective depth for a single sampling or denoising step of the generation process is then $N \times L$. Let $f_{\theta_i}(\mathbf{x})$ denote a single transformer layer with parameters $\theta_i$. We define a composite block $g_{\Theta}(x)$ consisting of $N$ unique transformer layers with parameters $\Theta = {\theta_1, \theta_2, \dots, \theta_N}$ as follows:

$ g_{\Theta}(\mathbf{x}) = f_{\theta_N}(f_{\theta_{N-1}}(\cdot s f_{\theta_1}(\mathbf{x}))) $

In a standard transformer model with total depth $\mathcal{D} = N \times L$, the effective transformation $F_{\mathcal{D} }(x)$ would require $\mathcal{D} $ sets of unique parameters. In looping, we reuse the same block of parameters $\Theta$ for $L$ successive applications, resulting in only $N$ unique layers of parameters. The effective transformation for a $N \times L$ configuration is given by:

$ F_{(N, L)} (\mathbf{x}) = \underbrace{g_{\Theta}\left(g_{\Theta} (\cdot s g_{\Theta}(\mathbf{x}))\right)}{L \text{ loops}} \equiv g{\Theta}^L(\mathbf{x}) $

This looping architecture decouples physical model size from computational depth (see Figure 3 for a visual overview). The parameter count ($\Theta$) is constrained by the number of unique blocks ($N$), while representational capacity and depth ($\mathcal{D}$) scale with loop count ($L$). This setup provides two primary advantages: extreme parameter efficiency and high throughput. See Section 5.2 for details.

flushleft Intra-Loop Self Distillation (ILSD): In a standard weight-tied (looped) transformer, the model is typically optimized only for its final output after fixed $L_{\text{max}}$ iterations i.e. the loop count for which it is trained. However, this creates a "black box" internal trajectory where intermediate loops ($L < L_{\text{max}}$) may produce suboptimal representations until the final projection layer. By treating the looped block $g_{\Theta}$ as an iterative refiner, we can motivate a training objective that ensures the model remains useful at multiple depths. This not only encourages the model to learn a more robust, incremental transformation but also allows for elastic inference, where the model can be exited early with minimal performance drop. Towards this end, we propose Intra-Loop Self Distillation (ILSD).

From a distillation perspective, ILSD leverages the fact that a model with more loops ($L_{\text{max}}$) naturally possesses a more mature and refined representational space than its shallower version ($L_{\text{int}}$), even though they have the same unique parameters. By treating the full-depth model as an internal teacher, we provide a high-fidelity, low-variance signal for the shallower student to follow (see Figure 3). This forces the shared parameters $\Theta$ to compress complex transformations into fewer steps, effectively distilling the knowledge of the deep model into the early stages of the computation.

flushleft Stochastic Student Sampling ($S^3$): We train with a fixed number of loops, $L_{\text{max}}$, for a block of $N$ layers with unique parameters. During each training step, we treat the model as a dual-path system sharing a single set of parameters $\Theta$. We define a teacher path that executes the full $L_{\text{max}}$ loops and a stochastic student path that exits early at an intermediate loop $L_{\text{int}}$. The intermediate path receives supervision from the ground truth labels as well as the online teacher with maximum training loop count $L_{\text{max}}$.

We denote the training loss for the output of a configuration $N \times L$ as $\mathcal{L}{\Theta}(F{(N, L)}(\mathbf{x}), \mathbf{y})$ where $\mathbf{y}$ denote the ground-truth for a certain input masked / denoise level. At every training iteration, we randomly sample an intermediate loop count $L_{\text{int}}$ from uniform distribution such that $L_{\text{min}} \leq L_{\text{int}} < L_{\text{max}}$. Note that $L_{\text{min}}$ is just used for constraining the student sampling distribution. The effective joint loss, $\mathcal{L}^{\text{ILSD}}_{\Theta}$, is computed as:

$ \begin{aligned} \mathcal{L}^{\text{ILSD}}{\Theta} = & \mathcal{L}^\text{GT}(F{(N, L_{\text{max}})}(\mathbf{x}), \mathbf{y}) && \text{(1) Ground-truth for teacher} \

• &\lambda \mathcal{L}^\text{GT}(F_{(N, L_{\text{int}})}(\mathbf{x}), \mathbf{y}) && \text{(2) Ground-truth for student} \
• & (1 - \lambda) \mathcal{L}^{\text{dist}}(F_{(N, L_{\text{int}})}(\mathbf{x}), \text{sg}(F_{(N, L_{\text{max}})}(\mathbf{x}))) && \text{(3) Intra-Loop Self Distillation} \ \text{with} \quad &L_\text{int} \sim \mathcal{U}(L_{\text{min}}, L_{\text{max}}) && \text{Stochastic Student Sampling} \end{aligned} $

where sg is stop-grad for teacher ($L_{\text{max}}$) in ILSD, $\lambda$ is hyperparameter controlling the weight between the ground truth and distillation losses. We introduce a curriculum for $\lambda$ and linearly decay it from $1$ to $0$ as training progresses. This initially anchors the student to reliable ground-truth labels while the teacher is still untrained and gradually shift to mimicking the teacher's predictions once they have matured. We found this linear schedule to be effective across all our experiments and did not observe sensitivity to the decay rate, as long as the transition is gradual.

flushleft Loss Formulation: The exact forms of $\mathcal{L}^\text{GT}$ and $\mathcal{L}^\text{dist}$ depend on the algorithm used for training. For masked generative models with discrete tokens, we use the cross-entropy loss for both ground-truth and distillation:

$ \begin{aligned} \mathcal{L}^\text{GT} &= -\sum_{i \in Mask} \log P_{(N, L_{\text{int}})}(y_i \mid \mathbf{x}{mask}) \ \mathcal{L}^\text{dist} &= -\sum{i \in Mask} \sum_{v \in \mathcal{V}} P_{(N, L_{\text{max}})}(v \mid \mathbf{x}{mask}) \log P{(N, L_{\text{int}})}(v \mid \mathbf{x}_{mask}) \end{aligned} $

where $\mathbf{x}{mask}$ is the masked input, $y = {y_i}{i \in \text{Mask}}$ represents the ground-truth tokens for the masked positions, and $\mathcal{V}$ is the full vocabulary of the tokenizer. For diffusion, we use the sigmoid-weighted Mean Squared Error (MSE) for both ground-truth and distillation losses. Let $\mathbf{x}_t$ be the noised version of ground truth latent $\mathbf{x}_0$, the ground-truth loss is:

$ \mathcal{L}^{\text{GT}}= w(t) \lVert F_{(N, L)}(\mathbf{x}_t) - \mathbf{x}_0 \rVert_2^2 $

where $L$ is $L_{\text{int}}$ for student and $L_{\text{max}}$ for teacher. $w(t)$ is a time-dependent sigmoid weighting term ([46]). The distillation loss is:

$ \mathcal{L}^{\text{dist}}= w(t) \lVert F_{(N, L_{\text{max}})}(\mathbf{x}t) - F{(N, L_{\text{int}})}(\mathbf{x}_t) \rVert_2^2 $

Note that gradients from both computational paths, corresponding to $L_{\text{int}}$ and $L_{\text{max}}$, update the single, shared set of block parameters $\Theta$. This joint optimization provides a significantly richer training signal; the shared block $g_{\Theta}$ is forced to learn a transformation that is not only effective at $L_{\text{int}}$ loops but remains incrementally useful for the subsequent iterations up to $L_{\text{max}}$ loops. This constraint prevents the model from learning a shortcut that might minimize loss at a specific depth but fail when composed further. Consequently, the shared block generalizes better to lower depths, leading to higher performance even with fewer loops. It is interesting to note that unlike traditional distillation, where we have to forward propagate through both the student and teacher models separately, in the proposed way of Intra-Loop Self Distillation (ILSD), the training overhead is minimal, as the computation of $F_{(N, L_{\text{int}})}(\mathbf{x})$ is a strictly required intermediate step for computing $F_{(N, L_{\text{max}})}(\mathbf{x})$ i.e. the student trajectory ($L_{\text{int}}$) is a strict prefix of the teacher trajectory ($L_{\text{max}}$). Refer Algorithm 1 and Algorithm 2 for details of our training and inference algorithms respectively.

5. Experiments and Results

Section Summary: The experiments evaluate the proposed approach on class-conditional image generation using ImageNet at 256x256 resolution and video generation on UCF-101, testing both masked generative transformers and diffusion transformers across multiple model scales. Researchers detail the datasets, tokenization methods, training procedures, and metrics such as FID for image quality and GFLOPs for computational efficiency, while exploring how varying the number of layers and inference loops affects performance. Results show that the method matches or improves upon baseline generation quality with substantially fewer parameters—for example, reaching the same FID score with four times fewer parameters—and enables flexible trade-offs between output fidelity and inference speed.

We conduct extensive experiments using masked generative transformers and diffusion transformers to demonstrate the generality and efficacy of our approach on class-conditional image generation. We also experiment with class-conditional video generation using masked generative transformers. We first detail our experimental setup and then present the results.

::: {caption="Table 1: Class-conditional Image Generation on ImageNet 256×256. “# steps” refers to the number of neural network runs. ^Delta denotes values taken from prior publications. * indicates usage of extra training data. ^g denotes use of classifier-free guidance ([47]). Note that L in (N × L) notation is inference loop count per sampling step."}

:::

5.1 Experimental Setup

flushleft Datasets: We experiment on ImageNet $256\times256$ ([64]) for image generation and UCF-101 ([65]) for class-conditional video generation.

flushleft Implementation Details: (i) Masked Generative Transformers: We use pretrained tokenizers from MaskGIT ([6]) (images) and MAGVIT ([7]) (videos) with a codebook size of 1024 tokens. Image models are trained at $256\times256$ resolution, compressed to $16\times16$ discrete tokens with an embedding dimension of 1024. Video models are trained for $16 \times 128 \times 128$ sequences, compressed to $4 \times 16 \times 16$ tokens. Following MaskGIT, we adopt the BERT ([66]) architecture as the transformer backbone and perform experiments at several model scales to understand the scaling behaviors of ELT. We train all models for 270 epochs unless otherwise specified. We employ a cosine schedule for unmasking tokens during inference. For image generation, we use classifier-free guidance by dropping class condition labels for $10%$ of the training batches.

(ii) Diffusion Transformers: We use a pretrained Stable Diffusion v1.4 VAE ([52]) model to map $256 \times 256$ images into a continuous $32 \times 32 \times 4$ latent space (8× spatial downsampling). We train a DDPM-style diffusion model which operates on these latents using a DiT architecture. We employ a shifted cosine noise schedule and sigmoid-weighted MSE loss for training ([46]). Models are trained for 500K steps with a batch size of $512$ using Adam. Unless mentioned otherwise, sampling uses 512-step DDPM with classifier-free guidance scale $3.0$. See Appendix for more details.

flushleft Evaluation Metrics and Efficiency: To evaluate the quality of synthesized content, we employ Fréchet Inception Distance (FID) ([67, 51]) and Inception Score (IS) ([68]) for image generation tasks, and Fréchet Video Distance (FVD) ([69]) for video generation. Beyond generative quality, we evaluate model efficiency using inference-time GFLOPs and throughput (samples generated per second). In the proposed $N \times L$ design space, for a fixed model scale and block size $N$, both computational complexity and latency scale linearly with the number of loops $L$. This relationship allows us to precisely navigate the trade-off between generation quality and inference speed by modulating the loop count.

5.2 Image Generation

flushleft Comparison with Baselines: We present the results for $256 \times 256$ image generation on ImageNet-1k in Table 1. Despite using 4 $\times$ less parameters, ELT-XL achieves same FID of 2.0 as MaskGIT-XL, which is the base setup for ELT. As shown in recent literature, using a superior tokenizer ([70, 9, 63]) or optimized training & inference configurations ([71, 72]) can further boost ELT's performance.

We also present comparisons of ELT using diffusion models in Table 2. Notably, ELT outperforms the baseline model with 32 layers (FID 3.43) using iso-inference-compute $8N \times 4L$ (FID 3.16) and $16N \times 2L$ (FID 2.83) settings, achieving $\textbf{4} \times$ and $\textbf{2} \times$ reduction in parameter count respectively. The $1N \times 32L$ configuration (FID 10.30) reveals that a single unique transformer layer, despite 32 effective passes, lacks the representational capacity for high-fidelity generation, highlighting that a minimum block size $N$ is necessary to provide sufficient architectural expressiveness within each iteration. While vanilla looping (without ILSD) gives competitive performance when running inference with same loops as training ($L=L_{max}$), performance degrades drastically for lower number of loops which is mitigated by ILSD as show in Figure 8b.

flushleft Qualitative Results: Figure 1, Figure 9, and Figure 10 compare ELT against vanilla looped transformers within a diffusion framework. It is clear that ELT unlocks Any-Time inference capability providing dynamic trade-off between generation quality and inference speed. Figure 11 and Figure 12 visualize the generation results of ELT in diffusion and masked generative framework respectively.

flushleft Scaling Inference GFLOPs and Pareto Front: Figure 4 illustrates the trade-off between generation quality (FID) and inference compute (GFLOPs). The pareto front (black curve) reveals that while increasing the loop count ($L$) consistently improves FID (faded points) for a fixed unique layers ($N$), the gains eventually diminish, where transitioning to the next architecture scale becomes more performant than over-looping smaller models. Crucially, ELT allows for Any-Time inference, we can traverse the pareto curve at test-time by simply adjusting $L$ to meet specific hardware constraints without retraining.

\begin{tabular}{lccc}
  \toprule
  Model & {FID $\downarrow$} & {# params} & $\mathcal{D}$ \\
  \midrule
  DiT - 16 layers & $3.87$ & 1.1B & 16 \\
  DiT - 32 layers & $3.43$ & 2.1B & 32 \\
  \midrule  
  ELT ($ 1N \times 32L$)& $10.30$ &69M & 32 \\
  ELT ($4N \times 8L$)& $3.96$ & 271M & 32\\
  \textbf{ELT} ($\bm{8N \times 4L}$)& $\textbf{3.16}$ & \textbf{539M}& 32\\
  \textbf{ELT} ($\bm{16N \times 2L}$)& $\textbf{2.83}$ &\textbf{1.1B}& 32\\
  \bottomrule
  \end{tabular}

\begin{tabular}{lcc}
  \toprule
  ELT & $d_{model}$ & Throughput Ratio \\ 
  \midrule
  $6N \times 2L$ (B) & 768 & 1.0 \\
  $8N \times 3L$ (L) & 1024 & 2.9 \\
  $7N \times 4L$ (XL) & 1152 & 3.3 \\
  $8N \times 4L$ (H) & 1280 & \textbf{3.5} \\
  \bottomrule
  \end{tabular}

flushleft Scaling Parameters: We further investigate the relationship between model capacity and performance in Figure 5. By plotting the best FID achieved at each parameter budget, we observe a clear power-law scaling trend across all model widths. While increasing the number of unique layers ($N$) reduces FID, the gains are most pronounced when accompanied by an increase in model width ($d$). Specifically, the $d=1536$ configuration ($G$) achieves the lowest overall FID with only $8$ unique layers, while the full $G$ model has $48$ unique layers. However, for on-device budgets, smaller looped variants (L and XL) remain highly efficient, providing a flexible scale-to-performance ratio that is critical for resource-constrained visual generation.

flushleftFaster Training Convergence: Our method demonstrates significantly faster convergence than standard DiT architectures ([10]). As illustrated in Figure 6, ELT-based diffusion models achieve $2\times$ and $1.4\times$ speedups when using configurations of $16N \times 2L_{\text{max}}$ and $8N \times 4L_{\text{max}}$, respectively, in iso-inference-compute settings with $N=32$ DiT baseline. Note that effective depth $\mathcal{D}$ is same for both ELT and DiT baseline (32).

flushleft ELT has high Throughput: ELT utilizes a compact set of shared parameters and maintain its major weight footprint closer to the accelerator computation unit. This avoids the cost of repeated memory transfers typically required in standard models. We evaluate the efficiency of our proposed architecture by measuring the throughput ratio relative to the baseline across various model scales with an inference batch size of 8, on 1 TPU slice. We choose the best performing ELT inference configuration from each model scale (refer Figure 5) and report their throughput gains in Table 3. Our method delivers significant performance gains across model scales L, XL and H. Note that these gains are contingent on the model size: as long as the shared parameters remain within memory capacity, the architecture eliminates redundant memory movement across iterations. For model scale B, the baseline (MaskGIT ([6])) is small enough to fit entirely within device memory capacity, eliminating the transfer penalty. Generally speaking, ELT offers extreme parameter efficiency, which would reduce significant memory transfers even for bigger models which need to be sharded along multiple devices.

5.3 Video Generation

\begin{tabular}{lcccccc}
  \toprule
  Method & Class & FVD $\downarrow$ & IS $\uparrow$ & # params & # steps & # GFlops\\
  \midrule
  RaMViD$^{\Delta*}$ ([74]) & & - & 21.71 $\pm$ \textcolor{gray}{0.21} & 308M & 500 & -\\
  StyleGAN-V$^{\Delta*}$([75]) & & - & 23.94 $\pm$ \textcolor{gray}{0.73} & - & 1 & -\\
  DIGAN$^{\Delta}$ ([76]) & & 577 $\pm$ \textcolor{gray}{21} & 32.70 $\pm$ \textcolor{gray}{0.35} & - & 1 & $\sim$ 148\\
  DVD-GAN$^{\Delta}$ ([77]) & $\checkmark$ & - & 32.97 $\pm$ \textcolor{gray}{1.70} & - & 1 & -\\
  Video Diffusion$^{\Delta*}$ ([78]) & & & 57.00 $\pm$ \textcolor{gray}{0.62} & 1.1B & 256 & -\\
  TATS$^{\Delta}$ ([79]) & & 420 $\pm$ \textcolor{gray}{18} & 57.63 $\pm$ \textcolor{gray}{0.24} & 321M & 1024 & -\\
  CCVS+StyleGAN$^{\Delta}$ ([80]) & & 386 $\pm$ \textcolor{gray}{15} & 24.47 $\pm$ \textcolor{gray}{0.13}& - & - & - \\
  Make-A-Video$^{\Delta*}$ ([81]) & $\checkmark$ & 367 & 33.00 & - & - & - \\
  TATS$^{\Delta}$ ([79]) & $\checkmark$ & 332 $\pm$ \textcolor{gray}{18} & 79.28 $\pm$ \textcolor{gray}{0.38} & 321M & 1024 & - \\
  \midrule
  \textcolor{gray}{CogVideo}$^{\Delta*}$ ([82]) & $\checkmark$ & 626 & 50.46 & 9.4B & - & - \\
  \textcolor{gray}{Make-A-Video}$^{\Delta*}$ ([81]) & $\checkmark$ & 81 & 82.55 & $\gg$ 3.5B & $\gg$ 250 & -\\
  PAR-4 $\times^{\Delta}$ [41] & $\checkmark$ & 99.5 & - & 792M & 323 & -\\
  PAR-16 $\times^{\Delta}$ [41] & $\checkmark$ & 103.4 & - & 792M & 95 & -\\
  MaGNeTS$^{\Delta}$ ([25]) & $\checkmark$ &96.4 $\pm$ \textcolor{gray}{2} & 88.53 $\pm$ \textcolor{gray}{0.20} & 306M & 12 & $\sim$ 1.7k \\
  \midrule
  MAGVIT-L$^{\Delta}$ ([7]) & $\checkmark$ & 76 $\pm$ \textcolor{gray}{2} & 89.27 $\pm$ \textcolor{gray}{0.15} & 306M & 12 & $\sim$ 4.3k \\ 

  \textbf{ELT} ($\bm{6N \times 4L}$) & $\checkmark$ &72.8 $\pm$ \textcolor{gray}{2.5} & 88.27 $\pm$ \textcolor{gray}{0.33} & \textbf{76M} & 12 & $\sim$ 4.3k \\

  \textbf{ELT} ($\bm{6N \times 6L}$) & $\checkmark$ &60.8 $\pm$ \textcolor{gray}{2.7} & 87.88 $\pm$ \textcolor{gray}{0.39} & \textbf{76M} & 24 & $\sim$ 13k \\
  \bottomrule
  \end{tabular}

We use the MAGVIT ([7]) framework to train parallel decoding based video generation models. We summarize the results for class-conditional video generation on UCF-101 in Table 4. Our compact 76M ELT model outperforms MAGVIT baseline (FVD 72.8 vs 76) on data-constrained settings of UCF-101 ($\sim$ 13.7M training tokens) in iso-inference-compute settings. Scaling the compute further with number of loops and sampling steps gives a boost in performance, reaching FVD of 60.8. This suggests that looped transformers can exhibit robustness against overfitting in data-constrained regimes like UCF-101, effectively regularizing the learning process while maintaining the expressive capacity for high-quality generation.

5.4 Intra-Loop Self Distillation (ILSD) drives Elasticity

We analyze the impact of ILSD for image generation across masked generative models (refer to Figure 8a) and diffusion models (refer to Figure 8b). Models trained without ILSD exhibit significant divergence from their fixed training depth ($L_{max}$). In contrast, ELT maintains stable, high-quality generation across the entire inference loop spectrum (see Figure 1). We further analyse class-conditional video generation on UCF-101 in Figure 7. Interestingly, ILSD enables the model to maintain reasonable quality even at unseen depths ($L > L_{\text{max}}$). On UCF-101, the model achieves a peak FVD of 69.20 at $L=6$ despite being trained with $L_{\text{max}}=4$, suggesting that ILSD regularizes the iterative process sufficiently for modest extrapolation beyond training depth. We note that this extrapolation behavior warrants further investigation across datasets and scales.

6. Conclusion

Section Summary: The authors introduce Elastic Looped Transformers, a parameter-efficient method that reuses transformer layers in a loop to generate images and videos. Their approach matches the performance of larger standard models while using roughly four times fewer parameters, and a self-distillation technique lets one trained model flexibly trade off quality against computation at inference time. Looking ahead, the method could enable dynamic compute allocation in diffusion models and support high-quality generation on resource-limited hardware.

We proposed a novel parameter-efficient approach to visual generation using recurrent transformers called Elastic Looped Transformers (ELT). Our approach achieves a strong empirical performance, similar to baselines, with $4\times$ fewer parameters in iso-inference-compute setting in both image and video generation tasks. Beyond significantly improved performance per parameter, we identified fundamental scaling properties of looped transformers: while increasing model width remains a primary driver of quality, recursive looping provides a unique "test-time" compute lever. Through our proposed Intra-Loop Self Distillation (ILSD) strategy, we train a single model that is performant across a variable number of iterations. This strategy effectively yields a continuous family of models from a single training run, enabling Any-Time inference where practitioners can traverse the pareto front to balance image quality and GFLOPs dynamically.

Looking forward, we note that ELTs can potentially unlock more efficient inference for diffusion models. While existing approaches rely on the same network (and hence allocate same compute) across denoising steps, ELT can dynamically allocate compute across denoising steps, spending more compute where it matters the most. Additionally, in the context of recent one-step generative modeling paradigms such as consistency models ([37]) and drifting models ([83]), ELT can enable true elasticity: since there is only one sampling step, one can dynamically control the quality of the model at inference by varying the number of loops, without having to pre-determine the number of sampling steps as is the case with traditional multi-step diffusion models. We believe this paradigm of flexible, weight-efficient scaling offers a promising direction for deploying high-fidelity generative models on resource-constrained hardware.

Appendix

Section Summary: The appendix provides detailed hyperparameters, model architectures, and training configurations for masked generative models on ImageNet and UCF-101 as well as diffusion models on ImageNet, covering aspects such as latent shapes, optimizers, sampling schedules, and scaling across different model sizes. It also describes the ELT training algorithm with intra-loop self-distillation and an anytime inference procedure, supported by pseudocode. Additional sections analyze efficiency gains in memory and speed, along with ablations comparing loss variants for image and video generation.

\begin{tabular}{lcccc}
\toprule
  {} & \multicolumn{2}{c}{Masked Generative Models} & & Diffusion Models \\
 \cmidrule(lr){2-3} \cmidrule(lr){5-5}
\textbf{Config} & ImageNet & UCF-101 & & ImageNet \\
\midrule
\textit{Model Input} & & & & \\
Latent shape & (1, 16, 16) & (4, 16, 16) & & (4, 32, 32) \\
Codebook Size & 1024 & 1024 & & --- \\
Embed / Latent dim & 1024 & 1024 & & 4096 \\
\midrule
\textit{Training / Optimization} & & & & \\
Batch Size & 512 & 256 & & 512 \\
Learning rate & $10^{-4}$ & $10^{-4}$ & & $10^{-4}$ \\
Optimizer & AdamW & AdamW & & AdamW \\
($\beta_1, \beta_2$) & (0.9, 0.96) & (0.9, 0.96) & & (0.9, 0.99) \\
Weight decay & 4.5e-2 & 4.5e-2 & & 0.01 \\
Warmup steps & 15K & 15K & & 10K \\
Training Epochs & 270 & 2000 & & 200 \\
Label drop (\%) & 10 & 0 & & 10 \\
Loss Function & Xent & Xent & & SigmoidELBO \\
Sampler / Prediction & --- & --- & & DDPM / $v$-pred \\

\midrule
\textit{Sampling} & & & & \\
Masking/Denoising Schedule & Cosine & Cosine & & Cosine \\
Sampling steps & 24 & 12 \& 24 & & 512 \\
CFG scale & $\checkmark$ & $\times$ & & 3.0 \\
Sampling Temp. & (0.5, 0.8) & (0.5, 0.8) & & --- \\

\bottomrule
\end{tabular}

: Table 6: Transformer Model Scaling. Architecture details for various model sizes.

Model Scale	$d_m$odel	MLP dim	Heads	Layers (in Baseline)
S	384	1536	6	12
B	768	3072	12	12
L	1024	4096	16	24
XL	1152	4608	16	28
H	1280	5120	16	32
g	1408	6144	16	40
G	1664	8192	16	48

: Table 7: Architecture Details for ELT in Diffusion framework.

Model Scale	$d_m$odel	MLP dim	Heads	Layers (in Baseline)
Small	2048	8192	16	16
Large	2048	8192	16	32

A. Implementation Details

We explain our training and inference algorithms in Algorithm 1 and Algorithm 2 respectively. We detail the ELT's implementation details below:

(a) Masked Generative Models: Refer Table 5 for details of configurations and hyperparameters of ELT-based masked generative models. We experiment with several model scales (refer Table 6 for architecture details). Note that for model scale $\geq$ H, we increase the weight decay for AdamW ([84]) optimizer by a factor of 5. We sweep over multiple classifier-free guidance (cfg) ([47]) scales for ELT as well as baseline. We do not use cfg for UCF-101 class-conditional video generation. Sampling Temperature (STemp) mentioned in Table 5 controls the randomness of the sampling from the categorical distribution of logits. Tokens are sampled from logits/STemp. STemp is calculated as:

$ \text{STemp} = bias + scale \times (1 - (k+1)/K)\tag{2} $

where $bias$ and $scale$ are hyperparameters, $k$ is current sampling step and $K$ is total number of sampling steps. Throughout our experiments, we use $bias = 0.5$ and $scale = 0.8$.

(b) Diffusion Models: Refer Table 5 for details of configurations and hyperparameters of ELT-based diffusion models. For diffusion models, we experiment with two model architectures (refer Table 7 for details).

B. Additional Experiments

flushleft Cost and Efficiency: ELT drastically reduces the unique param count, which in turn helps in following: (i) Hardware Costs: The primary bottleneck in training and deploying modern generative models is often memory (HBM/VRAM). ELT reduces the memory budget needed to train and serve larger effective depths. (ii) Training Speedup: As ELT trains intermediate loops using a single forward pass with negligible cost of the prediction head, training step time remains same. Table 8 (corresponding to Table 2 results) profiles DiTs training on TPU v6e with 8N × 4L looping config, using a TPU v6e with $8N\times4L$ looping config. $\mathcal{D}$ (effective depth) is 32 for all methods. This infrastructure efficiency compounds with the algorithmic convergence speed-up demonstrated in Figure 6, these complementary benefits yield a overall reduction in training wall-clock time.

\begin{tabular}{lcc}
  \toprule
  Method & \makecell{Step \\ Time (ms)} & \makecell{Peak \\ HBM (gb)} \\
  \midrule
  Baseline (32 layers) & 130 & 7.7 \\
  Just Looping & 130 & \textbf{5.1} \\
  ELT (w/ ILSD) & 130 & \textbf{5.1} \\
  ELT$^{opt}$ (w/ ILSD) & \textbf{118} & 5.8 \\
  \bottomrule
\end{tabular}

While matching the baseline's theoretical FLOPs, ELT inherently reduces peak HBM memory by 34%, which acts as a budget to speedup training. For instance, ELT$^{opt}$ uses unrolling ("for loop") instead of "jax.lax.scan" across layers axis. This generates a new computational graph for every layer, removing some sequential computation, utilizing the ELT's memory savings and reducing step time. Other orthogonal optimizations include increasing batch-size and gradient checkpointing. (iii) Inference Speedup: Please refer to Table 3 for quantitative comparison. ELT eliminates redundant memory transfers (HBM $\rightarrow$ VRAM), achieving 3.5x peak throughput speedup in iso-inference compute settings.

flushleftLosses Ablation: Figure 7 (video gen) and Figure 8 (image gen) show the ablation of vanilla looping vs ELT across masked generative models and DiTs. We further ablate the importance of ILSD loss in Figure 13.

flushleftIterative Refinement: We analyzed the per-iteration update magnitudes (L2 norm) across the loops and sampling steps (Figure 14). It shows that initial denoising steps have more updates for higher loops than later denoising steps. These observations can potentially be used to make the number of loops adaptive across denoising steps.

C. Limitations

While ELT demonstrates strong parameter efficiency and elastic inference capabilities, we identify some limitations of the current work.

flushleft Failure Cases: We observe that ELT performance degrades in two notable scenarios: (i) when the number of unique layers $N$ is too small (e.g., $1N \times 32L$ configuration in Table 2), a single layer lacks representational capacity regardless of loop count, and (ii) when loop count $L$ significantly exceeds $L_{\text{max}}$ at inference, quality can deteriorate as the shared block over-iterates beyond its trained convergence regime.

flushleft Deployment Considerations: While ELT offers significant parameter savings and throughput gains, practical deployment requires careful selection of the operating point ($N$, $L$) based on the target hardware. The optimal loop count $L$ for a given quality target is model-scale dependent and should be calibrated per deployment tier.

D. Acknowledgments

We thank Nikunj Saunshi, Thomas Mensink and Ashwini Pokle for their constructive comments and valuable suggestions, which enhanced the overall quality of this manuscript.

References

Section Summary: The references section compiles academic papers and preprints that investigate advanced neural network designs, with a strong emphasis on transformer models that incorporate loops, recurrence, or iterative processing to handle visual tasks like object recognition and image generation. These works explore how repeated computation steps can boost model performance and efficiency, often drawing inspiration from biological vision systems or optimization techniques. Many entries focus on making such architectures practical for real-world use in scalable AI applications.

[1] K. Kar, J. Kubilius, K. Schmidt, E. B. Issa, and J. J. DiCarlo. Evidence that recurrent circuits are critical to the ventral stream's execution of core object recognition behavior. Nature neuroscience, 22(6):974–983, 2019.

[2] T. C. Kietzmann, C. J. Spoerer, L. K. Sörensen, R. M. Cichy, O. Hauk, and N. Kriegeskorte. Recurrence is required to capture the representational dynamics of the human visual system. Proceedings of the National Academy of Sciences, 116(43):21854–21863, 2019.

[3] M. Dehghani, S. Gouws, O. Vinyals, J. Uszkoreit, and Ł. Kaiser. Universal transformers. arXiv preprint arXiv:1807.03819, 2018.

[4] N. Saunshi, N. Dikkala, Z. Li, S. Kumar, and S. J. Reddi. Reasoning with latent thoughts: On the power of looped transformers, 2025. URL https://arxiv.org/abs/2502.17416.

[5] L. Yang, K. Lee, R. Nowak, and D. Papailiopoulos. Looped transformers are better at learning learning algorithms. arXiv preprint arXiv:2311.12424, 2023.

[6] H. Chang, H. Zhang, L. Jiang, C. Liu, and W. T. Freeman. Maskgit: Masked generative image transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11315–11325, 2022.

[7] L. Yu, Y. Cheng, K. Sohn, J. Lezama, H. Zhang, H. Chang, A. G. Hauptmann, M.-H. Yang, Y. K. Hao, I. Essa, et al. Magvit: Masked generative video transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10459–10469, 2023a.

[8] S. Zilberstein. Using anytime algorithms in intelligent systems. AI magazine, 17(3):73–73, 1996.

[9] L. Yu, J. Lezama, N. B. Gundavarapu, L. Versari, K. Sohn, D. Minnen, Y. Cheng, A. Gupta, X. Gu, A. G. Hauptmann, et al. Language model beats diffusion–tokenizer is key to visual generation. arXiv preprint arXiv:2310.05737, 2023b.

[10] W. Peebles and S. Xie. Scalable diffusion models with transformers, 2023. URL https://arxiv.org/abs/2212.09748.

[11] K. Gatmiry, N. Saunshi, S. J. Reddi, S. Jegelka, and S. Kumar. Can looped transformers learn to implement multi-step gradient descent for in-context learning?, 2024. URL https://arxiv.org/abs/2410.08292.

[12] S. Bae, Y. Kim, R. Bayat, S. Kim, J. Ha, T. Schuster, A. Fisch, H. Harutyunyan, Z. Ji, A. Courville, and S.-Y. Yun. Mixture-of-recursions: Learning dynamic recursive depths for adaptive token-level computation, 2025. URL https://arxiv.org/abs/2507.10524.

[13] Y. Li. Mor-vit: Efficient vision transformer with mixture-of-recursions, 2025. URL https://arxiv.org/abs/2507.21761.

[14] Y. Fan, Y. Du, K. Ramchandran, and K. Lee. Looped transformers for length generalization. arXiv preprint arXiv:2409.15647, 2024.

[15] J. Geiping, S. McLeish, N. Jain, J. Kirchenbauer, S. Singh, B. R. Bartoldson, B. Kailkhura, A. Bhatele, and T. Goldstein. Scaling up test-time compute with latent reasoning: A recurrent depth approach, 2025. URL https://arxiv.org/abs/2502.05171.

[16] S. Bai, J. Z. Kolter, and V. Koltun. Deep equilibrium models, 2019. URL https://arxiv.org/abs/1909.01377.

[17] A. Pokle, Z. Geng, and Z. Kolter. Deep equilibrium approaches to diffusion models, 2022. URL https://arxiv.org/abs/2210.12867.

[18] Z. Geng, A. Pokle, and J. Z. Kolter. One-step diffusion distillation via deep equilibrium models, 2023. URL https://arxiv.org/abs/2401.08639.

[19] S. McCallum, K. Arora, and J. Foster. Reversible deep equilibrium models, 2025. URL https://arxiv.org/abs/2509.12917.

[20] C. Anil, A. Pokle, K. Liang, J. Treutlein, Y. Wu, S. Bai, Z. Kolter, and R. Grosse. Path independent equilibrium models can better exploit test-time computation, 2022. URL https://arxiv.org/abs/2211.09961.

[21] M. Gabor, T. Piotrowski, and R. L. G. Cavalcante. Positive concave deep equilibrium models, 2024. URL https://arxiv.org/abs/2402.04029.

[22] G. Menghani. Efficient deep learning: A survey on making deep learning models smaller, faster, and better. ACM Computing Surveys, 55(12):1–37, 2023.

[23] Y. Zhao, Y. Xu, Z. Xiao, H. Jia, and T. Hou. Mobilediffusion: Instant text-to-image generation on mobile devices, 2024b. URL https://arxiv.org/abs/2311.16567.

[24] T. Castells, H.-K. Song, T. Piao, S. Choi, B.-K. Kim, H. Yim, C. Lee, J. G. Kim, and T.-H. Kim. Edgefusion: On-device text-to-image generation, 2024. URL https://arxiv.org/abs/2404.11925.

[25] S. Goyal, D. Tula, G. Jain, P. Shenoy, P. Jain, and S. Paul. Masked generative nested transformers with decode time scaling, 2025. URL https://arxiv.org/abs/2502.00382.

[26] Devvrit, S. Kudugunta, A. Kusupati, T. Dettmers, K. Chen, I. Dhillon, Y. Tsvetkov, H. Hajishirzi, S. Kakade, A. Farhadi, P. Jain, et al. Matformer: Nested transformer for elastic inference. Advances in Neural Information Processing Systems, 2024.

[27] A. Kusupati, G. Bhatt, A. Rege, M. Wallingford, A. Sinha, V. Ramanujan, W. Howard-Snyder, K. Chen, S. Kakade, P. Jain, et al. Matryoshka representation learning. Advances in Neural Information Processing Systems, 35:30233–30249, 2022.

[28] J. Wang, Z. Lai, J. Chen, J. Guo, H. Guo, X. Li, X. Yue, and C. Guo. Elastic diffusion transformer, 2026. URL https://arxiv.org/abs/2602.13993.

[29] W.-J. Shu, X. Qiu, R.-J. Zhu, H. H. Chen, Y. Liu, and H. Yang. Loopvit: Scaling visual arc with looped transformers, 2026. URL https://arxiv.org/abs/2602.02156.

[30] H. He, J. Liang, X. Wang, P. Wan, D. Zhang, K. Gai, and L. Pan. Scaling image and video generation via test-time evolutionary search, 2025. URL https://arxiv.org/abs/2505.17618.

[31] S. Duggal, P. Isola, A. Torralba, and W. T. Freeman. Adaptive length image tokenization via recurrent allocation, 2024. URL https://arxiv.org/abs/2411.02393.

[32] R. Bachmann, J. Allardice, D. Mizrahi, E. Fini, O. F. Kar, E. Amirloo, A. El-Nouby, A. Zamir, and A. Dehghan. Flextok: Resampling images into 1d token sequences of flexible length, 2025. URL https://arxiv.org/abs/2502.13967.

[33] K. Miwa, K. Sasaki, H. Arai, T. Takahashi, and Y. Yamaguchi. One-d-piece: Image tokenizer meets quality-controllable compression, 2025. URL https://arxiv.org/abs/2501.10064.

[34] W. Yan, V. Mnih, A. Faust, M. Zaharia, P. Abbeel, and H. Liu. Elastictok: Adaptive tokenization for image and video, 2025. URL https://arxiv.org/abs/2410.08368.

[35] J. Shen, K. Tirumala, M. Yasunaga, I. Misra, L. Zettlemoyer, L. Yu, and C. Zhou. Cat: Content-adaptive image tokenization, 2025. URL https://arxiv.org/abs/2501.03120.

[36] A. Haridas, U. Saxena, P. A. Fashi, M. Rezagholizadeh, V. Appia, and E. Barsoum. DC-DiT: Adaptive compute and elastic inference for visual generation via dynamic chunking. arXiv preprint arXiv:2603.06351, 2026.

[37] Y. Song, P. Dhariwal, M. Chen, and I. Sutskever. Consistency models, 2023. URL https://arxiv.org/abs/2303.01469.

[38] T. Salimans and J. Ho. Progressive distillation for fast sampling of diffusion models, 2022. URL https://arxiv.org/abs/2202.00512.

[39] P. Esser, R. Rombach, and B. Ommer. Taming transformers for high-resolution image synthesis. In CVPR, pages 12873–12883, 2021.

[40] D. Kondratyuk, L. Yu, X. Gu, J. Lezama, J. Huang, R. Hornung, H. Adam, H. Akbari, Y. Alon, V. Birodkar, et al. Videopoet: A large language model for zero-shot video generation. ICML, 2024.

[41] Y. Wang, S. Ren, Z. Lin, Y. Han, H. Guo, Z. Yang, D. Zou, J. Feng, and X. Liu. Parallelized autoregressive visual generation, 2024. URL https://arxiv.org/abs/2412.15119.

[42] T. Li, Y. Tian, H. Li, M. Deng, and K. He. Autoregressive image generation without vector quantization, 2024. URL https://arxiv.org/abs/2406.11838.

[43] J. Ho, A. P. Jain, and P. Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020.

[44] Y. Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456, 2020.

[45] O. Ronneberger, P. Fischer, and T. Brox. U-net: Convolutional networks for biomedical image segmentation, 2015. URL https://arxiv.org/abs/1505.04597.

[46] E. Hoogeboom, T. Mensink, J. Heek, K. Lamerigts, R. Gao, and T. Salimans. Simpler diffusion: 1.5 fid on imagenet512 with pixel-space diffusion. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 18062–18071, 2025.

[47] J. Ho and T. Salimans. Classifier-free diffusion guidance, 2022. URL https://arxiv.org/abs/2207.12598.

[48] A. Brock. Large scale gan training for high fidelity natural image synthesis. arXiv preprint arXiv:1809.11096, 2018.

[49] A. Sauer, K. Schwarz, and A. Geiger. Stylegan-xl: Scaling stylegan to large diverse datasets, 2022. URL https://arxiv.org/abs/2202.00273.

[50] A. Q. Nichol and P. Dhariwal. Improved denoising diffusion probabilistic models. In International conference on machine learning, pages 8162–8171. PMLR, 2021.

[51] P. Dhariwal and A. Nichol. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34:8780–8794, 2021.

[52] R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer. High-resolution image synthesis with latent diffusion models, 2022a. URL https://arxiv.org/abs/2112.10752.

[53] S. Gao, P. Zhou, M.-M. Cheng, and S. Yan. Masked diffusion transformer is a strong image synthesizer. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 23164–23173, 2023.

[54] H. Zheng, W. Nie, A. Vahdat, and A. Anandkumar. Fast training of diffusion models with masked transformers. arXiv preprint arXiv:2306.09305, 2023.

[55] J. Ho, C. Saharia, W. Chan, D. J. Fleet, M. Norouzi, and T. Salimans. Cascaded diffusion models for high fidelity image generation. Journal of Machine Learning Research, 23(47):1–33, 2022a.

[56] A. Jabri, D. Fleet, and T. Chen. Scalable adaptive computation for iterative generation. arXiv preprint arXiv:2212.11972, 2022.

[57] E. Hoogeboom, J. Heek, and T. Salimans. simple diffusion: End-to-end diffusion for high resolution images. In International Conference on Machine Learning, pages 13213–13232. PMLR, 2023.

[58] D. P. Kingma and R. Gao. Understanding the diffusion objective as a weighted integral of elbos. arXiv preprint arXiv:2303.00848, 2, 2023.

[59] T. Hang, S. Gu, C. Li, J. Bao, D. Chen, H. Hu, X. Geng, and B. Guo. Efficient diffusion training via min-snr weighting strategy, 2024. URL https://arxiv.org/abs/2303.09556.

[60] Z. Wang, Y. Jiang, H. Zheng, P. Wang, P. He, Z. Wang, W. Chen, and M. Zhou. Patch diffusion: Faster and more data-efficient training of diffusion models, 2023. URL https://arxiv.org/abs/2304.12526.

[61] A. Razavi, A. Van den Oord, and O. Vinyals. Generating diverse high-fidelity images with vq-vae-2. Advances in neural information processing systems, 32, 2019.

[62] C. Zheng, L. T. Vuong, J. Cai, and D. Phung. Movq: Modulating quantized vectors for high-fidelity image generation, 2022. URL https://arxiv.org/abs/2209.09002.

[63] M. Weber, L. Yu, Q. Yu, X. Deng, X. Shen, D. Cremers, and L.-C. Chen. Maskbit: Embedding-free image generation via bit tokens, 2024. URL https://arxiv.org/abs/2409.16211.

[64] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009.

[65] K. Soomro, A. R. Zamir, and M. Shah. Ucf101: A dataset of 101 human actions classes from videos in the wild, 2012. URL https://arxiv.org/abs/1212.0402.

[66] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding, 2019. URL https://arxiv.org/abs/1810.04805.

[67] M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30, 2017.

[68] T. Salimans, I. Goodfellow, W. Zaremba, V. Cheung, A. Radford, and X. Chen. Improved techniques for training gans, 2016. URL https://arxiv.org/abs/1606.03498.

[69] T. Unterthiner, S. Van Steenkiste, K. Kurach, R. Marinier, M. Michalski, and S. Gelly. Towards accurate generative models of video: A new metric & challenges. arXiv preprint arXiv:1812.01717, 2018.

[70] Y. Zhao, Y. Xiong, and P. Krähenbühl. Image and video tokenization with binary spherical quantization, 2024a. URL https://arxiv.org/abs/2406.07548.

[71] Z. Ni, Y. Wang, R. Zhou, J. Guo, J. Hu, Z. Liu, S. Song, Y. Yao, and G. Huang. Revisiting non-autoregressive transformers for efficient image synthesis, 2024a. URL https://arxiv.org/abs/2406.05478.

[72] Z. Ni, Y. Wang, R. Zhou, Y. Han, J. Guo, Z. Liu, Y. Yao, and G. Huang. Enat: Rethinking spatial-temporal interactions in token-based image synthesis, 2024b. URL https://arxiv.org/abs/2411.06959.

[73] Google Cloud. TPU v6e (Trillium) Documentation. https://cloud.google.com/tpu/docs/v6e, 2024. Accessed: 2024-05-22.

[74] T. Höppe, A. Mehrjou, S. Bauer, D. Nielsen, and A. Dittadi. Diffusion models for video prediction and infilling. arXiv preprint arXiv:2206.07696, 2022.

[75] I. Skorokhodov, S. Tulyakov, and M. Elhoseiny. Stylegan-v: A continuous video generator with the price, image quality and perks of stylegan2. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 3626–3636, 2022.

[76] S. Yu, J. Tack, S. Mo, H. Kim, J. Kim, J.-W. Ha, and J. Shin. Generating videos with dynamics-aware implicit generative adversarial networks. arXiv preprint arXiv:2202.10571, 2022.

[77] A. Clark, J. Donahue, and K. Simonyan. Adversarial video generation on complex datasets. arXiv preprint arXiv:1907.06571, 2019.

[78] J. Ho, T. Salimans, A. Gritsenko, W. Chan, M. Norouzi, and D. J. Fleet. Video diffusion models. Advances in Neural Information Processing Systems, 35:8633–8646, 2022b.

[79] S. Ge, T. Hayes, H. Yang, X. Yin, G. Pang, D. Jacobs, J.-B. Huang, and D. Parikh. Long video generation with time-agnostic vqgan and time-sensitive transformer, 2022. URL https://arxiv.org/abs/2204.03638.

[80] G. Le Moing, J. Ponce, and C. Schmid. Ccvs: Context-aware controllable video synthesis. Advances in Neural Information Processing Systems, 34:14042–14055, 2021.

[81] U. Singer, A. Polyak, T. Hayes, X. Yin, J. An, S. Zhang, Q. Hu, H. Yang, O. Ashual, O. Gafni, et al. Make-a-video: Text-to-video generation without text-video data. arXiv preprint arXiv:2209.14792, 2022.

[82] W. Hong, M. Ding, W. Zheng, X. Liu, and J. Tang. Cogvideo: Large-scale pretraining for text-to-video generation via transformers. arXiv preprint arXiv:2205.15868, 2022.

[83] M. Deng, H. Li, T. Li, Y. Du, and K. He. Generative modeling via drifting. arXiv preprint arXiv:2602.04770, 2026.

[84] I. Loshchilov and F. Hutter. Decoupled weight decay regularization, 2019. URL https://arxiv.org/abs/1711.05101.