ReDi

ReDi: Rectified Discrete Flow

Jaehoon Yoo, Wonjung Kim, Seunghoon Hong
KAIST
Paper arXiv Code Model

TL;DR: We introduce Rectified Discrete Flow (ReDi), a novel iterative method that reduces factorization error by rectifying the coupling between source and target distribution.

ReDi reduces the underlying factorization error of discrete flow models, enabling high-quality few-step generation on ImageNet 256x256 and OpenWebText.

Abstract

Discrete Flow-based Models (DFMs) are powerful generative models for high-quality discrete data but typically suffer from slow sampling speeds due to their reliance on iterative decoding processes. This reliance on a multi-step process originates from the factorization approximation of DFMs, which is necessary for handling high-dimensional data. In this paper, we analyze the factorization approximation error using Conditional Total Correlation (TC), and reveal its dependence on the coupling. To address the challenge of efficient few-step generation, we propose Rectified Discrete Flow (ReDi), a novel iterative method that reduces the underlying factorization error (measured as Conditional TC) by rectifying the coupling between source and target distributions. We theoretically prove that each ReDi step guarantees a monotonic decreasing Conditional TC, ensuring its convergence. Empirically, ReDi significantly reduces Conditional TC and enables few-step generation. Moreover, we demonstrate that the rectified couplings are well-suited for training efficient one-step models on image generation. ReDi offers a simple and theoretically grounded approach for tackling the few-step challenge, providing a new perspective on efficient discrete data synthesis.

Key Innovations

  1. Factorization error of discrete flows quantified via Conditional Total Correlation.

    We analyze the factorization approximation error of Discrete Flow-based Models through the lens of Conditional Total Correlation (TC), and reveal that this error is governed by the coupling between source and target distributions — providing a clear, measurable handle on the few-step bottleneck.

  2. Rectified Discrete Flow (ReDi): coupling rectification with convergence guarantees.

    ReDi is a simple iterative procedure that rectifies the source–target coupling to reduce factorization error. We theoretically prove that each ReDi step monotonically decreases Conditional TC, guaranteeing convergence — without changing the underlying flow architecture.

  3. Few-step and one-step generation on text and image benchmarks.

    ReDi empirically reduces Conditional TC and enables strong few-step generation on ImageNet 256x256 (image) and OpenWebText (text). The rectified couplings further serve as effective targets for training efficient one-step image generators.

Image Generation on ImageNet 256x256

Rectified couplings produced by ReDi enable strong few-step and one-step image generation on ImageNet 256x256.

ReDi qualitative image generation comparison on ImageNet 256x256

Qualitative comparison of generated samples on ImageNet 256x256 across baselines and ReDi at small step counts.

Text Generation on OpenWebText

ReDi reduces Conditional TC and improves few-step text generation quality on OpenWebText.

ReDi generative perplexity and entropy on OpenWebText

Generative perplexity and entropy on OpenWebText across few-step regimes.

Citation

@misc{yoo2025redirectifieddiscreteflow,
      title={ReDi: Rectified Discrete Flow},
      author={Jaehoon Yoo and Wonjung Kim and Seunghoon Hong},
      year={2025},
      eprint={2507.15897},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2507.15897},
}