HazyResearch/ncm

Fast, Parallel Coarse-to-Fine Text Generation

Next-Concept Modeling (NCM) is a two-stage framework for parallel text generation. A multiscale residual VQ tokenizer represents a sequence as a hierarchy of discrete codes, and a transformer predicts that hierarchy from coarse to fine. Positions within a level are generated in parallel, so generation requires one forward evaluation per level.

NCM generates coarse to fine: each level predicts a longer sequence in parallel, and decoding every level shows a sample being refined from a rough sketch to the full output in a few steps.

NCM beats or matches strong diffusion baselines on different tasks & datasets while generating 9-31× faster.

Weights and tokenized datasets are hosted in the NCM Hugging Face collection. Each released model consists of a frozen tokenizer and a next-concept generator.

git clone https://github.com/HazyResearch/ncm.git
cd ncm
pip install -r requirements.txt

Python 3.10 or newer and a CUDA-capable PyTorch installation are recommended.

Download the checkpoints and evaluation inputs:

hf download hazyresearch/ncm-checkpoints --local-dir artifacts
hf download hazyresearch/ncm-datasets --repo-type dataset --local-dir data

Then run:

python evaluate.py --config config/eval/<task>.yaml --checkpoint-dir artifacts/<task> --data-dir data/<task>
# <task> is one of xsum, wmt14, sudoku-easy, sudoku-medium or sudoku-hard

python evaluate.py --config config/eval/<dataset>.yaml --checkpoint-dir artifacts/<dataset>
# <dataset> is tinystories, owt or lm1b-<size> with <size> one of small, medium or large

This runs one generation shard per visible GPU, merges them, and prints ROUGE-1/2/L and generative perplexity for XSum; cased and lowercased sacreBLEU, chrF and length ratio for WMT14; solved, cell and valid-grid rates for Sudoku; and generative perplexity under GPT-2 large with unigram entropy, over 5 seeds of 256 unconditional samples, for TinyStories, OpenWebText and LM1B, the latter under the FLM scoring protocol. Add --cfg <scale> for classifier-free guidance on XSum and WMT14; the guided WMT14 row uses --cfg 1.0.

• evaluate.py: evaluation entry point
• scripts/: per-GPU generation shard and the merge that scores it
• config/eval/: one config per released checkpoint, identical to the config.yaml shipped with it
• models/, tokenization/: generator and tokenizer implementations

@misc{kumbong2026nextconcept,
      title={Next-Concept Modeling: Fast, Parallel Coarse-to-Fine Text Generation},
      author={Hermann Kumbong and Ishan S. Khare and Iddah Mlauzi and Andrew Woen and Charles Ding and E. Kelly Buchanan and Simran Arora and Azalia Mirhoseini and Christopher Ré},
      year={2026},
      eprint={2610.14162},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2610.14162},
}

Released under the MIT license.