Junchao-cs/SolarWM

SolarWM: Open Data and Scalable Training for Long-Horizon Video World Models

Junchao Huang1,2   Guian Fang3   Shengju Qian4   Xianghao Kong5   Zhuoran Zhao5,6
Wei Huang7   Yihua Du6   Zixin Zhang6   Justin Cui8   Yuchao Gu7   Yukang Chen7   Xinting Hu
Tianyu He9   Shaoshuai Shi   Zhuotao Tian2   Xin Wang   Mike Zheng Shou3   Li Jiang1,2
1CUHK-SZ   2SLAI   3NUS   4CUHK   5HKUST   6HKUST-GZ   7NVIDIA   8UCLA   9MSRA

We present SolarWM, a fully open, end-to-end framework for building interactive video world models, from data preparation to scalable training and long-horizon inference.

• Four models. One codebase. A unified framework supports four 5B–33B models across Wan2.2, LTX-2.5, and MiniMax-H3, with shared data, training, and inference interfaces while preserving each backbone's native representation and objective.
• A simple three-stage training recipe. Bidirectional adaptation, teacher-forced autoregressive initialization, and distribution matching distillation form a shared route across heterogeneous video backbones, without specialized ODE or consistency-distillation initialization. The resulting causal models support few-step interaction over arbitrarily long horizons* through sliding-window inference.
• 1.43M clips · 25 TB. Fully open. We release the complete dataset and data-processing pipeline, unifying 14 datasets into a frame-aligned format with metric camera geometry, captions, quality metadata, and provenance. Source processing is decoupled from training-mixture design, allowing flexible data selection and mixing without reprocessing the source datasets.

• September 13, 2026: We release SolarWM-H3, the first MiniMax-H3-based world model for streaming, few-step interaction. Training and inference code, weights, and pre-encoded training data are now available for all three stages: Stage0.5, Stage1, and Stage2. Get started with the H3 guide.
• September 3, 2026 — We open-source the training and inference code, the complete dataset, the data pipeline, model weights for all SolarWM-5B training stages, and bidirectional weights for SolarWM-14B, SolarWM-LTX, and SolarWM-H3.

The staged workflow is Stage0.5 FM → Stage1 TF-AnyFlow → Stage2 DMD via SGF, turning a bidirectional video model into a camera-controlled few-step autoregressive model.

• Stage0.5 learns full-clip bidirectional flow matching and establishes the base video, text, and camera-conditioned representation.
• Stage1 combines teacher forcing with the AnyFlow loss in one training stage. Clean history conditions noisy target chunks while the model learns both denoising and finite-step flow maps. This removes the need for a separate ODE or consistency-distillation initialization before Stage2/DMD.
• Stage2 performs DMD via self-gradient forcing (SGF), training the causal student on its own autoregressive rollout with a frozen teacher and a trainable critic.

Wan, LTX, and MiniMax-H3 require separate runtime environments. Activate the environment for the selected backbone, then install the shared SolarWM source:

python -m pip install -e .
solarwm environment probe

Model weights are available from the SolarWM model collection. See the Wan2.2 TI2V-5B guide for Stage0.5, Stage1, and Stage2 commands. The MiniMax-H3 guide provides launch commands for Stage0.5, Stage1, and Stage2.

The public SolarWM-Data release contains release controls, licenses, recipe and test indexes, small format examples, and the SolarWM-Data-Annotation/ package. It does not include the full releases-v1/raw-wds/ or releases-v1/latent-wds/ payloads.

For a released recipe that uses preencoded data, download its matching latent generation. That is sufficient for training and does not require raw-WDS. Raw-WDS is needed only when you want the full processed video corpus, online encoding, your own latent generation, or another workflow whose index points to raw data.

1. Use preencoded latents. Each latent generation is published in a separate repository. See the latent-WDS release list for available downloads and generations that are still being uploaded.
2. Rebuild raw-WDS from annotations. SolarWM-Data-Annotation/ is an annotation-only release with no videos. It contains the released camera trajectories, captions, metadata, source identities, and reconstruction tools. Follow its README to download the original videos from their source publishers and process them into the expected raw-wds/ layout.
3. Request prepared raw-WDS. Submit the Dataset Access Form. Approved applicants receive download instructions by email.
4. Download the standalone test set. The SolarWM standalone test set v1 contains the current 1,300-example, 13-source evaluation set in 60 WebDataset tar files (approximately 77.4 GB), without the training payload.

See the dataset access guide for download commands and the payload required by each training, validation, and inference example.

# Validate and render the exact resolved configuration.
solarwm config resolve \
  --config configs/examples/minimax_h3/stage0p5-158f-lora384-sp2.yaml \
  --set distributed.world_size=8 \
  --set train.global_batch_size=4 \
  --set model.checkpoint_path=/path/to/SolarWM-models/SolarWM-h3-33B-base \
  --set data.index_root=/path/to/SolarWM-Data/releases-v1 \
  --set data.transport.root=/path/to/SolarWM-Data/releases-v1 \
  --set data.silence_latents_path=/path/to/SolarWM-Data/releases-v1/latent-wds/minimax-h3-158f-768p-nomind-v1/support/h3_silence_153_158_170.safetensors \
  --set data.encoder_contract_path=/path/to/SolarWM-Data/releases-v1/latent-wds/minimax-h3-158f-768p-nomind-v1/support/encoder_contract.json \
  --set runtime.output_dir=/path/to/output/h3-stage0p5-158f

# Train H3 Stage0.5 with preencoded latents.
torchrun --standalone --nproc-per-node=8 \
  -m solarwm train \
  --config configs/examples/minimax_h3/stage0p5-158f-lora384-sp2.yaml \
  --set distributed.world_size=8 \
  --set train.global_batch_size=4 \
  --set model.checkpoint_path=/path/to/SolarWM-models/SolarWM-h3-33B-base \
  --set data.index_root=/path/to/SolarWM-Data/releases-v1 \
  --set data.transport.root=/path/to/SolarWM-Data/releases-v1 \
  --set data.silence_latents_path=/path/to/SolarWM-Data/releases-v1/latent-wds/minimax-h3-158f-768p-nomind-v1/support/h3_silence_153_158_170.safetensors \
  --set data.encoder_contract_path=/path/to/SolarWM-Data/releases-v1/latent-wds/minimax-h3-158f-768p-nomind-v1/support/encoder_contract.json \
  --set runtime.output_dir=/path/to/output/h3-stage0p5-158f

# Run H3 Stage2 inference for 158 frames.
torchrun --standalone --nproc-per-node=8 -m solarwm infer \
  --config configs/examples/minimax_h3/infer-stage2-158f-sp4.yaml \
  --set model.checkpoint_path=/path/to/SolarWM-models/SolarWM-h3-33B-base \
  --set checkpoint.resume_from=/path/to/SolarWM-models/SolarWM-h3-33B-sgf-stage2-158f \
  --set checkpoint.weight_source=ema \
  --set data.index_root=/path/to/SolarWM-Data/releases-v1 \
  --set data.transport.root=/path/to/SolarWM-Data/releases-v1 \
  --set data.silence_latents_path=/path/to/SolarWM-Data/releases-v1/latent-wds/minimax-h3-158f-768p-nomind-v1/support/h3_silence_153_158_170.safetensors \
  --set data.encoder_contract_path=/path/to/SolarWM-Data/releases-v1/latent-wds/minimax-h3-158f-768p-nomind-v1/support/encoder_contract.json \
  --set runtime.output_dir=/path/to/output/h3-stage2-infer

# Preencode raw-WDS for the H3 158f recipe.
torchrun --standalone --nproc-per-node=8 \
  -m solarwm preencode \
  --config configs/examples/minimax_h3/preencode-158f.yaml \
  --set model.checkpoint_path=/path/to/SolarWM-models/SolarWM-h3-33B-base \
  --set data.index_root=/path/to/SolarWM-Data/releases-v1 \
  --set data.transport.root=/path/to/SolarWM-Data/releases-v1 \
  --set preencode.output_root=/path/to/output/preencoded/minimax-h3-158f-768p-nomind-v1 \
  --set runtime.output_dir=/path/to/output/h3-preencode

The quickstart covers H3 setup, training, and inference. For Stage1/Stage2 training and full-length inference, see the H3 guide. Other models have their own backend guides.

Every launch writes resolved-config.json and launch-manifest.json before model allocation. Config overrides are explicit and included in the resolved configuration identity.

Index rows always contain POSIX shard keys relative to the release directory:

{"sample_id":"...","shard":"raw-wds/abot/shards/kept-high-000001.tar"}

Only the runtime root changes:

# Locally mounted storage
data:
  index_root: /path/to/SolarWM-Data/releases-v1
  transport:
    kind: local
    root: /path/to/SolarWM-Data/releases-v1

# Object-store streaming uses the same release-relative rows. Supply the
# release root from the distribution channel or deployment environment.
data:
  index_root: /path/to/SolarWM-Data/releases-v1
  transport:
    kind: gcs
    root: ${SOLAR_RELEASE_ROOT}
    cache_dir: /path/to/solar-cache
    cache_max_gib: 256

• Quickstart
• Runtime environments
• Architecture
• Data contract
• Dataset overview and statistics
• Download and access
• Wan 2.2 TI2V-5B backend
• Wan 2.2 I2V-A14B backend
• LTX-2.5 backend
• MiniMax-H3 backend

We gratefully thank the teams behind Wan2.2, LTX-2.5, and MiniMax-H3 for releasing the code and pretrained models that make the SolarWM backbone family possible. We also thank the authors of all datasets and open-source projects listed in NOTICE.

If you use SolarWM-Data, the data engine, or the released models in your research, please cite our paper.

Paper: https://arxiv.org/abs/2609.02886

@misc{huang2026solarwmopendatascalable,
      title={SolarWM: Open Data and Scalable Training for Long-Horizon Video World Models}, 
      author={Junchao Huang and Guian Fang and Shengju Qian and Xianghao Kong and Zhuoran Zhao and Wei Huang and Yihua Du and Zixin Zhang and Justin Cui and Yuchao Gu and Yukang Chen and Xinting Hu and Tianyu He and Shaoshuai Shi and Zhuotao Tian and Xin Wang and Mike Zheng Shou and Li Jiang},
      year={2026},
      eprint={2609.02886},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2609.02886}, 
}

SolarWM is licensed under Apache-2.0. See LICENSE and NOTICE. Model weights and backbone packages may carry their own licenses; review the license and model card in the corresponding release package before use or redistribution. In particular, LTX-2.5 derivatives are subject to the LTX-2.x Community License, and the MiniMax H3 Community License contains territory restrictions. Those packages are not relicensed under the SolarWM code license.