Inductive Moment Matching

Official Implementation of Inductive Moment Matching

Linqi Zhou¹, Stefano Ermon² Jiaming Song¹

¹Luma AI, ²Stanford University

[Paper] [Blog]

Also check out our accompanying position paper that explains the motivation and ways of designing new generative paradigms.

Dependencies

To install all packages in this codebase along with their dependencies, run

conda env create -f env.yml

Pre-trained models

We provide pretrained checkpoints through our repo on Hugging Face:

• IMM on CIFAR-10: cifar10.pkl.
• IMM on ImageNet-256x256:
  1. t-s is passed as second time embedding, trained with a=2: imagenet256_ts_a2.pkl.
  2. s is passed as second time embedding directly, trained with a=1: imagenet256_s_a1.pkl.

Sampling

The checkpoints can be tested via

python generate_images.py --config-name=CONFIG_NAME eval.resume=CKPT_PATH REPLACEMENT_ARGS

where CONFIG_NAME is im256_generate_images.yaml or cifar10_generate_images.yaml and CKPT_PATH is the path to your checkpoint. When loading imagenet256_s_a1.pkl, REPLACEMENT_ARGS needs to be network.temb_type=identity. Otherwise, REPLACEMENT_ARGS is empty.

Checklist

• Add model weights and model definitions.
• Add inference scripts.
• Add evaluation scripts.
• Add training scripts.

Acknowledgements

Some of the utility functions are based on EDM, and thus parts of the code would apply under this license.

Citation

@article{zhou2025inductive,
  title={Inductive Moment Matching},
  author={Zhou, Linqi and Ermon, Stefano and Song, Jiaming},
  journal={arXiv preprint arXiv:2503.07565},
  year={2025}
}