NVIDIA Cosmos 3: four open world simulation models

What happened?

On November 5, NVIDIA published four new open-weight Cosmos 3 checkpoints on Hugging Face (Hugging Face). Each is a post-trained Cosmos 3 Nano example with 16 billion trainable parameters under OpenMDW 1.1, allowing commercial and non-commercial use under its terms.

The four models

• Cosmos3-Nano-Transfer-Auto: offline synthetic driving data across up to 11 camera views including fisheyes, joint RGB–LiDAR, with weather, lighting, and time-of-day controls.
• Cosmos3-Nano-Sim-Auto: the causally distilled streaming variant of Transfer-Auto for fast synchronized camera and LiDAR rollouts.
• Cosmos3-Nano-Sim-Transfer: turns a physics simulator’s depth, segmentation, edge, and blur controls plus a text prompt into photorealistic RGB video.
• Cosmos3-Nano-Sim-Bimanual: predicts next video frames from two-arm robot (YAM, AgiBot) or camera actions, driven by a policy or joysticks.

How streaming generation works

Cosmos 3’s Mixture-of-Transformers backbone links separate text and visual networks through attention. The generator predicts short chunks from the prompt, available controls, and generated history, with temporal-causal attention blocking future targets. Training runs in four stages: bidirectional adaptation, teacher forcing, consistency distillation, then self-forcing so the model learns under its own compounding errors.

Why it matters

Robotics and autonomous driving need environments where each action shapes the next observation. These checkpoints give an open starting point for multi-view driving data, streaming simulation, and bimanual manipulation, with post-training recipes for your own sensor rig. Unlike the earlier Cosmos3-H-Dreams surgical simulator, this release targets general interactive world simulation.