How does RT-2 differ from RT-1?

RT-1 was trained only on robot data. RT-2 co-fine-tunes a large vision-language model (PaLI-X or PaLM-E) on both web and robot data, enabling emergent reasoning capabilities that RT-1 lacks.

Can RT-2 handle objects it has never seen?

Yes. By leveraging web-scale visual knowledge, RT-2 can reason about and manipulate objects not present in its robot training data, a key emergent capability.

New: the Timeline is live. Track world model releases, papers, and benchmark updates in real time.

world-models.io

The Knowledge Hub for AI World Models

RT-2

RT-2 (Robotic Transformer 2) is a Vision-Language-Action model by Google DeepMind that transfers knowledge from web-scale vision-language pretraining directly to robotic control policies.

robotics model-based-rl simulation embodied-ai

world-models.io - The Knowledge Hub for AI World Models

world-models.io is a structured knowledge platform covering AI world models - from model-based reinforcement learning to foundation world models, embodied AI, and simulation engines.

Explore world models like DreamerV3, Genie 3, V-JEPA 2, NVIDIA Cosmos, MuZero, Sora, DIAMOND, OASIS, LeWorldModel, PixVerse R1, and more. Compare architectures, read research, and discover the labs building the future of AI.

Featured World Models

Genie 3 - Google DeepMind's real-time interactive 3D world generation from text at 24fps
V-JEPA 2 - Meta FAIR's self-supervised world model for visual understanding and zero-shot robot control
DreamerV3 - General algorithm mastering diverse domains through world model learning
NVIDIA Cosmos - World foundation model platform for physical AI
Genie 2 - Interactive 3D environment generation from single images
Sora - OpenAI's video generation model with emergent world understanding
MuZero - Superhuman game play through learned planning
LeWorldModel - Compact 15M-parameter JEPA that learns physics on a single GPU
V-JEPA - Self-supervised visual world model by Meta FAIR
DIAMOND - Diffusion-based world model for reinforcement learning
OASIS - Open-source real-time neural game engine
PixVerse R1 - First multiplayer real-time world model with shared worlds
TD-MPC2 - Scalable world model agent for 104+ tasks
IRIS - Autoregressive world model bridging language modeling and RL
GAIA-1 - Generative world model for autonomous driving
Copilot4D - 4D LiDAR world model for self-driving
I-JEPA - Image-based Joint Embedding Predictive Architecture
GameNGen - Neural game engine simulating DOOM in real-time
3D-VLA - 3D vision-language-action model for embodied AI
Pandora - Generative world model for open-ended environments
PlaNet - Deep Planning Network for latent space control
RSSM - Foundational architecture behind the Dreamer family
DreamerV2 - First model-based agent to achieve human-level Atari
World Models (Ha & Schmidhuber) - The seminal 2018 paper that popularized world models
Predictron - End-to-end learning and planning via abstract world models
UniSim - Universal simulator for real-world interactions
I2A - Imagination-augmented agents for deep RL
VPN - Value Prediction Network for abstract planning
Emu Video - Factorized video generation via image conditioning
AMI World Model - Multimodal foundation model for embodied AI
RT-2 - Vision-Language-Action model for robotic control
Gen-3 Alpha - Advanced video generation by Runway
Stable Video Diffusion - Open-source video generation foundation model
MILE - Model-based imitation learning for autonomous driving
LWM - Large World Model processing 1M+ tokens of video and text
Genie - First generative interactive environment from unlabeled video
STEVE-1 - Instruction-following agent for Minecraft

Explore

Home
What Are World Models
Models Database - Browse 37+ world models
Leaderboard - Rankings and performance index
Research - Topics and surveys
Comparisons - Side-by-side analyses
Guides - Beginner to advanced tutorials
Labs - Research labs and organizations
Timeline - Latest releases and updates
Glossary - Key terms and definitions
Methodology
Top World Models
World Models for Robotics
Research Papers
Video World Models
Autonomous Driving World Models
Game World Models

Research Topics

Comparisons

Guides

Labs & Organizations

Google DeepMind - Pioneers of DreamerV3, MuZero, Genie 2, and the RSSM architecture
NVIDIA Research - World foundation models for physical AI (Cosmos)
Meta FAIR - JEPA architectures and self-supervised world models
OpenAI - Sora and video generation as world simulation
Wayve - GAIA-1 for autonomous driving
Decart - Real-time neural game engines (OASIS)
Waabi - 4D LiDAR world models for self-driving
Microsoft Research - IRIS and diffusion-based world models
Google Research - Neural game engines (GameNGen)
AMI Labs - Multimodal foundation models for embodied AI
UC Berkeley
Stanford
Carnegie Mellon University
MIT CSAIL
Toyota Research Institute
Stability AI - Open-source generative AI (Stable Video Diffusion)
Runway - AI video generation (Gen-3 Alpha)
UT Austin - Game AI and instruction-following agents