Static research summary generated from local editorial content.
| Attribute | Value |
|---|---|
| Topic | Model-Based Reinforcement Learning |
| Summary | What model-based reinforcement learning is, how world models enable imagination-based planning, and why Dreamer, MuZero, PlaNet, and TD-MPC2 matter. |
| Related Models | 9 |
| Citations | 3 |
Editorial body section preserved directly in static HTML.
Model-based reinforcement learning is an approach where an agent learns a model of how the environment works (a world model) and uses it to simulate future outcomes. Rather than learning exclusively from real interactions, the agent can 'imagine' trajectories within its learned model, dramatically improving sample efficiency.
Editorial body section preserved directly in static HTML.
Model-based RL is dramatically more sample efficient than model-free approaches. In domains where real-world interaction is expensive or dangerous (robotics, autonomous driving, healthcare), the ability to learn from imagination rather than trial-and-error is transformative. It also enables long-horizon planning and safe exploration.
Editorial body section preserved directly in static HTML.
The field spans several paradigms: latent dynamics models (RSSM, Dreamer family) that learn in compressed representation space, learned simulators (UniSim, Cosmos) that generate realistic environments, abstract planning models (MuZero, Predictron) that optimize for decision-making, and hybrid methods (TD-MPC2) that combine model-based and model-free strengths.
Editorial body section preserved directly in static HTML.
DreamerV3 represents the current pinnacle for general model-based RL, mastering diverse domains with a single set of hyperparameters. TD-MPC2 extends this to multi-task settings with 104 tasks. Foundation world models like Cosmos and Genie 2 push toward general-purpose world simulation at scale. IRIS bridges language modeling and world modeling through autoregressive approaches.
Editorial body section preserved directly in static HTML.
The roots of model-based RL trace to Sutton's Dyna architecture (1991), which first proposed learning and planning with a model. Modern world models emerged with Ha & Schmidhuber (2018), followed by PlaNet (2019) introducing the RSSM, DreamerV2 (2021) achieving human-level Atari, and DreamerV3 (2023) achieving domain-general mastery.
Editorial body section preserved directly in static HTML.
Imagination: generating synthetic trajectories within a learned model. Latent dynamics: predicting state transitions in compressed space. Planning: evaluating imagined futures to select actions. Sample efficiency: learning from fewer real interactions by supplementing with imagined data.
| Model | Lab | Category | Index v1.1 |
|---|---|---|---|
| DreamerV3 | Google DeepMind | Model-Based RL | 88/100 |
| PlaNet | Model-Based RL | 57/100 | |
| MuZero | Google DeepMind | Model-Based RL | 78/100 |
| TD-MPC2 | MIT / Meta | Model-Based RL | 80/100 |
| Predictron | Google DeepMind | Model-Based RL | 43/100 |
| World Models (Ha & Schmidhuber) | Google Brain / IDSIA | Model-Based RL | 48/100 |
| IRIS | Microsoft Research | Model-Based RL | 65/100 |
| Imagination-Augmented Agents (I2A) | Google DeepMind | Model-Based RL | 45/100 |
| Value Prediction Network (VPN) | University of Michigan / Google Brain | Model-Based RL | 43/100 |
FAQ answers rendered directly into static HTML for extractable responses.
Model-based RL is generally more sample efficient but can be slower at inference due to planning. The best approach depends on domain constraints: model-based shines where data is expensive, while model-free can be simpler when data is abundant.
Imagination-based learning uses a world model to simulate future trajectories, allowing the agent to learn from these imagined experiences rather than requiring costly real-world interaction.
The Dreamer family (V1-V3) and MuZero are among the most impactful. DreamerV3 achieves state-of-the-art across diverse domains, while MuZero demonstrated superhuman game play without knowing game rules.
Short extractable summary preserved directly in static HTML.
Editorial provenance and refresh policy preserved directly in static HTML.
Published by world-models.io editorial board.
Lead editor Bernard Grenat.
This research page curates topic explanations, linked models, and citations grounded in primary research sources.
Each editorial page is assembled from primary sources, normalized into extractable summaries, checked for factual drift, and reviewed before publication or major refreshes. Last reviewed: 2026-06-21.
Pages are refreshed when a new paper, benchmark, release, architecture update, or stronger primary source materially changes the answer a reader or AI system should retrieve.
Each page links back to relevant primary sources and keeps a stable canonical URL so readers can verify claims, trace context, and reference the most up-to-date version. See the editorial policy.
Primary research citations embedded in static HTML.