Static research summary generated from local editorial content.
| Attribute | Value |
|---|---|
| Topic | World Model Evaluation |
| Summary | How to evaluate world models across rollout quality, benchmark performance, planning utility, and downstream transfer instead of relying on visual plausibility alone. |
| Related Models | 6 |
| Citations | 3 |
Editorial body section preserved directly in static HTML.
World models can look impressive while failing at the tasks they are meant to support. A rollout may appear realistic to a human observer yet still contain causal or task-level errors that break planning, value estimation, or control. That makes evaluation one of the hardest open problems in the field.
Editorial body section preserved directly in static HTML.
Researchers often distinguish intrinsic metrics such as reconstruction loss, prediction error, or horizon stability from extrinsic metrics such as task return, benchmark score, or real-world transfer. Strong world model evaluation requires both because low modeling loss does not guarantee useful downstream behavior.
Editorial body section preserved directly in static HTML.
Atari 100K, DMControl, Crafter, Minecraft, robot manipulation suites, and real-world driving datasets all probe different aspects of world model quality. Foundation models increasingly need evaluation across realism, controllability, robustness, and transfer rather than only narrow benchmark wins.
Editorial body section preserved directly in static HTML.
The field still suffers from cherry-picked rollouts, weak baselines, single-seed reporting, and overreliance on aesthetic quality. For large-scale video-native world models, dataset leakage and vague claims about physics understanding are also major risks.
Editorial body section preserved directly in static HTML.
Better evaluation combines multi-seed statistics, long-horizon stress tests, action-conditioned probing, human inspection of failure modes, and real downstream tasks. The goal is to measure whether a model is genuinely useful for prediction, planning, or transfer, not only whether it looks convincing.
| Model | Lab | Category | Index v1.1 |
|---|---|---|---|
| DreamerV3 | Google DeepMind | Model-Based RL | 88/100 |
| TD-MPC2 | MIT / Meta | Model-Based RL | 80/100 |
| DIAMOND | Microsoft Research / University of Geneva | Model-Based RL | 64/100 |
| IRIS | Microsoft Research | Model-Based RL | 65/100 |
| NVIDIA Cosmos | NVIDIA | Foundation World Model | 87/100 |
| V-JEPA 2 | Meta | Self-Supervised World Model | 87/100 |
FAQ answers rendered directly into static HTML for extractable responses.
There usually is no single best metric. The right evaluation depends on whether the model is used for planning, simulation, robotics transfer, or open-ended generation.
Because realism can hide causal errors. A world model that looks plausible but fails under action changes, long horizons, or task constraints may still be a poor planning tool.
Short extractable summary preserved directly in static HTML.
Editorial provenance and refresh policy preserved directly in static HTML.
Published by world-models.io editorial board.
Lead editor Bernard Grenat.
This research page curates topic explanations, linked models, and citations grounded in primary research sources.
Each editorial page is assembled from primary sources, normalized into extractable summaries, checked for factual drift, and reviewed before publication or major refreshes. Last reviewed: 2026-06-21.
Pages are refreshed when a new paper, benchmark, release, architecture update, or stronger primary source materially changes the answer a reader or AI system should retrieve.
Each page links back to relevant primary sources and keeps a stable canonical URL so readers can verify claims, trace context, and reference the most up-to-date version. See the editorial policy.
Primary research citations embedded in static HTML.