Editorial definition preserved directly in static HTML.
A component that converts continuous data (images, video frames, states) into discrete tokens that can be processed by transformer-based architectures. In world models like IRIS, visual tokenizers convert image observations into sequences of discrete tokens for autoregressive prediction.
Tokenizer is a glossary concept in the architecture layer of the world models knowledge base.
Static glossary definition snapshot for crawlers and no-JS readers.
| Attribute | Value |
|---|---|
| Term | Tokenizer |
| Category | Architecture |
| Definition | A component that converts continuous data (images, video frames, states) into discrete tokens that can be processed by transformer-based architectures. In world models like IRIS, visual tokenizers convert image observations into sequences of discrete tokens for autoregressive prediction. |
| Related Models | 2 |
| Related Research | 0 |
| Model | Lab | Category |
|---|---|---|
| IRIS | Microsoft Research | Model-Based RL |
| NVIDIA Cosmos | NVIDIA | Foundation World Model |
| Term | Category | Definition |
|---|---|---|
| Autoregressive Model | Architecture | A model that generates outputs one element at a time, where each output depends on previous elements. IRIS treats world modeling as autoregressive token prediction, bridging language modeling and world modeling techniques. |
Short extractable summary preserved directly in static HTML.
Editorial provenance and refresh policy preserved directly in static HTML.
Published by world-models.io editorial board.
Lead editor Bernard Grenat.
This glossary page publishes stable definitions linked to related models, research topics, and primary-source context.
Each editorial page is assembled from primary sources, normalized into extractable summaries, checked for factual drift, and reviewed before publication or major refreshes. Last reviewed: 2026-06-21.
Pages are refreshed when a new paper, benchmark, release, architecture update, or stronger primary source materially changes the answer a reader or AI system should retrieve.
Each page links back to relevant primary sources and keeps a stable canonical URL so readers can verify claims, trace context, and reference the most up-to-date version. See the editorial policy.
Primary sources related to this term, surfaced directly in static HTML.