Exploring Video Data for Training Agentic World Models

Session Overview

What does it actually take to build systems that can see, understand, and act?

Physical AI had its "Attention Is All You Need" moment with the rise of Vision-Language-Action models. The next bottleneck is data: not just more video, but the ability to find the exact real-world moments that teach models how the world works — gravity, motion, causality, human behavior, and object interactions.

This session explores a new approach: discovering specific scenes from the vastness of the web. We'll show how teams can search for moments like objects falling, people interacting with environments, or actions unfolding over time, then collect and structure only the relevant clips for training and evaluation.