From Research to Reality: The Hard Parts of Building Physical AI at Scale

Session Overview

The last few years of physical AI have produced breathtaking demos — robots folding laundry, making coffee, playing sports. Yet very few of these systems make it out of the lab. This talk is about the gap between a compelling demo and a robot you can actually deploy, and the four capabilities that gap is really made of: reliability, agility, on-device compute, and human-robot interaction.

Drawing on work from Gemini Robotics, Open X-Embodiment, Table Tennis Robots, and on building a robotics company from the ground up, Pannag argues that closing this gap means rethinking defaults the field has grown comfortable with — how we source data, how we teach skills, how we architect models for the edge, and how systems adapt once deployed. The aim is not a finished recipe but a sharper map of where the hard problems, and the opportunities, actually sit.

Key Takeaways

• The real reality gap is demo-to-deployment. The four things a lab lets you ignore — reliability, agility, edge compute, and HRI — are exactly what production demands.
• Data is the bottleneck, and tele-op is the wrong lever. Human and internet video are the underused axes for scale and diversity.
• Imitation gets you started; simulation-based RL is how you scale skills and earn robustness and agility.
• Edge compute and quick learnability are architecture problems, not afterthoughts. Shrinking a cloud model is not the same as designing for the robot.
• Continuous learning is the through-line. A robot that can't learn and adapt on the go — to new objects, scenes, and people — isn't deployable, however good the demo.