Meta
Machines that share a room
Illustrative scene and state labels · not product UI or recorded robot behavior.
The robotics work at FAIR kept coming back to one situation: a person and a machine in the same room, each trying to work out what the other is about to do.
At FAIR I set up design for the embodied AI work. I defined the scope, built the design and research teams, and drove the work alongside the robotics researchers.
SIRo
SIRo was a robot built on Boston Dynamics’ Spot, a four-legged robot, meant to help with ordinary tasks around people. Nobody next to it could see what it saw, which the team called the cave problem. The room and the tasks were the research team’s. We made a mixed-reality view down the robot’s line of sight, on monitors around the lab, so anyone could watch it think and act, and helped stage the demo.
A room that knew what you wanted
A separate project, with no robot in it, turned the question around: a set of instrumented rooms we designed and built. Walk in, and the room works out what you’re there to do, then does it. Most people saw a demo. Underneath was a product question, intent in and action out, that the technology couldn’t answer yet.

Bodies to learn with
Spot was a hard platform to learn on. A senior designer and I proposed building our own bodies with Reality Labs’ industrial design team, grabbers, arms and other shapes, so ideas could be tested quickly. It wasn’t approved, and the work stayed on Spot.

Hands and touch
Hands were the last piece: scenarios with the robotics team for grasping and touch, shown as demos that made sense without anyone explaining them.

Most of this can’t be shown, and some of it never left the lab. What stayed with me is the question I’d ask of any machine that works near people. Can the person next to it tell what it’s about to do?
