How Physical Intelligence Makes Robots Reliable Enough to Trust
Physical Intelligence's Chelsea Finn explains the RL recipe, human interventions, and value functions behind reliable autonomous robots.
Drafted with Claude from source material, checked by automated verification, and reviewed before release. How we make these.
Why robot reliability is a harder problem than chatbot accuracy
Physical Intelligence, the robotics company founded by Chelsea Finn, builds general-purpose models meant to let any robot perform any task in the real world. The core obstacle isn’t teaching a robot to attempt a task once. It’s getting that robot to succeed at the task over and over, without a person watching, at rates high enough that people can actually trust it. Finn’s team targets over 90% reliability on tasks like operating an espresso machine, a bar that most robot learning demos never have to clear because most demos only need to work for the camera once.
What makes physical AI different from other deployed machine learning
Most machine learning systems that have reached large-scale production, product recommendations, ad ranking, and now chat assistants and coding agents, share a common trait: a human sits between the model’s output and the real-world consequence. If a recommendation engine or a coding assistant gets something wrong, a person notices and corrects course. That safety margin is why imperfect models have still been enormously useful commercially.
Seven tools to build an app. Or just Remy.
Editor, preview, AI agents, deploy — all in one tab. Nothing to install.
Robots don’t get that margin. A robot arm pouring milk into a cup or inserting a portafilter is making direct physical decisions with no human in the loop to catch a mistake before it happens. That structural difference means physical AI systems need dramatically lower error rates than the AI products that came before them if they’re going to run autonomously for meaningful stretches of time. Finn points to Waymo crossing a quarter of a million weekly autonomous rides as evidence that trustworthy, autonomous, physical-world machine learning is achievable, but it took years of focused engineering to get there.
How does reinforcement learning improve robot reliability?
The standard machine learning loop, collect data, train, evaluate, rarely produces a reliable policy on the first pass. The usual fix is manual iteration: gather more data, clean up labels, target edge cases, rebalance the dataset. That works, but it’s slow and it depends on a person’s patience and judgment to keep finding the next weak spot.
Physical Intelligence’s alternative is to let the AI system find its own weak spots. That’s reinforcement learning in spirit: the robot attempts a task, learns from failures, and improves through repeated experience rather than through a human manually re-curating a dataset after every round. The appeal is that an automated search for failure cases can run far more iterations than a person ever could, which is the path toward pushing reliability from “usually works” to 99%-plus.
Why can’t robots just use the RL recipes built for language models?
Algorithms like PPO and GRPO have scaled large language models to complex reasoning by running millions, sometimes tens of millions, of attempts. That’s cheap when an “attempt” just means running a model in a data center. It’s a different story in robotics, where an attempt means physically operating a robot arm in real time. Finn estimates that even a modest one-minute task would need something like 700 robot-days of continuous operation to reach the trajectory counts that language model RL takes for granted. That’s not necessarily impossible, but it’s a very different cost structure, and it demands an algorithm that learns from far fewer physical attempts.
Physical Intelligence’s recipe attacks two specific inefficiencies in standard RL algorithms.
Dead-end trajectories waste real robot time. In language model training, a wasted rollout just burns compute. In robotics, it burns hardware-hours. If a robot grabs two cardboard boxes stuck together instead of one, and nobody intervenes, it will keep trying to fold the pair together indefinitely, generating data that teaches the model nothing useful. Physical Intelligence’s fix is to have a human step in mid-episode, teleoperate the robot back onto a productive path (in the box example, separating the two boxes), and then let the robot try to continue autonomously. If it still can’t recover, the human intervenes again. The goal is either a quick recovery demonstration or an early termination, so the robot isn’t burning real-world time on trajectories that lead nowhere.
Estimating value per-prompt is inefficient. Algorithms like PPO and GRPO typically sample many attempts, often 10 to 50, for a single prompt in order to judge which responses are good and which are bad. Physical Intelligence instead trains a single general-purpose value function across many different tasks and scenarios at once, amortizing that cost instead of re-estimating it separately for every task. This value function learns general patterns of progress and regression: undoing a fold while folding a shirt gets flagged as negative progress, while moving toward a goal state gets flagged as positive, and the same model can make analogous judgments about a completely different task, like retrieving an item from a fridge. In effect, it learns something like a general sense of “time to success” that transfers across tasks, which sharply reduces how many attempts are needed to know whether the robot is getting better.
What does the resulting training recipe look like end to end?
Put together, the recipe runs like this: start with a foundation model trained on a broad, diverse dataset of robot experience. Deploy it and collect fresh experience, with a human intervening as needed to prevent the robot from wasting time on dead ends. Use that experience to train a general-purpose value function that estimates progress across many tasks. Then use that value function to guide further improvement of the policy. It’s a loop of deploy, correct, evaluate, refine, but automated enough that it doesn’t depend on a person manually hunting for every weak spot the way traditional dataset iteration does.
How well does this work in practice?
Physical Intelligence tested the approach on a genuinely difficult composite task: making a latte in collaboration with a human, where the robot handles the espresso side (inserting the portafilter, timing the extraction, pouring steamed milk) while a person steams the milk. The hardest part isn’t the espresso mechanics, it’s transferring a full cup of latte to a coaster without spilling, which requires delicate, continuous balance control driven directly by the robot’s camera images.
To test reliability rather than just capability, the team ran the trained policy continuously for 13 hours rather than evaluating a handful of attempts. The policy held up over that stretch, suggesting the combination of intervention-based data collection and general value function training can push a policy from “works in a demo” to “works reliably enough to leave alone for long periods,” which is the actual bar for usefulness outside a lab.
Frequently Asked Questions
What is Physical Intelligence’s main approach to robot reliability?
It combines reinforcement learning with two efficiency fixes tailored to real-world hardware: human interventions that redirect robots away from dead-end trajectories, and a general-purpose value function trained across many tasks that estimates progress without needing dozens of rollouts per scenario.
Why do robots need higher reliability than other AI products?
Because robots act directly on the physical world with no human decision-maker in between. A chatbot or recommendation engine can be wrong and a person just ignores the bad suggestion. A robot pouring liquid or gripping an object has no equivalent safety net, so errors have immediate physical consequences.
Why can’t robotics just reuse RL algorithms like PPO and GRPO as-is?
Those algorithms were built assuming cheap, abundant rollouts, since running a language model in a data center costs compute, not physical time. Robot attempts happen in real time on real hardware, so an algorithm that needs millions of attempts per task is impractical without efficiency changes.
What is a general value function in this context?
It’s a single model trained on experience from many different robot tasks that learns to judge whether an action represents progress toward a goal or a regression away from it. Instead of estimating value separately for each task, it transfers a general notion of “getting closer to success” across tasks like folding laundry or retrieving items.
How reliable are Physical Intelligence’s robots right now?
The company has demonstrated tasks like making a latte with over 90% reliability as a target, and tested a latte-making policy over a continuous 13-hour run to confirm it held up over long stretches, rather than only succeeding in short demo clips.