Why Now Daily.

Published

Physical AI: Models That Perceive and Act in the World

Physical AI is an industry term for AI systems that perceive physical environments and influence them through robots, vehicles, cameras or other machines. It connects learned models with sensors, planning, control and actuators.

Timeline

  1. 2024-06-01: NIST published a research report on measurement challenges for manufacturing robotics, including embodied AI and safety.
  2. 2026-07-30: Google DeepMind announced Gemini Robotics 2 models for vision, language, reasoning and robot control.
  3. 2026-10-11: Current NIST, NVIDIA and Google DeepMind material was reviewed for this guide.

Physical AI is a broad industry term for artificial-intelligence systems that perceive, reason about and act in the physical world. Examples include robots, autonomous vehicles, machine-vision systems and instrumented spaces. The phrase overlaps with embodied AI, autonomous systems and intelligent robotics, and it is not one universally standardized product category. Its distinguishing feature is a loop between computation and the environment: observations enter through sensors, while decisions can change real objects or movement through actuators and controls. [1][3]

A physical-AI system is more than a language model attached to a robot. Cameras, depth sensors, force sensors or other instruments estimate what is happening; perception software represents objects and conditions; a planner or learned policy selects an action; and controllers translate that decision into motor commands. Feedback then reports the result so the system can adjust. Conventional robotics components, rules and safety controllers may operate beside learned models, so not every part of the stack is generative AI. [1][3]

Vision-language-action models illustrate the current direction. Google DeepMind describes Gemini Robotics 2 as converting vision and language input into motor control and its embodied-reasoning model as planning multistep tasks before coordinating execution. These are the developer’s reported capabilities for particular models and evaluations. They do not establish that a robot can safely perform every task in an uncontrolled home, factory or street, and availability may be limited to previews or selected partners. [2]

Simulation is important because real-world robot data are expensive, slow and sometimes hazardous to collect. Physics engines and digital twins can generate sensor observations, vary lighting and object placement, and let a policy practise many trials without damaging equipment. NVIDIA describes reinforcement learning, synthetic data and software-in-the-loop testing as parts of this workflow. Simulation remains an approximation: friction, wear, calibration, people and unusual environments can differ from the virtual model, creating a sim-to-real gap. [1][4]

The gap is why deployment still requires testing on physical hardware. Engineers can randomize simulated conditions, calibrate sensors, compare predicted and observed behavior, test failures and progressively expand an operating envelope. A result in one laboratory setup does not prove reliability on another robot body or environment. Even a shared model checkpoint can behave differently when cameras, grippers, loads, control timing or mechanical tolerances change. Performance claims should state the hardware, tasks, success measure and test conditions. [2][3][4]

Safety stakes are higher when software can move machinery. NIST’s manufacturing-robotics report notes that embodied AI needs extensive sensor data and control of physical entities, while unreliable behavior can cause severe physical harm. Risk controls can include force and speed limits, guarded zones, emergency stops, redundant sensing, collision checks, human supervision and validation against relevant machinery or vehicle standards. An AI safety benchmark by itself does not replace system-level engineering and workplace obligations. [3]

The label also covers systems with different degrees of autonomy. A fixed camera may provide perception without moving; a warehouse robot may navigate within a mapped facility; a manipulator may handle a narrow set of objects; and a vehicle may automate only under stated conditions. “Reasons” or “understands” in product descriptions can summarize functional behavior, but those words should not be treated as proof of human-like comprehension. The practical question is which inputs, decisions and actions were actually demonstrated. [1][2][3]

To evaluate a physical-AI claim, identify the machine and task, the sensors and actuators, where inference runs, the role of simulation, the tested environments, human oversight and the safe response to uncertainty. Ask whether results come from a video, controlled benchmark, pilot or production deployment. Physical AI is useful shorthand for the convergence of learned models and autonomous machines, but the phrase does not erase longstanding robotics requirements: precise control, measurable reliability, maintainability and protection for people near the system. [1][2][3][4]

Sources

  1. NVIDIA — What is Physical AI?
  2. Google DeepMind — Gemini Robotics 2 brings whole body intelligence to robots
  3. NIST — Research Opportunities for Advancing Measurement Science for Manufacturing Robotics
  4. NVIDIA Learning — Digital Twin Robotics: Software-in-the-Loop Testing

Related stories