Physical AI: Foundations, Architectures and the Simulation-to-Reality Gap

1. Introduction

Artificial intelligence has, until recently, been most visible in systems whose inputs and outputs are digital: text, images, code and structured data. Physical AI refers to systems that close the loop through the material world. They sense their surroundings, decide what to do, and act through motors, grippers, wheels or valves, and the consequences of their actions return as new sensor readings. The phrase came into wide use in 2024 and 2025; one widely cited formulation describes it as AI equipped with sensors and actuators and argues that such systems must be trained digitally before they are deployed physically [1]. The underlying idea is older. Brooks argued in 1991 that intelligent behavior is best studied in systems that are situated in, and act directly upon, the real world rather than on symbolic abstractions of it [2].

What has changed is the technical route. For decades, robots were programmed: engineers wrote explicit models of the robot and its task and tuned controllers by hand. Learning-based methods were studied throughout, but surveys of the field emphasized how hard they were to apply on real hardware, where samples are expensive, state and action spaces are continuous and high-dimensional, and simulation models are imperfect [3]. Three developments since about 2017 have moved learning from the laboratory toward practice: reinforcement learning in massively parallel simulation, imitation learning from teleoperated demonstrations with expressive generative policies, and robot foundation models that inherit knowledge from vision-language models trained on web-scale data [4].

This paper reviews the foundations of physical AI for an engineering readership. Section 2 defines the term and contrasts it with digital AI. Section 3 describes the perception, planning and control stack and the timescales at which its layers operate. Section 4 covers the two main learning paradigms and the compounding-error problem. Section 5 reviews vision-language-action models and the data on which they are trained. Section 6 discusses simulators and learned world models. Section 7 treats the simulation-to-reality gap, which is the central obstacle to training digitally and deploying physically, and Section 8 summarizes evaluation practice and open problems. Applications in process plants and factories, and questions of safety and economics, are treated in companion papers in this volume.

2. Definitions and Scope

2.1 What physical AI is

In this paper, physical AI denotes a learned decision-making system that (i) receives observations from physical sensors, (ii) produces actions that are executed by physical actuators, and (iii) operates in closed loop, so that each action influences later observations. Robots are the most familiar embodiment, including manipulator arms, mobile robots, legged machines, drones and humanoids, but the definition also covers autonomous vehicles and learned controllers for industrial equipment. The definition excludes systems that only perceive, such as a camera that classifies defects for a human to act upon, and systems that only recommend. Those systems are valuable, but their errors are filtered by a person before anything moves.

2.2 How physical AI differs from digital AI

The closed physical loop changes the engineering problem in four ways. First, errors have physical consequences: a wrong action can damage equipment or injure a person, and it cannot be retracted. Second, data are scarce. Language models learn from text that already exists on the internet, whereas robot actions must be recorded on robots; no comparable corpus of motor experience exists [4]. Third, the system must meet real-time deadlines, because the world does not pause while a model computes. Fourth, evaluation is slow, since every trial requires a robot, a scene and a reset. Table 1 summarizes the contrast. These differences explain why progress in physical AI has lagged progress in language and vision, and why techniques that reduce the need for real-world data dominate the field.

3. The Perception, Planning and Control Stack

3.1 Modular and end-to-end designs

A classical robot is organized as a pipeline. Perception converts sensor data into a state estimate, such as object poses or a map; planning computes a trajectory that achieves the task; and control tracks that trajectory with feedback at the joints. Each module is engineered and verified separately, which makes failures traceable, but the interfaces between modules discard information, and each module depends on models that must be built by hand. End-to-end learning replaces part or all of the pipeline with a neural network policy trained to map observations to actions. Levine and co-authors showed that a single convolutional network could be trained to map raw camera images directly to motor torques, and that training perception and control jointly gave better performance than training them separately [5]. Current systems lie between the two extremes: a learned policy produces target poses or joint positions, and conventional controllers execute them.

3.2 The policy as a conditional distribution

A modern robot policy can be written as a conditional distribution over a short sequence of future actions, given recent observations and an instruction:

where o denotes observations (camera images and joint readings), ℓ is a language instruction or goal, θ are the network parameters, k is the length of the observation history and H is the action horizon. Predicting a chunk of H actions at once, rather than one action per step, was introduced to reduce the effective number of decisions in a task and to cope with pauses and inconsistencies in human demonstrations [6]. Treating the policy as a distribution rather than a single-valued function matters because many tasks have several valid solutions, such as passing an obstacle on the left or on the right; averaging between them gives an invalid action.

3.3 Timescales

The layers of the stack operate at very different rates, and the rate at which a large model can run limits what it can control directly. The largest RT-2 model, with 55 billion parameters, ran at 1 to 3 Hz and had to be served from a multi-accelerator cloud service; a 5-billion-parameter variant reached about 5 Hz [7]. OpenVLA, with 7 billion parameters, runs at about 6 Hz on a single high-end graphics processor, and its authors note that this is too slow for bimanual systems controlled at 50 Hz [8]. Two designs address the mismatch. The first keeps a large vision-language backbone but replaces token-by-token decoding with a continuous action generator; π₀ uses flow matching to produce action chunks at up to 50 Hz [9]. The second separates a slow reasoning module from a fast action module. GR00T N1 runs its vision-language module at 10 Hz and a diffusion-transformer action module at 120 Hz, an arrangement its authors call System 2 and System 1 [10]. Below the learned policy, joint-level servo loops run faster again and remain conventional feedback controllers. Figure 1 shows the resulting stack: each layer runs faster than the one above it, and observations are fed back to every learned layer.

4. Learning Paradigms

4.1 Reinforcement learning

Reinforcement learning (RL) searches for a policy that maximizes the expected discounted sum of rewards collected through interaction:

where s is the state, r the reward, γ ∈ (0, 1] the discount factor and T the horizon. RL needs no demonstrations and can discover behaviors that exceed human skill, but it needs very large numbers of trials and a reward function that expresses the task without loopholes. Kober and co-authors identified the cost of real-world samples, the dimensionality of continuous states and actions, model error and reward specification as the main difficulties in robotics [3]. In practice, RL for robots is now run almost entirely in simulation. Its clearest successes are in legged locomotion and in-hand manipulation, where rewards are easy to state and simulation is tractable (Sections 6 and 7).

4.2 Imitation learning and compounding errors

Imitation learning trains the policy by supervised learning on demonstrations, usually collected by a person teleoperating the robot. It avoids reward design and exploration, and it is the basis of most manipulation systems today. Its weakness is distribution shift. A policy trained only on expert trajectories has seen only states that the expert visits; once it makes a small mistake, it enters a state it was not trained on, where further mistakes are more likely. Ross and co-authors formalized the effect: if a behavior-cloned policy errs with probability ε per step on the expert's state distribution, the expected number of mistakes over a horizon of T steps can grow as

The quadratic term applies to plain behavior cloning (BC); the linear rate is achieved when the learner's own states are relabeled by the expert and added to the training set, as in the DAgger algorithm [11]. The practical lesson is that long tasks are much harder than short ones for a cloned policy, and that data showing recovery from errors are worth more than further perfect demonstrations. Figure 2 plots the two rates for ε = 0.01: over a 50-step task the bound for behavior cloning is 25 mistakes, against 0.5 for the linear rate.

4.3 Generative action models

Two techniques reduced the compounding-error problem without interactive relabeling. Action chunking, Eq. (1), shortens the effective horizon. With it, the ACT method learned six fine bimanual tasks, such as opening a translucent condiment cup and slotting a battery, at 80 to 90% success from about 10 minutes of demonstrations each, on low-cost hardware [6]. Diffusion Policy represents the policy as a conditional denoising diffusion process over action sequences, which captures multi-modal action distributions; across 12 tasks from four benchmarks it improved on prior methods by an average of 46.9% [12]. Both ideas, chunked prediction and generative modeling of actions, were carried into the foundation models described next.

5. Robot Foundation Models

5.1 From task-specific policies to generalist models

The policies in Section 4 are trained for one task on one robot. Foundation models aim for a single model that performs many tasks, follows language instructions and adapts to a new robot with little data. RT-1 established that a transformer policy improves with data scale and diversity: it was trained on about 130,000 demonstrations covering more than 700 instructions, collected with 13 robots over 17 months, and performed 97% of more than 200 seen instructions successfully [13]. RT-2 took the next step by fine-tuning a vision-language model on robot data together with web data. Each action dimension is discretized into 256 bins and written as text tokens, so that the model emits actions in the same way it emits words. Its authors named this class vision-language-action (VLA) models and reported, over about 6,000 evaluation trials, better generalization to new objects and to instructions absent from the robot data [7].

A different use of language models keeps them out of the control loop. In SayCan, a language model proposes the next step of a long task in words, and learned value functions for a library of pretrained skills score which proposals are feasible in the current scene, so that the plan is grounded in what the robot can do [14]. This planner-plus-skills pattern remains common in industry because each skill can be validated separately.

5.2 Data

Because no web-scale corpus of robot actions exists, the field pooled data. The Open X-Embodiment collaboration assembled datasets from 22 robot types contributed by 21 institutions, covering 527 skills, and showed that a model trained on the pooled data improved the performance of several robots relative to models trained on each robot's data alone [15]. DROID added scene diversity: 76,000 trajectories, or 350 hours, collected in 564 scenes by 50 operators over 12 months [16]. These numbers remain small by the standards of language modeling, and collecting them required years of coordinated human effort, which is why simulation and video are attractive additional sources. GR00T N1 is trained on what its authors call a data pyramid, with web data and human video at the base, synthetic data in the middle and real-robot data at the top [10].

5.3 Current models

Table 2 lists representative models. Octo is an open transformer policy trained on 800,000 trajectories from Open X-Embodiment that can be fine-tuned to new sensors and action spaces within a few hours on consumer graphics hardware [17]. OpenVLA, trained on 970,000 demonstrations, outperformed the closed RT-2-X model by 16.5 percentage points in absolute success rate across 29 tasks with one-seventh of the parameters [8]. π₀ was pre-trained on more than 10,000 hours of data from seven robot configurations and was demonstrated on long, dexterous tasks such as folding laundry and assembling boxes [9]. These results show real progress in breadth. They do not yet show the reliability that industrial use requires: the OpenVLA authors state that success rates on their tested tasks are typically below 90% [8].

6. Simulation and World Models

6.1 Physics simulators

A physics simulator integrates the equations of motion of articulated bodies with contact and returns synthetic sensor readings. The decisive change for learning was to run both the simulation and the policy network on a graphics processor. Isaac Gym reported training times two to three orders of magnitude shorter than pipelines that simulate on a central processor [18], and with thousands of simulated robots in parallel on one processor, a walking policy for flat terrain was trained in under four minutes and transferred to a real quadruped [19]. Simulation at this speed converts the sample-inefficiency of RL from a barrier into a computing cost.

6.2 Learned world models

A world model is a learned predictor of how observations evolve under actions. Ha and Schmidhuber showed that an agent can learn a compressed model of its environment without supervision, train a compact policy entirely inside that model, and transfer the policy back to the real environment [20]. DreamerV3 developed the idea into a general algorithm that learns a model and improves its behavior by imagining future scenarios; with a single configuration it outperformed specialized methods on more than 150 tasks [21]. World foundation models extend the concept to video generators trained on large video corpora, which can be fine-tuned to a given robot and used to generate or augment training data; the Cosmos platform released such models with open weights for this purpose [1]. Learned world models complement physics simulators: they capture appearance and hard-to-model phenomena from data, but they offer no assurance that their predictions obey physical law, so policies trained inside them inherit the model's errors.

7. The Simulation-to-Reality Gap

7.1 Definition and sources

A policy that performs well in simulation often performs worse on hardware. The difference can be written as

where J is the performance measure of Eq. (2) or a task success rate, evaluated in simulation and on the real system. The gap arises because the simulator is a model, and optimization exploits model error: a learning algorithm will find and use any inaccuracy that raises simulated reward [22]. The main sources are (i) dynamics, including friction, contact stiffness, link masses and, above all, actuator behavior; (ii) timing, including sensing and actuation delays that are absent in an idealized simulation; (iii) perception, where rendered images differ from camera images in texture, lighting and noise; and (iv) unmodeled phenomena such as cable drag, backlash, deformable objects and wear. Actuators are a leading case. Hwangbo and co-authors replaced the analytic actuator model with a neural network trained on data from the real actuators, which allowed policies trained in simulation to run on the ANYmal robot [23].

7.2 Domain randomization

If the simulator cannot be made exact, the policy can be made insensitive to its errors. Domain randomization trains over a distribution of simulators instead of one:

where ξ is a vector of simulator parameters, such as friction coefficients, masses, delays, textures and lighting, drawn from a distribution p(ξ) at the start of each episode. Tobin and co-authors showed that an object detector trained only on simulated images with random, non-realistic textures localized real objects to within 1.5 cm, on the argument that with enough variability the real world appears to the model as one more variation [24]. Peng and co-authors applied the same idea to dynamics and obtained a pushing policy that kept its performance on a real arm without any training on hardware [25]. The method has a cost. If p(ξ) is too narrow, the real system lies outside it and transfer fails; if it is too wide, the policy becomes conservative or the task becomes unlearnable [22]. Automatic domain randomization widens the distribution progressively as the policy improves, and was used to train a five-fingered hand to manipulate a Rubik's Cube [26]. Figure 3 illustrates the narrow, suitable and wide cases.

7.3 Adaptation, privileged learning and real data

A robust policy that ignores the parameters ξ must compromise across all of them. Better performance is available if the policy infers the conditions it is operating in. In privileged or teacher–student learning, a teacher policy is trained in simulation with access to ground-truth quantities such as terrain shape and friction, and a student policy is then trained to imitate the teacher using only the sensors available on the robot; a quadruped trained this way crossed mud, snow, rubble and running water that it had never encountered in training [27]. Rapid Motor Adaptation adds a module that estimates a compact description of the environment from the recent history of states and actions, and adapts within fractions of a second to changes in terrain and payload, again without fine-tuning on hardware [28]. For manipulation, where contact and appearance are harder to simulate, the prevailing practice is to combine simulation with real demonstrations, or to pre-train broadly and fine-tune on a small amount of data from the target robot [17, 8]. Table 3 compares the methods.

Whether simulation alone can scale to general manipulation remains debated. A published discussion among researchers in the field recorded both positions: that sufficiently randomized simulation can cover reality, and that simulators will not capture contact and deformable materials well enough, so that real data remain necessary [29]. The evidence so far is that simulation-only training succeeds where the physics is dominated by rigid bodies and the reward is simple, and that tasks with rich contact and varied objects still need real data.

8. Evaluation and Open Problems

8.1 Evaluation

Results in this field are reported as success rates over physical trials, and three cautions apply when reading them. First, trial counts are small, so uncertainty is large: a success rate measured over 20 trials has a 95% confidence interval of about ±22 percentage points near 50% success. Second, results are specific to the robot, scene and objects tested, and the control rates in Table 2 were measured on different hardware. Third, a success rate that is impressive for research may be far from adequate in service. Section 4.2 showed that errors compound over the length of a task, and a companion paper in this volume treats the reliability levels that industrial deployment requires.

8.2 Open problems

Four problems limit current systems. The first is reliability over long horizons, for the reason just given. The second is distribution shift after deployment. Learned components can fail without warning on inputs unlike their training data, and Sinha and co-authors argue that this must be handled at the level of the whole system, with monitoring to detect such inputs and fallback behaviors when they are detected [30]. The third is data: teleoperated demonstrations cost operator time on real robots, and it is not yet established how far video and simulation can substitute for them [4]. The fourth is computation at the robot. Large models run at a few hertz on a graphics processor, as Section 3.3 showed, so that fast, contact-rich skills need either smaller action modules or hierarchical designs.

• Use simulation-trained RL where dynamics are rigid-body and the objective is simple, as in locomotion.
• Use imitation learning with chunked, generative action models for manipulation, and collect demonstrations that include recovery from errors.
• Treat a foundation model as a starting point to be fine-tuned and validated on the target robot and task, not as a finished controller.
• Measure the reality gap, Eq. (4), on the real system before relying on any simulated result.

9. Conclusion

Physical AI applies learned decision-making to systems that sense and act in the material world. Its methods are now well defined: reinforcement learning in fast parallel simulation, imitation learning with generative action models, and vision-language-action models that bring web-scale knowledge to robot control. Its central difficulty is also well defined. Robot experience is expensive, so systems are trained largely on substitutes for it, in simulators, on video and on pooled data from other robots, and each substitute introduces a gap that must be measured and closed on the target system. The techniques reviewed here, including system identification, domain randomization, privileged learning, online adaptation and fine-tuning on real data, have closed that gap for locomotion and for a growing set of manipulation tasks. The remaining distance to dependable general-purpose machines is mainly one of reliability over long tasks and under conditions not seen in training. For engineers assessing the technology, the appropriate questions are therefore how a system was trained, on what evidence its performance on the real task rests, and what it does when it meets a situation outside its experience.