Sim-to-Real: Closing the Reality Gap for Industrial Robots

Simulation offers something the physical world cannot: millions of trials, perfect state observability, no hardware damage, and no wall-clock cost. It also offers a policy that works beautifully in simulation and fails on the robot. Closing that gap is a specific engineering discipline, and it is largely about knowing which discrepancies matter.

The gap is not uniform — it is concentrated in contact and actuation

Rigid-body dynamics are well modelled. Given accurate mass, inertia and geometry, a simulator predicts free-space motion closely enough that the residual error rarely matters. Two things are not well modelled, and essentially all of the reality gap lives in them.

• Contact: friction is approximated by a Coulomb model with a friction cone that solvers linearise for tractability; real contact involves surface deformation, stick-slip transitions, and coefficients that vary with wear, temperature and contamination.
• Actuation: simulators typically apply commanded torque instantly. Real actuators have finite bandwidth, gear backlash, friction, thermal derating, current limits and control-loop dynamics of their own.

The distinction matters because it tells you where to spend effort. A policy that manipulates objects or walks is dominated by both. A policy that plans free-space trajectories transfers almost immediately, because neither term is on its critical path — which is why sim-to-real is discussed constantly in locomotion and manipulation and rarely in motion planning.

Domain randomisation: train on a distribution, not a point

Rather than attempting an exact model of one robot, randomise the parameters across training so the policy must be robust to any instance in the range. The real robot is then, in effect, one more sample from a distribution the policy has already seen.

The width of p(ξ) is the central trade-off and it is genuinely two-sided. Too narrow and the real robot falls outside the training support, so the policy has no reason to handle it. Too wide and the optimal policy becomes conservative — the best strategy that works across an implausibly broad range of robots is slow, stiff and cautious, because aggressive strategies fail on some members of the distribution.

Learn the actuator model rather than assuming one

Analytic actuator models with series elasticity and gearing are hard to identify accurately, and the residual error feeds directly into every contact interaction. A more reliable approach is empirical: drive the real actuator through a rich excitation trajectory, record commanded versus achieved torque along with the state history, and fit a small network that maps recent command and state history to realised torque. That network then replaces the simulator's ideal actuator.

This works because it captures the effects that resist analytic modelling — friction, backlash, control-loop bandwidth, thermal behaviour — without requiring them to be individually identified. It was the decisive component in the first widely reproduced demonstrations of learned locomotion transferring to hardware, and it remains among the highest-return interventions available.

Privileged learning: train with information the robot will never have

Simulation exposes state that no sensor can measure — exact friction coefficients, precise contact forces, true object mass. Discarding it wastes the main advantage of training in simulation; using it directly produces a policy that cannot run on hardware. The teacher-student construction resolves this.

• Train a teacher with full privileged state. It learns quickly, because the hard part of the problem — inferring hidden parameters — has been removed.
• Train a student that observes only real sensors, over a history window, to reproduce the teacher's actions.
• The student is forced to infer the privileged variables implicitly from observable history, which is exactly the estimation problem it will face on the robot.

The result is a policy that adapts online without any explicit system-identification module: it infers that the surface is slippery from how the last few contacts responded, and adjusts, because that inference was the only route to matching the teacher during training.

Measure the gap instead of discovering it

The most common process error is treating hardware deployment as a binary test. It should be a measurement. Replay identical command sequences in simulation and on the robot, and compare trajectories — the divergence tells you which parameters are wrong and by how much.

Feed those measurements back as tightened randomisation ranges centred on measured values, and the loop closes: simulate broadly, deploy, measure the discrepancy, narrow the distribution around reality, retrain. Each pass reduces the conservatism the policy must carry, which is where the remaining performance is.

What it costs and what it buys

Massively parallel GPU simulation now runs thousands of environments concurrently, which turns experiments that once took weeks of wall-clock into hours. That changes the economics: the expensive resource is no longer simulation time but the engineering judgement about what to randomise and how far. The teams that transfer reliably are not the ones with the most compute — they are the ones that measured their actuators, modelled their latency, and set their randomisation ranges from data rather than from intuition.