Sim2Real

The lecture motivates sim-to-real transfer by confronting a fundamental bottleneck in robotics RL: real-world data collection is painfully slow (a single robot arm action takes roughly 2 seconds), while training good policies can require the equivalent of millions of human years of experience — the price paid by AlphaGo and OpenAI Five. Simulators like MuJoCo, NVIDIA Isaac Sim, and RiseSim act as a kind of "time machine," accelerating data collection by 100–1000× relative to wall-clock time. That speed comes with a trade-off: simulators approximate rather than replicate reality, and fast GPU-parallel simulators (e.g., MuJoCo MJX) sacrifice physical accuracy—particularly for friction, contact dynamics, and soft or deformable objects—making the sim-to-real gap a central engineering challenge. The core technique for bridging that gap is domain randomization: instead of trying to perfectly model a single robot, you randomize physics parameters (mass, friction, ground springiness, actuator delays, IMU noise) across a wide distribution at the start of each training episode and hide those parameters from the policy. The goal is to make the distribution of simulated MDPs sufficiently large to include the real world as one of its points, so that the learned policy generalizes rather than overfits to any one simulator setting. Tasks with simple, well-understood rigid-body dynamics — quadruped locomotion, drone flight — transfer reliably under this approach; dexterous manipulation with dynamic or soft objects remains much harder because current simulators cannot accurately model the contact physics involved. In practice, the strongest sim-to-real results pair domain randomization with a small real-world data-collection stage for system identification, fine-tuning the simulator's parameter distribution around the actual hardware before final deployment.