Skip to content
BENIBIX Learn

Sim-to-Real Transfer

AI🦾 Robotics & VisionRobot Skill Learning

Also known as Sim2Real, Sim-to-Real, Simulation to Reality

A robot learns a skill in a fake world first. Then it uses the skill for real.

Think of it like

It is like practicing a dance in your head before the show. Then you do it for real on stage.

Example

Jack's team trains a robot hand to turn a cube in a simulation. It practices the equivalent of years in a few days, then does the same trick on a real hand on the first try.

Why it matters

Real practice is slow, costly and can break the robot. Simulation lets it fail millions of times for free.

Definition

Sim-to-real transfer means training a robot in a computer world first. Then the skill is moved to a real robot. In the computer, it can practice millions of times, fast, safely and for free. The catch is the reality gap: the real world has friction, light and wear the computer gets a bit wrong. A common fix is to change colors, weights and light at random while it practices. Then the real world looks like one more change.
Millions of tries in a computer world that keeps changing, then the same skill on a real arm.

Common mistake

Trusting a skill that only works in the computer. Test on the real robot early, since the gap often shows up there.

Key takeaways

  1. 1
    Practice in a computer. Fast and safe.
  2. 2
    The reality gap. Small differences.
  3. 3
    Randomize. To close the gap.