Glossary
The vocabulary of intelligent robotics, explained without jargon. Throughout the site, terms with a dotted underline link here.
Key concepts
- Benchmark
- A standardized test: the same tasks for everyone, so that models can be compared fairly.
- Bimanual
- Using two arms (or two hands) at the same time.
- Blind A/B comparison
- An evaluator watches two robots perform the same task without knowing which is which, then picks the better one.
- Compounded reliability
- If each step succeeds 90% of the time, a 30-step mission succeeds only 4 times out of 100: small errors multiply.
- DROID
- A large dataset of robot videos recorded in dozens of labs, and the standard platform (Franka arm + cameras) that goes with it.
- Elo rating
- A head-to-head score, as in chess: you gain points by beating others. A 100-point gap ≈ 64% chance of winning.
- It shows who is better than whom, not a success percentage.
- Fine-tuning
- Retraining a model on the data of a specific test before taking it.
- Like studying past exam papers: the score goes up, but it says less about versatility.
- Foundation model
- A very large model trained on a huge amount of varied data, then adapted to many tasks.
- Franka
- A German robotic arm widely used in research labs.
- Humanoid
- A human-shaped robot (two arms, often two legs), designed to operate in human environments.
- Long-horizon task
- A mission that chains many steps: every small error accumulates, which is what makes autonomy so difficult.
- Margin of uncertainty (±)
- The precision of the measurement: 1,600 ± 30 means “probably between 1,570 and 1,630.” Two scores whose margins overlap are tied.
- Open-source
- The model is published: anyone can download it, test it and improve it.
- Parameters (e.g. 7B)
- The “size” of the brain. 7B = 7 billion internal settings. Bigger is not always better.
- Policy
- The program that chooses the robot's action at every moment. On this site, “model” and “policy” mean the same thing.
- Published record
- The best score published by any model on a test, across all methods.
- Real-world evaluation
- The ultimate test: real robots, real objects, real rooms. Slower and more expensive, but impossible to game.
- Shared checkpoint
- The same model, without any adjustment, takes every test. It is the sign of a true generalist.
- Sim-to-real
- The move from virtual to real: does what works in simulation work on a real machine?
- Simulation
- A virtual world (like a realistic video game) where tests can be run thousands of times without breaking real hardware.
- A good score in simulation does not guarantee the same result on a real robot.
- Success rate
- Out of 100 attempts, how many times the robot completed the task.
- 90% sounds excellent, but a household robot that breaks one plate in 10 is still unusable.
- Teleoperation
- A human controls the robot remotely (virtual reality headset, controller). Useful for collecting data, misleading when a demo hides it.
- Questions to ask about any video: autonomous or piloted? Real time or sped up? How many failed attempts?
- Third-party measurement
- A score measured by a team other than the model's creators, often to compare it with their own method.
- VLA (Vision-Language-Action)
- A robot “brain” that looks (camera), reads an instruction and directly decides which movements to make.
- It is the equivalent of a ChatGPT whose answer is not text but movements.
- Zero-shot
- Completing a task never seen during training, on the first try.
The tests (benchmarks)
Grouped by capability. Plain-language name, then official name.
Basic skills — Can it carry out a simple instruction?
- Instruction followingLIBERO · simulation
- A virtual robot arm receives a written instruction (“put the bowl in the drawer”) and must carry it out. 40 basic tasks.
- In plain terms: A driving test in a parking lot: essential, but far from the open road.
- Precision skillsRLBench · simulation
- Fine 3D manipulation tasks: opening a jar, stacking cups, screwing in a light bulb.
- Grasping varied objectsManiSkill2 · simulation
- Grasping, stacking and moving objects with very different shapes.
Robustness — Does it hold up when conditions change?
- Robustness to the unexpectedLIBERO-Plus · simulation
- The LIBERO tasks, but with changes to the camera, lighting, scenery or instruction wording, or with added noise.
- In plain terms: Driving at night, in the rain, with a GPS that speaks another language.
- Traps and perturbationsLIBERO-Pro · simulation
- Objects moved or replaced, or instructions modified: checks that the robot truly understands instead of repeating by rote.
Realism — Would its results hold on a real robot?
- Small arm: realismSimplerEnv · WidowX VM · simulation
- Simulation calibrated to faithfully reproduce real experiments on a small WidowX arm (putting a carrot on a plate…).
- In plain terms: A flight simulator validated against real flights.
- Google robot: realismSimplerEnv · Google Robot VM · simulation
- Simulation calibrated on Google's robot: picking up a can, opening a drawer, moving two objects closer together.
- Google robot: varied scenerySimplerEnv · Google Robot VA · simulation
- Same tasks, with changing scenery, lighting and textures.
Two arms — Can it coordinate two hands?
- Two arms, clean sceneRoboTwin 2.0 · Easy · simulation
- Tasks that require coordinating two arms (passing an object from hand to hand, opening a lid while holding the jar).
- Two arms, cluttered sceneRoboTwin 2.0 · Hard · simulation
- The same two-arm tasks with clutter and randomized lighting and textures.
Home — Can it manage in an unfamiliar home?
- Virtual kitchen (mobile arm)RoboCasa · Panda · simulation
- An arm on a mobile base performs 24 kitchen tasks in hundreds of different kitchens.
- In plain terms: Helping out in a stranger's kitchen.
- Virtual kitchen (humanoid)RoboCasa · GR1 (humanoïde) · simulation
- A humanoid robot (Fourier GR1) picks up and puts away objects on a countertop.
- Common sense and general knowledgeVLABench · simulation
- Tasks that require common sense or knowledge (“put the fruit rich in vitamin C in the basket”).
Memory — Does it remember what it has seen or done?
- Working memoryRoboMME · simulation
- Counting, remembering where an object was hidden, imitating a sequence seen earlier.
- In plain terms: The shell game.
- Short-term memoryMIKASA-Robo · simulation
- Remembering a color or position that then disappears from view, and acting accordingly.
- Long-term memoryLIBERO-Mem · simulation
- Chaining sub-steps while remembering what has already been done.
Long missions — Can it carry out a long mission on its own?
- Long, unpredictable missionsRoboCerebra · simulation
- Long household missions with obstacles, perturbations and missing objects: the robot must plan and replan.
- In plain terms: Cooking a whole meal while someone tidies the kitchen behind you.