Skip to content
BENIBIX Learn

Hand Tracking

Design🧊 3D & ImmersiveSpatial Design

Also known as Hand Tracking (XR), Controller-Free Input, Hand Input

A headset that sees your hands. You tap and pinch with no controller.

Think of it like

It is like a mirror that copies your hands. Then you can touch things in a game.

Example

In Emma's design app, a pinch picks a color and a pull stretches a box. Nobody needs a manual to try it.

Why it matters

It removes the controller, so new users start at once. It also changes how big and how near buttons must be.

Definition

Hand tracking lets a headset see your hands and use them as input. Cameras watch your hands and work out where each finger joint is. You can then poke, pinch, grab and swipe with no controller. A pinch often works like a click. It feels natural, but there is nothing to feel when you touch, and hands can be out of the cameras' view.
Downward cameras track the joints of each hand, and a pinch acts as a click.

Common mistake

Making small buttons that need exact taps. Hands shake and there is no touch feedback, so make targets large.

Key takeaways

  1. 1
    Cameras see hands. Every finger joint is tracked.
  2. 2
    Pinch to click. Simple, common gestures.
  3. 3
    Big targets. No touch, so allow for shake.