EGO1GS and EgoHand: Egocentric Capture with Real-Time, On-Device Hand Detection

EGO1GS is the most advanced headset in our Egocentric Headset family, and it advances on EGO1 in two ways. First, the capture hardware takes a full step up: global-shutter sensors keep fast motion geometrically straight, a new stereo audio stream joins the recording, and the IMU doubles to 400 Hz — all synced on one hardware clock. Second, the headset now understands what it's recording: EgoHand, a purpose-built hand detection model, runs on the device itself and finds the wearer's hands in every frame while the session is still being captured.

The hardware: a step up across the board

The biggest step is the one in the name. EGO1GS's two cameras are global shutter: every row of the frame is exposed at the same instant. A rolling-shutter sensor reads the frame out line by line, so a fast head turn or a quick reach smears geometry across the readout — skew that a SLAM or pose pipeline has to model out before the data is usable. With global shutter there is nothing to model out. Fast head turns and quick hand motion stay straight, which is exactly the regime egocentric manipulation footage lives in.

Around that sensor change, nearly every capture spec moved up:

• Wider field of view. Each camera now covers 157° × 84°, up from EGO1's 120° × 72°. The wider cone keeps both hands in frame through the full reach envelope — including the edge-of-reach positions where EGO1's detector coverage was hardest won.
• A faster IMU. The 9-DoF IMU now samples at 400 Hz, double EGO1's 200 Hz, and is clocked against the cameras in hardware.
• Stereo audio. EGO1GS adds a dual-microphone stereo audio stream, recorded on-device on the same hardware clock as everything else — and a built-in speaker announces record start/stop, low battery, and storage faults, so the wearer gets feedback without reaching for a phone.

EgoHand: egocentric hand detection at capture time

Physical AI learns manipulation from human hands. That makes hand data the heart of every egocentric recording, and it puts one simple demand on every session: the wearer's hands should be in frame, doing the task, for as much of the recording as possible. The demand sounds trivial, but at fleet scale it is the difference between a dataset and a pile of video — and a pipeline that checks in post-processing discovers the misses only after the footage has been uploaded, converted, and inspected. A real-time hand detection model on the device moves that check to capture time. Because the headset knows, frame by frame, whether both hands are in view, it can enforce that demand on the spot: the moment both hands leave the frame, EGO1GS warns the operator by voice through its on-board speaker, so a framing problem is fixed in seconds rather than discovered in review. Sessions segment themselves into the intervals where hands are actually working, and idle footage is dropped before it ever leaves the SD card.

State-of-the-art hand detectors answer the wrong question. MediaPipe Hands^[1]1.Zhang et al., "MediaPipe Hands: On-device Real-time Hand Tracking," CVPR Workshop on Computer Vision for AR/VR, 2020. is the standard for real-time on-device hand detection, and on its own terms it works: run it over egocentric footage and it finds hands wherever they appear. That is exactly the problem. The question capture-time quality needs answered is not "is this a hand?" but "is this the wearer's hand?" — and to measure the gap between the two, we ran MediaPipe over 3,128 held-out frames from our own recordings and labeled every box it proposed against human ground truth:

One box in three is not the wearer's hand, and nothing in the output tells them apart: the closest ownership proxy MediaPipe exposes, its handedness score, ranks own-hand boxes above the rest with an AUC of 0.70 — barely better than chance. To be clear, this is not MediaPipe failing at its job; ownership is not a task it attempts. But it means capture-time quality cannot be built on an off-the-shelf detector. And whatever model does the job must share the headset's NPU with dual-1080p30 recording, which rules out anything cloud-scale from the start.

EgoHand is our answer: a compact detector with an architecture inspired by the YOLO series of models,^[2]2.Redmon et al., "You Only Look Once: Unified, Real-Time Object Detection," CVPR, 2016. trained on our own egocentric footage where hand ownership is the label. It carries no separate classification head — the detection confidence is the ownership answer, and a box the model declines to emit is the model saying "not yours." On the held-out test set it separates the wearer's hands from everything else at an AUC of 0.9925 [0.9899, 0.9948]; at its deployed operating point it recognizes 99.6% of the wearer's hands at 97.8% precision. And going small costs nothing where it counts: a 7×-larger cloud-side model evaluated on the same benchmark lands inside the same confidence interval — on ownership, running on-device gives up nothing.

Get one

EGO1GS ships as a kit: the headset, a pre-formatted 128 GB MicroSD card, and a USB cable for power and direct transfer — bring any USB power bank. See the full details on the product page. Sample recordings are available on request. If you're collecting manipulation data at scale and want to talk, book a demo or reach us at hello@gilabs.xyz.