Rooted in 3D vision and brain-inspired science, we turn real-world motion, interaction and manipulation into data and skills machines can learn and execute — research and product run on one pipeline.
Replacing studio motion capture with a video-in, data-out pipeline: from monocular to multi-shot, from body pose to finger contact and facial expression — whole-body 3D motion with semantic labels, the academic foundation of the UNIData million-hour asset.
From understanding objects to grasping them: category-level detection, pose estimation and 3D reconstruction from stereo vision in a single stage, with a frozen VLM perception layer focused purely on manipulation — so robots succeed on objects and scenes they have never seen.
Human demonstrations as the data source: hand-object interaction videos are physically refined and retargeted across embodiments into executable trajectories for robot arms, dexterous hands and humanoids — turning one person’s motion into a skill for every body.
15.6M frame-level whole-body pose annotations across 81.1K motion sequences, plus a fully automatic video-to-motion-and-text pipeline that moves capture out of the lab and into the wild.
Scaled further: 19.5M pose annotations and 120.5K sequences, with the first inclusion of 80.8K RGB videos and 45.3K audio clips — enabling audio-driven motion generation.
Recovering world-coordinate continuous motion from multi-shot in-the-wild videos: shot-transition detection, cross-shot alignment and an anti-foot-sliding integrator — so edited footage still yields long-sequence motion data.
One-stage category-level detection, pose estimation and 3D reconstruction from stereo images, including transparent and specular materials; state-of-the-art on the public TOD dataset with simulation-to-real transfer to robot grasping.
No end-to-end fine-tuning: freeze the VLM’s general perception, extract task-critical features via a Mixture of Layers (MoL), and let a Meta-Skill Module carry trajectory learning — zero-shot generalization with manipulation accuracy.
HOI videos → high-precision reconstruction → RL-based physical refinement → cross-embodiment retargeting → Isaac Sim domain randomization; the first quantitative evidence that HOI can supervise robotic learning, on par with teleoperation.
Whole-body motion datasets and an automatic annotation pipeline
Motion-X · Motion-X++ · HumanMMUNIData
Million-hour real-world operation dataset and skill assets
View →Category-level object understanding + frozen-VLM manipulation
CODERS · M²-VLAE²-CAP-V / CogniBrain OS
Stereo embodied capture device and cognition engine, down to grasping and manipulation
View →Human demonstration → physical refinement → cross-embodiment retargeting
RoboWheelWIOS · SKILL
Compiling motion data into deterministic, cross-embodiment standardized SKILLs
View →Joint research, data co-building and engineering deployment — open to researchers and industry partners.