Markerless full-body + hand tracking
AI pose estimation detects 33 body landmarks plus 21 landmarks per hand (all five fingers) every frame, with no suit, gloves, or markers. Just a webcam.
Local Only·No Cloudv1.3.4
Your body is the controller.
Markerless full-body webcam tracking that turns your movement into computer input, entirely on-device, no cloud.
Windows 10/11 · One installer, auto-detects GPU/CPU · Bundles Python 3.11 + .NET 8, no separate installs.
Capabilities
Everything tracked. Nothing uploaded.
A complete on-device tracking stack, from raw webcam frames to real OS-level input, with privacy guarantees built directly into the architecture.
On-device privacy
Everything happens locally: no cloud calls, no telemetry, no uploads. The app’s two internal processes only ever talk to each other over loopback (127.0.0.1).
Two modes, one camera
Accessibility Mode turns hand gestures into real cursor movement, clicks and keyboard input. Gaming Mode creates a virtual Xbox 360 controller, so your body controls the game directly.
Multi-camera fusion
Add a second camera (front and back), and the app automatically picks whichever one has a clear view of each joint, giving you full-body occlusion coverage.
Occlusion prediction
When a camera loses sight of a joint, a One-Euro + constant-velocity Kalman + IK pipeline predicts where it should be, shown ghosted/dashed on screen, so you always know what’s real and what’s a prediction.
Gesture-to-action mapping
A data-driven gesture → action table you can edit live: pinch, pointer, fist, pose, swipe and more, each mapped to cursor, mouse, key, scroll or controller actions.
Recording & playback
Record the live landmark stream to disk, replay it through the full engine pipeline with no camera, and export to CSV or BVH for animation tools.
Tuned for real hardware
Runs RTMW whole-body pose estimation on CPU or CUDA, with a TensorRT FP16 fast path reaching ~90 fps on an RTX 2060, plus MediaPipe as the zero-setup CPU fallback.
Two modes
Control your desktop, or control your game
The same skeleton drives two different output routes. Switch profiles per app, or let mHumanTrackAI auto-switch when a game takes focus.
Accessibility Mode
Cursor · clicks · scroll · keyboard via Win32 SendInput
Control the real mouse and keyboard with hand gestures, including dwell-click (hover to click) and an adjustable cursor-sensitivity multiplier.
| Gesture | Detection | Action |
|---|---|---|
| Point + move | Index tip position | Move cursor (relative, trackpad-style) |
| Pinch tap (thumb + index) | Brief contact | Left click |
| Pinch hold (thumb + index, >300 ms) | Sustained | Left-hold → drag |
| Thumb + middle | Brief contact | Right click |
| Thumb + ring | Brief contact | Middle click |
| Thumb + pinky | Brief contact | Double-click |
| Index + middle vertical move | Both extended | Scroll |
| Fist | All curled | Freeze cursor |
| Both open palms held 2 s | Safety pose | Toggle tracking (panic) |
Pinch = clickPoint = moveFist = freezeBoth palms = panic
Gaming Mode
Virtual Xbox 360 controller via ViGEm
Map body movement to a virtual gamepad: pinch to fire, raise a hand to jump, lean to strafe, pan your head to look. Requires the ViGEmBus driver.
| Gesture | Input | Controller output |
|---|---|---|
| Right pinch (thumb + index) | Hand | RT: primary fire |
| Left pinch (thumb + index) | Hand | LT: secondary / aim |
| Right fist | Hand | RB: grab / hold |
| Left fist | Hand | LB: grab / hold |
| Either hand raised above head | Body | A: jump (edge pulse) |
| Torso compression (crouch) | Body | B: crouch (held) |
| Body lean left / right | Body | Left stick X: strafe |
| Head pan left / right | Body | Right stick X: camera look |
Pinch = triggerLean = stickCrouch = BHand up = jump
By the numbers
Real-time tracking, measured
- 0
- Whole-body keypoints
- 0
- Body landmarks
- 0
- Hand landmarks
- ~0 fps
- TensorRT FP16
- 0
- Cloud calls
- 0
- Processes
RTMW COCO-WholeBody
Pose skeleton
Per hand, all 5 fingers
RTX 2060, verified
No telemetry, no uploads
Loopback 127.0.0.1 only
Privacy by architecture
Built to never phone home
Privacy here isn't a policy you have to trust. It's how the software is wired. The title bar says it plainly: Local Only · No Cloud.
No cloud calls
Capture, inference, gestures and input injection all run on your machine, so there’s no server to send anything to.
No telemetry, no uploads
Nothing about your camera, your movement, or your usage is collected or transmitted. Ever.
Loopback only
The Python engine and the C# app talk to each other over 127.0.0.1 via ZeroMQ: no external network needed for tracking.
Skeleton-only privacy view
Prefer not to see the camera feed? A skeleton-only view mode hides the video entirely while you work.
No outbound connection
Your machine
Python engine
cameras · ML · fusion
127.0.0.1ZeroMQ · loopback
C# / WPF app
UI · input drivers
Inside the app
A dense, dark productivity UI
Built in the spirit of Discord and VS Code: layered navy surfaces, orange brand accents, blue for interaction.
Local Only·No Cloud
Turn movement into input.
Free for Windows 10/11. One installer detects your hardware and bundles its own Python 3.11 and .NET 8 runtime, so there’s nothing else to install.