Skip to content
BetaThis project is currently in its beta version.Updates might take a while to roll out.There's still plenty of work left to do.Thanks for your patience while we keep building.
BetaThis project is currently in its beta version.Updates might take a while to roll out.There's still plenty of work left to do.Thanks for your patience while we keep building.
mHumanTrackAImHumanTrackAI

Introduction

mHumanTrackAI is a Windows desktop app that performs real-time, markerless full-body and hand tracking from an ordinary webcam, and translates your movement into real computer input — cursor, clicks, keyboard, or a virtual Xbox 360 gamepad.

It runs entirely on your own machine. There are no cloud calls, no telemetry, and no uploads. The title bar says it plainly: Local Only · No Cloud.

Privacy by architecture

The Python tracking engine and the C#/WPF app run as two separate processes that communicate only over loopback (127.0.0.1). No external network is required to track your movement.

What it is

mHumanTrackAI watches a standard USB or built-in webcam and runs AI pose estimation to detect, every frame:

  • 33 body landmarks (a full-body pose skeleton)
  • 21 hand landmarks per hand — all five fingers
  • optional face landmarks

It recognises gestures from those landmarks and turns them into OS-level input. A single camera gives reliable 2D tracking; adding a second camera (for example front + back) enables multi-view fusion that handles occlusion. Joints that no camera can see are gracefully extrapolated by a One-Euro + Kalman + IK predictor and drawn ghosted/dashed in the UI.

Who it is for

  • People who cannot use a mouse or keyboard with precision — Accessibility Mode maps gestures to cursor movement, clicks, scroll and keys, with dwell-click and adjustable sensitivity.
  • People who want to control a PC or game with body movement alone — Gaming Mode emits a virtual Xbox 360 controller.
  • Developers and tinkerers — a clean two-process architecture, a data-driven gesture table, recording/playback, and CSV/BVH export.

How it works

The pipeline has four stages:

StageWhat happens
1 · Webcam captureOpenCV grabs frames (MJPG fast path for 60 fps). Add a second camera for fusion.
2 · AI pose estimationRTMW (or MediaPipe) infers a 133-keypoint whole-body skeleton each frame.
3 · Gesture recognitionThe gesture engine evaluates landmarks against your editable rules table.
4 · Real OS inputWin32 SendInput drives the desktop; a ViGEm virtual controller drives games.

New here? Head to Installation to get set up, then Getting Started for your first launch and calibration.