Architecture

mHumanTrackAI is built as two processes, designed to be independently testable:

• Python engine (engine/) — owns cameras, ML inference, occlusion prediction, multi-camera fusion, gesture evaluation, and recording. Knows nothing about Win32/WPF.
• C# / WPF app (app/) — UI, visualization, OS-level input drivers, and process lifecycle. Knows nothing about ML.

Two-process design

The C# app owns the lifecycle of the Python engine. EngineHost spawns the engine as a hidden child process inside a Win32 Job Object (KILL_ON_JOB_CLOSE), which guarantees the Python child dies with the app even on a hard crash. If an engine is already listening on the control port (for example, a developer started one manually), EngineHost attaches to it instead of spawning a new one.

The engine runs a main publish loop on the main thread and a control loop on a daemon thread, so commands never block the tracking pipeline.

IPC channels

The two processes communicate over ZeroMQ (NetMQ on C#, pyzmq on Python) across TCP loopback. The wire format is newline-less JSON, schema version 1, defined in shared/schema/ipc.schema.json.

The control channel handles commands such as ping, getConfig, setConfig, setMode, setCamera, captureTPose, startRecording, exportRecording, playRecording, listCameras, and shutdown.

Key design decisions

• The engine binds all three sockets; the app connects. This makes the engine the single source of truth for port state.
• Preview is rate-decoupled (small high-water mark; stale frames dropped) so JPEG encoding never adds latency to the landmark stream.
• Job Object lifecycle ensures the Python child cannot outlive the app.
• Config hot-reload — setConfig / setCamera / loadProfile set a dirty flag; the next loop iteration reopens the camera/source with new settings.