1. Introduction: Kinetic Games Without Wearable Hardware
Historically, digital athletic measurement and motion-based combat games have required dedicated hardware: Nintendo Wii MotionPlus controllers, Sony PlayStation Move wands, or expensive IMU (Inertial Measurement Unit) straps. While effective, hardware dependencies impose prohibitive financial barriers and calibration friction.
Shadow Boxer transforms any commodity laptop or smartphone webcam into a full-body athletic motion capture arena. Using real-time upper-body computer vision compiled to WebAssembly (WASM), the engine derives punch acceleration, velocity vectors, and virtual hitbox collisions directly from raw pixel streams with sub-15ms perceptual latency.
Strike velocity derivation operates on MediaPipe's 3D poseWorldLandmarks. Unlike screen-space coordinates that suffer from camera distance foreshortening, world landmarks estimate coordinates in real-world metric meters with origin (0, 0, 0) centered at the pelvis/hips, ensuring punch speed measurements (m/s) remain invariant regardless of how far the boxer stands from the webcam.
2. Instantaneous Velocity & Acceleration Derivation
Let P_wrist(t) = [x(t), y(t), z(t)] represent the metric 3D position of the boxer's wrist from poseWorldLandmarks in physical meters at timestamp t.
Instantaneous velocity v(t) is derived via finite differences across consecutive timestamps:
Δx = x(t) - x(t - Δt) Δy = y(t) - y(t - Δt) Δz = z(t) - z(t - Δt) Distance_meters = sqrt(Δx² + Δy² + Δz²) Velocity_m_s = Distance_meters / Δt
A punch event is classified as valid when three simultaneous thresholds are satisfied:
- Forward Z-Velocity: v_z < -1.8 m/s (rapid extension toward camera).
- Elbow Extension Angle: Angle formed by Shoulder-Elbow-Wrist exceeds 145 degrees.
- Torso Counter-Rotation: Shoulder yaw velocity opposes the punching arm direction.
3. Hitbox Collision & Screen Shake Shaders
Virtual opponents are represented as dynamic hierarchical bounding volumes (Oriented Bounding Boxes - OBB). When a strike vector intersects the opponent's guard or target regions, the engine dispatches a kinetic impact:
| Strike Type | Velocity Threshold | Kinetic Impact Score | Visual / Audio Feedback |
|---|---|---|---|
| JAB | > 2.5 m/s | 250 Pts | Light chromatic aberration, snap audio pulse |
| CROSS | > 3.8 m/s | 500 Pts | Radial screen blur, deep bass thump |
| HOOK | > 4.5 m/s (lateral vector) | 750 Pts | Directional screen tilt, particle burst |
| UPPERCUT | > 4.0 m/s (vertical vector) | 1,000 Pts | Full screen vibration, time-dilation slowmo |
4. Open-Source NPM Package Documentation: @animatrous/shadow-boxer
Developers building fitness apps, motion games, or martial arts training software can install the core kinematics engine:
npm install @animatrous/shadow-boxer
Quickstart Integration
import { BiomechanicsEngine, useShadowBoxer } from "@animatrous/shadow-boxer";
// 1. Process pose frames with both screen landmarks and metric worldLandmarks
const { processPose, state } = useShadowBoxer({
speedThreshold: 1.5, // m/s
filterCutoff: 1.5, // Hz
});
// 2. In your MediaPipe Pose Landmarker render loop:
const results = poseLandmarker.detectForVideo(videoEl, performance.now());
if (results.landmarks?.length > 0) {
// Screen landmarks for 2D UI; worldLandmarks for camera-invariant m/s physics
processPose(results.landmarks[0], Date.now(), results.worldLandmarks[0]);
}
// 3. Direct velocity computation utility
const speedMs = BiomechanicsEngine.computeVelocity(
prevWristWorld,
currWristWorld,
dtSeconds,
true // isWorldCoordinates = true (real-world metric meters)
);5. Conclusion
Shadow Boxer proves that browser-native computer vision can deliver console-grade kinetic combat mechanics without peripheral hardware. Future work includes integrating ML-based form grading to provide real-time coaching corrections for professional boxing trainees.
