Headset-Less Browser VR: Multi-Modal FaceMesh & Hand Kinematics for 3D Motion Gaming

1. Introduction: Eliminating the Hardware Barrier in Spatial Gaming

Virtual and augmented reality experiences have traditionally required dedicated head-mounted displays (HMDs) such as the Meta Quest, Apple Vision Pro, or PlayStation VR. While providing high-fidelity tracking, dedicated hardware introduces severe barriers: high monetary cost ($400 to $3,500+), physical fatigue, thermal discomfort, and friction in app onboarding.

Void Runner (void-runner) demonstrates that full 3-axis motion gaming can be delivered inside a standard desktop or mobile web browser using commodity monocular webcams. By combining two distinct neural computer vision pipelines running concurrently via WebAssembly—Google MediaPipe's 468-pointFaceMesh and 21-point Hands—the engine constructs a continuous spatial control surface without requiring the player to hold any controller or strap on any optical headset.

2. Multi-Modal Vision Architecture: FaceMesh + Hand Landmark Synchronization

A central engineering challenge in monocular motion gaming is avoiding state collision between navigation controls and combat actions. In Void Runner, this is resolved by decoupling degrees of freedom across two biological focal points:

3. Head Kinematics: Deadband Normalization & Jump Dynamics

Raw facial landmarks are subject to high-frequency micro-saccades, breathing drift, and distance variations. Void Runner applies an adaptive bounding box normalization heuristic to convert absolute pixel coordinates into calibrated control deltas:

// Extract calibrated centroid from key nasal and ocular landmarks
const headX = (noseTip.x + leftEye.x + rightEye.x) / 3;
const headY = (noseTip.y + forehead.y) / 2;

// Calculate normalized displacement relative to calibrated baseline
const deltaX = headX - calibrationCenter.x;
const deltaY = calibrationCenter.y - headY; // Invert for upward velocity

// Deadband filtering: Suppress micro-jitter around neutral posture
let targetLane = currentLane;
if (Math.abs(deltaX) > DEADBAND_LATERAL) {
  targetLane = deltaX < 0 ? -1 : 1; // Left or Right 3D track lane
} else {
  targetLane = 0; // Center lane
}

// Vertical impulse triggering
if (deltaY > JUMP_THRESHOLD && !isJumping && isGrounded) {
  verticalVelocity = JUMP_FORCE; // e.g. 15.0 px/frame
  isJumping = true;
}

// Integrate gravitational deceleration per tick
verticalVelocity -= GRAVITY; // e.g. 0.8 px/frame²

This prevents false-positive jumps caused by gentle user swaying, while guaranteeing that a deliberate upward head jerk immediately launches the player avatar over obstacle barriers with zero physical button presses.

4. Spatial Laser Sword: Ray-Segment Mid-Air Collision

When the camera detects hand landmarks, Void Runner activates Sword Mode. The vector between the wrist landmark (Joint 0) and the index fingertip (Joint 8) is extruded into a 2D/3D energy beam.

Obstacle destruction uses minimum distance calculations between the dynamic laser segmentS = [(x1, y1), (x2, y2)] and the target obstacle center P = (px, py):

export function pointLineDistance(
  px: number,
  py: number,
  x1: number,
  y1: number,
  x2: number,
  y2: number
): number {
  const dx = x2 - x1;
  const dy = y2 - y1;
  if (dx === 0 && dy === 0) return Math.hypot(px - x1, py - y1);
  
  // Project point onto line segment clamped to [0, 1]
  let t = ((px - x1) * dx + (py - y1) * dy) / (dx * dx + dy * dy);
  t = Math.max(0, Math.min(1, t));
  
  // Return Euclidean distance to closest segment point
  return Math.hypot(px - (x1 + t * dx), py - (y1 + t * dy));
}

If pointLineDistance(px, py, x1, y1, x2, y2) < HITBOX_RADIUS, a slice event triggers: the cyber-drone or obstacle splits into procedural particle debris with reactive screen-shake and audio synthesis.

5. Procedural 3D Perspective Projection Engine

Void Runner utilizes a custom, lightweight 3D perspective projection pipeline optimized for Canvas 2D / WebGL contexts, eliminating the memory footprint of full monolithic 3D game engines while preserving true depth parallax:

export function project(
  x: number, y: number, z: number,
  camX: number, camY: number, camZ: number,
  canvasWidth: number, canvasHeight: number
) {
  const relX = x - camX;
  const relY = y - camY;
  const relZ = z - camZ;
  
  // Near plane clipping
  if (relZ <= 0.1) return null;
  
  const fov = 500;
  const scale = fov / relZ;
  
  return {
    x: canvasWidth / 2 + relX * scale,
    y: canvasHeight / 2 - relY * scale,
    scale,
  };
}

6. Official NPM Package Quickstart & Component API

The entire game is packaged as an open-source React / Next.js component distributed on NPM asvoid-runner (v0.1.1) under the AGPL-3.0 license.

npm install void-runner

Embed the complete headset-less VR runner directly into any React application with a few lines of code:

import { VoidRunner } from "void-runner";
import "void-runner/dist/styles.css";

export default function ArcadePage() {
  return (
    <div style={{ width: "100vw", height: "100vh", background: "#050014" }}>
      <VoidRunner
        onExit={() => {
          console.log("Player navigated back to arcade lobby");
        }}
        onGameOver={(finalScore, coins) => {
          console.log(`Game Over! Score: ${finalScore} | Coins: ${coins}`);
        }}
        onScoreChange={(score) => {
          // Sync with global leaderboards or state
        }}
      />
    </div>
  );
}

Component Props Specification

7. Verification & Open Source Repositories