Babylon.js

The Babylon.js connector. @babylonjs/core is a peer dependency — the connector reads your existing scene and never bundles or mutates it.

Install

npm install @uptimizr/babylon @babylonjs/core

Usage

The quickest integration is a single call:

import { trackScene } from "@uptimizr/babylon";

const tracker = trackScene(scene, {
  projectId: "your-project",
  endpoint: "https://collect.example.com",
});

// later, on teardown
tracker.dispose();

trackScene creates the client, registers the Babylon collector, reads device/GPU caps, and starts the session. The return value exposes the session id, custom events, and lifecycle:

tracker.track("add_to_cart", { sku: "ABC-123" });
tracker.setScene("level-2");
tracker.dispose();

Choosing the camera

By default the connector records the engine’s active camera. For multi-camera scenes (picture-in-picture, split-screen, render-target rigs) set camera explicitly — otherwise the “active” camera is ambiguous and the view-direction heatmap can collapse to a single direction:

trackScene(scene, { projectId, endpoint, camera: mainCamera });

Each camera_sample also carries the camera’s projection intrinsics — vertical fov, viewport aspect, and the near-plane distance near — captured automatically when the engine exposes them. They let the click-ray heatmap reconstruct a flat pointer’s ray origin on the near plane instead of collapsing it to the camera point (ADR 0043); they’re optional, so a camera that doesn’t expose them simply falls back to the camera position.

First-person scenes (pointer lock)

When the rendering canvas holds the browser Pointer Lock (engine.enterPointerlock(), first-person/FPS navigation), the OS cursor is hidden and the aim point is the fixed crosshair at the viewport centre. The connector detects pointer lock and reports pointer_move / pointer_down / pointer_up / pointer_click from screen centre (screen = [0.5, 0.5]), re-picking at the render-target centre — the crosshair the visitor actually aims with. The 2D pointer heatmap therefore clusters at the centre for locked scenes; read the cursor-independent gaze / floor-plan heatmaps instead. Cursor (orbit/viewer) scenes are unaffected. See Concepts → pointer lock (ADR 0034).

Capture fidelity

Continuous channels are sampled at conservative defaults (≈1 Hz camera, ≈4 Hz pointer, ≈0.5 Hz perf). Raise or lower them per channel with sampling (rates in Hz, "frame" for every tick, or 0 to disable):

trackScene(scene, {
  projectId,
  endpoint,
  sampling: {
    camera: 10, // 10 Hz camera pose
    pointerMove: 60, // 60 Hz pointer movement
    perf: 0.5, // a perf sample every 2 s
  },
});

Discrete events (clicks, mesh interactions, scene changes, custom events) are always captured at 100% and cannot be rate-limited.

Opt-in channels

Off by default for privacy and cost — enable as needed:

trackScene(scene, {
  projectId,
  endpoint,
  capture: { hoverDwell: true, resourceSample: true, gaze: true },
  meshVisibility: { meshes: ["product-hero"], windowMs: 5000 },
});

• meshVisibility — per-object dwell summaries (mesh_visibility).
• capture.hoverDwell — hover hesitation episodes (hover_dwell).
• capture.resourceSample — GPU/memory footprint (resource_sample).
• capture.gaze — world-space gaze hit points (camera_sample.hitPoint).
• captureErrors — opt-in runtime_error capture (not auto-redacted).

Scene proxy (3D heatmap backdrop)

To draw 3D heatmaps against a recognizable backdrop without shipping your real geometry, register a lightweight proxy (per-mesh bounding boxes):

import { scanSceneProxy } from "@uptimizr/babylon";

const proxy = scanSceneProxy(scene, { sceneId: "lobby" });
// PUT it to /api/v1/scenes/lobby/representation (see the HTTP API docs)

For a custom transport or a beforeSend hook, see sdk-core (advanced).

AR placement (WebXR “view in your room”)

Retail AR placement is app-driven, so babylonArPlacementCollector turns three Observables from your placement UI into exactly one ar_placement event per settle — never per frame. It counts re-placement attempts, times how long placement took, records the final scale, and classifies the surface coarsely from the WebXR hit-test normal (floor / wall / table / ceiling / unknown):

import { UptimizrClient } from "@uptimizr/sdk-core";
import { babylonCollector, babylonArPlacementCollector } from "@uptimizr/babylon";

const client = new UptimizrClient({ projectId, endpoint });
client.use(babylonCollector({ scene }));
client.use(
  babylonArPlacementCollector({
    mesh: "Sofa",
    onPlacementStartObservable, // { mesh } — user entered placement mode
    onPlaceObservable, // { position, normal? } — one fire per (re-)place
    onSettleObservable, // { position, scale?, final? } — the commit
    hitTest: xrHitTest, // optional Babylon WebXRHitTest feature
  }),
);
client.start();

Signals are coarse and on-device only (privacy): no plane polygons, no room dimensions, no PII — only the coarse surface bucket, counts, and durations leave the client. The data powers the dashboard’s AR placement funnel panel and the /api/v1/ar/placement/* endpoints.