Custom events & input

The built-in channels cover pointer, camera, mesh, and performance. These APIs let your app contribute the rest of the picture — domain events, non-pointer input, capability fallbacks, and scene changes. All of them are methods on the client returned by trackScene(...).

Custom events

Record your own discrete domain events. They’re always captured at 100% — never rate-limited.

const client = trackScene(scene, { projectId, endpoint });

client.track("add_to_cart", { sku: "ABC-123", price: 49 });

Keep the property map free of PII. Read per-type counts from GET /api/v1/event-counts.

Input actions (keyboard, gamepad, XR)

Mouse and touch are captured automatically as pointer events. Discrete input actions from other devices — keyboard shortcuts, gamepad buttons, XR controller buttons — are recorded as input_action events. Each carries a semantic action label (what the input did) plus the originating source and the raw code/button token.

Emit one explicitly whenever you handle a binding:

// In your own keydown / gamepad handler:
client.trackInput("next-camera", { source: "keyboard", code: "KeyN", pressed: true });
client.trackInput("jump", { source: "gamepad", button: 0 });

source defaults to "keyboard". These events are discrete and always captured at 100%.

keyBindings allowlist

The Babylon, three.js, and PlayCanvas connectors can capture bound keys for you. Pass keyBindings mapping a physical KeyboardEvent.code to an action label — only the listed keys are recorded (privacy-first), arbitrary typing is never captured, and auto-repeat is suppressed:

trackScene(scene, {
  projectId,
  endpoint,
  keyBindings: { KeyW: "move-forward", KeyS: "move-back", Space: "jump" },
});

three.js / PlayCanvas (and react-three-fiber via @uptimizr/r3f) accept the same option. They have no keyboard observable, so they listen on window — handy for pointer-lock / FPS scenes where the canvas rarely holds focus.

Capability changes (fallbacks & recovery)

Rendering capability isn’t constant: some visitors run WebGPU, others fall back to WebGL2; weaker devices auto-downgrade quality/LOD; a lost GPU device may re-initialise at a different capability. These transitions otherwise look like unexplained noise in aggregate metrics.

Engines decide their backend at init and expose no reliable runtime hook, so connectors do not auto-capture this — report it from your app whenever you perform a fallback or recovery:

// after a WebGPU init fails and you fall back:
client.reportCapabilityChange({ kind: "graphics-backend", from: "webgpu", to: "webgl2" });
// or a runtime quality/LOD auto-downgrade:
client.reportCapabilityChange({ kind: "quality", from: "high", to: "low", reason: "low-fps" });
// or a completed XR tracking-degradation episode (ADR 0048):
client.reportCapabilityChange({
  kind: "tracking",
  from: "hand",
  to: "lost",
  reason: "signal-lost",
  source: "hand",
  handedness: "left",
  durationMs: 1200,
});

kind is one of graphics-backend / quality / device-recovery / tracking / feature / other; from / to / reason are optional, low-cardinality, app-defined tokens (never raw device strings or PII). The tracking kind additionally carries the input source / handedness that degraded and an optional durationMs (the completed degraded-episode length — one event per episode, emitted on recovery), which powers the tracking-quality timeline (GET /api/v1/xr/tracking). The Babylon connector reports coarse XR tracking loss/recovery automatically when a hand or controller drops out of the input registry mid-session (toggle via the XR capture tracking option, default on). This pairs with the raw context_lost / context_restored events — it’s the higher-level “what we ran as” signal. Read the rollup from GET /api/v1/capabilities.

Rendering technology (always-on)

Each session’s session_start.graphics block records the rendering API surface (api), the real backend beneath it (backend, e.g. WebGPU → Metal), the API/driver version (apiVersion), and the shadingLanguage — captured once per session as non-PII, low-cardinality metadata (ADR 0021 / ADR 0046). It is always-on: every connector reports it, so no opt-in is needed.

The dashboard’s Rendering technology panel aggregates the mix across the selected window — counts by API, backend, and shading language — backed by GET /api/v1/rendering-technology, the always-on sibling of the opt-in Engine diagnostics panel. Blank fields surface as “unknown”.

Engine diagnostics (opt-in GPU health)

graphics_diagnostic carries engine-authored GPU-health signals — GPU errors/warnings, shader-compile/link failures, richer context-loss reasons, WebGPU uncapturederror, and sampled gl.getError() — in one engine-agnostic shape (severity, category, optional backend, length-capped message/code, and a count rollup-or-marker discriminator).

It is off by default and gated by the captureGraphicsDiagnostics option — like runtime_error, the text can leak application IP, so you opt in and redact via beforeSend. The default emission is a rate-limited per-session rollup so an error storm can’t flood ingestion. context_lost / context_restored are exempt and stay always-on; engine-driven backend fallback stays in capability_change above.

Once captured, these incidents surface in the dashboard’s Engine diagnostics panel — counts by severity, category, and backend — backed by GET /api/v1/graphics-diagnostics, which folds discrete markers and per-session rollups into one honest total. With capture off, the panel shows an explicit opt-in empty state rather than reading as broken.

Wired today in the Babylon (@uptimizr/babylon) and three (@uptimizr/three) connectors: WebGPU device.lost → category: device-lost (info for a requested loss, reason: "destroyed"; fatal otherwise; WebGL is a no-op — its interruption is the always-on context_lost); WebGPU uncapturederror → a rate-limited rollup (category: validation / out-of-memory, count + first message); WebGL/WebGPU context-creation failure → category: context-loss (severity: fatal, backend: unknown when undetermined; fires once at connector init and queues before the first flush); shader compile/link failures → category: shader-compile (error; WebGL info logs on failure, WebGPU shader-module compilation info); and sampled WebGL gl.getError() → category: validation (a low-rate rollup, never per-frame — it forces a sync GPU stall; no-op on WebGPU). Shader source can hide in the error log, so raw source is stripped unless the separate captureShaderSource sub-opt-in is set (off by default — application IP). All length-capped text runs through beforeSend.

Spatial error heatmap (where errors happen)

Both runtime_error and graphics_diagnostic carry an optional position — the best-effort camera position ([x, y, z]) at the moment the error or diagnostic fired. Connectors stamp it automatically: @uptimizr/babylon reads the tracked camera’s globalPosition; connectors that can’t resolve a camera simply omit it, and errors on pages without a 3D connector never carry one. Nothing is required of your app, and the field is additive — older events just omit it.

With positions attached, the dashboard’s Error heatmap (3D) panel voxel-bins them into the same world-space grid as the pointer/gaze heatmaps, so you can see where in the scene things break — errors clustering around specific geometry, a shader-heavy area, or a level region — instead of only when. It’s backed by GET /api/v1/heatmaps/errors, which accepts optional severity/category filters (narrowing to engine diagnostics) and errorKind (narrowing to JS errors). Position reuses the existing promoted column shared with camera_sample, so it inherits the same privacy posture — no new PII surface.

Guardian / boundary-touch heatmap (room-scale VR comfort)

In room-scale WebXR, visitors move inside a physical guardian / play-space boundary. When the headset comes within a short near threshold of that boundary, @uptimizr/babylon’s opt-in babylonBoundaryCollector emits an xr_boundary_proximity event — one per approach — carrying only a coarse voxel-binned position (the HMD position at the closest approach) and durationMs (how long the pose stayed inside the near zone). Count is implied by frequency, not a running counter.

import { trackScene, babylonBoundaryCollector } from "@uptimizr/babylon";

const client = trackScene(scene, { projectId, endpoint });
// After you create the WebXR experience (bounded-floor reference space):
client.use(babylonBoundaryCollector({ experience: xr, nearMeters: 0.5 }));

The recommended nearMeters default is 0.5 m (with ~0.1 m exit hysteresis and ~100 ms sampling) — enough to catch a real reach-for-the-wall moment without firing on normal room-centre movement.

Privacy (ADR 0003 / ADR 0048): the boundary polygon and room geometry are never transmitted. The bounds check runs entirely on-device; only the outcome (position + duration) leaves the headset. position reuses the promoted world-space column, so it inherits the same privacy posture as the pointer/gaze/error heatmaps — no new PII surface.

The dashboard’s Boundary-touch heatmap (3D) panel voxel-bins these positions into the same world-space grid as the pointer/gaze/error heatmaps (backed by GET /api/v1/heatmaps/boundary), and a Guardian boundary contacts panel lists per-session approach counts + near-zone time (backed by GET /api/v1/xr/boundary-contacts) as a comfort signal alongside the VR locomotion dashboard.

Changing scenes / levels (setScene)

A single session can span multiple scenes, areas, or levels — game levels, a viewer swapping models, or a multi-room walkthrough. You do not stop and restart tracking when the visitor moves between them; you keep one client alive and mark the transition:

const client = trackScene(scene, {
  projectId,
  endpoint,
  meta: { sceneId: "level-1" }, // initial scene/area
});

client.setScene("level-2"); // when the next scene loads

See the multi-scene experiences guide for the full patterns (levels, viewers, rooms), per-scene querying, and how replay crosses scene changes.