Skip to content

Session replay for 3D scenes

Replay re-drives a captured session in your own scene: camera pose is applied directly, and pointer / mesh / custom / input events are surfaced to callbacks so you can draw a cursor, highlight a mesh, or annotate a timeline.

Terminal window
npm install @uptimizr/replay
import { fetchSessionEvents, ReplayPlayer } from "@uptimizr/replay";
import { createBabylonReplayDriver } from "@uptimizr/replay/babylon";
const events = await fetchSessionEvents({ endpoint, apiKey, sessionId });
const driver = createBabylonReplayDriver({
scene,
onPointer: (screen, hitPoint, hitMesh, type) => { /* draw cursor / flash */ },
onMeshInteraction: (mesh, kind) => { /* highlight */ },
onCustom: (name, props) => { /* timeline marker */ },
onInputAction: (input, ts) => { /* input.action / input.source / input.code / input.button */ },
onLifecycle: (event, ts) => { /* viewport_resize / focus_change / context_lost / … */ },
onError: (error, ts) => { /* mark a runtime_error (only if captureErrors was on) */ },
});
const player = new ReplayPlayer(events, driver, { speed: 1 });
player.play();
// player.pause(); player.seek(ms); player.stop();

ReplayPlayer is deterministic — seeking backward resets the driver and replays from the start. player.durationMs gives the total length, and player.isPlaying the current state. Besides speed, the third argument takes onProgress(elapsedMs, durationMs) and onComplete().

All four drivers take the same callbacks (onPointer, onMeshInteraction, onCustom, onInputAction, onLifecycle, onError) plus nodes and onNodeTransform.

fetchSessionEvents buffers the whole session. fetchSessionEventsStream takes the same options and yields events as they arrive, over NDJSON when the collector supports it and from the buffered array when it does not. A malformed line is skipped and reported through the optional onMalformedLine(count).

import { fetchSessionEventsStream } from "@uptimizr/replay";
for await (const event of fetchSessionEventsStream({ endpoint, apiKey, sessionId })) {
// one validated event at a time, in server order
}

To replay scene actors (node_transform), pass a nodes map from each recorded nodeId to the engine node to drive, and/or an onNodeTransform callback to observe every sample:

const driver = createBabylonReplayDriver({
scene,
nodes: {
"npc-guard": () => scene.getMeshByName("Guard_root"), // resolver, name, or ref
},
onNodeTransform: (nodeId, transform, ts) => {
/* transform.boneId? / transform.childPath? / transform.position / transform.rotation /
transform.scale? */
},
});

The Babylon, three, and PlayCanvas drivers re-apply Tier-1 root transforms and Tier-2 skeleton bones (matching each bone by name on the node’s skeleton). The babylon-lite driver drives the Tier-1 root and forwards bone samples to the callback only. Unknown nodeId / boneId are skipped without error (forward/back-compatible).

To re-pose a whole rigid sub-hierarchy from one captured node, reconstructRigidSubtree combines the sampled root pose with the scene proxy’s scan-time path / world fields. It is pure and engine-agnostic:

import { reconstructRigidSubtree } from "@uptimizr/replay";
const meshes = reconstructRigidSubtree({
proxy, // the registered SceneProxy
rootPath: "Forklift/Mast", // the captured node's path
rootWorld: sampledRootTransform, // its live { position, rotation, scale }
// rootWorldAtScan is optional; falls back to the proxy mesh at rootPath
});
// → [{ name, path, world: { position, rotation, scale } }, …] for strict descendants

It returns only strict descendants of rootPath, and [] when no scan-time root transform is available. Non-rigid deformation isn’t reconstructed, and a captured childPath sample wins over a reconstructed pose.

Replay normally re-drives into the scene you already have. When you only have the captured stream and no scene to host it — a hosted drag-and-drop viewer, say — load an arbitrary asset as a backdrop first, then replay over it. The Babylon helper accepts a URL or a dropped File:

import { loadSceneBackdrop } from "@uptimizr/replay/babylon";
const backdrop = await loadSceneBackdrop(scene, urlOrFile); // ".glb" / ".gltf"
console.log(`${backdrop.meshes.length} meshes added`);
// swap one dropped model for the next:
backdrop.dispose();

It returns a handle ({ rootNodes, meshes, container, dispose() }) whose dispose() removes everything it added and releases the GPU resources. The default loader lazily imports Babylon’s glTF SceneLoader, so the lean replay path never pulls it in unless a backdrop is requested; pass options.load to supply your own loader or options.pluginExtension to force a parser. Actor / subtree nodes from the loaded model re-drive exactly like any other scene node (see above).

The global build exposes window.UptimizrReplay, with a one-call replayInScene convenience that fetches and plays a session:

const r = document.createElement("script");
r.src = "https://cdn.jsdelivr.net/npm/@uptimizr/replay/dist/uptimizr-replay.global.js";
r.onload = () => {
UptimizrReplay.replayInScene({
scene,
endpoint: "https://collect.example.com",
apiKey: "your-project-api-key",
sessionId: "<copy from the dashboard Sessions table>",
backdropUrl: "https://example.com/room.glb", // optional — load a model first
debug: true, // log fetch/play progress to the console
});
};
document.head.appendChild(r);

replayInScene starts playback immediately — it does not wait for the scene to be “ready”, so call it once scene exists and has an activeCamera. It always logs a concise summary and warns about the common “nothing happens” causes: an empty session, a session with no camera_sample events (camera won’t move), or a scene with no active camera. (pnpm playground prints this snippet pre-filled and serves the bundle at /uptimizr-replay.global.js.) It resolves to a handle with play, pause, stop, seek, durationMs and isPlaying.

Beyond the options shown, replayInScene takes camera (defaults to scene.activeCamera), speed, onPointer, onMeshInteraction, onCustom, onLifecycle, onError, onProgress, onComplete, loadBackdrop and backdropPluginExtension.

backdropUrl loads a .glb/.gltf into the scene before replay. To keep the global bundle from shipping a second copy of Babylon’s SceneLoader, it reuses the host page’s loader: expose Babylon as window.BABYLON (with LoadAssetContainerAsync and a glTF loader registered) or pass an explicit loadBackdrop callback. When no loader is found it warns and replays without a backdrop.

  • 403 from the events endpoint — ENABLE_RAW_SESSION_RETENTION is off, or the key lacks query:raw. The response body says which.
  • 200 with 0 events — usually an API-key / project mismatch: reads are scoped to the key’s project, so a valid session id looked up with another project’s key returns nothing. Copy the session id and the API key from the same dashboard project.
  • Camera doesn’t move — the session has no camera_sample events.

The dashboard’s Session replay birdview is the same engine, no code required. Open a session from the Sessions table to scrub its camera path, interaction rays, and moving scene actors, with a colour-coded event timeline you can click to seek. Each tracked actor is drawn as one labelled marker per root — a subtree actor (an articulated character) shows a single dot for “where it is” rather than a marker per joint; its full per-joint motion still reconstructs when you replay it in your own scene.

When the session is live right now (it has produced an event within the active-now window), the same window switches to live follow: new camera moves and interactions stream in and the timeline grows in real time. A ● LIVE control pins the playhead to the live edge; scrub back (or press Play) to review what already happened, then press ● LIVE to jump back to the edge. Live follow uses the same gate as replay — with ENABLE_RAW_SESSION_RETENTION off, the window shows nothing to follow.

By default the birdview draws a wireframe box per registered proxy mesh so the recorded motion reads against your scene’s rough shape. To see the session re-driven over the real geometry, use Load model (.glb) under the timeline and pick a .glb/.gltf. The model loads as a backdrop and replaces the wireframe boxes; the camera path, interaction rays, and actor markers keep re-driving over it. Replace model swaps in a different file and Remove model restores the boxes. The model is held only in your browser for the current view — nothing is uploaded.