In-browser assistant (LLM)

The in-browser assistant lets anyone ask natural-language questions of their 3D analytics (“what were the most-clicked meshes this week, and how’s the average FPS?”) without installing an MCP client. The agent loop runs entirely in the browser against the same read-only query API a human dashboard user sees — one project, aggregate-only, no raw events, no PII.

It ships no model and no key. You pick a backend, and the choice persists per-user in localStorage. Everything is user-controlled, so there is no Uptimizr-operated backend and no default egress (ADR 0050 §4/§5).

The two backends are provider adapters for @uptimizr/agent-core, imported from code-split subpaths so consumers who never use the assistant pay nothing for it:

import { createWebLlmProvider } from "@uptimizr/agent-core/providers/webllm";
import { createHostedProvider } from "@uptimizr/agent-core/providers/hosted";

Choosing a backend

Backend	Where inference runs	What leaves the browser	Requirements
Local (WebLLM / WebGPU)	Your GPU, in the browser	Nothing — zero egress	A WebGPU-capable browser
Bring-your-own hosted	Your chosen LLM provider	Prompt + aggregated results only	Provider key + CORS

There is no default backend — the first time you open the <AssistantPanel> it presents an explicit first-run chooser with both options side by side and their honest tradeoffs, and nothing is loaded or downloaded until you pick (ADR 0050 §4, amended). This avoids surprising a first-time user with a multi-GB local model download. When WebGPU is present the local option is highlighted as Recommended (it is the zero-egress choice); when it isn’t (older Safari, Firefox-on-Android, low-RAM devices) the local option is shown disabled with a “requires a WebGPU browser” note, and the hosted backend provides broader reach at the cost of sending aggregated analytics to your own provider. Your choice persists, so returning users go straight to the chat. You can change it — including switching between local and hosted — at any time via Change backend, which returns you to the same side-by-side selection cards; picking the other backend releases the previous model (freeing GPU memory) and activates the new one. Switching between local models also reclaims the previous model’s cached weights by default (see the storage note below).

import { defaultBackendKind, isWebGpuAvailable } from "@uptimizr/agent-core/providers";

if (isWebGpuAvailable()) {
  // Offer (and, if you like, highlight) the local, zero-egress backend — but let
  // the user choose; don't auto-select it.
}
const suggested = defaultBackendKind(); // "local" when WebGPU is present, else "hosted"

Local backend (WebLLM / WebGPU)

The heavy @mlc-ai/web-llm runtime is an optional dependency loaded via a lazy import() only when you actually run the assistant. Model weights download on first use, behind an explicit consent prompt, and are cached by the runtime in the browser’s Cache Storage — they are never part of any precache (including the demo’s “Prepare demo” step, ADR 0050 §6). Inference runs on your GPU; nothing leaves the browser.

import { CURATED_MODELS, createWebLlmProvider } from "@uptimizr/agent-core/providers/webllm";

const provider = createWebLlmProvider({
  model: "Hermes-3-Llama-3.1-8B-q4f16_1-MLC",
  // Called once, before any weights download. Show the size disclosure and
  // return false to abort — no data is downloaded if the user declines.
  confirmDownload: (model) =>
    confirm(`Download ${model.label} (${model.downloadSize})? It runs 100% locally.`),
  onInitProgress: ({ progress, text }) => updateProgressBar(progress, text),
  // Optional. "active-only" (the default) evicts the OTHER curated models' cached
  // weights when this one loads, so switching never stacks ~4 GB caches;
  // "keep-all" keeps every download for fast switching at the cost of disk.
  cachePolicy: "active-only",
  onCacheEvicted: (ids) => console.info("reclaimed cached models:", ids),
});

// Reclaim every cached model on demand (the same space "clear site data" frees).
const reclaimed = await provider.clearCachedModels(); // → the model ids removed

Install the runtime alongside the assistant (it is an optional peer dependency):

npm install @mlc-ai/web-llm

Curated models

WebLLM only supports the tool-calling (function calling) the assistant relies on for the 7–8B Hermes family, so the curated set is limited to those variants — ordered strongest-first, so the default is the best tool-caller (which most reliably answers even simple single-step questions on a small 4-bit local model). Sizes are approximate (sourced from WebLLM’s prebuiltAppConfig) and shown to the user before any download:

Model	Download	VRAM	Notes
Hermes 3 (Llama 3.1 8B)	~4.5 GB	~4.9 GB	Highest quality; the default.
Hermes 2 Pro (Llama 3 8B)	~4.6 GB	~5.0 GB	Stronger Llama-3 base; needs a capable GPU.
Hermes 2 Pro (Mistral 7B)	~3.9 GB	~4.0 GB	Smallest tool-calling model; least-VRAM.

Local mode needs a capable GPU. WebLLM hard-codes function calling to the Hermes-2-Pro / Hermes-3 family (its tool-call prompt and output parser are Hermes-specific), and the smallest of those is a 7B model. There is no small (<3 GB) tool-calling model in WebLLM, so local mode has an inherent floor: a WebGPU device with roughly 5 GB of free VRAM. On devices that can’t meet it, use the hosted backend instead. The provider validates the selected model up front and throws UnsupportedToolCallingModelError before any weights download if it isn’t tool-calling-capable.

Small in-browser models do tool-calling adequately but not perfectly — expect good summaries, not deep analytics (ADR 0050 trade-offs). Call provider.unload() to release GPU memory when done.

Getting good answers from the local model

Small local models shine at single-step questions and struggle with long, multi-tool analysis. The assistant is tuned for that reality:

• Current-time grounding. The system prompt is stamped with the current time (ISO 8601 + epoch ms) at send time, so relative ranges like “today”, “this week”, or “the last 24 hours” resolve to concrete since/until epoch-millisecond arguments instead of being dropped or guessed. The stamp is refreshed on every send — the conversation’s single system message is updated in place, never duplicated — so a long-lived conversation that crosses midnight keeps resolving “today” against the real current day. The clock is injectable via useAssistant({ now }) for deterministic tests, and the pure refreshSystemPrompt() helper (messages, basePrompt, nowMs) is exported for custom loops.
• A focused core tool set. The full catalog is 77 read tools, generated from the metric registry (ADR 0051); sending them all would overwhelm a 4-bit 7–8B model’s function-calling prompt many times over. For the local backend the assistant exposes a focused core subset of the most common single-step tools — list_sessions, list_scenes, top_meshes, perf_summary, event_counts, timeseries, and camera_heatmap. The hosted backend gets the full catalog (frontier models handle it). The core set is a filtered view of the same tool definitions — nothing is redefined (selectReadTools("core") / coreReadTools in @uptimizr/agent-core).
• Pin your own tool list. Pass useAssistant({ tools: ["perf_summary", "jank_rate"] }) to override the per-backend default with a deliberate subset — useful for a focused panel, a tight token budget, or a model that chooses better from fewer options. Names that are not in the catalog are ignored, and an empty (or entirely unknown) list falls back to the default selection rather than leaving the model with no tools.
• Guided example prompts. <AssistantPanel> shows a few starter questions (e.g. “What are my top meshes this week?”, “How’s my average FPS?”) in the empty conversation; each maps to a single core tool. Clicking one sends it — a reliable first-run path that also demonstrates the agent working.
• Keep tool results small — pass format=summary. A tool hands the model every row it asked for, and a big heatmap can fill a local model’s whole context on its own. Every generated aggregate tool takes a format argument (result formats): summary returns a bounded digest — top rows, a trend or merged spatial clusters, with shares, the metric’s caveats and a templated reading sentence — capped at the metric’s maxSummaryRows, so a 500-bin heatmap costs the same as a 5-bin one. The tools default to table, which keeps every row and adds the meta block (metric, range, applied filters, sample size, row count, truncation flag) so the model can tell what it is looking at; that envelope costs a fixed ~200 characters, it does not bound the result, so ask for summary when the answer could be large. Prefer questions that map to a small result and pass limit / scene / a tight range as well.
• An evaluated tool catalog. The tools the assistant hands your model are not just generated — they are measured. A bank of ~48 real analytics questions is run against a deterministic fixture set through this same catalog whenever it changes, and each answer is scored on tool selection, argument correctness and accuracy, so a tool description that makes a model pick badly shows up as a failed case. The harness is in the repository at oss/packages/agent-eval.

Local mode is for quick, single-metric answers. Use it for “top meshes this week”, “average FPS”, or “events in the last 24h”. For deeper, multi-step analysis, switch to a hosted backend with your own key at any time via Change backend — the frontier model gets the full tool catalog. This is a capability trade-off, not an error.

The local context window is raised to fit the assistant’s prompt. WebLLM loads the curated Hermes model records with a default context_window_size of 4096 tokens, but the assistant’s prompt — its system instructions plus the tool JSON schemas and any tool results — runs past that (a “Prompt tokens exceed context window size” error). The Hermes 7–8B models natively support 8k+ context, so the adapter loads the engine with an 8192-token window (DEFAULT_LOCAL_CONTEXT_WINDOW), overriding the model-record default. Hosts can tune it via createWebLlmProvider({ contextWindowSize }) — raise it only up to what the selected model actually supports.

Local models need browser storage — plan for ~4–5 GB per model. WebLLM caches a model’s weights in your browser’s Cache Storage on first use, and each curated model is roughly 4–5 GB. To keep that from piling up, the adapter keeps only the active model cached by default: when you switch to a different curated model, the previously downloaded model’s weights are evicted (via WebLLM’s hasModelInCache / deleteModelAllInfoInCache) before the new download starts, so caches never stack and switching back simply re-downloads. Hosts who prefer fast switching at the cost of disk can opt out with cachePolicy: "keep-all" (on createWebLlmProvider, useAssistant, or <AssistantPanel>). Eviction is scoped strictly to the curated/known model ids — nothing else cached on the origin is touched. Should the origin’s quota still fill up, the browser’s Cache API throws a storage-quota error — a browser storage limit, not an LLM API quota (the local backend has zero network egress) — and the assistant surfaces it with a clear remedy plus a one-click Clear cached models action (also shown in the panel’s local-backend footer) that deletes every cached curated model and reports what it reclaimed. The same is available programmatically as clearCachedModels() from @uptimizr/agent-core/providers/webllm, provider.clearCachedModels(), and useAssistant().clearCachedModels(). Alternatively free up disk space or clear this site’s cached data in your browser settings, pick the smallest model, or switch to a hosted backend. Before a large download the adapter also runs a best-effort navigator.storage.estimate() preflight (after the eviction, so freed space counts) and fails fast when free space is clearly insufficient, avoiding a corrupt half-download.

Local models manage their own function-calling system prompt. WebLLM injects its own tool-calling system prompt for the Hermes family and rejects a caller-supplied system message whenever tools are present. So for the local backend the assistant’s system instructions are merged into the first user turn instead of sent as a system role — a WebLLM constraint that is transparent to you. Hosted backends receive the system prompt as usual, which is why the local vs hosted framing can differ slightly under the hood.

Bring-your-own hosted backend

You supply an OpenAI-compatible or Anthropic endpoint + key. The key and endpoint are stored only in your browser (localStorage) and the browser calls your own provider directly — Uptimizr operates no proxy. Only the prompt and the aggregated tool results the loop produces leave, and only to the provider you chose (never raw events or PII, ADR 0050 §5).

import { createHostedProvider } from "@uptimizr/agent-core/providers/hosted";

// OpenAI-compatible
const openai = createHostedProvider({
  api: "openai",
  endpoint: "https://api.openai.com/v1", // or a self-hosted / gateway URL
  apiKey: "sk-…",
  model: "gpt-4o-mini",
});

// Anthropic
const anthropic = createHostedProvider({
  api: "anthropic",
  endpoint: "https://api.anthropic.com/v1",
  apiKey: "sk-ant-…",
  model: "claude-3-5-haiku-latest",
});

Required provider CORS

Because the request originates in the browser, the provider must allow cross-origin calls:

• Anthropic — the adapter sends the anthropic-dangerous-direct-browser-access: true header, which enables Anthropic’s browser CORS path. No proxy needed.
• OpenAI-compatible — api.openai.com does not send permissive CORS headers, so calling it directly from a browser is blocked. Use a provider/gateway that returns Access-Control-Allow-Origin for your dashboard’s origin (many self-hosted servers and LLM gateways do), or run the assistant against such an endpoint.

Persisting the choice

import { loadBackendConfig, saveBackendConfig } from "@uptimizr/agent-core/providers";

saveBackendConfig({
  backend: "hosted",
  hosted: {
    api: "anthropic",
    endpoint: "https://api.anthropic.com/v1",
    apiKey: "sk-ant-…",
    model: "claude-3-5-haiku-latest",
  },
});

const config = loadBackendConfig(); // null until the user picks a backend

The selection is read back on the next visit so users don’t re-choose each time. Clearing it (clearBackendConfig()) forgets the backend and any stored key.

Embed in a React app

Prefer not to wire the provider adapters and the tool-calling loop by hand? The @uptimizr/react component catalog ships a drop-in <AssistantPanel> and a headless useAssistant() hook from a dedicated, code-split subpath (ADR 0047). Importing the core @uptimizr/react barrel pulls no assistant or LLM code; only @uptimizr/react/assistant does, and even then @mlc-ai/web-llm stays lazy until a local model runs — so consumers who never open the assistant pay nothing.

import { AssistantPanel } from "@uptimizr/react/assistant";

// Reuses an ambient <UptimizrProvider>, or pass the collector connection directly.
export function Analytics() {
  return <AssistantPanel collectorUrl="http://localhost:4318" apiKey="proj_…" />;
}

<AssistantPanel> renders the whole surface — on first open, the backend chooser (both options with their tradeoffs); once a backend is picked, the message list, input, the local-vs-hosted backend and model picker, the WebLLM download-consent prompt and progress bar, and the privacy note. It reuses the same read-only CollectorApi client the panels use, so there is no second transport.

Knowing when it’s working

Local generation is not instant: a single answer on a 7–8B Hermes model can take anywhere from a few seconds to a couple of minutes on modest hardware — with nothing sent to a server in the meantime. Two things keep that from looking frozen.

Answers stream in. Both backends stream tokens, so the reply appears in the conversation word by word as the model produces it (“watch it type”) instead of landing all at once at the end. Hosted providers stream over Server-Sent Events (OpenAI-compatible stream: true and the Anthropic Messages stream — parsed with plain string scans, never a regex over model output); the local WebLLM backend streams the answer turn directly from the GPU. Tool-calling turns are not shown as text: WebLLM’s function-calling mode emits a JSON tool-call array as its whole output, and a hosted model’s pre-tool commentary is dropped when that turn ends — only the answer turn renders live. If a provider or gateway ignores the stream request and returns one JSON body, the answer simply lands in one piece as before.

A status label is always visible while a turn is in flight (a small spinner in an aria-live region, so it’s announced to screen readers — the streamed tokens themselves are deliberately kept out of that region so a screen reader isn’t read every fragment):

• Loading model… while a local model downloads/initializes (a progress bar replaces it once download progress is available),
• Running analytics… while a read-only tool call is executing (the per-tool list is shown too),
• Thinking… while the model is composing its answer, becoming
• Streaming… once the first tokens of the answer arrive and the live text is rendering.

Small local models sometimes gather the data but then stall — either returning an empty answer or tool-calling until the step cap without ever writing a reply. To fix that at the source, when a run would otherwise end with no usable answer the loop makes one final pass with tools disabled, which forces the model to compose a plain-text answer from the tool results it already gathered rather than reaching for another tool call. This forced pass is still local for the local backend, so there is no new data egress. To protect the local model’s context window, very large tool results are truncated (with a clear marker) before being fed back — full fidelity is kept below the cap. Both behaviors are on by default in @uptimizr/agent-core (runAgent({ forceFinalAnswer, maxToolResultChars })). Even so, hosted backends handle complex, multi-step questions more reliably than the small local models — reach for one when a local model keeps coming up short.

If a turn ever finishes without a natural-language answer, the panel says so explicitly instead of rendering nothing, so a reply is never silently dropped. It distinguishes two cases from the agent loop’s own signals: the model simply stopped with no text, or it kept calling tools and hit the step cap (it reports how many steps it took and suggests rephrasing, asking for a summary, or switching to a hosted model). The in-browser panel allows a few more tool-calling turns than the shared default (12 vs. @uptimizr/agent-core’s 8) so a small local model has room to wrap up; tune it with useAssistant({ maxSteps }). The conversation area scrolls and auto-follows the newest message, so answers stay in view inside a fixed-height drawer.

For a custom UI, drive the headless hook instead and render your own chat:

import { useAssistant } from "@uptimizr/react/assistant";

function MyAssistant() {
  const { messages, send, status, toolActivity, backend, setBackend, isReady } = useAssistant({
    collectorUrl: "http://localhost:4318",
    apiKey: "proj_…",
    // Optional: pass an explicit backend, or omit to load the persisted choice.
    // With neither, `backend` stays `null` on first run (no auto-select) so you
    // can render your own chooser before anything loads.
  });
  // messages: the transcript · send(text): run a turn · status: "idle" | "initializing" |
  // "thinking" | "error" · isBusy: a turn is in flight (show a working indicator) · partialText:
  // the answer streamed so far for the in-flight turn (null when nothing is streaming; render it
  // as the live assistant bubble — it clears in the same render the final turn lands in
  // `messages`, so never both) · toolActivity:
  // live tool-call progress · notice: {kind:"no_answer"|"stopped_on_max_steps",steps?} when a turn
  // produced no written answer · setBackend(cfg): switch + persist · clearCachedModels(): delete
  // every cached local model's weights and resolve with the ids reclaimed. Options: cachePolicy
  // ("active-only" default | "keep-all") and onCacheEvicted(ids) for the local backend.
}

The hook wraps @uptimizr/agent-core’s runAgent loop, manages message history and per-turn state, tracks tool-call and WebLLM download progress, and persists the backend choice via the config helpers above. The loop runs client-side, so it works against both a real collector and the demo’s in-browser DuckDB-Wasm query layer with no server. Point Tailwind at the package source (as the panels require) so the component’s utility classes aren’t tree-shaken out.

In the OSS dashboard

The self-hostable dashboard has the assistant built in — no wiring required. Once you’re connected to a project, open the Analytics assistant card at the top of the overview and click Ask the assistant. The panel mounts against that project’s existing collector connection (the same read-only query API and key the panels use) and answers are grounded in real tool calls against your data.

The assistant is loaded exactly like the portable component above: a lazy import() pulls @uptimizr/react/assistant (and, only when a local model runs, @mlc-ai/web-llm) on first open, so the dashboard’s main bundle is unchanged for anyone who never opens it. The first time you open it, the panel asks you to choose a backend — local WebLLM (zero egress) or your own hosted key — before anything loads; the choice is remembered, and you can change it — or switch between local and hosted — at any time via Change backend.

Keeping an answer: “Annotate this” and “Save this analysis”

An answer you have to re-derive next week is half an answer. Under every reply the panel offers two actions:

• Annotate this stores the answer as a project note, pinned to whatever the dashboard is currently filtered to — the scene you are looking at, or the time window you are showing. The note then appears as a marker on the event-volume time axis, with its text as the tooltip, so the next person to look at that spike reads the explanation instead of re-deriving it.
• Save this analysis stores the turn as a titled record: a title (pre-filled with the question you asked), the collector reads the model actually made, and the answer as the conclusion.

Both actions are shown only when the connected key holds the annotate capability — the panel asks GET /api/v1/whoami once and hides them otherwise, rather than offering a button that would be refused. A key minted by uptimizr init / uptimizr new-project carries it; a read-only key (uptimizr new-key’s default) does not.

Rows written this way are recorded as authored by an agent, because the text is the model’s — the collector decides that from the calling client, never from the payload. They are bounded, audited, and metadata only: nothing here can write, alter or delete an event. See Metadata endpoints.

In the backend-less demo

The live demo embeds this same dashboard build, and its /api/v1/* reads are served entirely in the browser by a service worker backed by DuckDB-Wasm (no server, no account). So the assistant works there too: choose the Local (WebLLM) backend and ask away — the tool calls read from the in-browser store and the model runs on your GPU, with no server and no API key. The model weights download on demand behind the consent prompt the first time you open it, and are never part of the demo’s one-time “Prepare demo” precache (ADR 0050 §6) — a visitor who never opens the assistant downloads nothing extra.

The demo updates itself automatically: its service worker serves the app shell network-first, so reloading the page always picks up the latest deploy (and its latest assistant build) while staying usable offline after “Prepare demo”. If you ever seem stuck on an old build, reload once more, or force a clean copy via your browser’s Clear site data / a hard refresh / an incognito window.

Project context in the system prompt

Before its first answer the assistant reads the collector’s project context document and folds a compact rendering of it into the system prompt. That is what lets a small local model use your names instead of plausible ones:

Project context (read from this collector; prefer these real names over any you infer):

Scenes (use these exact ids for the `scene` filter; region ids for `region`):
- lobby "Main Lobby" [regions: counter, entrance]
- arena

Custom events this app emits (name ×count {props}) — use these exact names:
- add_to_cart ×311 {sku: string, qty: number}

Most-interacted meshes: checkout_button, door_left.

No data is captured for these metrics, so they WILL return empty — say the channel is off rather
than reporting a zero: mesh_dwell, hover_dwell.

Data: last event 4 min ago, 91 sessions in the last 24 h.
Raw per-session retention is OFF: session timelines and replay are unavailable by design.

The block is deliberately short (≈1.5 k characters at most, truncated on a line boundary if a project is unusually large), because the same prompt has to carry the tool schemas for a 1–3 B local model. It is re-stamped on every send, so a context that arrives mid-conversation still reaches the model, and it sits after the current-time line so the original prompt is unchanged.

The document is fetched once per collector connection and cached server-side for ~30 s, so it costs almost nothing. A collector too old to serve /api/v1/context is not an error: the read fails silently and the assistant runs with exactly the prompt it had before.

useAssistant also returns the raw document as projectContext (null while loading, or when the endpoint is unavailable), so a host UI can show what the assistant knows:

const { projectContext } = useAssistant({ collectorUrl, apiKey });
const scenes = projectContext?.scenes?.map((s) => s.id) ?? [];

If you drive @uptimizr/agent-core yourself, renderContextForPrompt(document) produces the same block.

See also

• MCP server (AI agents) — the same read-only tool catalog for external/local agents.
• Project context — the document the assistant injects, and the endpoint behind it.
• ADR 0050 — design rationale and trust boundary.