feat(live): human live view via CDP screencast over SSE (C1)

README.md

@@ -10,7 +10,7 @@ Shadow DOM + iframes), multi-step plans, structured extraction, visual diff, and
 guardrails** for payments and bookings. It drives real Chromium, so it reads **Next.js / SPA**
 pages after hydration — not just static HTML.
 
-> 30 MCP tools · stealth + rotating proxies · virtualized-list scraping · HAR record/replay · pixel visual-diff · human handoff.
+> 32 MCP tools · stealth + rotating proxies · virtualized-list scraping · HAR record/replay · pixel visual-diff · human handoff + live view.
 
 ## Install
 
@@ -55,15 +55,15 @@ agent passes `humanApproved`.
 - **Sees everything** — open Shadow DOM, same/cross-origin iframes, and **virtualized/infinite lists** (`browser_collect`).
 - **Fast-path** — `browser_fetch` impersonates a real Chrome TLS fingerprint for server-rendered HTML, no browser launch — returns clean **markdown** and optional **contacts** (`extractContacts`) at ~HTTP speed.
 - **Data out** — multi-currency prices, typed CSS extraction, **contact extraction** (emails/phones E.164, `fastPathFirst` cascade), a clean→validate→dedupe→emit pipeline, CSV export, Google SERP rank tracking.
-- **Ops** — persistent sessions, **auto crash recovery** (a crashed page is recreated in the same context and restored to its last URL between calls), opt-in **per-host circuit breaker** + **bounded probe queue/budget** for mass scraping, `storageState` auto-save, HAR record/replay, pixel `visual_diff`, human handoff for login/2FA.
+- **Ops** — persistent sessions, **auto crash recovery** (a crashed page is recreated in the same context and restored to its last URL between calls), opt-in **per-host circuit breaker** + **bounded probe queue/budget** + **`browser_metrics`** for mass scraping, **live view** (watch any session — even headless — in your browser), `storageState` auto-save, HAR record/replay, pixel `visual_diff`, human handoff for login/2FA.
 
 ## Documentation
 
 Full reference in **[`docs/`](./docs/README.md)**:
 
 [Installation](./docs/installation.md) ·
 [CLI](./docs/cli.md) ·
-[MCP tools (30)](./docs/mcp-tools.md) ·
+[MCP tools (32)](./docs/mcp-tools.md) ·
 [Configuration](./docs/configuration.md) ·
 [Sessions](./docs/sessions.md) ·
 [Extraction](./docs/extraction.md) ·

docs/README.md

@@ -7,7 +7,7 @@ New here? Start with the root [README](../README.md), then dive in:
 | --- | --- |
 | [Installation](./installation.md) | Requirements, install, Chromium, MCP registration, the three ways to get a browser |
 | [CLI](./cli.md) | `probe` / `fetch` / `serp-batch` / `shots` + every flag |
-| [MCP tools](./mcp-tools.md) | All 30 tools with parameters and examples |
+| [MCP tools](./mcp-tools.md) | All 32 tools with parameters and examples |
 | [Configuration](./configuration.md) | `AgentOptions`, `FUSE_*` env vars, identity, retry, output location |
 | [Sessions](./sessions.md) | Session lifecycle, auto crash recovery, `storageState` auto-save, HAR record/replay, CDP attach |
 | [Extraction](./extraction.md) | `browser_extract` / `extract_schema` / `collect` + the clean→validate→dedupe→emit pipeline |

docs/mcp-tools.md

@@ -1,6 +1,6 @@
 # MCP tools
 
-Complete reference for the 30 `browser_*` tools exposed by the fuse-browser MCP server.
+Complete reference for the 32 `browser_*` tools exposed by the fuse-browser MCP server.
 
 Tools fall into two families:
 
@@ -560,3 +560,35 @@ Returned fields: `uptimeMs`, `probesOk`, `probesFailed`, `avgDurationMs`, `minDu
 ```json
 { "reset": false }
 ```
+
+---
+
+## Live view
+
+Watch a session's browser in real time from a normal web page — works even for **headless** sessions. A CDP screencast streams JPEG frames over an ephemeral, token-gated `127.0.0.1` server (Server-Sent Events) to a `<canvas>`/`<img>` viewer. **Read-only** (no click-through). The frame stream binds to the page at start; after a crash recovery it does not auto-reattach — call `browser_live_view` again.
+
+### browser_live_view
+
+Start the live view for a session and return the viewer URL (token embedded). Opens it in the OS default browser unless `open:false`.
+
+| Param | Type | Required | Description |
+| --- | --- | --- | --- |
+| `sessionId` | string | yes | Target session. |
+| `quality` | integer | no | JPEG quality 1–100 (default `60`). |
+| `maxWidth` | integer | no | Max frame width in px (default `1280`). |
+| `maxHeight` | integer | no | Max frame height in px (default `720`). |
+| `open` | boolean | no | Open the URL in the default browser (default `true`). |
+
+Returns `{ url, note }`. The server auto-closes when the page closes or on `browser_live_view_stop`.
+
+```json
+{ "sessionId": "s_abc123", "quality": 60, "open": true }
+```
+
+### browser_live_view_stop
+
+Stop a session's live view and shut down its local server. Returns `{ stopped: true|false }` (`false` if none was running).
+
+```json
+{ "sessionId": "s_abc123" }
+```

package.json

@@ -54,7 +54,7 @@
     "build": "tsc -p tsconfig.json",
     "typecheck": "tsc -p tsconfig.json --noEmit",
     "test": "bun test tests/unit",
-    "test:integration": "node --test --import tsx tests/integration/mcp.test.ts tests/integration/probe.test.ts tests/integration/snapshot.test.ts tests/integration/snapshot-frames.test.ts tests/integration/collect.test.ts tests/integration/selectors.test.ts tests/integration/visual-diff.test.ts tests/integration/session-state.test.ts tests/integration/pipeline.test.ts tests/integration/run.test.ts tests/integration/extract-schema.test.ts tests/integration/recovery.test.ts",
+    "test:integration": "node --test --import tsx tests/integration/mcp.test.ts tests/integration/probe.test.ts tests/integration/snapshot.test.ts tests/integration/snapshot-frames.test.ts tests/integration/collect.test.ts tests/integration/selectors.test.ts tests/integration/visual-diff.test.ts tests/integration/session-state.test.ts tests/integration/pipeline.test.ts tests/integration/run.test.ts tests/integration/extract-schema.test.ts tests/integration/recovery.test.ts tests/integration/live-view.test.ts",
     "browsers": "patchright install chromium",
     "mcp": "node --import tsx src/bin/mcp.ts",
     "cli": "node --import tsx src/bin/cli.ts"

src/live/manager.ts

@@ -0,0 +1,60 @@
+/**
+ * Live-view manager: ties a session to an ephemeral SSE server fed by a CDP
+ * screencast of its page. One live view per session; starting again replaces
+ * the previous one. Auto-stops when the page closes.
+ * @module live/manager
+ */
+import { randomBytes } from "node:crypto";
+import type { SessionData } from "../session/session.js";
+import { openUrl } from "./open-url.js";
+import { startScreencast, type ScreencastOptions } from "./screencast.js";
+import { type LiveServer, startSseServer } from "./sse-server.js";
+
+interface LiveView {
+  url: string;
+  stop: () => Promise<void>;
+}
+
+const views = new Map<string, LiveView>();
+
+/** Options for {@link startLiveView}. */
+export interface LiveViewOptions extends ScreencastOptions {
+  /** Open the viewer in the OS default browser (default true). */
+  open: boolean;
+}
+
+/**
+ * Start (or restart) the live view for a session and return the viewer URL.
+ *
+ * @param session - The live session whose page to stream.
+ * @param opts - Frame quality/size and whether to auto-open the viewer.
+ * @remarks The screencast binds to the page live at call time. If the page is
+ * later recreated by crash recovery (B1), the stream tears down with the old
+ * page (no silent leak) but does not auto-reattach — call `browser_live_view`
+ * again after a recovery to resume watching.
+ */
+export async function startLiveView(session: SessionData, opts: LiveViewOptions): Promise<string> {
+  await stopLiveView(session.id);
+  const token = randomBytes(16).toString("hex");
+  const server: LiveServer = await startSseServer(token);
+  const stopCast = await startScreencast(session.page, opts, (b64) => server.broadcast(b64));
+  views.set(session.id, {
+    url: server.url,
+    stop: async () => {
+      await stopCast();
+      await server.close();
+    },
+  });
+  session.page.once("close", () => void stopLiveView(session.id));
+  if (opts.open) openUrl(server.url);
+  return server.url;
+}
+
+/** Stop a session's live view; returns false if none was running. */
+export async function stopLiveView(id: string): Promise<boolean> {
+  const view = views.get(id);
+  if (!view) return false;
+  views.delete(id);
+  await view.stop();
+  return true;
+}

src/live/open-url.ts

@@ -0,0 +1,17 @@
+/**
+ * Best-effort: open a URL in the OS default browser. Never throws.
+ * @module live/open-url
+ */
+import { spawn } from "node:child_process";
+
+/** Spawn the platform opener for `url`, detached; failures are ignored. */
+export function openUrl(url: string): void {
+  const platform = process.platform;
+  const cmd = platform === "darwin" ? "open" : platform === "win32" ? "cmd" : "xdg-open";
+  const args = platform === "win32" ? ["/c", "start", "", url] : [url];
+  try {
+    spawn(cmd, args, { detached: true, stdio: "ignore" }).unref();
+  } catch {
+    /* best-effort: the URL is also returned to the caller */
+  }
+}

src/live/screencast.ts

@@ -0,0 +1,59 @@
+/**
+ * CDP screencast on a live page: emits base64 JPEG frames to a callback. Each
+ * frame is acked with its own `sessionId` (mandatory — the stream stalls after
+ * frame 1 otherwise), and the screencast is re-issued on navigation (it stops
+ * on a hard nav). Chromium only; works headless.
+ * @module live/screencast
+ */
+import type { CDPSession, Page } from "playwright";
+
+/** Screencast frame quality/size knobs. */
+export interface ScreencastOptions {
+  quality: number;
+  maxWidth: number;
+  maxHeight: number;
+}
+
+interface ScreencastFrame {
+  data: string;
+  sessionId: number;
+}
+
+/**
+ * Start a CDP screencast on `page`, invoking `onFrame` with each base64 JPEG.
+ *
+ * @param page - The live Playwright page (Chromium).
+ * @param opts - Frame quality and max dimensions.
+ * @param onFrame - Called with the base64 JPEG of every frame.
+ * @returns A stop function that ends the screencast and detaches the session.
+ */
+export async function startScreencast(
+  page: Page,
+  opts: ScreencastOptions,
+  onFrame: (base64: string) => void,
+): Promise<() => Promise<void>> {
+  const cdp: CDPSession = await page.context().newCDPSession(page);
+  const begin = (): Promise<unknown> =>
+    cdp
+      .send("Page.startScreencast", {
+        format: "jpeg",
+        quality: opts.quality,
+        maxWidth: opts.maxWidth,
+        maxHeight: opts.maxHeight,
+        everyNthFrame: 1,
+      })
+      .catch(() => undefined);
+  cdp.on("Page.screencastFrame", (frame: ScreencastFrame) => {
+    // Ack FIRST (with the frame's own sessionId) or the stream stalls.
+    void cdp.send("Page.screencastFrameAck", { sessionId: frame.sessionId }).catch(() => undefined);
+    onFrame(frame.data);
+  });
+  const onLoad = (): void => void begin();
+  page.on("load", onLoad);
+  await begin();
+  return async () => {
+    page.off("load", onLoad);
+    await cdp.send("Page.stopScreencast").catch(() => undefined);
+    await cdp.detach().catch(() => undefined);
+  };
+}

src/live/sse-server.ts

@@ -0,0 +1,66 @@
+/**
+ * Ephemeral localhost SSE server for the live view. Binds 127.0.0.1 on an
+ * OS-assigned port, gates every request on a token, serves the viewer HTML at
+ * `/` and a `text/event-stream` of base64 JPEG frames at `/stream`.
+ * @module live/sse-server
+ */
+import { createServer, type ServerResponse } from "node:http";
+import { VIEWER_HTML } from "./viewer-html.js";
+
+/** A running live-view server. */
+export interface LiveServer {
+  /** Viewer URL (includes the access token). */
+  url: string;
+  /** Push a base64 JPEG frame to every connected viewer. */
+  broadcast(base64: string): void;
+  /** Close all streams and stop the server. */
+  close(): Promise<void>;
+}
+
+/** Start the token-gated SSE server bound to loopback. */
+export async function startSseServer(token: string): Promise<LiveServer> {
+  const clients = new Set<ServerResponse>();
+  const server = createServer((req, res) => {
+    const u = new URL(req.url ?? "/", "http://127.0.0.1");
+    if (u.searchParams.get("token") !== token) {
+      res.writeHead(403).end("forbidden");
+      return;
+    }
+    if (u.pathname === "/stream") {
+      res.writeHead(200, {
+        "content-type": "text/event-stream",
+        "cache-control": "no-cache",
+        connection: "keep-alive",
+      });
+      clients.add(res);
+      req.on("close", () => clients.delete(res));
+      return;
+    }
+    res.writeHead(200, { "content-type": "text/html; charset=utf-8" }).end(VIEWER_HTML);
+  });
+  await new Promise<void>((resolve) => server.listen(0, "127.0.0.1", resolve));
+  const addr = server.address();
+  const port = typeof addr === "object" && addr ? addr.port : 0;
+  return {
+    url: `http://127.0.0.1:${port}/?token=${token}`,
+    broadcast(base64) {
+      const chunk = `data: ${base64}\n\n`;
+      for (const res of clients) {
+        if (res.writableEnded || res.destroyed) {
+          clients.delete(res);
+          continue;
+        }
+        // Async EPIPE/ECONNRESET (client gone mid-frame) arrives in the callback,
+        // not as a throw — handle it here so it never crashes the process.
+        res.write(chunk, (err) => {
+          if (err) clients.delete(res);
+        });
+      }
+    },
+    async close() {
+      for (const res of clients) res.end();
+      clients.clear();
+      await new Promise<void>((resolve) => server.close(() => resolve()));
+    },
+  };
+}

src/live/viewer-html.ts

@@ -0,0 +1,35 @@
+/**
+ * Self-contained HTML viewer for the live screencast. Connects to the SSE
+ * `/stream` endpoint (token from the URL) and paints each base64 JPEG frame
+ * into a full-window <img>. Read-only, no dependencies.
+ * @module live/viewer-html
+ */
+
+/** The viewer page served at `/` (inlined; no file reads). */
+export const VIEWER_HTML = `<!doctype html>
+<html lang="en">
+<head>
+<meta charset="utf-8" />
+<meta name="viewport" content="width=device-width, initial-scale=1" />
+<title>fuse-browser · live view</title>
+<style>
+  html,body{margin:0;height:100%;background:#0b0b0e;color:#9aa;font:13px system-ui}
+  #wrap{display:flex;align-items:center;justify-content:center;height:100%}
+  #frame{max-width:100%;max-height:100%;object-fit:contain;box-shadow:0 0 40px #0008}
+  #status{position:fixed;top:8px;left:10px;opacity:.7}
+</style>
+</head>
+<body>
+<div id="status">connecting…</div>
+<div id="wrap"><img id="frame" alt="live view" /></div>
+<script>
+  const token = new URLSearchParams(location.search).get("token");
+  const img = document.getElementById("frame");
+  const status = document.getElementById("status");
+  const es = new EventSource("/stream?token=" + encodeURIComponent(token));
+  es.onopen = () => { status.textContent = "live"; };
+  es.onmessage = (e) => { img.src = "data:image/jpeg;base64," + e.data; };
+  es.onerror = () => { status.textContent = "disconnected"; };
+</script>
+</body>
+</html>`;

src/server/server.ts

@@ -13,6 +13,7 @@ import { registerExtractTool } from "./tools/extract.js";
 import { registerExtractSchemaTool } from "./tools/extract-schema.js";
 import { registerHandoffTool } from "./tools/handoff.js";
 import { registerInspectTool } from "./tools/inspect.js";
+import { registerLiveViewTool } from "./tools/live-view.js";
 import { registerMetricsTool } from "./tools/metrics.js";
 import { registerNavigateTool } from "./tools/navigate.js";
 import { registerFetchTool } from "./tools/fetch.js";
@@ -52,6 +53,7 @@ export function createServer(): BuiltServer {
   registerInspectTool(server, sessions);
   registerVisualDiffTool(server, sessions);
   registerHandoffTool(server, sessions);
+  registerLiveViewTool(server, sessions);
   registerMetricsTool(server);
   registerResources(server);
   return { server, sessions };