feat(packages): add dsh-plugin-browserskill — DeepSeek Harness tools for bsk - #91
Draft
BB-fat wants to merge 22 commits into
Draft
feat(packages): add dsh-plugin-browserskill — DeepSeek Harness tools for bsk#91BB-fat wants to merge 22 commits into
BB-fat wants to merge 22 commits into
Conversation
…for bsk Add a new monorepo package that registers BrowserSkill (bsk) browser automation as model-visible tools in DeepSeek Harness (dsh): - 11 tools mapping to bsk CLI --json commands: session start/stop/list, navigate, snapshot, observe, click, fill, press, screenshot, emulate - multi-session support: optional session arg with a current-session pointer, session id echoed in every result, configurable concurrency cap (default 5), and dispose-time cleanup of plugin-started sessions - cancellation: exec.signal aborts kill the underlying bsk child process - terminal-style UI cards (command line as the call card, output as the result card); screenshots inline the PNG through the host attachment store when the model route accepts image input, else return the path - config: bskPath / defaultTimeoutMs / maxSessions via Schemastery schema - install guidance when the bsk binary is missing; bsk JSON error envelopes (code/message/hint) surfaced to the model - unit tests mock the bsk runner (no real browser required); the root lint script now also runs the package's typecheck + tests so the existing CI frontend job covers the new package (a dedicated CI job diff is provided in the PR; the pushing token lacks workflow scope)
The concurrency cap was checked in registry.add() AFTER `bsk session start` had already created the session, so a rejected start leaked a live session outside the plugin's tracking (found by the real-browser e2e: the rejected third session stayed in `bsk session list` after dispose). Check capacity before spawning; registry.add() keeps the same check as a backstop.
…n message The daemon's JSON error envelope carries an actionable hint (e.g. 'choose an input, textarea, or contenteditable element from the latest snapshot'), but the thrown BskError only embedded the message field, so the model-facing error text never showed the hint. Append it.
…-block results browser_screenshot on an image-capable route returns text + image blocks; the shared result presenter required exactly one text block and bailed to the generic card (tool name + raw args). Project the first text block instead so the terminal card shows in both modes.
…_screenshot toolview Make the package dual-face: a dsh.client declaration (platform 'web') plus an exports["./client"] bundle built in dsh's closure-factory shape (window.__ModuleLoader__.load handoff, platform modules external, everything else inlined, CSS Modules via lightningcss). The client registers a keyed 'tool.call.toolview' view for browser_screenshot that keeps the terminal block (command + output) and, when the settled result carries an image block, resolves the durable attachment through the client session's authorized readAttachment RPC and renders it with the shared MessageImage thumbnail/lightbox atoms. Path-only results render unchanged; every other browser_* tool keeps the stock terminal card. Tests: view-model derivation (running/ok/error, image vs path-only), component rendering through @testing-library/react + happy-dom, and the registration key. biome.json now excludes the package's lib/ build output (same treatment as dist/).
Collaborator
|
@BB-fat 你再检查一下，就是如果有别的agent工具在使用当前bsk在做一些事情，agent window激活，然后插件完成工作，session关闭，会不会影响之前的 agent window或者session |
…emon Review hardening for daemon-sharing deployments (other agents, terminals, or dsh instances on the same bsk daemon): - Ownership split: only sessions created by browser_session_start are 'owned'. Explicitly referenced foreign sessions are still tracked for the current-session pointer, but browser_session_stop refuses them and unload cleanup stops exactly the owned set — no path can stop a session this plugin did not create. - Start race: the capacity check is now a synchronous reserveStart/completeStart/abandonStart protocol, so two concurrent starts can never both pass the cap (check-and-reserve is atomic on the event loop); a rejected or failed start never leaks a session. - Stale handles: dispose already swallowed per-stop errors; the owned set is computed at dispose time so an externally stopped session is a no-op, not a failure. Tests (+7): reservation race (two concurrent starts, cap 1 -> exactly one spawn), foreign-stop refusal (no stop command reaches the daemon), reference adoption never becoming owned, resolveForStop pointer stability, and an apply-level dispose test proving only owned sessions are stopped while stale stops are tolerated. e2e: a manually started foreign session survived the plugin's full lifecycle while both plugin sessions were cleaned up.
BB-fat
force-pushed
the
feat/dsh-plugin
branch
from
August 14, 2026 03:57
17a3d3c to
afdc398
Compare
Collaborator
Author
|
Effect screenshots — scripted model driving real tool execution (real bsk + Chromium behind):
|
BB-fat
marked this pull request as draft
August 14, 2026 04:06
…ssions are invisible Tighten the shared-daemon boundary from 'cleanup only touches owned sessions' to 'the plugin only ever sees and operates on sessions it created': - browser_session_list no longer queries the daemon at all; it returns the plugin's own session table (with the current marker), so foreign sessions cannot even be enumerated through the plugin. - Every tool's optional session argument must name a plugin-created session; foreign/unknown ids are rejected with a clear error before any command reaches the daemon (the reference-adoption channel is gone). - Unchanged: dispose stops exactly the owned set, stop requires ownership, the start reservation protocol, and handle-precise child kills. Tests updated (59): explicit-foreign rejection without daemon contact, registry-only listing, reservation race, stale-handle-tolerant dispose. e2e: a manually started foreign session was rejected on reference, absent from session_list, refused on stop, and survived dispose while both plugin sessions were cleaned up.
Collaborator
Author
|
Feature idea: live BSK observation overlay with one-click interrupt When the agent is driving the browser, the user has no ambient visibility into BSK's control state, and interrupting means hunting for the Stop button in the chat flow. Proposal: a video-conference-style mini-window. Interaction — a real Picture-in-Picture window (Document PiP API), always on top:
Phasing
Open point: in-app floating overlay as fallback when PiP is unavailable. Feedback welcome — happy to prototype this behind the plugin's client half. |
…iP overlay (Phase 1-A) Per-session live observation for owned sessions only, feeding the upcoming client overlay: - State model: SessionObservation (sessionId/url/action/since/ thumbnailAttachmentId/lastError), one entry per owned session; added on start, removed on stop, cleared on dispose. - Instrumentation: the shared runBsk wrapper begins/ends an action (tool name mapped to a verb) around every model-facing call and tags the child with its session id; action end triggers an immediate thumbnail refresh. - Thumbnail loop: 1.5s cadence while active, 8s idle downclock, x3 failure backoff, silent frame retention on errors; captures run OUTSIDE the instrumentation (no action events, no current-pointer movement) and land in the attachment store by reference. - Interrupt: runner.killFor(tag) kills exactly the tagged in-flight bsk children — chat-Stop-equivalent semantics for the current or a named owned session; foreign ids and the no-in-flight case return false. - Remote seam: dsh 0.1's Typert Remote pipeline and forwarded-event allowlist are closed to out-of-tree packages, so state/events/ interrupt are served over the documented webServer route seam: GET /bsk-observation/state, GET /bsk-observation/events (SSE), POST /bsk-observation/interrupt; routes mount only when a webServer service exists (headless compositions skip them). - Config: observationEnabled (true), thumbnailIntervalMs (1500), idleIntervalMs (8000). Tests (+13, 72 total): state machine + event stream, cadence fast/idle/ backoff, headless no-store silence, observation-traffic isolation, interrupt routing (default/specified/none/foreign), HTTP route handlers, disabled-webServer no-op.
…nterrupt, and PiP (Phase 1-B) Client half of the PiP observation window, wired to the host ObservationService over the /bsk-observation HTTP+SSE seam: - ObservationClientStore: initial state fetch + SSE increments + on-demand thumbnail blob loading (session-authorized readAttachment, same path as the toolview) + interrupt POST. All I/O injectable. - ObservationOverlay registered into the shell.overlay list seat (the sanctioned frame-wide surface; root is off-limits). Hidden with no owned sessions; appears on first start; vanishes when all stop. - Focus view: status row (green/grey/red dot + session + action + mm:ss ticking), breathing thumbnail (new frames fade in, failures keep the last frame with a warning badge, pre-navigate placeholder), and the action area. - Interrupt: single click, no confirm; greys into 'Interrupting…' and back; disabled when nothing is in flight; one-time tooltip explains the semantics (stops the current action only; the run continues). - Card: drag-move via the header, corner-handle resize (min 240x180, max 80% viewport), both clamped and kept for the page lifetime; collapsible status capsule. - PiP: Pop out (user gesture) opens Document PiP with the card's current size, portals the same content in, and clones the document's style nodes; pagehide falls back to the card with state intact; unsupported browsers hide the button; nothing auto-pops. Tests (+17, 89 total): store (fetch/SSE apply/malformed frames/ thumbnail lifecycle/interrupt wire/stop cleanup) and overlay (hidden-> visible lifecycle, status row, thumbnail via loader, interrupt disabled/active/hint-once, capsule, resize clamps, move clamps, PiP unsupported, PiP pop/fallback). Styling follows web-styling (dsw-alias tokens, CSS Modules, focus-visible, reduced-motion).
…ge states (Phase 1-C)
- Strip (meeting-style multi-session layout): one item per session
(mini frame + id + status dot), horizontal row with scroll overflow;
hover reveals a per-item interrupt button that acts without
refocusing; click pins the focus view (pin badge, click again to
release); auto-follow picks the most recently active session but
never steals focus for errored or dead sessions (red edge / greyed
item instead).
- Edge states: the host flips an availability flag after repeated
global capture failures (client shows 'browser unavailable', keeps
the last frame, greys interrupt, and recovers on the next success);
a session_not_found envelope marks the session dead (no more frame
requests, grey strip item, removable as usual); lastError now clears
when the next action starts or succeeds.
- State/SSE wire gains { available } and an 'availability' event;
SessionObservation gains dead?.
Tests (+8, 97 total): host availability flip, dead marking +
instrumentation drop + clean removal, lastError lifecycle; client
strip render/pin/unpin/auto-follow, error no-steal, unavailable strip,
dead grey, strip-item interrupt without refocus.
…ver appears
ctx.get('webServer') at apply time raced the web composition's service
registration (the fallback SPA then answered /bsk-observation/* with
index.html). Ride ctx.inject(['webServer'], …) instead: the callback
runs when the service is provided and never runs in headless
compositions, keeping route registration timing-safe and optional.
…tion vs tool calls) The daemon accepts only one unfinished command per session; the thumbnail loop racing a model tool call produced 'session already has an unfinished command' failures on both sides (found by the Phase 1-C e2e: navigate/snapshot/click all errored while captures kept running). Add a KeyedExecutor: every command for a session — tool calls and observation captures alike — runs FIFO; queued tasks reject early on abort, running tasks keep the runner's signal-driven kill.
…lugin's own route The session-authorized client RPC (readAttachment) refuses images that no session log references — observation frames are plugin-owned runtime data, so every overlay thumbnail failed with ATTACHMENT_NOT_REFERENCED. The host now keeps the full attachment ref per frame and serves bytes through GET /bsk-observation/thumbnail/<id> (verified readImage path); the client loader is a plain fetch of that route.
…ng slash The webserver prefix matcher joins prefix + '/', so a registered '/bsk-observation/thumbnail/' never matched real paths.
…errupted A child killed via killFor (overlay interrupt) exits with a null code and empty output, which used to render as a duplicated 'bsk x failed: bsk x failed'. Report it as interrupted instead.
… dsh-native styling Review follow-ups: the card/capsule now default to the top-right (clear of the composer), and the overlay reads as part of the shell instead of a foreign widget: - shared primitives everywhere: Button (outline/ghost sm) for Interrupt/Pop out with the shell's danger hover token, Tooltip for the one-time interrupt semantics hint (native span anchor — Button does not forward refs), StateDot for status, and outline icons (Stop/RightUp/ChevronDown/Warning) replacing text glyphs; - every color/surface/shadow/label now uses --dsw-alias-* semantic tokens (bg-overlay, border-l1/l2, state-*-primary, label-*, interactive-bg-hover[-danger], brand-primary focus rings, dsw-shadow-lv1/lv2) — no invented tokens, no literal colors (dsh 0.1 ships no spacing/radius scale; those stay px like dsh's own code); - strip item focus ring uses brand-primary; focus-visible and reduced-motion preserved.
The plain top:16 default overlapped the shell's Session log action; dock the card and capsule at the content area's top-right (64px) so neither collides with the header controls.
… own UI system The previous pass aligned the observation overlay with the dsh shell; the product call is the opposite direction — the floating card should read as a BrowserSkill surface, not a shell-native widget. - Reuse @browser-skill/ui directly (Button, cn); status dots spec'd after the extension popup's ConnectionStatusIndicator; Remix icons for interrupt/pin/warn/pop-out. - Compile the BSK tailwind utility sheet scoped under the .bsk-obs root class (scripts/build-client-css.mjs + postcss-prefix-selector) and ship the oklch design tokens on the same scope, so nothing leaks into the host shell and the shell theme cannot bleed back. tsdown injects .nomodule.css verbatim (minified, unhashed) alongside the hashed CSS modules. - BSK ships no shared tooltip, so the one-time interrupt semantics hint is a card-spec bubble (bg-card/border/12px radius), retired after first use. - Tests: pin react/react-dom to this package's 18.x copies in vitest — the ui package sources peer on react ^19 and would otherwise emit react-19 (transitional) elements that the react-18 renderer rejects. Production is unaffected (react stays external, provided by the shell). - stylelint: ignore the generated bsk-ui.nomodule.css.
The client entry imports the generated bsk-ui.nomodule.css; the file is gitignored, so a fresh CI checkout could not resolve it. Generate it as part of the test script.
…ence, queue race) Blocking: - B1: delete screenshot scratch PNGs — observation frames reuse one fixed per-session path and unlink after every read (finally); browser_screenshot unlinks once the bytes are in the attachment store (kept only when the file itself is the model-facing artifact). - B2: replicate dsh's browser-trust fence on /bsk-observation/*: loopback Host only, Origin must match Host, sec-fetch-site: cross-site refused, POST requires application/json; README documents the loopback trust premise and the 0.0.0.0 warning. Major: - M1: KeyedExecutor tail chains previous+task (allSettled) so a task aborted while queued cannot release the next one into the running session. - M2: client tracks the live frame per session — replacing/removing/reset revokes the old blob URL at once; loads settling after replacement never resurrect it. Minor: drop replaced thumbRefs (+on remove); beginAction fires inside the queue (no label overwrite while queued, no idle flash); half-initialized session cleanup stops through the queue with one retry; SSE (re)open refetches /state (+ snapshot flag renamed subscribed); install probe uses --version (no daemon spawn); emulate mobile documents the width+height requirement (the daemon refuses it alone — verified). Nit: single BskRunOptions declaration; tails map entry dropped on drain; SSE cleanup on res close; popOut catches requestWindow rejections; resize handle gains keyboard control (arrows, 16px steps, aria value attrs). Tests: +14 (fence rules, scratch lifecycle, queue race, blob revocation, instrumentation timing, SSE resync). 113/113 green.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add
dsh-plugin-browserskill: DeepSeek Harness tools for BrowserSkillMotivation
DeepSeek Harness (dsh) is DeepSeek's open-source agent harness where everything is a Cordis plugin and tools are registered via
ctx.tools.register(defineTool({...})). This PR adds a new monorepo package,packages/dsh-plugin-browserskill, that exposes bsk's browser automation as model-visible dsh tools — so any dsh agent can drive a browser through BrowserSkill.The package is a standard dsh bundle (
dsh.bundlemanifest +cordis.patch.yml), installable withdsh plugin --profile <name> add <package>. It does not change any existing BrowserSkill code.Design
Each tool maps to one
bsk <cmd> --jsoninvocation: spawn the CLI → parse the structured JSON → return a canonical typed value (output.schemafor programmatic use vs.output.renderfor model-facing text, per the dsh tool contract).11 tools:
browser_session_start/browser_session_stop/browser_session_list/browser_navigate/browser_snapshot/browser_observe/browser_click/browser_fill/browser_press/browser_screenshot/browser_emulate.browser_session_startreturns the session id and makes it the current session; every operation tool takes an optionalsessionarg (explicit wins and becomes current; omitted falls back to current), and every result echoes the session it acted on. Concurrent starts go through a synchronous reservation protocol so the cap (maxSessions, default 5) holds even under parallel calls. Window size (--width/--height) and mobile device emulation (bsk emulate --device …) are exposed as tool parameters.sessionarg naming a foreign or unknown id is rejected before any command reaches the daemon,browser_session_listreturns only plugin-created sessions (no daemon-wide view), and stop/unload cleanup can never touch a session owned by another program.exec.signal) kills the underlying bsk child process, aligned with the cooperative cancellation model landed in fix(extension): make tool cancellation cooperative #89.browser_screenshotadditionally commits the PNG through the host attachment store and attaches the image itself only when an attachment service is mounted and the active model route declares image input (mirroringdsh-tool-fs read_image's gate); otherwise it returns the file path.Config):bskPath(defaultbskfrom PATH),defaultTimeoutMs(120s),maxSessions(5).code/message/hint) to the model; a missing bsk binary produces install guidance (also probed once at plugin activation).Verification
AbortError, JSON error-envelope mapping, screenshot attachment gating (path-only vs. inlined image), terminal card presenters.@deepseek-ai/dsh@0.1.0-rc.6): the bundle installs viadsh plugin add, its layer composes in--dump-config,applyruns at boot (verified through config override + install probe), and all 11 tools register. AdefineToolround-trip through the real@deepseek-ai/dsh-toolsregistry path (schema validation → execute → render) passes using a mockbskexecutable. A real-browser end-to-end run was not possible in the CI-like test environment (no Chrome/extension); the bridge layer is fully covered by the mocked tests above.CI note (maintainer action requested)
The pushing token lacks the
workflowscope, so this PR deliberately does not touch.github/workflows/. As a stopgap, the rootlintscript now also runs the new package'stypecheck+test, so the existing frontend CI job covers it. A maintainer with workflow scope may prefer to revert that one-line change and apply this instead:Follow-ups (not in this PR)
dsh-plugin-browserskill(currentlyprivate: true) and adding thedsh-plugintopic for discoverability.bsk record) throughctx.jobs.browser_console,browser_network,browser_evaluate,browser_get_html, tab management).Update: Web client half —
browser_screenshottoolview (6b96fcc)The package is now dual-face: a
dsh.clientdeclaration (platformweb) plusexports["./client"]→lib/client.js, built in dsh's closure-factory contract (window.__ModuleLoader__.loadhandoff;react,dsh-client-ui-*platform modules external and answered by the loader's frozen module table; everything else inlined; CSS Modules compiled by lightningcss; a build-time purity gate rejects cross-plugin value imports, mirroringpackages/client/tsdown.client.ts).The client half registers a keyed
tool.call.toolviewview forbrowser_screenshot(the same extension pointdsh-client-ui-skilluses for theskilltool):TerminalBlockprimitive) — no information the stock card showed is lost.readAttachmentRPC and renders it with the sharedMessageImagethumbnail +ImageLightboxatoms. Bytes never enter the session log or the page upfront — the reference is resolved on demand into a blob URL.browser_*tool stays on the stock terminal card; only thebrowser_screenshotkey is registered.docs/web-styling.md:--dsw-alias-*semantic tokens, CSS Modules, no component library, keyboard-focus andprefers-reduced-motionpreserved.Tests (+8, 50 total): view-model derivation (running / ok / error, image vs path-only), component rendering through @testing-library/react + happy-dom (image loads through the loader; path-only never touches it), and the keyed registration itself.
Verified live in the real Web UI (scripted OpenAI-compatible mock provider deciding tool calls; every tool executes against a real bsk + headless Chromium): the screenshot card shows the captured image inline in both collapsed and expanded states, the lightbox opens the original, the text-only route degrades to the path form, and the other tools' terminal cards are unaffected.
Update: live observation overlay (PiP mini-window) — host + client
The plugin now ships a live observation overlay for the dsh Web UI (Phases 1-A/1-B/1-C):
ObservationServicetracks one record per owned session (action,since,url,thumbnailAttachmentId,lastError,dead), instrumented at the sharedrunBskwrapper (tool entry/exit → action events; action end → immediate frame refresh). A throttled loop (1.5s active / 8s idle / 3-strike backoff) captures frames into the attachment store; observation traffic is fully isolated (no action events, never moves the current pointer) and — like every other command — flows through a per-session FIFO (KeyedExecutor), because the daemon accepts only one unfinished command per session. Edge states: a global failure streak flips anavailableflag (client shows "browser unavailable", keeps the last frame, greys interrupt); asession_not_foundenvelope marks the session dead.webServerroute seam:GET /bsk-observation/state,GET /bsk-observation/events(SSE),POST /bsk-observation/interrupt,GET /bsk-observation/thumbnail/<id>(routes mount viactx.inject(['webServer'], …), so headless compositions are unaffected). Frames are plugin-owned (no session-log reference), so the session-authorizedreadAttachmentRPC rightly refuses them — hence the plugin's own thumbnail route.shell.overlay(the sanctioned frame-wide seat): auto-appearing floating card (drag-move, corner resize min 240×180 / max 80% viewport, remembered for the page), collapsible capsule, focus view (status dot + session + action + ticking elapsed, breathing thumbnail with fade-in and failure badge), one-click Interrupt (no confirm, greys while settling, one-time semantics tooltip), multi-session meeting-style strip (per-session tile with hover interrupt, click-to-pin focus, red-edge errors never steal focus, dead sessions greyed), and Document-PiP pop-out (gesture-gated, inherits the card size, styles cloned,pagehidefalls back; unsupported browsers hide the button).observationEnabled(true),thumbnailIntervalMs(1500),idleIntervalMs(8000).