Skip to content

feat(app): surface browser tools as extensions - #1861

Merged
benjaminshafii merged 10 commits into
devfrom
feat/extension-browser-tools
May 22, 2026
Merged

feat(app): surface browser tools as extensions#1861
benjaminshafii merged 10 commits into
devfrom
feat/extension-browser-tools

Conversation

@benjaminshafii

@benjaminshafii benjaminshafii commented May 21, 2026

Copy link
Copy Markdown
Member

Summary

  • add OpenWork Browser and Chrome as extension catalog entries with real icons and extension detail content
  • add an Extensions section to the composer tool menu that inserts extension prompts without @ mentions
  • add lightweight enable/disable state for default OpenWork extensions so disconnecting removes them from the composer extension list

Tests

Local typecheck

  • pnpm --filter @openwork/app typecheck: all errors are pre-existing on dev (motion/react, react-resizable-panels, @heroicons, xlsx modules missing). No new errors introduced by this PR.

Daytona Electron test (sandbox: openwork-eval-20260520-185458)

Flow 1: Create workspace - PASS

  • Created sandbox, checked out feat/extension-browser-tools, installed deps
  • Started VNC + Vite + Electron services
  • CDP connected at https://9825-wfwvlrjxrefuym5a.daytonaproxy01.net
  • Welcome page rendered correctly
  • Injected folder path via React fiber dispatch
  • Workspace "hello" created, navigated to #/workspace/ws_b0a42e5bae15/session
  • opencode sidecar running on port 34935
  • Status bar shows "Ready for new tasks"

Extensions page - PASS

  • Settings > Extensions shows all catalog entries (Notion, Linear, Sentry, Stripe, Context7, OpenWork Cloud Control, OpenWork UI Control)
  • OpenWork Browser shows as "Connected" with correct icon and description
  • Chrome shows as "Connected" with correct icon and description
  • OpenAI Image Gen and Ollama also visible

OpenWork Browser detail modal - PASS

  • Type: OpenWork Extension, Status: Connected
  • "Ready by default" section renders correctly
  • Disconnect/Close buttons visible

Chrome detail modal - PASS

  • Type: OpenWork Extension, Status: Connected
  • Shows chrome://inspect/#remote-debugging instructions
  • "Debugging Port" configuration section visible
  • Disconnect/Close buttons visible

Composer tool menu - PASS

  • Tool menu shows Commands, Skills, Extensions, MCPs sections
  • Extensions submenu lists: OpenWork Browser, Chrome, OpenAI Image Gen, Ollama
  • Each entry shows icon and description

Browser panel - PASS

  • Clicking Browser icon opens BrowserView with Google homepage
  • Close browser panel button works correctly
  • BrowserView hides properly when modals are shown (hasNativeBrowserOccluder)

Right-pane extensions - PASS

  • Extensions icon in right sidebar opens inline settings pane
  • Tabs: Extensions, AI Providers, Permissions
  • Lists all available extensions with connect/connected status

"Connect an extension" starter card - PASS

  • Clicking navigates correctly to Settings > Extensions page

AI Providers (regression check) - PASS

  • AI Providers page renders correctly, shows 1 provider connected
  • Connect provider flow accessible

Electron logs - CLEAN

  • No errors in electron logs, only normal API calls (config, health, capabilities)

Commands run

pnpm --filter @openwork/app typecheck
daytona create --name openwork-eval-20260520-185458 --dockerfile .devcontainer/Dockerfile.daytona-vnc --context ... --class large --memory 8 --disk 10 --auto-stop 60 --public --target us
daytona exec ... -- "bash -lc 'cd /workspace && git checkout feat/extension-browser-tools && pnpm install'"
browser_list / browser_eval / browser_screenshot via CDP

@vercel

vercel Bot commented May 21, 2026

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
openwork-app Ready Ready Preview, Comment May 22, 2026 12:51am
openwork-den Ready Ready Preview, Comment May 22, 2026 12:51am
openwork-den-worker-proxy Ready Ready Preview, Comment May 22, 2026 12:51am
openwork-landing Ready Ready Preview, Comment, Open in v0 May 22, 2026 12:51am
openwork-share Ready Ready Preview, Comment May 22, 2026 12:51am

benjaminshafii and others added 8 commits May 21, 2026 16:58
- ensureBrowserPlugin() adds opencode-chrome-devtools to workspace opencode.jsonc
- resolveAgentTemplate() bakes the Electron CDP port into agent instructions
- embedded.ts passes OPENCODE_BROWSER_URL env to managed OpenCode sidecar
- Agent instructions now include explicit browser_url and tool names
@benjaminshafii

Copy link
Copy Markdown
Member Author

End-to-end test results (browser extension tools)

Fixes in this update

  1. workspace-init.ts: ensureBrowserPlugin() adds opencode-chrome-devtools to workspace opencode.jsonc — without this, browser tools were not available
  2. workspace-init.ts: resolveAgentTemplate() bakes http://127.0.0.1:{CDP_PORT} into agent instructions so the agent knows the exact browser_url parameter
  3. embedded.ts: passes OPENCODE_BROWSER_URL env to managed OpenCode sidecar

Test results (dev instance, pnpm dev app with OPENWORK_ELECTRON_REMOTE_DEBUG_PORT=9223)

Test Result Details
Test 1: Built-in Browser navigate PASS Agent used browser_listbrowser_navigate with correct browser_url: "http://127.0.0.1:9223". Browser panel navigated to example.com. Response: "Example Domain"
Test 2: Built-in Browser interaction PASS Full tool chain: browser_listbrowser_navigatebrowser_snapshotbrowser_evalbrowser_fillbrowser_evalbrowser_snapshot. Searched Google for "opencode ai", returned first result: "OpenCode | The open source AI coding agent"
Test 3: Chrome CDP Skipped Requires user Chrome with remote debugging enabled
Test 4: Extension toggle PASS Disabling OpenWork Browser via localStorage hides it from composer Extensions menu (4→3). Re-enabling restores it

Root cause of previous failure

The opencode-chrome-devtools plugin was npm-installed but not declared in the workspace opencode.jsonc, so OpenCode never loaded it. The agent fell back to curl/webfetch because browser_navigate etc. did not exist as tools.

- ensureBrowserPlugin now removes legacy MCP entries (openwork-browser,
  chrome, chrome-devtools, control-chrome) and adds the plugin
- Only touches OpenWork-owned workspaces or workspaces with legacy MCPs
- Preserves non-browser MCP entries (e.g. openwork-ui)
- Electron always exposes CDP on port 9223 (or OPENWORK_ELECTRON_REMOTE_DEBUG_PORT)
  so the browser plugin can connect in production builds
… evals

- Remove unused OPENCODE_BROWSER_URL from embedded server env
- Add evals/browser-extension-flows.md with 7 flows covering plugin
  loading, browser navigation, composer extensions, toggle, and
  stale MCP migration
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant