In-app browser panel with agent control - #88
Merged
Conversation
Threads enter the deck when they go live (working, awaiting input, pending approval, plan ready, background), have an unseen completion, or are pinned. Rows keep insertion order - activity only changes the status dot, never the position - and stay until dismissed, archived, or auto-trimmed (settled, seen, unpinned rows beyond the cap). Deck membership and dismissals persist in ui-state localStorage. Deck rows lead the jump-key and thread-traversal order, deduped against their project-tree rows.
- Add interactive HTML mockups comparing browser panel layouts and sidebar variations - Demonstrate expanded, collapsed, overlay, and deck-style sidebar states
The sidebar collapsed to nothing, so reclaiming width for a wide panel meant losing every signal about the threads you were not looking at. Collapse now lands on a 44px rail carrying the question the sidebar answers while you are heads-down in one thread: is anything else running, finished, or stuck? The rail is a second density of the same column rather than a companion to it — expanded shows the pane, collapsed shows the rail, never both — so nothing is rendered twice. On Deck rows keep their expanded order, so a dot never moves under the cursor, and an aggregate badge stands in for the per-row attention signal that collapsing takes away. Project glyphs are deliberately absent: per-project status is derived inside each project row, and duplicating that derivation for the rail would be worse than shipping without it.
With a wide panel open, source control can sit far from its toggle, and a bare icon says nothing about whether there is anything in the working tree worth opening it for. The toggle now carries its own status. The counts appear only while the panel is closed — once it is open it lists per-file counts and repeating the total is noise — and only for a dirty repository, so a clean tree stays visually quiet and an unloaded status never reads as "no changes".
A thread on deck rendered a second row under its project, so the two most prominent parts of the sidebar showed the same work twice and it read as duplicate threads rather than one thread in two places. Deck rows now own those threads outright. The filter runs before the preview window so a deck thread does not consume a preview slot and then vanish, which would leave "Show more" counts disagreeing with the rows on screen, and the jump-label pass filters identically so keyboard indices keep matching what is rendered. Two things deliberately keep using the unfiltered list: the project's aggregate status dot, because a project with a running thread is still busy wherever that thread is displayed; and the "no threads yet" empty state, because a project whose only threads are on deck does have threads and simply renders no rows of its own.
General Chats was a fixed place in the app wearing a project's clothes: a tree group that expanded, carried a thread count, and sat directly under On Deck. On Deck changes height as threads come and go, so that was the one slot in the sidebar that moved the most. Fixed navigation now lives in a band above the deck, where it can stay in the same pixel every time, and general chats reach it as a Chats destination with its own list view. The band is deliberately plain -- one line, no counts, no disclosure -- so it reads as chrome and never competes with the live work below it. The collapsed rail renders the same destinations as icons, so both densities offer the same places in the same order. The band holds one destination for now; Pull requests and Scheduled are meant to join it, which is why the list is data-driven rather than hand-rolled rows. Two consequences worth naming: general chat threads no longer take jump indices, because they no longer render a row in the tree; and the rail keeps its own New thread button rather than promoting "New chat" into the band, since the frequent action here starts a project thread, not a general chat.
A deck row shortens the title to fit 218px and the rail keeps nothing but a status dot, so the sidebar's most compressed surfaces were also the ones that said least about what they pointed at. Hovering a thread now restores exactly what the row had to drop: the full title, its status and last activity, the project it belongs to, its branch or worktree, the machine when the thread is remote, and the model. Built on Base UI's preview card rather than the tooltip, because a tooltip is meant to be a short string and this is a small surface with structure. Its open delay is longer than a tooltip's so the card does not fire while the pointer is only crossing the rail on its way somewhere else. Deck rows also gained the project favicon inline. The project name was already there, but as mono text it read as metadata rather than as the project, which made a cross-project deck harder to scan than it should have been. The model is only known once a thread's shell has loaded, so the card reports the provider until then, and it reports it the way the model picker does -- "Claude", never the internal "claudeAgent".
Two fixes to how the sidebar's top reads and behaves. The destination band had no boundary of its own. Every other block in the sidebar opens with a section label, so a lone row sitting in the gap between search and the deck read as something that had lost its section rather than as navigation. Measuring showed the rows were already aligned to the same grid, so the fix is the missing boundary, not indentation: the band now sits tight under search and a rule closes it, which groups the two as the top chrome they are. Hover cards now share a delay group per list. The delay exists to stop a card firing while the pointer is only crossing the list on its way elsewhere; once one card is open that intent is no longer in doubt, so re-imposing the wait on the next row just punishes deliberate scanning. Neighbouring rows now swap instantly and the group stays warm briefly after the last card closes. Measured: 346ms for the first card, 2ms for the next. That grouping is why these are tooltips rather than preview cards -- only the tooltip primitive ships the shared-delay provider. The trigger keeps an aria-label carrying the thread title, so the accessible name stays complete even though the card's structure is exposed as a tooltip's description.
On Deck is the working set for project work. A general chat belongs to no project and answers no question about a repo, so it has no business sitting between two threads that do -- it made the deck look like a mixed inbox rather than the small stable set it is meant to be. The deck sync now only ever sees project threads, and because the sync drops keys absent from its input, a chat already on the deck evicts itself without a migration. Also two adjustments from looking at it in use. The hover card's title was a size larger than everything below it and dominated a card whose job is to be scanned; the hierarchy now comes from weight and colour at a single size. And the destination band, having been given a rule last round, was still pinned against the search field with no air on either side -- it now has padding of its own so the rule reads as the edge of a band rather than a line drawn under a cramped row.
General chats are backed by a hidden project, and that implementation detail was surfacing everywhere the user looks: the crumb read "General Chats / New thread", a chat's hover card named the project as though it had been chosen, the recent list under a chat advertised project threads, and a terminal toggle offered to run commands in a checkout that does not exist. Chats now present as a mode. The crumb reads "Chats / New chat", the hover card omits the backing project, recent lists no longer cross the divide in either direction (a project draft stops advertising chats too), and the terminal control is absent rather than disabled -- absent because a chat has no checkout at all, where a project thread without a checkout keeps the disabled toggle that explains itself. Because chats no longer reach the deck, a running chat had nowhere left to signal from, so the Chats destination carries the aggregate status of the chats behind it. That is a badge on a band argued to hold none, but the alternative was activity with no representation anywhere in the sidebar, and the same "the control carries its own status" shape already serves the branch toggle and the rail's attention badge.
Removing the terminal from general chats rested on a false premise. The claim was that a chat has no checkout to run anything in, but general chats run in a scratch workspace, so the terminal was working the whole time and was useful there. It is back, and the prop added to gate it is gone rather than left behind always true. "Chats" also asked users to hold a distinction the word does not carry: a thread inside a project is a conversation too, so a list called Chats invites them to look for project work in it. "General chats" says the thing that actually separates them -- these belong to no project -- and it fits the sidebar without truncating.
The rail dropped the project tree entirely, so collapsing the sidebar meant losing every signal about projects you were not currently inside. Each project now keeps a glyph below the deck dots, carrying its favicon and the same aggregate status its expanded row shows; clicking one opens the pane with that project already unfolded rather than navigating somewhere unasked. That status was the reason the glyphs were cut in the first place: the derivation lived inside the project row and duplicating it would have let the two drift. It is now a shared helper both callers use, so the glyph and the row can only ever agree. The rail is also narrower than macOS traffic lights, so painting the rail colour behind them split the titlebar into two greys that met mid-button. The rail now starts below the titlebar in Electron, leaving one continuous strip across the top. Scoped to the left sidebar: the right panel is a collapsible container too and would otherwise have picked up the same gradient.
The previous attempt at this only looked correct in the computed style. The rail surface is painted by the sidebar's inner element, not the container the cutout was applied to, so the inner element went on covering the titlebar and the traffic lights kept sitting on rail grey. The cutout now applies where the colour is actually drawn. The rail's right border was the other half of it: a border spans the element's full height, so the edge ran up through the titlebar and met the yellow button. It is drawn as a background layer instead, starting at the same offset as the surface, leaving the top band uninterrupted across the whole window. One residual: the band ends at the titlebar height while the chat header's rule sits about three pixels higher, so the two horizontal edges do not quite meet at the corner. Aligning them means keying off --chat-header-bottom, which is measured at runtime and is zero until a chat header mounts -- on a route without one the rail would paint to the top again and bring the original problem back.
Four fixes to the general chats surfaces. The list read as a continuation of the page title: a heading, a subtitle, a rule, then rows of nearly the same weight. A count label now heads the list the way section labels do elsewhere, with room above it, and the row titles sit a shade back from the title so the hierarchy is legible rather than implied. Starting a chat also meant opening the destination first, which project rows never ask of you. The Chats row now carries the same hover-revealed action its project siblings do; the status dot yields that corner while the action is showing rather than the two stacking. The collapsed rail advertised a resize it cannot perform -- the drag handle was still mounted at a width the rail does not honour. It is hidden while collapsed, so the only thing on that edge is the border. That border is now macOS-only in its offset form. Only macOS draws window controls over the app's top-left corner; Windows and Linux put them on the right, so there the rail is free to run to the very top.
A project row shows a name and a count; the rail shows a favicon and nothing else. Neither says where the project lives, how much of it is moving, or when it last did anything -- which is most of what you want before opening one. Hovering a project now answers all three, in both densities. The card hangs off the row's sticky wrapper rather than the row button. The button is where dnd-kit puts its activator ref, and Base UI merges event handlers but explicitly does not merge refs, so triggering from the button would have quietly broken project drag-reorder. The wrapper carries no drag props and covers the same area. The rail's top edge now lands on the chat header's rule rather than the titlebar height, so the divider reads as one line across the window instead of stopping where the rail begins. That needed the header's measured position, so --chat-header-bottom now defaults to the titlebar height instead of zero: the rail stays correctly inset before any header mounts and on routes that have none, which zero would not have survived. The summary builder moved to Sidebar.logic where the pure sidebar logic lives and where it could be tested -- its counting, its notion of "live", and its empty-project case were all untested while it sat in the card.
The divider left an unpainted pixel exactly where it met the chat header's rule. The horizontal line was painted by the rail's inner element, which spans the content box, while the vertical edge was painted by the container starting below the divider -- so the one pixel belonging to both was claimed by neither. The divider is drawn on the container now, where it reaches across the edge strip. Hover cards were also grouped per list, so the delay reset whenever you crossed from one list to another: deck row to project row, or project to project, each waited again. There is now one group per sidebar surface, which is the honest scope -- the delay exists to survive a pointer crossing the sidebar, and the sidebar is one thing to cross. Measured: 359ms for the first card, 41ms to a project card, 36ms back to a thread.
Groundwork for the center split. The panel's open state, address and viewport preset are per-thread because what you are looking at belongs to the work you are doing: opening a second thread should not inherit the first one's page. The split ratio is deliberately not per-thread. It is a window preference, and columns that jump width when you switch threads would be worse than one ratio you set once. It clamps to a third either way, since a browser squeezed to a sliver is worse than a closed one.
The browser sits beside the chat rather than in the right rail: it is the other half of the work surface, not an annotation of it, and the rail belongs to source control. Both panes honour one ratio, stored as a fraction so the split survives a window resize, and clamped so a hard drag cannot collapse either side to a sliver. The preview is an Electron <webview> rather than an iframe. An iframe cannot be navigated, inspected or driven cross-origin, which is the entire point of the panel; a <webview> is a real Chromium tab that CDP can attach to later. Because it lives in the renderer rather than as a native view over the window, it stays inside normal CSS layout and dialogs still stack above it. It runs in one persistent partition, so signing in to the app you are building is something you do once rather than once per thread -- and so the agent will later drive a tab that is already signed in. That session is kept apart from the app's own, and preview content gets no Node, no preload and a short permission allowlist. Manual only for now: no agent control yet.
Verifying the webview in the desktop app showed it was running in the app's default session: shared cookies and storage with Threadlines itself, which is the isolation the panel was supposed to have. The cause was assigning params.partition in will-attach-webview. Electron resolves the guest's session from the element's attribute before that event fires, so the assignment silently did nothing while reading as though it had. The renderer now sets the attribute, and the main process refuses any attach that asks for a different partition -- enforcement rather than assignment, so the attribute is load-bearing instead of advisory. The constant is shared because both ends have to agree on it. Verified in the running desktop app: the preview loads a real page and reports its own partition directory, and a webview requesting another partition does not attach.
Chrome DevTools Protocol rather than executeJavaScript: it operates at the browser level instead of inside the page's JavaScript sandbox, so it is unaffected by same-origin policy, and the input events it will later dispatch are real ones -- hover, focus and blur behave as they do for a person, which synthetic DOM events do not. Console output and failed requests are collected from the moment a tab attaches rather than gathered on request, because by the time anyone asks what went wrong the interesting line has already been printed. Both buffers reset on main-frame navigation so they describe the page being looked at, and are capped so a chatty page cannot grow without bound. Tabs are addressed by webContents id. The renderer owns the <webview> and is the only side that knows which element belongs to which thread, so it registers the pairing and the main process stays an executor that cannot act on a tab nobody claimed. Verified in the running desktop app: attach succeeds, document.title and element text read back through CDP, a console log and error are captured, and a 404 is recorded with its status.
Electron writes security warnings straight into an unpackaged renderer's console, so they landed in the buffer whose whole purpose is telling an agent what the page under development is doing wrong. They are about how Electron hosts the page, not about the page. Filtered rather than silenced with ELECTRON_DISABLE_SECURITY_WARNINGS: that variable is process-wide and would also disable the warnings for Threadlines' own renderer, giving up a real safety net on our own code to tidy someone else's console. Matching the brand string is stable across versions, and if it ever changes a warning reappears in the buffer -- today's behaviour -- rather than anything breaking. Only warnings are dropped, never errors, so a page debugging its own Electron integration still sees everything it logged.
Network.loadingFailed carries only a request id, and it was being stored in the url field. A 404 hid the mistake, because that path arrives via Network.responseReceived which does carry an address; only failures below HTTP -- DNS, connection refused, a blocked port -- take the loadingFailed path, and those were reported as an opaque id. Request addresses are now remembered from Network.requestWillBeSent and resolved on failure, then forgotten, so the map cannot grow with the page. Also capture the Log domain. Runtime.consoleAPICalled only sees what the page itself printed, which misses the browser's own complaints: CSP violations, CORS refusals, blocked ports, mixed content. Those are precisely the failures a developer asks an agent about, and the page never logs them. Verified against a page whose requests fail both ways: the blocked-port request now reports http://127.0.0.1:9/x rather than a request id.
A snapshot from the accessibility tree, and clicks and typing addressed by the refs that snapshot hands back. Targeting by ref rather than by a selector string means there is nothing to parse and nothing ambiguous: the agent can only act on something it was actually shown. The tree is filtered to roles worth acting on, plus a few landmarks for orientation. A real page's full tree runs to thousands of generic containers, which would bury the handful of things that can be pressed. Nameless controls are dropped because an element that cannot be described cannot be chosen deliberately. Input goes through CDP rather than element.click(): hover, focus and blur fire as they would for a person, and handlers that check isTrusted behave the same way. Clearing a field uses the editing command directly. Dispatching Meta+A as a raw key does not work -- the browser resolves shortcuts to editing commands above the layer CDP injects at, so the chord selects nothing and the insert lands wherever the caret was. A click centres the caret, so the text arrived mid-word: old@examp + new@example.com + le.com.
A blank preview asks for the one thing nobody remembers: the port a dev server happened to pick. It now lists what is actually listening, so opening the thing you are working on is a click rather than a guess. Discovery parses lsof's -F field output. The human-readable table changes shape between versions and locales; the tagged format does not. Tags arrive in sets, so every address belongs to the process last named, and a process with no command clears the previous name rather than inheriting it -- attributing a port to the wrong program is worse than showing none. Ports listed once per address family collapse to one row, established connections are skipped since they are not listeners, and addresses this machine's browser cannot reach are dropped. Local by design: the preview is a browser on this machine, so a remote environment's servers would not be reachable from it anyway. Also adds screenshot capture, taken through CDP rather than capturePage so the image is the page itself even when the window is occluded.
Every tab keeps its own element, mounted whether or not it is showing. Switching tabs would otherwise tear down the guest and take the CDP attachment with it, losing the page and the console and network history collected from it -- and reloading on the way back, which is the one thing a preview must not do while you are watching something happen. Inactive tabs are hidden with visibility rather than display so the guest keeps its layout: a page collapsed to nothing reports a meaningless viewport to a screenshot or to an agent measuring an element. Only the visible tab drives the toolbar, so a background tab finishing a load cannot repaint controls describing a different page. Closing the active tab selects its right-hand neighbour, falling back to the left at the end of the strip, so closing several in a row walks along instead of jumping to an end. Closing the last one leaves a blank tab rather than an empty frame. Imperative webview calls are guarded: a guest rejects them until it has attached and fired dom-ready, so a tab opened and immediately pointed somewhere threw. Navigation happens anyway through the bound src, which is why the failure was invisible except in the console.
Docking to the bottom, expanding over the chat, and a menu for the things that belong to the whole tab rather than the page: open in the default browser, copy the address, developer tools, and clearing the preview's cookies. The layout controls sit with the tabs and the page controls stay in the toolbar, because they act on different things -- one moves the panel, the other moves the page. Dock side is a window preference like the split ratio, so it is remembered; expansion is not. Reopening the app to a hidden chat would look like the thread had vanished, which is a worse first impression than re-expanding. Clearing cookies reloads the tab afterwards: signing out is the point, and a page left on screen still looking signed in would suggest it had not worked. Developer tools open undocked, since the guest is a small pane and docking them into it would leave no page to inspect.
Three things, all visible the moment a dark page loaded. The guest was painted white. That was a literal bg-white on the element, so a page with a dark background sat inside a white frame and the empty state -- which overlays the guest -- rendered our light-on-dark text onto white, leaving it nearly unreadable. The guest now uses the app's background and the empty state paints its own, so neither depends on what the page underneath happens to be. The white line around a dark page was the same white showing at the element's edges past the page's own painting. Gone with it. Pages that respect prefers-color-scheme now follow the app rather than the OS, emulated through CDP so a page that ignores the query is left alone. A dark app hosting a stubbornly light page is the jarring part, and it is the app the page is embedded in. Docking to the bottom is removed. The bottom edge belongs to the terminal, and a browser that could take it made two things compete for one place. The browser splits the centre; that is the whole story.
A machine has many listeners that are not web servers, and showing them all buried the two that mattered. Measured on a real machine: a dev server answers 200 text/html, an AirPlay receiver answers 403 with no content type, and a Handoff daemon does not speak HTTP at all. Serving HTML is the line that separates them, so each candidate is now asked for a page and kept only if it returns one. Fourteen entries became five. That also fixes the labels. Three dev servers all called "node" are indistinguishable; their titles are exactly what you were looking for, so the page's title leads and the process name is the fallback. Probes are cached per port and pid, so a stable server is not re-fetched on every poll while a restart on the same port still updates within a few seconds. Redirects are followed, since a dev server pointing at its app shell is still a web server. The trade-off: a server that answers with JSON rather than HTML no longer appears. This panel opens pages, so that seems the right side to err on, but it is a real exclusion rather than an oversight. Also gives the status dot room; it was sitting against the row's edge.
Two doc comments describing constants that no longer exist, and one comment duplicated where the file was stitched back together. All of it invisible to the compiler and to lint, which is why it survived a typecheck, a lint pass and a full test run. Found by asking for a second pair of eyes on the restoration rather than trusting that the diff being the right size meant the diff was right.
Five days and twenty-eight commits behind. Three files conflicted and none of them badly: main deleted a draft-prompt helper we happened to have added a callback beside, reordered the recent-threads list to put in-flight threads first, and added a new-thread button to a project header we had wrapped in a hover card. Resolved toward main throughout. Their thread ordering replaces ours in that list -- putting working threads at the top is a deliberate change and our sort order was incidental to it -- and we keep only the scope filter, which is what that component is for. Their new-thread button keeps our hover card around it. Their focus-ring convention wins everywhere, including in three files this branch wrote after their sweep landed and which nothing had reconciled. Also here: effectiveCwd on two sidebar test fixtures, which main added to the thread projection while this branch was away, and the removal of resolveProjectStatusForThreads -- written early on this branch, imported once, never called. The project hover card reads a status the projection already carries.
The tool returned base64 inside a JSON string. Thousands of tokens for
something the model cannot look at -- in our own agent's words, strictly
worse than the tool not existing.
MCP has an image content type for exactly this, and a toolkit cannot
reach it: whatever a tool returns becomes structuredContent, so a picture
arrives as a description of a picture. The screenshot is registered by
hand instead, which means doing what the toolkit does internally and then
splitting the image out into a content block, leaving the dimensions
behind as text so the model can reason about size without being handed
the bytes twice.
Verified over the protocol rather than by reading the code:
structuredContent is now {width, height}, and the content carries an
image block of image/png.
The thread comes off the request's own fiber. A hand-registered handler
does not run under the router middleware that provides it, so reading it
from the surrounding scope would get whichever request happened to build
the layer -- one thread driving another thread's browser, which is the
thing the credential exists to prevent.
Also fixed, and worse than the thing I set out to fix: Codex refuses
bearer_token on a streamable HTTP MCP server, and refuses it by failing
to start the thread at all. Every Codex session on this branch was dead
on arrival and I had only ever tested with Claude. Its own config schema
takes http_headers, which is what we send now.
Two bugs with one cause: the tab was resolved fresh on every request, so whichever tab you had just clicked became the one the agent acted on, and a navigation moved the page out from under whatever you were reading. The agent now pins the tab it first acted in and keeps it. Automation never changes which tab is selected -- the agent moves its own page, not your view. The pin lets go only when the tab does, and then the next action picks up whatever you are actually looking at. Which leaves the other half: an agent working in a tab you cannot see is indistinguishable from a page misbehaving on its own. So the agent's tab carries a dot that pulses while a call is in flight, and a line under the toolbar says what it just did in words -- clicked "Sign in", not browser_click with a locator. Off its tab the line becomes the way over. Nothing follows for you; a mode that moved your selection would be the yanking this was built to stop. Verified the pin against a live agent: it snapshotted and navigated its own tab while the address bar stayed on the other one. The line and the dot are covered by tests rather than by driving the app, after the drive turned out to be measuring a second dev stack I had left running.
With source control and the browser both open, the chat column is what pays, and it is the column the work is in. The rail gives back around two hundred pixels and loses nothing: the destinations and the projects are all still there as glyphs. The restraint is the feature. It folds once, on the way in, and never fights you -- expand it yourself and it stays expanded, with both panels still open. Leaving the squeeze resets the arrangement, so the next time is a new situation rather than a continuation of the one you overruled. The rule is a pure function with the awkward cases written down, because the second unrequested collapse is the one that makes somebody stop trusting the control.
Asked to click something "in our browser", Codex drove the user's real Chrome, reported on a signed-in personal session, and said it was using the in-app browser while doing it. It was not being careless. Codex ships a bundled browser plugin whose skill file claims "in-app browser" as one of its own aliases and then says outright: do not use external MCP browser-control tools for this surface. Having claimed the words, it looked for the ChatGPT in-app browser, did not find one, and fell back to Chrome -- still calling it the in-app browser, which is why the message read as though it had done what was asked. A tool description cannot answer that, because the model commits to the skill before it reads the tool list. Developer instructions can: developer-role text outranks a skill file, which arrives as tool output. So the thread is told which surface the user is looking at, that Chrome and Codex's own browser are different surfaces that cannot see it, and which tools reach it. Deliberately narrow, and it disables nothing. Asked for Chrome, the model should still drive Chrome -- that is what those plugins are for. Turning them off was the first thing I tried and it is the wrong shape of answer: it fixes our problem by breaking somebody else's feature on a machine that is not ours. The tool descriptions now name the panel too, as insurance for the case where the model does read them first. Claude was measured on the same prompt and the same page before any of this: it clicked our panel correctly, so this changes nothing for it. The Codex side is reasoned from the skill's own text and is not yet confirmed against a live run.
"I can't access the Threadlines preview panel right now -- the in-app browser connection isn't available." It was telling the truth. The server was never there. MCP servers are read when the app server starts, the same as plugins. We were naming ours in the thread/start config overlay, which arrives long after that -- so every Codex thread began with no browser tools at all, and the model, correctly refusing to substitute Chrome, had nothing left to use. The same mistake twice in one evening, in the same place, for the same reason. We spawn one app server per thread, so the server is named in its argv instead, which is early enough. The credential goes through the environment rather than argv, because argv is readable by anything that can run ps. The thread-level path is gone rather than left as a fallback. It never worked, and two mechanisms where one is dead is how the next person loses an evening. What the user saw is the half that already worked: told which surface they meant, the model declined to reach for Chrome. That was the part that needed persuading. This is the part that needed plumbing.
A JSON-RPC notification has no reply, so the transport answered with an empty body -- but it answered 200 with content-type application/json, promising a JSON document that was not there. Codex took the promise seriously, tried to parse nothing, failed with "EOF while parsing a value", and abandoned the handshake. It registered threadlines_browser with zero tools, which is why the model kept saying the in-app browser was unavailable. It was. 202 Accepted is what the spec asks for on a notification and it says what is true: received, nothing to return. Measured before and after -- notifications/initialized moved from 200 to 202, and tools/list still answers 200 with all eleven. Diagnosed by asking the model in the running app to troubleshoot its own missing tools, which it did by reading the wire. Worth recording that the same fix was sitting in the competitor's source I had already read tonight, in a function whose name told me exactly what it was for. I did not connect it until somebody else found the failure it prevents.
The bundler bundles code and copies nothing, so the 311KB Playwright script under src/ never reached dist-electron, and the bundled main resolves it relative to itself. The file was only ever where the runtime was not looking. Every browser_snapshot failed with ENOENT. What makes that worth a commit message is how it presented. The model hit the error, said "the semantic page reader hit a local packaging error," fell back to screenshots and status reads, and answered the question correctly anyway. It clicked the right button and read the right word off the page. Nothing looked broken. The most capable tool we ship was missing for hours behind a graceful degradation. So the copy runs on every dev launch, because pack --watch clears its output directory and a rebuild restart would otherwise start an Electron whose assets had just been deleted -- and the release build refuses to stage an app without it, since a shipped binary would have failed the same quiet way. Four places needed to agree on where this file lives: the extractor that writes it, the runtime that reads it, the build step that copies it, and the packager that checks it. Two of them are repo scripts and two are in the desktop app, so the module moved to @threadlines/shared, which is how everything else here crosses that line. It was importing across package roots before, which typecheck had been failing on quietly under a stale cache.
Three complaints, one shape. It looked like everyone else's because it was painted like everyone else's: a hollow arrow with a coloured outline, which is what every system cursor on every platform looks like. Same silhouette now, inverted paint -- solid in our colour with a rim around it, round joins and caps instead of mitres, stroke under fill so the rim sits outside the shape rather than eating it. Familiar as a pointer, obviously not yours. It vanished after 1.4 seconds, which answered "did the click land?" only if you happened to be looking when it happened. It rests now, dimmed, and goes when the page navigates -- the coordinates are viewport-relative, so resting through a navigation would mean pointing confidently at the wrong thing. And it teleported, because it remounted on every action. One element now, gliding, with the duration scaled to the distance and clamped at both ends: below the floor it reads as a jump, above the ceiling it is still in flight after the page has changed. Typing moves it too. `type` was already resolving the same centre point as `click` and dropping it on the floor, so the pointer sat on the last thing clicked while the agent filled in a field somewhere else. The click ring runs once and unmounts rather than looping. Tailwind's ping is infinite, and a permanently animating SVG over a live webview is the exact shape of the idle churn that cost a quarter of a core per window the last time it got into this app.
There was no mouseMoved anywhere in this codebase. Press and release, and nothing in between. That ruled out every menu that opens on hover, every tooltip, every slider, and selecting text by dragging across it -- so when asked to select a word, the model reached for a DOM Range through evaluate, which touches no mouse at all and left the pointer sitting on the last thing it clicked. It said so plainly in its answer. It was right. browser_move dispatches a hover. browser_drag presses, steps ten interpolated moves, and releases. The intermediate moves are the feature: a press followed straight by a release produces no selection and no drag-and-drop, and every move carries buttons:1 or the page reads a hover where a drag was meant. A drag reports both ends, not just where it finished. The gesture is over by the time the renderer hears about it, so the mark replays it -- presses at one end, travels, releases at the other. The ring stays still while the button is down. A pulsing ring during travel would read as repeated clicking, and one continuous gesture is what happened. The activity line names both ends too. `dragged` on its own says nothing about what moved, and that line exists to answer exactly that. Threading written by codex against a spec; the shape of the gesture, the copy, and the replay are mine.
The pick outline sat about two pixels down and to the right of whatever it was pointing at. Measured in the app's own Chromium, against an element spanning 100..300 horizontally: the highlight ran 100..304, flush on the left and four pixels long on the right, and the same again vertically. A border. The overlay lives in a shadow root under `all:initial`, so box-sizing is content-box, and setting width to the element's width gives a border box four pixels wider than that. The frame grows outward on the right and bottom while overlapping the element on the left and top, and the eye reads the difference as the whole thing being nudged diagonally. Region members at 1.5px skewed by a different amount than a single pick at 2px, which is why it never looked consistently wrong. An outline occupies no space and surrounds the box evenly. Same in the reveal flash, which had it too. The drag band keeps its border and gets border-box instead, since that line belongs inside the rectangle you dragged rather than around it. Found by Will noticing it looked off, which it had since the day it was written.
Asked about a tab that had just been opened, the agent answered about a different one and said no other page existed. Both halves of that were our doing. It pinned itself to a tab so a turn in flight could not lose its page when you glanced elsewhere, and then never let go -- the pin only cleared when the tab closed. And it had no way to ask what else was open, so "there is no panel at that domain" was the honest report of everything it could see. Now: an idle agent releases its pin when you switch tabs, so its next action lands on the page in front of you. Mid-turn it still holds, which is what the pin was for. browser_tabs lists every open page with the one you are looking at and the one it acts on both marked. browser_select_tab moves it deliberately, and brings that tab to the front while it does. That last part is the rule worth keeping: the page the agent works on is the page you can see. An agent clicking something destructive on a tab nobody is watching is a bad way to find out we allowed it.
Clicking a link is the most common way a page navigates, and the mark was cleared the instant the URL changed -- so the one action you most want confirmed was the only one that left nothing behind. It looked broken, and it was the old "pops in and disappears" complaint wearing a new coat. It still cannot stay. The coordinates are viewport-relative, so once the page changes the mark would be pointing confidently at whatever now occupies that spot. But it can leave over four hundred milliseconds instead of within a frame, which reads as "that happened, and then the page changed" -- the true account. A fresh action outranks a fade still in flight.
Roughly 1.7x slower, and eased at both ends rather than only the far one. A pointer that starts at full speed and merely decelerates reads as something being dragged; gathering pace and then settling is how a hand moves, and that is most of the difference between smooth and animated. Short hops go from 140ms to 220, the long haul is capped at 560 rather than 320. The cap still exists and still matters: the point arrives only after the click has been dispatched, so a click that navigates has the page changing underneath a mark still on its way there. Slower is better to watch right up until the mark is reporting on a page that has gone.
Asked to drag across two words of a heading, the agent turned the drag
tool down and scripted a DOM Range instead. Its reasoning was right: drag
took elements and went centre to centre, and "Fire Alarm" is the first two
words inside a larger span, so there was no element whose centre was where
the selection had to start.
A mouse works in coordinates. Naming only elements quietly ruled out
everything that happens between them -- selecting part of a line, drawing
on a canvas, dropping onto empty space -- and left the tool looking
capable while being unable to do the thing it is named after.
Targets now include `{point: {x, y}}`, and the pointer operations take it.
Anything needing a real node refuses it by name rather than hit-testing
the coordinates, because whatever is under a point is as likely to be an
overlay or the body as the thing meant, and that is wrong in the way that
looks like it worked.
Typing facpmanuals.com in full, every time, for the sake of a browser that had already been there twenty times. Not history in the browser sense: no list to browse, nothing to manage, no UI at all. Just the part of it people actually use. Three letters and the rest is there, with the part you did not type selected -- so carrying on typing overwrites the guess and Enter accepts it, which is why a real address bar never feels like it is arguing with you. Ranked by visits first and recency second. The site you go to daily should beat the one you opened once an hour ago, and among sites you use equally the recent one is the one you mean. When the list fills it drops the lowest-ranked rather than the oldest, so a morning spent somewhere else cannot cost you the site you live in. Nothing under two characters, because a guess that changes on every keystroke reads as the field being possessed. Nothing while you are deleting, because deleting is a correction and putting the guess back makes Backspace look broken. Visits are recorded where navigation lands rather than where you aimed, so a redirect teaches it the real address.
Two things, both about the app having outgrown its own defaults. The agent's tab was being taken from it. Opening a tab in the gap between two of its own actions handed it the new one mid-task, because the release rule read "idle" from the last operation rather than the turn -- and every operation reports done the moment it finishes. It survived only because switching back handed it the right tab again. The rule is gone rather than corrected. It existed because the agent could not see other tabs when I wrote it, so trailing the user was the only way it could ever end up somewhere useful. It can list them now and move itself deliberately, which is the arrangement people already expect from tabs: yours are yours to browse, its tab is marked in the strip and one click from the activity line, and it goes only where it decides to go. And the first window was 1100x780, set when this was a chat with a sidebar. It now opens onto four columns, which at that size arrive already fighting for width -- so the first thing anyone does is drag it bigger. It opens at nine tenths of the screen instead, capped so a 6K display does not get a window the size of a wall and floored so a small laptop still gets something usable. Remembered sizes are untouched; this is only what happens on a machine that has never opened one, which is exactly what the dev build is every time it gets a fresh state directory.
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
github-actions
Bot
added
vouch:trusted
Three browser tests looked for a pinned thread's row in the project tree. It is not there, and that is the point of the deck: a pinned or running thread gets a deck row, and a thread on deck is drawn once rather than twice. They were asserting the shape of the sidebar this branch replaced. The hover-group test keeps its subject and loses its pin, since the row it is about only exists for an ordinary thread. The two that asserted pin and archive on a running thread's tree row are gone, replaced by one that checks what is true now: the thread moved to the deck rather than vanishing, and a run in progress offers no dismiss. Found by CI, not locally -- `vp run test` is the unit projects, and the browser project runs behind `test:browser`. Worth remembering before calling a branch green.
badcuban
added a commit
that referenced
this pull request
Jul 28, 2026
vp run test is the unit projects. The web browser tests live behind test:browser, CI runs them, and a branch called green on unit tests alone shipped three failures to PR #88 before CI caught it. The gap belongs in the task-completion list, where every agent reads before claiming done.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds an in-app browser panel that the agent can drive, and reworks the sidebar
around it.
The browser
A tabbed browser panel beside the chat, sharing the centre column. Tabs, an
address bar that finishes URLs you have typed before, a device toolbar for
sizing, and the page tools (annotate, region, draw, erase) that put a screenshot
and the elements you circled into the composer.
Agent control
Eleven-plus MCP tools reach the panel over an authenticated local HTTP endpoint:
snapshot, click, move, drag, type, press, scroll, evaluate, navigate, screenshot,
resize, tabs, select tab. Both Codex and Claude have driven it end to end against
a live page.
Two things worth calling out, because they are the load-bearing decisions:
The page is described by Playwright's own injected script, vendored at build
time from the installed
playwright-corerather than pasted in.scripts/extract-playwright-injected.tsextracts it, records a hash, and CI fails if the two drift. The alternative was
our own accessibility walk, which kept only named actionable nodes -- an agent
could see every button and not a word of what the page said. It runs in an
isolated world, so the page cannot read our handle or notice it is being
automated.
The agent is visible while it works. A pointer marks where it last acted,
travels between actions, replays drags as press-travel-release, and rests where
it finished. An activity line says what it just did in words. Its tab is marked
in the strip. The rule underneath all of it: the page the agent works on is a
page you can see it working on.
The sidebar
Icon-collapsible with a rail, a destinations band, and a diffstat chip on the
source control toggle. It folds to its rail on its own when both side panels
squeeze the chat, once, and never again after you overrule it.
Testing
vp fmt,vp lint,vp run typecheckand the full suite pass. Beyond unitcoverage, every browser tool has been exercised against a real page in the dev
app by both providers -- that is what turned up the ones that were shipped and
broken: an MCP notification answered with an empty 200 that Codex could not
parse, a vendored asset the desktop build never copied, and a drag tool that only
accepted elements when the thing it is named after needs coordinates.