Skip to content

feat(context): add controllable OpenCode prompt observability - #3029

Draft
reachjalil wants to merge 5 commits into
devfrom
feature/context-observability-skill-visibility
Draft

feat(context): add controllable OpenCode prompt observability#3029
reachjalil wants to merge 5 commits into
devfrom
feature/context-observability-skill-visibility

Conversation

@reachjalil

@reachjalil reachjalil commented Jul 23, 2026

Copy link
Copy Markdown
Collaborator

Overview

This is the consolidated feature PR for controllable OpenCode context and prompt observability in OpenWork. Current head 4453a4da9 contains upstream/dev at 3da9003e5 through merge commit ef0039492.

The original integration combined three implementation slices:

Since then, dev independently merged the definitive skill architecture and marketplace fix. This PR now treats those changes as its base and keeps only the complementary observability, diagnostics, and contract hardening described below.

Already on dev — not owned by this PR

#3025: single-skill-plugin architecture

  • Removes local and Den Skill Hub concepts and routes.
  • Migrates standalone skills to one-skill plugins on the plugin/config-object architecture.
  • Uses only plugin:<pluginId>:<configObjectId> for remote marketplace skill execution.
  • Removes the obsolete skill: execution rail, hub browsing/install UI, tables, and cloud skill catalog.

#3030: marketplace skill discovery fix

  • Preserves human title, normalized description, marketplaceName, and pluginName after authorization filtering.
  • Publishes every authorized descriptor and renders every skill; the former 100-entry and 32,000-character prompt limits are gone.
  • Tells the agent to call the exact listed capability and retry that same capability once for transient HTTP 502/503/504 failures.
  • Escapes catalog values and marks catalog/retrieved skill content as untrusted.

Functionality this PR adds

1. Ordered context contributor registry

  • Replaces independent OpenWork context plugins with one ordered contributor registry and bundled context plugin.
  • Gives each contributor a stable identity, ordering, include/omit/failure outcome, timing, size, and hash.
  • Correlates contributor output with the final post-system-hook array and reports text correspondence without claiming causal provenance OpenCode does not expose.
  • Adds fixture-backed baseline tests for the complete prepared system-block sequence.

2. Controllable observability levels

  • Supports off, metadata, and exact modes with fail-closed precedence.
  • Desktop Developer Mode enables metadata observability.
  • Exact prepared-prompt content requires a separate explicit consent toggle and a managed-engine restart.
  • Turning Developer Mode off revokes exact consent.
  • Managed engine bootstrap records when the engine is instantiated and which observability mode was applied.

3. Safe Electron DevTools bridge

  • Mirrors only allowlisted OpenWork observability records to the renderer console.
  • Metadata mode exposes event names, hashes, sizes, contributor outcomes, block deltas, and MCP lifecycle transitions.
  • Exact mode can show prepared prompt content only after explicit consent.
  • URLs, credentials, payloads, raw exceptions, MCP error text, and arbitrary child-process output are excluded.

4. Prompt and system-array change tracing

  • Records when contributor output changes and when the final system array changes.
  • Emits stable hashes and character counts for each block and the final prepared array.
  • Produces per-trace provenance rows so an operator can follow a block from contributor decision to final text correspondence.
  • Logs MCP status changes and prompt preparation without relying on uncontrolled console.log calls.

5. Resilient Connect skill-index diagnostics

  • Establishes a shared browser-safe OpenWork MCP skill://index.json contract used by Den and the server.
  • Parses descriptors independently so one malformed entry cannot hide valid neighboring skills.
  • Reports bounded, sanitized failure classes including response-too-large, invalid UTF-8, unsupported schema/envelope, all-invalid, partial filtering, JSON-RPC phase failure, cache state, and candidate selection.
  • Uses a bounded stale-while-revalidate cache and deterministic candidate priority without mutating server-scoped Connect state during reads.
  • Preserves fix(connect): preserve marketplace skill discovery metadata #3030's complete catalog behavior: all authorized skills are rendered with no 100-entry or 32k prompt cap. The independent 512 KiB MCP response safety limit remains explicit and observable.

6. Core plugin visibility and ownership

  • Exposes OpenWork-managed core plugins in the effective plugin inventory.
  • Deduplicates stale runtime/config copies against core identities.
  • Shows core ownership in Settings and prevents removing managed core plugins.

7. Operator surfaces and documentation

  • Adds Developer Mode controls for metadata and exact tracing.
  • Extends agent-context diagnostics with the effective observability state.
  • Documents the event vocabulary, safety model, prompt tracing workflow, and remote-skill debugging chain in:
    • docs/context-observability.md
    • docs/skills-visibility-runbook.md

Reconciliation with latest dev

The sync deliberately follows #3025's plugin-only architecture and #3030's complete-catalog behavior:

  • obsolete hub-skill code and skill-capabilities.ts remain deleted;
  • marketplace descriptors come from the plugin/config-object rail;
  • producer and prompt entry/character truncation were removed;
  • fix(connect): preserve marketplace skill discovery metadata #3030's exact capability and one-retry guidance is retained;
  • its direct raw Developer Mode skill logger is superseded by this PR's controlled metadata/exact observability framework;
  • shared schema validation, per-entry resilience, sanitized diagnostics, and the transport envelope guard remain.

Validation after syncing with dev

  • Complete clean-source server suite: 633 passed, 2 skipped, 0 failed
  • Server focused regression set: 35 passed, 0 failed
    • Connect catalog and complete-catalog rendering
    • shared skill-index schema parity
    • core plugin ownership/immutability
  • Den MCP focused set: 14 passed, 0 failed
    • standards-shaped index
    • 150-skill complete producer regression
    • resource registration and capability contracts
  • App focused set: 6 passed, 0 failed
    • core plugin visibility
    • Developer Mode metadata/exact controls
  • apps/server, apps/app, and ee/apps/den-api TypeScript checks: passed
  • Linux and macOS OpenWork workflows: passed
  • Electron IPC typecheck and e2e in both platform workflows: passed
  • Den API, Den web, and inference container builds: passed
  • Helm validation, i18n audit, and all Vercel preview contexts: passed
  • diff whitespace/conflict-marker checks: passed

The first post-sync CI run found one stale assertion that still expected the pre-#3030 prompt wording. Commit 4453a4da9 updates that test only; the full source suite and all fresh GitHub checks pass with the reconciled wording.

Root cause addressed

OpenWork previously assembled context through independently registered plugins without a safe way to correlate contributor decisions to the final prepared system array. Operators could not distinguish absent, filtered, malformed, cached, or changed context from a prompt that OpenCode actually received. The feature centralizes assembly, gives every decision a traceable identity, and exposes only the level of detail explicitly enabled by the desktop operator.

…3026)

* feat(context): centralize OpenCode observability

* docs(context): add observability and skill triage guides
)

* feat(context): centralize OpenCode observability

* docs(context): add observability and skill triage guides

* refactor(observability): remove duplicate prompt plumbing

* fix(connect): harden marketplace skill index handling
…3028)

* feat(context): centralize OpenCode observability

* docs(context): add observability and skill triage guides

* refactor(observability): remove duplicate prompt plumbing

* fix(connect): harden marketplace skill index handling

* fix(connect): preserve marketplace skill metadata

* test(context): update remote skill prompt contract
@vercel

vercel Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
openwork-app Ready Ready Preview, Comment Jul 23, 2026 6:52am
openwork-den Ready Ready Preview, Comment Jul 23, 2026 6:52am
openwork-den-worker-proxy Ready Ready Preview, Comment Jul 23, 2026 6:52am
openwork-landing Ready Ready Preview, Comment, Open in v0 Jul 23, 2026 6:52am
1 Skipped Deployment
Project Deployment Actions Updated (UTC)
openwork-diagnostics Skipped Skipped Jul 23, 2026 6:52am

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant