Skip to content

skills: cache orchestrator resources per thread#28336

Merged
jif-oai merged 3 commits into
mainfrom
jif/thread-cache-remote
Jun 15, 2026
Merged

skills: cache orchestrator resources per thread#28336
jif-oai merged 3 commits into
mainfrom
jif/thread-cache-remote

Conversation

@jif-oai

@jif-oai jif-oai commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

Why

Hosted orchestrator skills are read through the remote MCP resource server. Within one thread, the same catalog or skill resource can be requested multiple times by prompt injection and the skills.list / skills.read tools. Re-fetching adds latency and can make those surfaces observe different remote contents during the same thread.

This is a follow-up to #28333: orchestrator skills remain limited to threads without a local executor, and those threads now get a stable per-thread view of the remote skill data they use.

What changed

  • Reuse the existing per-thread orchestrator catalog snapshot for skills.list and skills.read availability checks.
  • Cache successful orchestrator resource reads by authority, package, and resource so prompt injection and tool calls share the same contents.
  • Keep the cache memory-only and bounded to 100 resources and 8 MiB per thread.
  • Leave host and executor skill reads unchanged, and do not cache failed remote reads.

Verification

  • Extended the app-server MCP resource integration test to read the same hosted skill resource twice and verify that the remote server receives one read.
  • The same test verifies that catalog discovery and the selected skill's main prompt are each fetched only once per thread.

@jif-oai jif-oai changed the title feat: cache remote skills skills: cache orchestrator resources per thread Jun 15, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

self.orchestrator_catalog
.get_or_init(|| async {
initialize.await.unwrap_or_else(|err| SkillCatalog {
warnings: vec![err.message],
..Default::default()
})
})

P2 Badge Invalidate orchestrator skill cache on MCP refresh

The catalog is now stored in a per-thread OnceCell, but MCP resources are intentionally hot-swapped when account/plugin changes queue a refresh (Session::refresh_mcp_servers_inner installs a new manager, and the app-server plugin/account processors clear skill caches). If discovery happens before apps are available, or a plugin/account change alters codex_apps, subsequent turns keep serving the stale empty/old catalog for this thread instead of observing the refreshed server. Please key this cache by a manager generation or clear it during refresh.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread codex-rs/ext/skills/src/state.rs Outdated
@jif-oai

jif-oai commented Jun 15, 2026

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown
Contributor

Codex Review: Didn't find any major issues. Another round soon, please!

Reviewed commit: 173510103e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@jif-oai
jif-oai merged commit 0afe559 into main Jun 15, 2026
31 checks passed
@jif-oai
jif-oai deleted the jif/thread-cache-remote branch June 15, 2026 18:20
@github-actions github-actions Bot locked and limited conversation to collaborators Jun 15, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant