feat: agent tools — OpenAI Agents SDK, Claude Agent SDK, and any framework, local & cloud

[rejojer]

Stacked on #389 (the SDK with local mode). Adds an agent integration layer to the client: the PageIndex cloud MCP tool contract, runnable in-process against either mode.

What

One line per framework, identical across local and cloud — the mode is decided by the client constructor alone:

client = PageIndexLocalClient()                        # or PageIndexCloudClient(api_key=...)
client.submit_document("doc.pdf", wait=True)

# OpenAI Agents SDK        (pip install "pageindex[openai]")
agent = Agent(name="PageIndex",
              instructions=client.agent_instructions(),
              tools=client.as_openai_tools())   # local: in-process tools; cloud: full cloud MCP tool set
result = await Runner.run(agent, "What was total revenue this quarter, vs last year?")
print(result.final_output)

# Claude Agent SDK         (pip install "pageindex[claude]")
options = ClaudeAgentOptions(
    system_prompt=client.agent_instructions(),
    mcp_servers={"pageindex": client.as_claude_mcp()},
    allowed_tools=["mcp__pageindex__*"],
)
async for message in query(prompt="What was total revenue this quarter?", options=options):
    if isinstance(message, ResultMessage):
        print(message.result)

# Any other framework
tools = client.agent_tools()   # plain functions — local: built-in tools; cloud: full live tool set over MCP

Document QA, end to end

An actual run (local mode, OpenAI Agents SDK, over examples/documents/q1-fy25-earnings.pdf):

Q: What was Disney's total revenue in Q1 FY2025, and how did it compare to the prior-year quarter? Cite the page you found it on.

[tool] get_document_structure({"doc_name": "q1-fy25-earnings.pdf", "part": 1})
[tool] get_page_content({"doc_name": "q1-fy25-earnings.pdf", "pages": "1,3"})

A: Disney's total revenue in Q1 FY2025 was $24.7 billion, up 5% from $23.5 billion in the prior-year quarter.
Citations — Page 1: "Revenues increased 5% for Q1 to $24.7 billion from $23.5 billion in Q1 fiscal 2024"; Page 3: table shows $24,690M vs $23,549M, +5%.

This is the reasoning-based retrieval loop working as designed: the agent reads the tree structure first, picks tight page ranges, and answers strictly from tool output with page citations — no vector index, no chunking, and the retrieval "intelligence" is the host agent's own model (the navigation tools themselves make no LLM calls).

Design

• The tool surface is the cloud MCP contract: browse_documents / get_document / get_document_structure / get_page_content, doc_name-addressed, same input schemas, descriptions, and JSON response envelopes as the hosted MCP server's tools/list — agent prompts port unchanged between the cloud MCP connection and these in-process tools. tests/data/cloud_mcp_contract.json freezes the contract; a parity test guards drift.
• Local is an honest subset: tools that don't exist locally (folders, search_documents, get_document_image) are not registered, mirroring the server's gating semantics. Cloud-only parameters (folder_id, sort/query, recursive) are hidden from the local surface entirely — strict-schema frameworks then cannot express the dead-end calls, and the call_tool/MCP path still answers direct calls with a guided "works on PageIndex cloud" envelope as the backstop (lenient framework argument models drop the hidden params pre-call, degrading to the bare call). Local descriptions and instructions teach only that surface: the exposed schema is the contract minus the documented hidden set (mechanically asserted), and a dead-reference test keeps local guidance from naming cloud-only tools. remove_document is behind include_management=False.
• Cloud always serves the full cloud tool set: all three tool methods are polymorphic. On a cloud client, as_claude_mcp() hands the framework the remote MCP config, as_openai_tools() serves the full set as plain function tools from your process (any model backend; hosted=True opts into a hosted MCP tool with server-side execution on the Responses API, for OpenAI models), and agent_tools() discovers the live tools/list through a built-in minimal MCP client and synthesizes one plain function per tool (signatures and docstrings from the server's schemas, calls proxied from your process) — so the plan-gated tool set, including new server-side tools, arrives without an SDK release, and works from any framework or model backend. The default is the compatible path; hosted=True is the explicit latency optimization (the framework's own MCPServerStreamableHttp against api.pageindex.ai/mcp remains the async-native alternative). On a local client, all three serve the in-process contract subset. Because plain functions have no framework permission layer, agent_tools() applies its management gate in both modes: by default only tools the server marks read-only (readOnlyHint) are exposed, and include_management=True opens the complete list — the same switch that gates remove_document locally.
• Tools never raise — every failure returns the same {"error", "errorCode", "next_steps"} envelope the cloud emits, so agent behavior is uniform across frameworks.
• Zero new hard dependencies: openai-agents / claude-agent-sdk are imported at call time with actionable errors; pageindex[openai] / pageindex[claude] extras carry floor-only pins. import pageindex and every existing client feature work with neither installed (covered by tests).
• submit_document(wait=True): polls get_document status with growing intervals; returns on completed, raises on failed or after 30 minutes. The manual polling loop cloud callers write today spins forever on a failed document; default stays wait=False so batch submission is unaffected.
• agent_instructions(doc_id=None) supplies the retrieval playbook for the agent's system prompt. Cloud: the live instructions the MCP server serves for the key's tool set, captured from the initialize handshake over the same bridge session as agent_tools() — server-side guidance updates arrive without an SDK release, and an empty server response raises instead of silently substituting the subset text. Local: the built-in playbook for the in-process tools (structure-first over 20 pages, tight page ranges, persistence protocol) — a trimmed subset of the server's instructions, with a consistency test that every tool it names exists locally. doc_id (str or list, same shape as chat_completions) appends the target documents — in the run above it is what let the agent skip discovery and go straight to the named document.

Verification

• An independent adversarial review round (runtime probes against real framework versions, source bisects) surfaced and fixed: optional-no-default params now annotate Optional so strict schemas don't force values (browse.query was unusable for time-sort on the cloud+OpenAI path); a server annotation regression can no longer silently zero out the toolset (loud error instead); exec synthesis hardened (fixed internal def name + dict-literal args — tool/param names can no longer recurse or shadow builtins); SSE parsing handles CRLF multi-message bodies; transport failures wrap into PageIndexAPIError per the documented contract; hosted=True now routes non-read-only tools through the Responses API approval flow ({"never": {"read_only": true}}) instead of auto-approving everything; local get_document_structure serves the raw stored tree so nodes carry start_index/end_index like the cloud (live-verified shape); openai-agents floor raised to >=0.8.0 (older versions run sync tools inline on the event loop); tool annotations added to the frozen contract and passed to the Claude in-process server; wait= tolerates transient poll failures; failed documents get a real "processing failed" message instead of "still processing".
• 148 tests green + key-gated live tests that diff the frozen contract — descriptions, schemas, required, annotations — against the real server's tools/list, assert the server serves non-empty initialize.instructions, and check envelope field parity (every field the local tools emit exists in the live cloud response for the analogous call): contract parity vs the frozen snapshot, tool behavior against a seeded local store (no LLM calls), framework-missing/-installed behavior both ways, wait= completed/failed/timeout semantics.
• Live smoke against the real cloud MCP server: agent_tools() discovered this key's gated tool set (7 read-only tools; include_management=True adds remove_document; upload tools correctly absent per plan), a real browse_documents call returned the expected envelope, and a live parity check matched the frozen contract letter-for-letter on all shared tools — after catching and fixing one real bug (SSE responses decoded as latin-1 by requests' charset guess; SSE is UTF-8 by spec).
• The live QA run above, plus examples/agentic_vectorless_rag_demo.py rewritten to the new API (its inline tool definitions collapse into client.as_openai_tools()).

Follow-ups (not in this PR): a stdio pageindex-mcp entry point for non-Python MCP hosts (pageindex[mcp]), and the docs-site agent-integration page.

[rejojer]

Code review

Found 5 issues:

1. get_document_structure pagination sizes chunks with compact JSON but the envelope is emitted with indent=2, so parts overshoot TOOL_RESPONSE_CHAR_LIMIT (bug due to _serialized_size using json.dumps(value, separators=(",", ":")) against _CHAR_BUDGET, while call_tool returns _dumps(payload) with indent=2 at line 315). A 40x12 outline emits a 113,676-char part against the 100,000 limit; a 3000-node outline emits 123,831 chars with 4 of 5 parts over. The existing pagination test uses 4000-char summaries, the one shape where indent inflation is ~1%, so it passes while the guardrail is defeated.

PageIndex/pageindex/agent_tools.py

Lines 580 to 583 in 7d37c3f

	def _serialized_size(value: Any) -> int:
	return len(json.dumps(value, ensure_ascii=False))

2. sort="relevance" discards documents that score zero instead of ranking them lower, while the schema tells the model it performs semantic ranking (bug due to if score: scored.append((score, doc)) after a substring token count over name + description). A document named Q3-financials.pdf described "Revenue and profit for the third quarter" returns "documents": [] for query="quarterly earnings". _PERSISTENCE step 2 tells the agent to rephrase with synonyms, which a substring matcher cannot benefit from. Sorting zero-score documents last would satisfy the frozen contract's "orders" wording without touching the snapshot.

PageIndex/pageindex/agent_tools.py

Lines 657 to 671 in 7d37c3f

	if sort == "relevance":
	tokens = [token for token in (query or "").lower().split() if token]
	scored = []
	for doc in _all_documents(client):
	haystack = f"{doc.get('name') or ''} {doc.get('description') or ''}".lower()
	score = sum(1 for token in tokens if token in haystack)
	if score:
	scored.append((score, doc))
	# Stable sort: equal scores keep the newest-first listing order.
	scored.sort(key=lambda pair: pair[0], reverse=True)
	ranked = [doc for _, doc in scored]
	window = ranked[offset:offset + limit]
	has_more = offset + limit < len(ranked)
	else:

3. get_page_content expands every out-of-range page number into the message text (bug due to ", ".join(map(str, out_of_range)), while _format_page_spec — which compresses the same list — is used four lines below for requested_pages). On a 2-page document, pages="1-10000" returns a 59,461-char envelope of which 58,930 chars are comma-separated digits. This is additive on top of the content budget: with a full page payload the response reached 153,906 chars, 54% over TOOL_RESPONSE_CHAR_LIMIT. The same expansion exists on the success path at L1012-L1014.

PageIndex/pageindex/agent_tools.py

Lines 968 to 976 in 7d37c3f

	if out_of_range and not valid_pages:
	return _failure(
	f"All requested pages are out of range. Document has {max_page} "
	f"pages, but you requested pages: {', '.join(map(str, out_of_range))}",
	{
	"doc_name": doc_name,
	"max_pages": max_page,
	"requested_pages": _format_page_spec(out_of_range),
	},

4. The structure field alternates between a JSON array and a bare object across parts, and pagination is omitted on single-part responses (bug due to chunks.append(group if len(group) > 1 else group[0])). A document with one root node wrapping many sections returns structure as a dict on every part; a flat tree returns a list. The tool description tells the agent to iterate until pagination.has_more is false, but that key is absent whenever total_parts == 1. Line 623 already applies chunk if isinstance(chunk, list) else [chunk] for the inner nodes field — the same normalization never reached the response boundary.

PageIndex/pageindex/agent_tools.py

Lines 594 to 611 in 7d37c3f

	group_size = 0
	for node in nodes:
	size = _serialized_size(node)
	if size > budget:
	if group:
	chunks.append(group if len(group) > 1 else group[0])
	group, group_size = [], 0
	chunks.extend(_split_oversized_node(node, budget))
	continue
	if group and group_size + size > budget:
	chunks.append(group if len(group) > 1 else group[0])
	group, group_size = [], 0
	group.append(node)
	group_size += size
	if group:
	chunks.append(group if len(group) > 1 else group[0])
	return chunks or [structure]

5. as_claude_mcp(include_management=False) ignores the flag on the cloud path, and the shipped examples defeat the documented mitigation (bug due to build_claude_mcp returning the HTTP config without consulting include_management, which is read only on the local branch at line 57). Both sibling adapters do enforce it on cloud — agent_tools() filters on readOnlyHint and raises if the filter empties, and as_openai_tools(hosted=True) sets require_approval={"never": {"read_only": True}}. The docstring prescribes listing read tools in allowed_tools rather than the * wildcard, but both the docstring's own usage block and the README snippet ship allowed_tools=["mcp__pageindex__*"], which auto-approves mcp__pageindex__remove_document. The same copy-pasted snippet is read-only on a local client and delete-capable on a cloud one.

PageIndex/pageindex/integrations/claude_agent_sdk.py

Lines 15 to 23 in 7d37c3f

	def build_claude_mcp(client, include_management: bool = False):
	if getattr(client, "api_key", None):
	return {
	"type": "http",
	"url": f"{client.BASE_URL}/mcp",
	"headers": {"Authorization": f"Bearer {client.api_key}"},
	}

🤖 Generated with Claude Code

_{- If this code review was useful, please react with 👍. Otherwise, react with 👎.}