Skip to content

Add cross-SDK RPC E2E coverage#1424

Merged
stephentoub merged 7 commits into
mainfrom
stephentoub/e2e-coverage-gaps
May 26, 2026
Merged

Add cross-SDK RPC E2E coverage#1424
stephentoub merged 7 commits into
mainfrom
stephentoub/e2e-coverage-gaps

Conversation

@stephentoub
Copy link
Copy Markdown
Collaborator

Why

The SDKs expose a large public RPC surface, but E2E coverage had gaps across several server and session RPC groups. This PR adds meaningful non-Canvas RPC E2E coverage across the supported SDKs so regressions in typed RPC behavior are caught consistently.

What changed

  • Adds and expands C# E2E tests for server/session RPCs, event log, queue, remote, schedule, workspace checkpoints, commands, permissions, MCP/skills, tasks/handlers, and compaction.
  • Ports the same non-Canvas coverage strategy to Node/TypeScript, Python, Go, and Rust.
  • Uses assertions for stable semantics: return shapes, state changes, event delivery, idempotent/no-op behavior, handler round trips, and capability-gated error paths.
  • Leaves Canvas coverage and Java intentionally out of scope.

Validation

  • C#: build passed; non-Canvas RPC coverage set passed 5 consecutive clean runs after the fix-first review.
  • Node/TypeScript: targeted Vitest E2E suite passed 5 consecutive runs; ESLint passed for changed tests.
  • Python: targeted pytest suite passed; repeated stress runs completed 3 consecutive passes before PR creation was requested.
  • Go: gofmt, compile smoke test, and targeted E2E suites passed.
  • Rust: cargo +nightly-2026-04-14 fmt --check, cargo check --all-features --tests, and targeted E2E filters passed.
  • git diff --check passed.

Notes

Some remote and MCP app paths are capability-gated in the local E2E environment, so those tests assert stable implemented errors or no-op result shapes rather than pretending unavailable infrastructure exists.

Add non-Canvas RPC E2E coverage across C#, Node, Python, Go, and Rust. The tests exercise server and session RPC surfaces with assertions for stable return shapes, state transitions, events, no-op behavior, and capability-gated error paths.

Java is intentionally excluded.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings May 26, 2026 02:44
@stephentoub stephentoub requested a review from a team as a code owner May 26, 2026 02:44
Comment thread python/e2e/test_rpc_queue_e2e.py Fixed
Comment thread python/e2e/test_rpc_server_e2e.py Fixed
Comment thread python/e2e/test_rpc_server_e2e.py Fixed
@github-actions

This comment has been minimized.

Comment thread dotnet/test/E2E/RpcSessionStateE2ETests.cs Fixed
Comment thread dotnet/test/E2E/RpcMcpAndSkillsE2ETests.cs Fixed
Comment thread dotnet/test/E2E/RpcRemoteE2ETests.cs Fixed
Comment thread dotnet/test/E2E/RpcRemoteE2ETests.cs Fixed
Comment thread dotnet/test/E2E/RpcWorkspaceCheckpointsE2ETests.cs Fixed
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR expands end-to-end JSON-RPC coverage across the SDKs (C#, Node/TS, Python, Go, Rust) by adding/porting non-Canvas RPC E2E tests that validate typed request/response shapes, stable semantics (no-ops/idempotency), and capability-gated error behavior.

Changes:

  • Added new cross-SDK E2E tests for additional RPC groups (event log, queue, remote, schedule, workspace checkpoints, compaction handoff summary, commands, MCP apps/skills, and expanded tasks/UI handlers).
  • Updated existing E2E harness/helpers (Node wait helper; Python harness directory prep) to support more reliable multi-RPC tests.
  • Extended existing RPC E2E suites (tasks/handlers, permissions, server-scoped RPC, MCP/skills) with additional typed coverage paths.
Show a summary per file
File Description
rust/tests/e2e/rpc_workspace_checkpoints.rs New Rust E2E coverage for workspace checkpoints/diff/large paste.
rust/tests/e2e/rpc_tasks_and_handlers.rs Expands Rust tasks + UI pending-handler RPC coverage.
rust/tests/e2e/rpc_schedule.rs New Rust E2E coverage for session.schedule RPC.
rust/tests/e2e/rpc_remote.rs New Rust E2E coverage for session.remote RPC and steerable persistence.
rust/tests/e2e/rpc_queue.rs New Rust E2E coverage for session.queue RPC.
rust/tests/e2e/rpc_mcp_and_skills.rs Expands Rust MCP apps + skills RPC coverage.
rust/tests/e2e/rpc_event_log.rs New Rust E2E coverage for session.eventLog RPC.
rust/tests/e2e/compaction.rs Adds Rust E2E coverage for handoff summary + compaction cancel/abort.
rust/tests/e2e/commands.rs Adds Rust E2E coverage for commands list/invoke/execute/queue response.
rust/tests/e2e.rs Registers newly added Rust E2E modules.
python/e2e/testharness/context.py Ensures test harness base directories exist before cleanup.
python/e2e/test_rpc_workspace_checkpoints_e2e.py New Python E2E coverage for workspace checkpoints/diff/large paste.
python/e2e/test_rpc_tasks_and_handlers_e2e.py Expands Python tasks + UI pending handler + elicitation/auto-mode coverage.
python/e2e/test_rpc_server_e2e.py Expands Python server-scoped RPC coverage (sessions, secrets, sessionFs, etc.).
python/e2e/test_rpc_schedule_e2e.py New Python E2E coverage for schedule RPC.
python/e2e/test_rpc_remote_e2e.py New Python E2E coverage for remote RPC + persisted steerable.
python/e2e/test_rpc_queue_e2e.py New Python E2E coverage for queue pending items + mutations.
python/e2e/test_rpc_mcp_and_skills_e2e.py Expands Python MCP/skills and MCP apps host-context/capability tests.
python/e2e/test_rpc_event_log_e2e.py New Python E2E coverage for event log read/tail/interest/long-poll.
python/e2e/test_rpc_commands_e2e.py New Python E2E coverage for commands list/invoke/execute/queued response.
nodejs/test/e2e/rpc_workspace_checkpoints.e2e.test.ts New Node E2E coverage for workspace checkpoints/diff/large paste.
nodejs/test/e2e/rpc_tasks_and_handlers.e2e.test.ts Expands Node tasks + UI pending handler coverage + direct auto mode switch.
nodejs/test/e2e/rpc_server.e2e.test.ts Expands Node server-scoped RPC coverage and adds polling helper usage.
nodejs/test/e2e/rpc_schedule.e2e.test.ts New Node E2E coverage for schedule RPC.
nodejs/test/e2e/rpc_remote.e2e.test.ts New Node E2E coverage for remote RPC + persisted steerable.
nodejs/test/e2e/rpc_queue.e2e.test.ts New Node E2E coverage for queue pending items + mutations.
nodejs/test/e2e/rpc_mcp_and_skills.e2e.test.ts Expands Node MCP/skills and MCP apps host-context/capability tests.
nodejs/test/e2e/rpc_event_log.e2e.test.ts New Node E2E coverage for event log read/tail/interest/long-poll.
nodejs/test/e2e/permissions.e2e.test.ts Adds Node E2E coverage for permission paths/state/locations/folder trust.
nodejs/test/e2e/harness/sdkTestHelper.ts Adds reusable waitForCondition helper for polling-based assertions.
nodejs/test/e2e/compaction.e2e.test.ts Adds Node E2E coverage for handoff summary and compaction cancel/abort.
nodejs/test/e2e/commands.e2e.test.ts Adds Node E2E coverage for commands list/invoke/execute/enqueue/respond.
go/internal/e2e/rpc_workspace_checkpoints_e2e_test.go New Go E2E coverage for workspace checkpoints/diff/large paste.
go/internal/e2e/rpc_schedule_e2e_test.go New Go E2E coverage for schedule RPC.
go/internal/e2e/rpc_remote_e2e_test.go New Go E2E coverage for remote RPC + persisted steerable.
go/internal/e2e/rpc_queue_e2e_test.go New Go E2E coverage for queue pending items + mutations.
go/internal/e2e/rpc_event_log_e2e_test.go New Go E2E coverage for event log read/tail/interest/long-poll.
go/internal/e2e/rpc_coverage_helpers_test.go Adds shared Go E2E helpers for pointers, paths, and polling.
go/internal/e2e/permissions_e2e_test.go Expands Go permission RPC E2E coverage (paths/state/locations/folder trust).
go/internal/e2e/compaction_e2e_test.go Expands Go compaction E2E coverage (handoff summary, cancel/abort).
go/internal/e2e/commands_and_elicitation_e2e_test.go Expands Go commands E2E coverage (list/invoke/execute/enqueue/respond).
dotnet/test/E2E/RpcWorkspaceCheckpointsE2ETests.cs New .NET E2E coverage for workspace checkpoints/diff/large paste.
dotnet/test/E2E/RpcTasksAndHandlersE2ETests.cs Expands .NET tasks + UI pending handlers + elicitation/auto-mode coverage.
dotnet/test/E2E/RpcScheduleE2ETests.cs New .NET E2E coverage for schedule RPC.
dotnet/test/E2E/RpcRemoteE2ETests.cs New .NET E2E coverage for remote RPC + persisted steerable.
dotnet/test/E2E/RpcQueueE2ETests.cs New .NET E2E coverage for queue pending items + mutations.
dotnet/test/E2E/RpcMcpAndSkillsE2ETests.cs Expands .NET MCP/skills and MCP apps host-context/capability tests.
dotnet/test/E2E/RpcEventLogE2ETests.cs New .NET E2E coverage for event log read/tail/interest/long-poll.
dotnet/test/E2E/PermissionE2ETests.cs Expands .NET permission RPC E2E coverage (paths/state/locations/folder trust).
dotnet/test/E2E/CompactionE2ETests.cs Expands .NET compaction E2E coverage (handoff summary, cancel/abort).
dotnet/test/E2E/CommandsE2ETests.cs Expands .NET commands E2E coverage (list/invoke/execute/enqueue/respond).

Copilot's findings

  • Files reviewed: 61/61 changed files
  • Comments generated: 4

Comment thread rust/tests/e2e/compaction.rs
Comment thread rust/tests/e2e/compaction.rs
Comment thread rust/tests/e2e/compaction.rs
Comment thread rust/tests/e2e/rpc_queue.rs Outdated
Apply Ruff formatting and import ordering to the new Python RPC E2E tests so the Python SDK checks pass.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions

This comment has been minimized.

Relax model switch assertions for runtime differences, remove unused Go E2E helper, align Rust fork prompt with the replay snapshot, and normalize Windows permission paths.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Comment thread dotnet/test/E2E/RpcSessionStateE2ETests.cs Fixed
@github-actions

This comment has been minimized.

Use SendAndWaitAsync for the compaction setup turn so the test waits through the SDK send path instead of racing metadata polling and assistant event delivery.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions

This comment has been minimized.

Give the Go SDK race test run enough time for the expanded E2E suite on slower Windows runners.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions github-actions Bot mentioned this pull request May 26, 2026
Tighten review-commented RPC E2E tests across .NET, Python, and Rust by narrowing exception handling, making nullability expectations explicit, avoiding Python static-analysis false positives, and fixing Rust test artifact collisions and queue assertions.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Comment thread dotnet/test/E2E/RpcSessionStateE2ETests.cs Fixed
Comment thread dotnet/test/E2E/RpcRemoteE2ETests.cs Fixed
@github-actions

This comment has been minimized.

Make the context host type assertion explicit and document the intentional remote RPC fallback path.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions
Copy link
Copy Markdown
Contributor

Cross-SDK Consistency Review ✅

This PR adds E2E test coverage across .NET, Go, Node.js, Python, and Rust — no changes to SDK public API surfaces. From a cross-SDK consistency standpoint, there are no issues to flag.

Coverage symmetry: The new rpc_* test files (event log, queue, remote, schedule, server, session state, tasks/handlers, workspace checkpoints, commands, MCP/skills) were added for all five languages. Python's test_compaction_e2e.py and test_permissions_e2e.py already pre-existed and weren't in scope here, which is consistent with how Rust's permissions.rs was also not touched. Java is explicitly excluded as noted in the PR description.

API parity: All RPC methods exercised by the new tests (e.g., tasks.get_progress, tasks.send_message, ui.handle_pending_user_input, ui.register_direct_auto_mode_switch_handler) exist symmetrically across all SDK implementations.

No cross-SDK consistency concerns found.

Generated by SDK Consistency Review Agent for issue #1424 · ● 5.3M ·

@stephentoub stephentoub merged commit ac6de48 into main May 26, 2026
45 checks passed
@stephentoub stephentoub deleted the stephentoub/e2e-coverage-gaps branch May 26, 2026 14:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants