Skip to content

feat: add MCP 2025-11 conformance validation - #21

Merged
Esquetta merged 20 commits into
mainfrom
feature/v1.51-mcp-conformance-design
Jul 24, 2026
Merged

feat: add MCP 2025-11 conformance validation#21
Esquetta merged 20 commits into
mainfrom
feature/v1.51-mcp-conformance-design

Conversation

@Esquetta

Copy link
Copy Markdown
Owner

Summary

  • add deterministic, version-aware MCP 2025-11 conformance evaluation
  • add explicit read-only runtime probing with bounded tasks/list pagination
  • extend scorecards, reports, rule catalog, output contract, and public docs
  • harden runtime cwd containment and redact task/error data

Verification

  • npm test: 655 passed, 6 skipped
  • npm run build
  • npm run release-check
  • git diff --check
  • privacy and prohibited-method scans

@qodo-code-review

Copy link
Copy Markdown

Qodo reviews are paused for this user.

Troubleshooting steps vary by plan Learn more →

On a Teams plan?
Reviews resume once this user has a paid seat and their Git account is linked in Qodo.
Link Git account →

Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center?
These require an Enterprise plan - Contact us
Contact us →

@Esquetta
Esquetta merged commit 577a151 into main Jul 24, 2026
1 check passed

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 617c13717e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/core/runtime-probe.ts
Comment on lines +1640 to +1644
"tools/call",
{
name: callableTool.tool.name,
arguments: callableTool.args
},

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Skip required task tools during plain probes

When a 2025-11 server exposes only tools with execution.taskSupport: "required", that declaration is accepted by the new conformance evaluator when capabilities.tasks.requests.tools.call is present, but this path still sends a normal tools/call containing only name and arguments. Servers that require task augmentation are allowed to reject that non-task call, so the doctor will report plugin.runtime.tool_call.invalid for an otherwise valid task-only tool; skip required tools for the plain safe probe or use only an optional/non-task tool here.

Useful? React with 👍 / 👎.

Comment thread src/core/runtime-probe.ts
Comment on lines +1374 to +1377
if (
!Array.isArray(tasks) ||
!tasks.every(isPlainObject) ||
nextCursor === "__invalid__"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Validate task entries before passing tasks/list

When a declared tasks/list returns tasks: [{}] or entries with non-string taskId/invalid status, this condition still treats the page as valid because it only checks that each entry is an object. The new conformance scorecard will report tasksList: pass, so malformed task catalogs slip through the 2025-11 validation; validate the required Task fields and status values before incrementing the counts.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant