feat: agent deployment compatibility validation — server-side checks with auto-fix (#668)#1295
Merged
Conversation
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Run ~100 best-practice checks (11 categories) against a running agent's workspace, surfaced non-blocking in the Overview tab with one-click auto-fix for the 10 gitignore checks, plus an MCP tool. - services/compatibility/ package (spec/collector/static_checks/ai_checks/fixes): ONE docker exec -> in-container python -> JSON snapshot (secret files existence-only, size/binary caps); pure STATIC checks (HARD-only) + category-batched AI checks (Haiku, iterate-expected, fail-open, capped at SOFT, secret-redacted); runtime-aware (claude-only checks skipped for Codex/Gemini). - GET/POST endpoints (read AuthorizedAgentByName; fix OwnedAgentByName, gitignore only, per-agent Redis lock, atomic write, uncommitted until next sync; include_ai path rate-limited). agent_compatibility_results table (dual-track SQLite + Alembic) persists the latest snapshot; cascade/rename via AGENT_REFS. - CompatibilityPanel.vue (two-phase fetch, grouped checklist, per-check fix, re-run) in OverviewPanel; get_agent_compatibility_report MCP tool. - 35 fixture-driven unit tests; spec sync-tested against docs/agent-validation-spec.md. Persistence departs from the issue's "no DB table" note so AI verdicts show without re-spend + enable fleet aggregation (see requirements section 41). Fixes #668 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
dolho
force-pushed
the
feature/668-compat-validation
branch
from
June 22, 2026 09:09
825bd6b to
4d4063e
Compare
CodeQL py/polynomial-redos (high): `_ASSIGN_RE` captured the value as `[ \t]*(.+?)[ \t]*$`. The lazy `.+?` and the surrounding `[ \t]*` can both match a tab, giving polynomial backtracking when `_redact()` runs the pattern over up to 48 KB of agent-supplied file text. Capture the value greedily to end-of-line (`(.*)$`) and let the callers strip — both `_looks_placeholder()` callers already `.strip()`, so secret detection and redaction are behaviourally identical (verified: `=`, `:`, `export`, and indented forms still match). 35 unit tests pass.
dolho
approved these changes
Jun 22, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
CompatibilityPanel.vue, "needs attention" idiom + grouped checklist), with one-click auto-fix for the 10 gitignore checks, plus an MCP toolget_agent_compatibility_report.docs/agent-validation-spec.md(single source of truth, sync-tested againstservices/compatibility/spec.py).Changes
services/compatibility/package (spec/collector/static_checks/ai_checks/fixes): ONEdocker exec→ in-containerpython3→ one JSON snapshot (secret files existence-only, per-file binary/size caps); pure(snapshot)→[Check]STATIC checks (HARD reserved for STATIC) + category-batched AI checks (Claude Haiku, tool-use, iterate-expected so omitted→skipped, fail-open, capped at SOFT, secret-redacted, secret files never sent). Runtime-aware — claude-only checks skipped for Codex/Gemini (Codex harness MVP — pluggable agentic execution engine alongside Claude Code #1187).routers/compatibility.py:GET …/compatibility?include_ai=(read,AuthorizedAgentByName; theinclude_aifan-out is rate-limited) andPOST …/compatibility/fix(owner/admin,OwnedAgentByName; gitignore-only, per-agent Redis lock, atomic write, uncommitted until next git sync;check_idwhitelisted → 400/409).agent_compatibility_results(latest-snapshot-per-agent, upsert) — dual-track migration (SQLitedb/migrations.py+ Alembic0003_*), DDL inschema.py/tables.py, ops indb/compatibility.py, cascade/rename viaAGENT_REFS.CompatibilityPanel.vue(two-phase fetch — STATIC instant, AI async/cached; grouped checklist; per-check Fix; Re-run) inOverviewPanel.vue;agents.jsstore actions.get_agent_compatibility_report(agents.ts+client.ts+types.ts), three surfaces in sync (Invariant feat: SMARTS trading pipeline with Telegram notifications and Miro visualization #13).Design note — persistence (deliberate deviation)
The issue specified "no DB table (transient)". This PR adds
agent_compatibility_resultsbecause AI verdicts aren't cheaply recomputable (they cost API calls): persistence lets AI findings show on every Overview load without re-spending tokens, unlocks fleet aggregation, and enables cheap post-fix re-checks. STATIC recomputes live each read; persisted AI verdicts merge in until a re-run. Rationale recorded in requirements §41 + the spec's deviation note.Test Plan
pytest tests/unit/test_compatibility_checks.py— spec consistency + spec↔doc sync, STATIC checks (good/bad/empty), gitignore fix transforms (CRLF/dup/comment/.claude/projects/survival/idempotent), AI batching (no-key skip, omitted→skipped, redaction),build_reportorchestration (+ tmp-DB persistence, codex omits claude_only, stopped→unavailable), collector script vs a temp ROOT.vite build✓, MCPtsc --noEmit✓./review: 0 critical./cso --diff: 1 MEDIUM (LLM cost amplification oninclude_ai) — fixed (rate-limited); report indocs/security-reports/.Fixes #668
🤖 Generated with Claude Code