fix(compliance): heartbeat pre-stamp TTL, dry_run correctness, manual eval → dashboard, requeue endpoint#4265
Merged
Conversation
… eval → dashboard status, requeue endpoint Fixes three bugs reported in #4253 (comply re-runner stale status) plus adds a member-facing requeue workaround: 1. compliance-heartbeat: pre-stamp uses NOW()+30min lock TTL instead of NOW() so a mid-loop process crash re-queues within 30 min rather than blocking for the full check_interval (default 12 h). 2. complianceResultToDbInput: remove hardcoded dry_run:true; heartbeat paths now set dry_run:false explicitly so scheduled runs are marked authoritative. 3. evaluate_agent_quality: call complianceDb.recordComplianceRun() after agentContextDb.recordTest() so manual runs update the dashboard comply status immediately (dry_run:false so they count alongside heartbeat runs). 4. New POST /api/registry/agents/{url}/monitoring/requeue endpoint + dashboard "Requeue comply" button: clears last_checked_at so the agent is picked up on the next heartbeat cycle (~1 hour). Owner-auth + 60s per-agent rate limit. Refs #4253 https://claude.ai/code/session_01DafX6UtByExTqbPcJHXFnT
Bare UPDATE silently no-ops when agent has no existing row in agent_compliance_status. Switch to INSERT ... ON CONFLICT DO UPDATE so the requeue endpoint works even for agents that have never been through the heartbeat cycle. https://claude.ai/code/session_01DafX6UtByExTqbPcJHXFnT
bokelley
added
the
claude-triaged
Generated build artifacts reflecting schema changes already merged to main (#4235 agent type required, onboarding schema addition). Committed to keep dist/schemas in sync with source. https://claude.ai/code/session_01DafX6UtByExTqbPcJHXFnT
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes three compounding bugs that caused stale
degradedcompliance status on the dashboard after manual evaluation runs, and adds a member-facing requeue mechanism so agents can be force-queued for the next heartbeat cycle.Refs #4253
Root causes fixed
1. Pre-stamp lock window too wide (
compliance-heartbeat.ts)last_checked_atwas stamped toNOW()beforecomply()executed. On a 12-hour check interval, a crash between stamp andrecordComplianceRunsilenced the agent for up to 12 hours per cycle. Fixed to stampNOW() + INTERVAL '30 minutes'(maximum realistic comply() duration), so a crash only silences for 30 min.2.
dry_runalwaystrueon heartbeat runs (compliance-heartbeat.ts,compliance-testing.ts)complianceResultToDbInputhardcodeddry_run: truein its return value, meaning all heartbeat runs were recorded as non-authoritative and never surfaced on the dashboard. Removed the hardcoded value from the shared helper; callers now setdry_run = falseexplicitly.3. Manual
evaluate_agent_qualitynever updatedagent_compliance_status(member-tools.ts)The MCP tool recorded to
agent_context_testsonly. Dashboard status is driven byagent_compliance_status. Added acomplianceDb.recordComplianceRun()call (withdry_run = false) inside the existing org-ownership gate so manual evals propagate to the dashboard.New feature: requeue endpoint + dashboard button
POST /api/registry/agents/{encodedUrl}/monitoring/requeue— clearslast_checked_at = NULLso the agent is eligible on the next heartbeat tick (~1 hour). UsescomplianceWriteMiddleware,verifyAgentOwnership, and a 60-second per-agent in-memory rate limit.requeueForHeartbeatincompliance-db.tsuses an upsert so agents with no prior heartbeat row are handled correctly.Non-breaking justification
All changes are server-side only (no protocol schema changes). No API surface changes are backwards-incompatible: the new endpoint is additive, and the
dry_runfix only affects rows that were previously silently wrong.Pre-PR review sign-offs
requeueForHeartbeatbare UPDATE → upsert) anddry_runundefined path were fixed.Test plan
comply()against a registered agent via heartbeat tick — confirmagent_compliance_status.last_checked_atadvances by ~30 min, not current timeagent_compliance_runs.dry_run = falsefor heartbeat-triggered rows post-deployevaluate_agent_qualityvia MCP — confirm dashboard status updates within ~30sPOST /monitoring/requeueas an org member — confirmlast_checked_at = NULLand agent appears in next heartbeat batchhttps://claude.ai/code/session_01DafX6UtByExTqbPcJHXFnT
Generated by Claude Code