Skip to content

feat(ops): eval-canary liveness probe + Supabase advisor triage docs (DB easy wins)#1009

Merged
BigSimmo merged 3 commits into
mainfrom
claude/clinical-kb-pwa-review-asi3wb
Jul 20, 2026
Merged

feat(ops): eval-canary liveness probe + Supabase advisor triage docs (DB easy wins)#1009
BigSimmo merged 3 commits into
mainfrom
claude/clinical-kb-pwa-review-asi3wb

Conversation

@BigSimmo

@BigSimmo BigSimmo commented Jul 20, 2026

Copy link
Copy Markdown
Owner

Summary

User-requested database + process easy-wins pass. Discovery source: a read-only, user-authorized Supabase advisor sweep of the live project (security + performance) plus repo inspection. Live mutations were deliberately withheld — they're listed below for per-item confirmation.

  • Eval-canary liveness probe (ci.yml static-pr, warn-only): GitHub cron fires are best-effort and the canary's failure-issue step only reacts to runs that happen and fail — since Account governance: publish local safeguards and workflow catalog #923 (2026-07-19) moved the cadence from daily to weekly (Sunday 18:00 UTC), a dropped fire would go unnoticed for a week or more. A pinned github-script step with job-scoped actions: read now warns on any PR when the last completed canary is > 8 days old, surfacing a dropped weekly canary within hours instead of weeks. API errors degrade to a warning; the step can never fail a PR. (Review correction, 43788f4: this PR originally claimed the 2026-07-20 "Sunday" slot was silently dropped — that incident never happened. 2026-07-20 is a Monday; the Sunday 2026-07-19 slot fired as scheduled run chore(deps): bump lucide-react from 1.18.0 to 1.21.0 #48, 19:03 UTC, and succeeded. The probe stands as proactive hardening for the weekly cadence; staleness is now measured from updated_at behind a finite-timestamp guard.)
  • docs/db-maintenance.md (new): the 2026-07-20 advisor snapshot with standing dispositions, so advisor output is never re-triaged from scratch or "fixed" against design intent:
    • Security: clean — the single INFO (document_title_words RLS-enabled/no-policies) is the deliberate fail-closed service-role-only pattern (all client roles revoked); recorded so nobody adds policies to silence the linter. A schema.sql comment was deliberately skipped: the repo's drift-manifest guard hashes the file and regeneration needs Docker — the docs entry is the durable record instead.
    • Performance: ~33 unused_index INFOs recorded as a triage list, not a delete list — retrieval-surface trigram indexes flagged RAG-protected (dropping them changes the retrieval search space → full canary protocol), owner-scoped indexes retained for the multi-tenant design (live corpus is single-tenant so they legitimately read "unused"), operational candidates deferred as negligible-benefit at this corpus size.
    • Telemetry retention recorded as an open decision (see below).

Verification

  • npm run check:github-actions + npm run check:ci-scope — pass (new step pinned to the existing github-script SHA; job permissions minimal)
  • npm run check:gate-manifest — pass (re-run after merging main's ci: enforce the local verify:cheap gate set in CI + gate-manifest self-test (maturity L3) #1002 into the branch)
  • npm run check:function-grants — 28/28 pass (no schema change shipped)
  • npm run docs:check-links — 1034 refs resolve (new doc included)
  • tests/supabase-schema.test.ts — 66/66; lint + typecheck + prettier clean
  • Supabase access in this task was read-only (advisors); no migrations, no config changes, no data mutations

Risk and rollout

  • Risk: low — a warn-only CI step with read-only permissions plus documentation. The probe cannot block merges by construction.
  • Rollback: revert.
  • Provider or production effects: None shipped. The advisor reads were the only live interaction.

Clinical Governance Preflight

  • Source-backed claims still require linked source verification before clinical use — no behaviour change
  • No patient-identifiable document workflow was introduced or expanded
  • Supabase target remains Clinical KB Database (sjrfecxgysukkwxsowpy) — read-only advisor access only
  • Service-role keys and private document access remain server-only — the fail-closed RLS pattern is documented, not altered
  • Demo/synthetic content remains clearly separated from real clinical sources
  • Source metadata, review status, and outdated/unknown-source behavior remain conservative
  • Deployment classification/TGA SaMD impact was checked: no clinical decision-support behaviour changed

Notes — decisions awaiting your confirmation (not implemented)

  1. Scheduled telemetry retention: purge:query-logs exists (owner-scoped, --dry-run, default 90 days) but runs nowhere — telemetry grows unbounded. To wire a weekly workflow I need your policy: retention window (90 days?), scope (eval-owner only, or a global variant needing a small script change?). Say the word with a window and I'll ship it. Update: authorized at the 90-day recommendation ("implement your recommendation for all decisions") — shipping as a follow-up PR after this one merges.
  2. Auth percentage connection strategy (advisor): dashboard config change; only matters at the next instance resize. Low priority.
  3. Operational unused-index drops: possible but negligible benefit; recommend leaving until storage/write latency is a measured problem.

🤖 Generated with Claude Code

https://claude.ai/code/session_01EXsJcLrbZUXwnBeG91cVo9


Generated by Claude Code

Summary by CodeRabbit

  • CI Improvements

    • Added an eval-canary staleness check that warns (instead of failing) when the latest completed run is older than eight days.
    • Expanded CI job read permissions to support a liveness probe; liveness-check data issues now degrade to warnings.
  • Documentation

    • Added database maintenance guidance for the Clinical KB database, including security posture, unused-index triage, telemetry retention notes, and health-check references.
    • Updated the branch review ledger with a new dated entry covering advisor sweep outcomes and related validation results.

- ci.yml static-pr gains a warn-only canary-staleness step (read-only
  actions:read job permission): GitHub cron fires are best-effort — the
  2026-07-20 Sunday 18:00 slot silently never ran and nothing noticed. PR
  traffic now surfaces a >8-day canary drought within hours; API errors
  degrade to a warning, never a failure
- docs/db-maintenance.md (new): 2026-07-20 Supabase advisor snapshot with
  standing dispositions — security clean by design (document_title_words
  RLS-no-policy = the deliberate fail-closed service-role-only pattern,
  recorded here; schema comment skipped to avoid drift-manifest churn);
  ~33 unused-index INFOs recorded as a TRIAGE list (retrieval-surface trgm
  indexes flagged RAG-protected, owner-scoped indexes kept for the
  multi-tenant design, operational candidates deferred); auth
  connection-strategy note; telemetry-retention decision recorded as open
  (purge:query-logs exists, deliberately unscheduled pending explicit policy)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EXsJcLrbZUXwnBeG91cVo9
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

@BigSimmo
BigSimmo enabled auto-merge (squash) July 20, 2026 20:55
@supabase

supabase Bot commented Jul 20, 2026

Copy link
Copy Markdown

This pull request has been ignored for the connected project sjrfecxgysukkwxsowpy because there are no changes detected in supabase directory. You can change this behaviour in Project Integrations Settings ↗︎.


Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.

@coderabbitai

coderabbitai Bot commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

The PR adds a warning-only CI check for stale eval-canary runs and documents Supabase advisor findings, database maintenance decisions, telemetry retention status, and related health-check references.

Changes

Canary liveness and maintenance documentation

Layer / File(s) Summary
Eval-canary liveness probe
.github/workflows/ci.yml, docs/branch-review-ledger.md
The static PR job queries the latest completed eval-canary run, warns when it exceeds eight days or the query fails, and records the change in the review ledger.
Database advisor snapshot and maintenance guidance
docs/db-maintenance.md
Documents RLS and index dispositions, auth connection notes, telemetry retention status, and related observability and health-check references.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant static-pr
  participant GitHub API
  participant eval-canary workflow
  static-pr->>GitHub API: Request latest completed eval-canary run
  GitHub API-->>static-pr: Return run timestamp
  static-pr->>static-pr: Evaluate 8-day staleness threshold
  static-pr-->>eval-canary workflow: Emit warning or info status
Loading

Possibly related PRs

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title is concise and accurately reflects the main changes: a warn-only eval-canary liveness probe and Supabase advisor triage docs.
Description check ✅ Passed The description follows the template well, covering summary, verification, risk/rollback, governance preflight, and notes with concrete details.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch claude/clinical-kb-pwa-review-asi3wb

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/ci.yml:
- Around line 142-146: Correct the missed-canary incident date from July 20,
2026 to Sunday, July 19, 2026 in the workflow comment at
.github/workflows/ci.yml lines 142-146, the corresponding ledger row at
docs/branch-review-ledger.md line 662, and the cross-reference at
docs/db-maintenance.md lines 52-54, keeping all records consistent.
- Around line 159-164: Update the Eval-canary liveness check around latest to
calculate staleness from latest.updated_at instead of latest.run_started_at.
Validate that the timestamp exists and parses to a finite value before comparing
its age against eight days; handle missing or invalid timestamps safely without
treating them as stale.

In `@docs/branch-review-ledger.md`:
- Line 662: Update the date or weekday reference in the ledger entry so they are
consistent: use July 19, 2026 for the Sunday 18:00 canary slot, or change the
slot to Monday if retaining 2026-07-20. Preserve the rest of the entry
unchanged.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 0a091b97-0c8b-4949-9531-3f2f72e26148

📥 Commits

Reviewing files that changed from the base of the PR and between 929f728 and d42e253.

📒 Files selected for processing (3)
  • .github/workflows/ci.yml
  • docs/branch-review-ledger.md
  • docs/db-maintenance.md

Comment thread .github/workflows/ci.yml Outdated
Comment thread .github/workflows/ci.yml
Comment thread docs/branch-review-ledger.md Outdated
BigSimmo and others added 2 commits July 21, 2026 05:05
…amps

CodeRabbit review findings on #1009, verified against the live run history:
2026-07-20 is a Monday, and the 'dropped Sunday fire' never happened - the
Sunday 2026-07-19 slot fired as scheduled run #48 (19:03 UTC, success), and
Monday has no slot since #923 moved the cron from daily to weekly. Rewrote the
workflow comment, ledger row, and db-maintenance cross-reference to the truth
(the probe stands as proactive hardening for the weekly cadence). Probe now
measures staleness from updated_at (~completion time; payload has no
completed_at) with a finite-timestamp guard instead of run_started_at.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EXsJcLrbZUXwnBeG91cVo9
@BigSimmo
BigSimmo merged commit e6cd6cb into main Jul 20, 2026
66 checks passed
@BigSimmo
BigSimmo deleted the claude/clinical-kb-pwa-review-asi3wb branch July 20, 2026 21:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants