ci: ratchet-only floors in storyboard matrix; lower-then-raise pattern erodes the floor file as a conformance signal

[bokelley]

The product-expert review of #3974 + #3976 flagged the floor ratchet pattern as the highest-leverage concern in the post-#3962 cluster:

Floor cadence — landing-wrong as a pattern, even if each individual PR is correct. The lower→raise→fix→raise sequence makes the floor file a lagging indicator of PR ordering rather than conformance. A reader of git blame on floors can't tell "real regression" from "shipping artifact." Alternative: floors move once per cluster close, not per PR. Use a scratch/observed file during the cluster, ratchet the canonical floor only when the cluster closes. Or: floors only ratchet up, and #3962 should have been a feature flag / waiver rather than a floor lower.

Concrete this cycle:

• feat(compliance): bump @adcp/sdk to 6.7.0 + adopt upstream_traffic on 5 storyboards #3962 lowered creative 58 → 51, creative-builder 55 → 49, brand 59 → 58 (to ship the @adcp/sdk@6.7.0 bump).
• fix(compliance): close 3965 Class B + D — deterministic_testing error widening, idempotency dead capture removal, raise floors #3974 raised them back: creative 51 → 56, creative-builder 49 → 52, brand 58 → 65.
• fix(training-agent): wire listCreativeFormats on /creative and /creative-builder tenants (closes #3965 Class F) #3976 lifted past them in observed numbers (+8/+10 creative, +6/+10 creative-builder) but didn't touch floors to avoid conflicts.
• A follow-up will ratchet again to capture fix(training-agent): wire listCreativeFormats on /creative and /creative-builder tenants (closes #3965 Class F) #3976.

Four floor-touching PRs in one cluster. Future contributors reading git blame on `.github/workflows/training-agent-storyboards.yml:39-56` get noise instead of signal — the lowered floor in #3962 wasn't a regression, but reads like one.

Proposals (pick one or compose)

1. Ratchet-up-only invariant. Floor numbers MAY only increase. A PR that needs to ship despite a regression files a waiver issue listing the affected storyboards and sets a waivers: block on the matrix entry instead of touching the floor. The workflow checks the waiver list against actual failures and accepts known-failing storyboards without lowering the global floor.

2. Cluster-close-only floor changes. Floors move once per regression-cluster-close (i.e., once per Training-agent baseline regressions exposed by @adcp/sdk@6.7.0 bump (PR #3962) #3965-class tracker), not per individual fix-up PR. Individual PRs touching the same cluster reference an observed_floors.yaml scratch file that the workflow reads; the canonical floor file only updates when the cluster closes.

3. Lock floor changes to a separate review path. Edits to the floor numbers require a floor-change label and dual approval from a codeowner of .github/workflows/training-agent-storyboards.yml. Forces a moment of "do we actually want to lower this?" friction.

The first option is cheapest and closest to how most CI suites handle this. Worth pairing with a build-time lint that compares the new floor to the prior floor on every PR and rejects decreases without a waiver.

References

• feat(compliance): bump @adcp/sdk to 6.7.0 + adopt upstream_traffic on 5 storyboards #3962 — bump that lowered floors.
• fix(compliance): close 3965 Class B + D — deterministic_testing error widening, idempotency dead capture removal, raise floors #3974 — first raise.
• fix(training-agent): wire listCreativeFormats on /creative and /creative-builder tenants (closes #3965 Class F) #3976 — fix-up that should bump floors but didn't (defers to this issue).
• Training-agent baseline regressions exposed by @adcp/sdk@6.7.0 bump (PR #3962) #3965 — broader regression cluster tracker.
• Three-expert review of fix(compliance): close 3965 Class B + D — deterministic_testing error widening, idempotency dead capture removal, raise floors #3974 + fix(training-agent): wire listCreativeFormats on /creative and /creative-builder tenants (closes #3965 Class F) #3976 (product expert) flagged this pattern.

[bokelley]

Triage

Classification: Feature request
Bucket(s): infra/agents, compliance-suite
Status: ready-for-human

What the experts said:

• dx-expert: Option 1 recommend — git blame on the floor rows becomes a clean monotonic signal; the waivers: block is self-documenting and reviewable. Option 2 avoid — two files of truth with a manual cluster-close trigger is exactly the skip pattern that caused this issue. Option 3 acceptable as a complement (not standalone).
• code-reviewer: Option 1 recommend — build-step enforcement is mechanical; waiver validation slots into the existing passed/clean parse step. Both experts flagged waiver accumulation as the main failure mode: mitigate by requiring a closes: #N reference on each waiver entry and a stale-waiver lint. Option 3 acceptable as defense-in-depth alongside Option 1.

My take: Strong consensus on Option 1 + Option 3 as defense-in-depth: ratchet-up-only invariant with a waivers: block and build-time lint rejecting floor decreases without a waiver, plus CODEOWNERS gate on the workflow file path for second eyes on any floor-touching PR. Flagging rather than drafting a PR because this lands in .github/workflows/ — outside the triage routine's edit scope. @bokelley, confirm direction (or compose/adjust) and comment /triage execute to re-trigger, or implement directly.

---

Triaged by Claude Code. Session: https://claude.ai/code/session_017xXxTsexsCByyL2B2puodm

---

Generated by Claude Code