Fix what five page-level audits found, including two self-contradictions - #172
Merged
Conversation
The five section audits killed when the machine slept were rerun. Their findings, worst first. /forecasts headlined an accuracy number, which is the one thing its own generator says it must never do. The record band's third tile read "0.12pp — latest absolute error", level with the round and scored counts, while score.py's module docstring argues that "a track record that reports an impressive figure derived from one observation is worse than an empty one", and the disclaimer saying so sat below the tile. The tile now shows the next scored period. The error still appears on the scored card, beside its benchmarks. The same page called its one favourable result "beats naive" using the driftless walk that its own section 06 calls too weak for a trending series. The footer now names the benchmark: "beats driftless-naive error". frb-us/index promised a "shock explorer on this page". There is none, on that page or anywhere on the site — the page contains no form, input or button. A reader was told to do something impossible. /economy carried the only hand-typed live number on the page, and it had already rotted: "the model expects it back above 3% by Q4", from the 2026-07-21 round, against a current round putting 2026Q4 at 2.94% and an inflation topic one click away saying not until 2027Q2. It now describes the note as the dated artifact it is and points at the live view. svar/index cited the 49-origin rolling evaluation as a validation credential. That study is precisely what found the model no better than naive: against a random walk with drift, UK GDP is not statistically distinguishable (p=0.38-0.67). The page published a GDP fan with no qualifier. Both now sit in the fact. obr/index presented 0.15% anchored MAPE as validation. Anchored accuracy is a by-construction invariant — the page's own validation page says so — and free-running the same equations miss by 5.75%. Both now stated. Every equation and code block on fourteen pages was invisible to screen readers. .meth-inline and .scrolly-sticky are a responsive pair, exactly one visible per viewport, and both carried aria-hidden="true" — so in every viewport the only visible copy was hidden from assistive technology. On pe/code and define/code, whose entire subject is the commands, not one was exposed. 58 blocks across 14 pages now are; the copy buttons in the hidden aside are no longer focusable. The sticky column blanked through step 1 on five pages, and that one is mine: my simulation of the data-step pairing skipped the active === 0 case, which is exactly the case that fails. Panels now pin from the first available step. models/index called all 284 FRB/US equations behavioural; the paper says 120 behavioural and 164 identities. It counted seven model classes where there are six — obr-macro and frb-us share one. Its HMRC link said June 2025 and pointed at the January bulletin. And it attached "machine precision" to the ~1e-8 cross-validation figure, which is solver tolerance, not the 5.6e-17 tracking invariant. /connect said "everything else here runs hosted"; the three define_* tools return run instructions, as the unlicensed upstream is never hosted. /models#score landed on a closed accordion because the id sits on the <details> element, which browsers do not auto-expand — breaking the primary "score a reform" journey from three high-traffic links. reveal.js now opens a details element a fragment targets. olg/validation published debt-to-GDP as 95.1% for May 2026. The committed store says May is 94.8 and June 94.9, and the site's own economy page says 94.9. Corrected, with the deviation sign flipped to match. And the DEFINE clean-room status was stale everywhere, which is my doing: I merged §3.3.1 and §3.3.2 today and updated neither the site nor the registry. Four surfaces said "§3.2, 23 equations, three defects" against a reality of 118 equations and some thirty-five. The §3.3.2 blocker — six untabulated parameters and two variables defined nowhere, so the section as published cannot be simulated forward — was published nowhere at all, while the same pages call the clean room the path to hosting. Now stated. One audit finding was wrong and is not acted on: define/validation cites upstream issue DEFINE_UK_1.1#1 for the emissions divergence, flagged as unsupported. The issue exists, is open, and its title matches the finding exactly. Verified before deleting anything. Twelve gates, 1038 site tests, 274 integration tests. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The five section audits that died when the machine slept were rerun, covering every page. Their top findings, fixed.
Two self-contradictions — the site arguing against itself
/forecastsheadlined an accuracy number, the one thing its own generator says it must never do. The record band's third tile read "0.12pp — latest absolute error", level with the round and scored counts, whilescore.py's docstring argues "a track record that reports an impressive figure derived from one observation is worse than an empty one" — and the disclaimer saying so sat below the tile. Now shows the next scored period; the error stays on the scored card beside its benchmarks.A promise of something that does not exist
frb-us/indexsaid "Run the shock explorer on this page." There is no explorer — the page contains no form, input or button, and neither does any other page on the site.Every equation and code block on 14 pages was invisible to screen readers
.meth-inlineand.scrolly-stickyare a responsive pair — exactly one visible per viewport — and both carriedaria-hidden="true", so in every viewport the only visible copy was hidden from assistive technology. Onpe/codeanddefine/code, whose entire subject is the commands, not one was exposed. 58 blocks fixed; the copy buttons in the still-hidden aside are no longer focusable.Stale and wrong numbers
/economycarried the only hand-typed live number on the page, and it had rotted: "the model expects it back above 3% by Q4", from the 2026-07-21 round, against a current round putting 2026Q4 at 2.94% and an inflation topic one click away saying not until 2027Q2.olg/validationpublished debt-to-GDP as 95.1% (May 2026). The committed store says May = 94.8, June = 94.9, and the site's own economy page says 94.9. Corrected, deviation sign flipped to match.models/indexcalled all 284 FRB/US equations behavioural (the paper says 120 behavioural + 164 identities); counted seven model classes where there are six (obr-macro and frb-us share one); linked the January HMRC bulletin under June 2025 text; and attached "machine precision" to the ~1e-8 cross-validation figure, which is solver tolerance, not the 5.6e-17 tracking invariant./connectsaid "everything else here runs hosted" — the threedefine_*tools return run instructions, since the unlicensed upstream is never hosted.Missing disclosure on two overview pages
svar/indexcited the 49-origin rolling evaluation as a validation credential. That study is precisely what found the model no better than naive — against a random walk with drift, UK GDP is not statistically distinguishable (p = 0.38–0.67) — and the page published a GDP fan with no qualifier.obr/indexpresented 0.15% anchored MAPE as validation. Anchored accuracy is a by-construction invariant (its own validation page says so) and free-running the same equations miss by 5.75%.A broken primary journey
/models#scorelanded on a closed accordion, because the id sits on the<details>element and browsers only auto-expand when the fragment targets a descendant. Three high-traffic links point there.reveal.jsnow opens a targeted details element.Two findings that were mine
data-steppairing skipped theactive === 0case — exactly the case that fails — so I reported "no page blanks mid-article", which was narrowly true and missed the defect.One finding not acted on, because it was wrong
define/validationcites upstream issueDEFINE_UK_1.1#1for the emissions divergence; an audit flagged it as unsupported, "the same failure class as the fabricated figures". The issue exists, is open, and its title matches the finding exactly. Verified before deleting.Verification
Twelve generator and contract gates, 1038 site tests, 274 integration tests.
🤖 Generated with Claude Code