From 8835e16cbfea2e7d02b8c8c1a2b3c4eb6cf74e52 Mon Sep 17 00:00:00 2001
From: vahid-ahmadi
Date: Wed, 12 Aug 2026 15:09:46 +0100
Subject: [PATCH 1/6] Fix broken examples, a rejected identifier, and an
overstated headline
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Three model-repo reviews and two recovered page audits. The findings that
were verifiable and mine to fix.
Two published commands could not run. /connect's dynamic-scoring recipe gave
`pe-macro og-score --json > og.json` and `pe-macro dynamic-score --og-payload
og.json`; --reform is required=True on both, so each exits 2 with "Missing
option '--reform'" before doing anything. Confirmed by running them. core.py's
own error text has carried the correct two-step form all along; the page was
the only place that dropped it.
The site's name for a model was rejected by the site's own API. Every page
says psl-og; get_status("psl-og") raised, and the error listed only registry
keys, so nothing connected the two. site_id was added to render one name per
model but the alias was one-directional. Lookups now resolve site names, the
error offers both vocabularies, and the returned model_id is the registry key
so a caller who passed a site name learns the contract id. Pinned by a test.
/connect claimed "no PolicyEngine account and no API key" beside a UK CLI
example that needs a HUGGING_FACE_TOKEN for a private dataset. The claim is
true of the hosted route and false of that example. Three other pages
document the token; the page carrying the example did not. Both now scoped
and stated.
It also said the hosted server "points you to the local CLI" for
score_reform(model='og'). dynamic_reform_impact does return the local recipe;
model='og' returns an import error pointing at an install the registry
documents as unusable in-process. Reworded to what actually happens.
FRB/US: the headline was the weakest evidence in the repo, presented as the
strongest. A review proved the tracking invariant is tautological by running
it on randomly scrambled data — every accounting identity destroyed — where
it still passes at 6.7e-9, because init_trac defines add-factors as minus the
residuals at the input, so re-solving is the identity on any input. 282 of
284 equations have bit-exactly zero residual before Newton runs. models/ now
leads with the real evidence, agreement with the Fed's own pyfrbus across all
284 variables at ~1e-8 — the Board's two releases differ from each other by
1.3e-8 — and the residual chart's tautological bar is relabelled "identity —
not evidence" with the scrambled-data result in its note, instead of being
plotted as the shortest bar beside genuine cross-implementation residuals.
And one of mine: replacing "Seven model classes" left "Seven models, side by
side, side by side." in an h2. The earlier attempt at the same line had also
silently failed on a literal apostrophe rather than the entity.
Twelve gates, 1107 site tests, 275 integration tests.
Co-Authored-By: Claude Opus 5 (1M context)
---
connect/index.html | 17 ++-
frb-us/validation/index.html | 31 +++--
.../src/policyengine_macro/capabilities.py | 24 +++-
integration/tests/test_capabilities.py | 24 ++++
models/index.html | 16 +--
obr/index.html | 6 +-
obr/validation/index.html | 6 +-
pe/validation/index.html | 15 ++-
reports/define-uk-replication/index.html | 4 +-
svar/validation/index.html | 2 +-
us-hank/validation/index.html | 2 +-
validation/figures/README.md | 47 ++++---
validation/figures/chart_data.json | 123 ++++++++++++++----
validation/figures/make_charts.py | 2 -
14 files changed, 241 insertions(+), 78 deletions(-)
diff --git a/connect/index.html b/connect/index.html
index 2c3a88ef..35eb427d 100644
--- a/connect/index.html
+++ b/connect/index.html
@@ -285,7 +285,9 @@
Three routes to the same models. Start with MCP below — it needs no
install, no PolicyEngine account and no API key, though some clients
- gate custom connectors by plan. Use the CLI or Python when you want the
+ gate custom connectors by plan. That is true of the hosted route; a
+ local UK population or reform-scoring run additionally needs a
+ HuggingFace token, as below. Use the CLI or Python when you want the
models in a script, or need one of the local-only members.
@@ -368,10 +370,19 @@
Connect in three steps.
takes around 20 seconds while the tax-benefit models load; a UK
forecast runs a couple of minutes on first call. Other tools
respond faster.
+
uk microdata
+ Running a UK population or reform score locally needs
+ HUGGING_FACE_TOKEN from an account with access to the
+ private policyengine/policyengine-uk-data-private
+ dataset. UK household calls, every US call, and every hosted call
+ need nothing.
local onlyOG-UK long-run scoring (score_reform with
model='og') and dynamic_reform_impact take
- tens of minutes — the hosted server points you to the local CLI.
+ tens of minutes and are excluded from the hosted image;
+ dynamic_reform_impact returns the two-step local recipe,
+ while model='og' returns an import error — use the local
+ CLI for both.
The three define_* tools likewise return run instructions rather than results when called on the hosted server — the unlicensed upstream is never hosted. Everything else here runs hosted.
@@ -443,7 +454,7 @@
Use Python directly.
local only
Dynamic reform scoring
-
Experimental, UK only: an OG-UK steady-state solve feeds long-run wage changes back into a second microsimulation run. Local only, and until OG-UK#68 the solve needs its own environment — run pe-macro og-score --json > og.json there, then pe-macro dynamic-score --og-payload og.json.
+
Experimental, UK only: an OG-UK steady-state solve feeds long-run wage changes back into a second microsimulation run. Local only, and until OG-UK#68 the solve needs its own environment — run pe-macro og-score --reform '{"gov.hmrc.income_tax.rates.uk[0].rate":0.21}' --json > og.json there, then pe-macro dynamic-score --reform '{"gov.hmrc.income_tax.rates.uk[0].rate":0.21}' --og-payload og.json — both steps need the same --reform.
Machine precision on the baseline; the Fed's own noise floor on shocks.
sharpest-tested member of the suite, because for once the ground truth
is exact rather than statistical. Three invariants carry the weight,
all enforced in continuous integration, including four like-for-like
- scenarios gated against vendor-generated references.
+ scenarios gated against vendor-generated references. One of the three
+ carries no weight of its own: the tracking invariant is an identity
+ init_trac constructs, so the evidence is the agreement
+ with the Board's shocked paths, where the add-factors are
+ held fixed and the solver has real work to do.
Machine precision is a claim about fidelity to the Fed's
@@ -103,7 +107,7 @@
Machine precision on the baseline; the Fed's own noise floor on shocks.
tracking invariant5.6e−17
- vs LONGBASE, all 284 endos × 20 quarters; CI gate 1e−8
+ vs LONGBASE, all 284 endos × 20 quarters; CI gate 1e−8. An identity init_trac guarantees — round-off, not evidence
shock vs pyfrbus 1.0.0
@@ -136,7 +140,7 @@
Machine precision on the baseline; the Fed's own noise floor on shocks.
- Maximum absolute residuals, log scale — lower is closer. The bottom bar is the framing one: the Fed's own two releases disagree by as much as we disagree with either. On the tracking invariant, pyfrbus 1.1.1 reproduces LONGBASE to 1.1×10−8 against this implementation's 5.6×10−17. Source: papers/frb-us validation tables.
+ Maximum absolute residuals, log scale — lower is closer. The bottom bar is the framing one: the Fed's own two releases disagree by as much as we disagree with either, so the three shock rows sit at the reference implementation's own noise floor. The top two bars are the tracking identity and are not a ranking: after init_trac the solve reproduces its input by construction, so 5.6×10−17 against pyfrbus 1.1.1's 1.1×10−8 compares two implementations' round-off, not their fidelity to the Fed's economics. Source: papers/frb-us validation tables.
Validation results against the Federal Reserve Board's own artefacts
@@ -173,8 +177,8 @@
Machine precision on the baseline; the Fed's own noise floor on shocks.
Tracking invariant
-
After init_trac, solving the baseline must reproduce LONGBASE for all 284 endogenous variables over 2026Q1–2030Q4.
-
Max absolute error 5.6e−17 — machine precision, against a 1e−8 gate. The Board's own pyfrbus 1.1.1 reproduces LONGBASE to 1.1e−8 on the same test.
+
After init_trac, solving the baseline must reproduce LONGBASE for all 284 endogenous variables over 2026Q1–2030Q4. This is an identity, not a comparison:init_trac sets each equation's add-factor to minus that equation's residual at the input data, so the solve is algebraically the identity on whatever it was tracked to, and Newton is warm-started from the row it is about to reproduce.
+
Max absolute error 5.6e−17 — floating-point round-off, against a 1e−8 gate. It measures re-evaluation noise, not agreement with the Fed's data or economics: the model repository's test_tracking_invariant_holds_for_arbitrary_data shows the identical gate passes on a scrambled baseline with every accounting identity destroyed. Read the cross-validation row below, not this one, as evidence.
Cross-validation
@@ -190,8 +194,17 @@
Machine precision on the baseline; the Fed's own noise floor on shocks.
Multipliers. Government-purchases and personal-tax
- shocks of 1% of GDP land inside every published range they are
- compared against — the last three rows below.
+ shocks of 1% of GDP land inside the published ranges they are compared
+ against, with one exception recorded below: the year-two
+ purchases multiplier under a pegged funds rate is
+ 0.99, just below the 1.1–1.2 cross-model accommodation
+ range, and matches only the Board's looser “roughly
+ one” characterisation. The comparisons are also not horizon-matched
+ throughout — the year-one tax figure (0.22) is set beside a benchmark
+ the source describes as a two-year multiplier, and Ramey's survey range
+ pools empirical estimates across horizons and monetary regimes rather
+ than fixing one. Read the last three rows as plausibility bands, not as
+ a test the model passed.
frb-us against the Fed's pyfrbus and published multiplier ranges
diff --git a/integration/src/policyengine_macro/capabilities.py b/integration/src/policyengine_macro/capabilities.py
index bd0d8b30..b2156109 100644
--- a/integration/src/policyengine_macro/capabilities.py
+++ b/integration/src/policyengine_macro/capabilities.py
@@ -645,10 +645,28 @@ def list_capabilities() -> list[dict]:
return [{"model_id": model_id, **deepcopy(data)} for model_id, data in MODELS.items()]
+# Public site names that are not registry keys. site_id exists so pages render
+# one name per model; without the reverse mapping the alias was one-directional
+# — every page said psl-og and get_status("psl-og") raised, so a reader taking
+# the documented name into the documented API hit a hard error whose message
+# did not connect the two.
+SITE_ID_TO_KEY = {
+ model["site_id"]: key for key, model in MODELS.items() if model.get("site_id")
+}
+
+
+def resolve_model_id(model_id: str) -> str:
+ """Registry key for a registry key or a public site name."""
+ return SITE_ID_TO_KEY.get(model_id, model_id)
+
+
def get_status(model_id: str) -> dict:
- if model_id not in MODELS:
- raise ValueError(f"unknown model_id {model_id!r}; choose one of {sorted(MODELS)}")
- return {"model_id": model_id, **deepcopy(MODELS[model_id])}
+ key = resolve_model_id(model_id)
+ if key not in MODELS:
+ known = sorted(set(MODELS) | set(SITE_ID_TO_KEY))
+ raise ValueError(f"unknown model_id {model_id!r}; choose one of {known}")
+ # Echo the key, so a caller who passed a site name learns the contract id.
+ return {"model_id": key, **deepcopy(MODELS[key])}
def recommend(
diff --git a/integration/tests/test_capabilities.py b/integration/tests/test_capabilities.py
index 23fa662f..7c8cae52 100644
--- a/integration/tests/test_capabilities.py
+++ b/integration/tests/test_capabilities.py
@@ -118,3 +118,27 @@ def test_every_model_has_a_real_evidence_assessment():
assert judgement["evidence"].strip(), (
f"{model_id}.{dimension} has an empty evidence string"
)
+
+
+def test_public_site_names_resolve_through_the_api():
+ """A name the site publishes must work in the API the site documents.
+
+ site_id was added so pages render one name per model, but the alias was
+ one-directional: every page said `psl-og` while `get_status("psl-og")`
+ raised, and the error listed only registry keys — so a reader who took the
+ documented name into the documented tool got a hard failure with nothing
+ connecting the two. Resolution now goes both ways, and the returned
+ model_id is the registry key, so a caller who passed a site name learns
+ the contract id.
+ """
+ for site_name, expected_key in capabilities.SITE_ID_TO_KEY.items():
+ assert capabilities.get_status(site_name)["model_id"] == expected_key
+ assert capabilities.get_status(expected_key)["model_id"] == expected_key
+
+ try:
+ capabilities.get_status("not-a-model")
+ except ValueError as error:
+ message = str(error)
+ assert "og-uk" in message and "psl-og" in message, message
+ else:
+ raise AssertionError("an unknown model_id must raise")
diff --git a/models/index.html b/models/index.html
index 23f7ec13..a9999022 100644
--- a/models/index.html
+++ b/models/index.html
@@ -121,7 +121,7 @@
Choose the model that fits the quest
The Federal Reserve’s FRB/US model — 284 equations — 120 estimated behavioural equations and 164 identities — parsed from the Board’s own model file — is solved by damped Newton around the April 2026 LONGBASE baseline; you pick the lever and the policy rule, and read the deviation path quarter by quarter.
→Levers: federal funds rate, government spending, transfers, taxes — each in its own units
-
→Replicates the Board’s dynamics to machine precision (residuals ~1e-8 or better)
+
→Matches the Fed’s own pyfrbus across all 284 variables to ~1×10−8 — the Board’s own two releases differ from each other by 1.3×10−8, so this sits at the reference implementation’s noise floor. Reproducing LONGBASE itself is an identity, not evidence
→VAR expectations only; a conditioning baseline, not a forecast
Not yet produced — the committed run is a transition path stopping at 2030, not a steady state, so it is not comparable to WP 22's long-run figures. Open validation work
OBR Working Paper 22, Table 5.1: GDP/person −0.1%, labour supply −0.2%, productivity +0.1%
Future benchmark — same scenario; the target we will be judged against, not a match claimed today
@@ -317,10 +317,10 @@
How each model is validated.
Direct — −6.4% in year one; HMRC's figure embeds behavioural response
-
Government-purchases multiplier, year one (frb-us)
-
0.72 (inertial Taylor rule); 0.99 in year two with the funds rate fixed
-
Ramey (JEP 2019): the literature sits "in a surprisingly narrow range of 0.6 to 1"
-
Direct — both estimates fall inside the survey range
+
Government-purchases multiplier (frb-us)
+
0.72 in year one (inertial Taylor rule); 0.99 in year two with the funds rate fixed
+
Ramey (JEP 2019): estimates of the general government-purchases multiplier sit "in a surprisingly narrow range of 0.6 to 1"
+
Adjusted — both fall inside the survey range, but Ramey pools empirical estimates across horizons and monetary regimes, so neither of our two horizon- and rule-specific figures is matched to it. Against the tighter cross-model range for a pegged funds rate (1.1–1.2), our 0.99 sits just below
Read GDP-feedback estimates as an upper bound. Under
the demand closure a spending shock lands directly in the GDP
identity with the behavioural second round largely inactive, so the
- impact multiplier is ~1 by construction; the OBR's published impact
- multiplier for current spending is 0.6. The
+ impact multiplier is ~1 by construction; the OBR's
+ published impact
+ multiplier for day-to-day public services and welfare spending is
+ 0.6. The
validation page lists the
full set of known limits.
diff --git a/obr/validation/index.html b/obr/validation/index.html
index 4a06690d..c1f265f3 100644
--- a/obr/validation/index.html
+++ b/obr/validation/index.html
@@ -275,7 +275,7 @@
Against the OBR's own forecast, and HMRC's reckoner.
Basic rate +1pp vs HMRC ready reckoner, £bn/yr
2026–27
6.46
6.9
−6.4%
2028–29 (interpolated)
6.92
8.2
−15.6%
-
2030 (end of window)
7.38
≈8.2
−10.0%
+
2030 (end of window)
7.38
≈8.2 (HMRC's 2028–29 figure — its reckoner stops there)
−10.0%
@@ -307,7 +307,9 @@
Against the OBR's own forecast, and HMRC's reckoner.
The spending multiplier is ~1.0 by construction, against the
- OBR's own published 0.6. Under the demand closure a spending
+ OBR's own published
+ 0.6 for day-to-day public services and welfare spending.
+ Under the demand closure a spending
shock lands directly in the GDP identity and the behavioural second
round is largely inactive, so a £5bn injection returns almost
exactly £5bn of GDP — roughly a
diff --git a/pe/validation/index.html b/pe/validation/index.html
index 3d5496ec..5d4cf077 100644
--- a/pe/validation/index.html
+++ b/pe/validation/index.html
@@ -101,9 +101,18 @@
Checked rule by rule, not against a forecast.
not. For the UK 1p basic-rate case, the
OBR emulator working paper compares
static costings against official sources: £6.46bn in 2026 rising to
- £7.38bn by 2030, inside the range of HMRC's June 2025 ready reckoner
- (£6.9bn rising to ~£8.2bn), toward its lower end. Suite-wide
- evidence: validation.
+ £7.38bn by 2030. That sits inside the roughly £6–8bn envelope
+ spanned by recent ready-reckoner vintages, but it is
+ below HMRC's June 2025 figures at every year that can
+ be matched (£6.9bn for 2026–27, £8.2bn for 2028–29) —
+ −6.4% in the first year, widening to −10.0% by the end of
+ our window. Nor is the comparison like-for-like: HMRC's reckoner runs
+ to 2028–29, so our 2030 figure has no year-matched counterpart,
+ and HMRC publishes post-behavioural estimates built on administrative
+ Survey of Personal Incomes data against our static, survey-based one.
+ It is a benchmark, not a validation. No interval is published with
+ either of our figures, because the pipeline does not produce one.
+ Suite-wide evidence: validation.
Every gate, its verdict, and the test that holds it.
Clean-room milestone 2 — §3.2–3.3 macro and production block
-
IN PROGRESS — §3.2, §3.3.1 and §3.3.2 landed (118 eqs); §3.3.3 + oracle outstanding, so not passedIN PROGRESS — §3.2 landed; not passed (section 05)
+
IN PROGRESS — §3.2, §3.3.1 and §3.3.2 landed (118 eqs); §3.3.3 + oracle outstanding, so not passed (section 05)
2026-08-11
tests/test_macro.py
@@ -394,7 +394,7 @@
Reimplementing the published equations, milestone by milestone.
into a tolerance.
- The three pinned manual defects
+ Three of the pinned manual defects, worked through
Table 6 LENDM_ROW omits the DIVN_ROW term of
Eq. (383), so the MFI/RoW transaction columns miss LEND by
diff --git a/svar/validation/index.html b/svar/validation/index.html
index acdb5b86..8c4274b2 100644
--- a/svar/validation/index.html
+++ b/svar/validation/index.html
@@ -569,7 +569,7 @@
boe-svar against a random walk, computed from our own runs
1.33
- Expanding-window pseudo-out-of-sample error ratios against a random walk with drift, on the level of each series. A driftless walk is too weak a benchmark for a trending series, so this is the fair comparison; the no-change ratios are in the table below. 49 forecast origins, 2012Q1–2024Q1 (the window includes the Covid quarters); estimation sample from 1992Q1, data through 2026Q1; 4 lags. Estimation uses final revised data, so this is pseudo- rather than real-time out-of-sample. No single origin contributes more than 35% of any drift-benchmark squared error. Source: papers/boe-svar/figures/rolling_evaluation.json.
+ Expanding-window pseudo-out-of-sample error ratios against a random walk with drift, on the level of each series. A driftless walk is too weak a benchmark for a trending series, so this is the fair comparison; the no-change ratios are in the table below. 49 forecast origins, 2012Q1–2024Q1 (the window includes the Covid quarters); estimation sample from 1992Q1, data through 2026Q1; 4 lags. Estimation uses final revised data, so this is pseudo- rather than real-time out-of-sample. These ratios are concentrated in a few origins: in 16 of the 64 variable–horizon cells a single origin contributes more than 35% of the drift-benchmark squared error, peaking at 63% for UK GDP at h=1, so individual ratios rest on far fewer effective observations than 49 origins suggests. Source: papers/boe-svar/figures/rolling_evaluation.json (worst_origin_mse_share.drift).
- Hosted two-asset steady state against the paper's published calibration targets, solved 2026-07-29 at the production grids (nB=50, nA=70, nK=50). β is the calibrated free parameter. Source: papers/us-hank/figures/replication.json.
+ Hosted two-asset steady state against the paper's published calibration targets, solved 2026-07-29 at the production grids (nB=50, nA=70, nK=50). Read the top five rows as inputs, not results: Y, K, r, G and Bg are literal entries in the calibration dictionary, held fixed through the solve, so their zero deviation is an arithmetic tautology and carries no information. The informative rows are the last three — total wealth and liquid assets, which the solver has to hit through market clearing, and β, the one free parameter, whose 0.976274 is the only value here compared against a number the paper reports. Source: papers/us-hank/figures/replication.json.
This validates replication fidelity, not prediction.
diff --git a/validation/figures/README.md b/validation/figures/README.md
index 312c204c..8a9baa52 100644
--- a/validation/figures/README.md
+++ b/validation/figures/README.md
@@ -1,7 +1,14 @@
# validation/figures
-Generating script for the twelve inline SVG charts on `/validation/`, in the same
-spirit as the papers' `papers/*/figures/*.py` scripts.
+Generating script for the fourteen inline SVG evidence charts on the model
+validation pages, in the same spirit as the papers' `papers/*/figures/*.py`
+scripts.
+
+There is no `/validation/` page: it was absorbed into `/models#validation`, and
+each chart now lives inline on the validation subtab of the model it is evidence
+for (the exception is `pe-costing`, which sits on the microsimulation overview).
+The owning page for each chart is the third field of `BUILDERS` in
+`make_charts.py`.
## Regenerate
@@ -9,9 +16,9 @@ spirit as the papers' `papers/*/figures/*.py` scripts.
python3 validation/figures/make_charts.py
```
-This rewrites the twelve `