Common ScoreResult schema across scoring adapters + OBR microsim static-costing bridge - #22
Merged
Merged
Conversation
) ScoreResult (pydantic, emitted as plain dicts): every scoring adapter (og_score_reform, obr_score_reform, pe_population_impact) now returns its existing payload unchanged plus an additive "score" block — model id and class (microsim / semi-structural / olg-ge), country, reform, horizon, per-quantity deltas with units and basis, assumptions, caveats, and an optional distributional block (filled by the microsim only). OBR bridge: PolicyEngine population static costings per year in the window enter the OBR emulator as a quarterly HHDI shock path (sign-corrected: revenue raised lowers disposable income; £bn/yr -> £m/quarter, flat within each year) via run_reform's path-shock interface; second-round demand effects on GDP/consumption/investment come out. Corporation-tax reforms are refused with a pointer to the direct TCPRO lever (not household-borne in the microsim); supply-side channels stay the OG member's job — both stated in the payload's caveats, not just the docs. score_reform now dispatches og / obr / microsim; new `macromod compare` renders the same reform through several model classes in one table from the common blocks. Fast tests cover the pure costing->shock translation, the schema converters, bridge wiring with both heavy deps stubbed, and the compare command; the real pipeline test is slow-marked and skips cleanly without HUGGING_FACE_TOKEN / obr_macro. Closes #9 Closes #10 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
vahid-ahmadi
added a commit
that referenced
this pull request
Jul 17, 2026
- Replace the stale 'OBR arm pending' remote test (score_reform model=obr no longer errors — the bridge landed in #22) with the corp-tax refusal contract: a corporation_tax reform must be refused with a pointer to the direct obr_shock TCPRO lever. - Add a fast post-deploy remote test for score_reform(model='microsim'): sign of the costing, ScoreResult coherence, distributional block. - Add a slow end-to-end remote test of the full bridge pipeline (microsim costing -> HHDI path -> OBR second-round effects) with sign and magnitude gates; runs in the scheduled full validation. - Bump validate-deployment timeout 20->35 min: the slow leg gains the bridge test and already timed out once at 20. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
vahid-ahmadi
added a commit
that referenced
this pull request
Aug 12, 2026
…e the two levers that do not work (#176) * Refresh OBR figures that had been stale since the ONS anchor, and name the two levers that do not work A fresh review of the emulator recomputed every published number and probed each fiscal instrument. Two things needed changing here. **The free-running scorecard had been stale since PR #22 upstream.** The `OSHH` ONS anchor landed and these pages went on publishing the pre-anchor figures, which *understated* the model's accuracy on every line: real GDP 4.48% not 5.75%, consumption 7.49% not 9.56%, household income 6.27% not 14.15%, real household income 6.03% not 13.86%, business investment 15.73% not 16.12%, company profits 63.29% not 79.80%, current account 3.60% of GDP not 4.17%. Six of the eleven computed variables land within band, not four. Regenerated `fig_free_running_data.csv` from the model and recomputed the MAPEs from the CSV rather than transcribing them, so the chart and the prose now come from the same place. **Two of the four fiscal levers do not work, and nothing said so.** `CGIPS` moves business investment by exactly 0.0 in all twelve quarters and leaves a wrong-signed residual GDP effect, against the OBR's published 1.0 — the channel is absent, not weak. `TCPRO` never converges: for a sustained +5pp rise the investment response runs £97m at q3, £2,876m at q12 and £43,387m at q25, compounding 1.21–1.27x every quarter for 25 quarters, because the stabiliser anchors the level rather than the deviation. Truncating the shock from twelve quarters to eight barely moves the q12 answer, so the published number is carried by accumulated drift rather than by the tax rate. Both now carry a caveat in the `obr_shock` payload — a docstring does not reach an MCP caller reading JSON — and on the lever list at /obr/code. The site already said the government-consumption multiplier is ~1 by construction against the OBR's published 0.6, and the review confirmed it is exactly 1.0000 on impact at both £0.1bn and £10bn and still 0.9995 at quarter 12, with consumption moving £0.64m on a £1,250m shock. That leaves the household-income lever as the one instrument here with behaviour in it. Upstream: PolicyEngine/obr-macroeconomic-model#27, which also fixes three ways to get a confidently wrong answer out of this model — a silent all-NaN shock reading as "no effect", a corporation-tax rate above 1 returning a wrong-signed finite answer, and a transmission audit that had published +53.99% GDP for a 1pp corporation-tax rise. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Re-pin the OBR emulator to the merged review PolicyEngine/obr-macroeconomic-model#27: the multiplier documentation, the TCPRO domain guard, the transmission-audit fix and the corrected WB/WP/WV/RDELTA seeds. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #9
Closes #10
ScoreResult schema (#10)
One pydantic
ScoreResultfor every scoring entry point, emitted per the module's plain-dicts-out convention and additively:og_score_reform, the newobr_score_reform, andpe_population_impactkeep their existing payloads unchanged and gain a"score"key containing:model(id) +model_class(microsim/semi-structural/olg-ge),country,reform,horizon(steady-state/quarterly window …/annual …)quantities— gdp, consumption, investment, government, revenue, debt, each aScoreQuantitywithlevel_bn/delta_bn/delta_pctplus requiredunitsandbasis; a model fills only what it genuinely produces (the static microsim fills revenue only, and says why incaveats)assumptions,caveats, optionaluncertainty, optionaldistributionalblock (deciles / winners / losers — microsim only for now)Payoff wired:
macromod compare --reform '…' --models microsim,obr(orog) renders one table from N ScoreResults;--jsonemits the list. The table footer states explicitly that steady-state and budget-window deltas are each model's own concept.OBR bridge (#9)
score_reform(model="obr")/obr_score_reform: static costing in → second-round effects out.pe_population_impact(country="uk", reform, year=y)for each year in[start_year, start_year+years)→ annualbudgetary_impact_bnpath (baseline sims cached per year).obr_costing_to_shock(pure arithmetic, unit-tested with no heavy deps): annual £bn → quarterly £m, flat within each year (declared), sign-corrected (revenue raised ⇒ disposable income falls).run_reform(var="HHDI", shock=[…])using the upstream path-shock interface (Path-valued shocks and an add-factor injection point for externally costed reforms obr-macroeconomic-model#9's path-valued shocks are on obr main). HHDI is the demand-side incidence channel: HHDI → RHHDI → CONS → GDPM.Declared boundaries — in the result payload (
caveats), not just docstrings:run_reformexogenises HHDI identically in baseline and shocked runs, so the delta isolates the shock, at the stated cost that the economy's feedback onto disposable income itself is not recycled (this is a plain exogenous path, not upstream's proposed held add-factor injection point, which is not implemented there yet).ValueErrorpointing atobr_shock(var="TCPRO"), before any heavy import.revenuequantity in the OBR score block echoes the static costing and itsbasissays it is the bridge INPUT, not an emulator output.Also
score_reformgains amicrosimmodel so the compare table can include the static costing itself;country=usworks there.score_reformdocstring/params updated (years,dataset); CLIscoregains--years/--datasetand renders the common block for non-OG models.compare; the real UK pipeline test isslow-marked and skips cleanly withoutHUGGING_FACE_TOKEN/obr_macro. Fast suite: 60 passed, 35 skipped (the pre-existingtest_summary_parsesneedsboe_varorMACROMOD_BOE_VAR_REPO, unrelated to this change and green with either).Deliberately deferred: the site/docs page (#10 mentions it; this PR is the integration layer), any
uncertaintyband producer (SVAR is not a reform-scoring adapter yet), and the upstream held-add-factor injection point.🤖 Generated with Claude Code