Skip to content

Common ScoreResult schema across scoring adapters + OBR microsim static-costing bridge - #22

Merged
vahid-ahmadi merged 1 commit into
mainfrom
score-schema-obr-bridge
Jul 17, 2026
Merged

Common ScoreResult schema across scoring adapters + OBR microsim static-costing bridge#22
vahid-ahmadi merged 1 commit into
mainfrom
score-schema-obr-bridge

Conversation

@vahid-ahmadi

Copy link
Copy Markdown
Contributor

Closes #9
Closes #10

ScoreResult schema (#10)

One pydantic ScoreResult for every scoring entry point, emitted per the module's plain-dicts-out convention and additively: og_score_reform, the new obr_score_reform, and pe_population_impact keep their existing payloads unchanged and gain a "score" key containing:

  • model (id) + model_class (microsim / semi-structural / olg-ge), country, reform, horizon (steady-state / quarterly window … / annual …)
  • quantities — gdp, consumption, investment, government, revenue, debt, each a ScoreQuantity with level_bn / delta_bn / delta_pct plus required units and basis; a model fills only what it genuinely produces (the static microsim fills revenue only, and says why in caveats)
  • assumptions, caveats, optional uncertainty, optional distributional block (deciles / winners / losers — microsim only for now)

Payoff wired: macromod compare --reform '…' --models microsim,obr (or og) renders one table from N ScoreResults; --json emits the list. The table footer states explicitly that steady-state and budget-window deltas are each model's own concept.

OBR bridge (#9)

score_reform(model="obr") / obr_score_reform: static costing in → second-round effects out.

  1. pe_population_impact(country="uk", reform, year=y) for each year in [start_year, start_year+years) → annual budgetary_impact_bn path (baseline sims cached per year).
  2. obr_costing_to_shock (pure arithmetic, unit-tested with no heavy deps): annual £bn → quarterly £m, flat within each year (declared), sign-corrected (revenue raised ⇒ disposable income falls).
  3. Injected via run_reform(var="HHDI", shock=[…]) using the upstream path-shock interface (Path-valued shocks and an add-factor injection point for externally costed reforms obr-macroeconomic-model#9's path-valued shocks are on obr main). HHDI is the demand-side incidence channel: HHDI → RHHDI → CONS → GDPM.

Declared boundaries — in the result payload (caveats), not just docstrings:

  • run_reform exogenises HHDI identically in baseline and shocked runs, so the delta isolates the shock, at the stated cost that the economy's feedback onto disposable income itself is not recycled (this is a plain exogenous path, not upstream's proposed held add-factor injection point, which is not implemented there yet).
  • Corporation tax is not household-borne in the microsim: refused with a ValueError pointing at obr_shock(var="TCPRO"), before any heavy import.
  • Demand-side incidence only; supply-side channels belong to the OG member.
  • The revenue quantity in the OBR score block echoes the static costing and its basis says it is the bridge INPUT, not an emulator output.

Also

  • score_reform gains a microsim model so the compare table can include the static costing itself; country=us works there.
  • MCP score_reform docstring/params updated (years, dataset); CLI score gains --years/--dataset and renders the common block for non-OG models.
  • Tests: fast unit tests for the costing→shock translation, schema converters, bridge wiring (both heavy deps stubbed), corp-tax boundary, and compare; the real UK pipeline test is slow-marked and skips cleanly without HUGGING_FACE_TOKEN / obr_macro. Fast suite: 60 passed, 35 skipped (the pre-existing test_summary_parses needs boe_var or MACROMOD_BOE_VAR_REPO, unrelated to this change and green with either).

Deliberately deferred: the site/docs page (#10 mentions it; this PR is the integration layer), any uncertainty band producer (SVAR is not a reform-scoring adapter yet), and the upstream held-add-factor injection point.

🤖 Generated with Claude Code

)

ScoreResult (pydantic, emitted as plain dicts): every scoring adapter
(og_score_reform, obr_score_reform, pe_population_impact) now returns its
existing payload unchanged plus an additive "score" block — model id and
class (microsim / semi-structural / olg-ge), country, reform, horizon,
per-quantity deltas with units and basis, assumptions, caveats, and an
optional distributional block (filled by the microsim only).

OBR bridge: PolicyEngine population static costings per year in the window
enter the OBR emulator as a quarterly HHDI shock path (sign-corrected:
revenue raised lowers disposable income; £bn/yr -> £m/quarter, flat within
each year) via run_reform's path-shock interface; second-round demand
effects on GDP/consumption/investment come out. Corporation-tax reforms are
refused with a pointer to the direct TCPRO lever (not household-borne in
the microsim); supply-side channels stay the OG member's job — both stated
in the payload's caveats, not just the docs.

score_reform now dispatches og / obr / microsim; new `macromod compare`
renders the same reform through several model classes in one table from the
common blocks. Fast tests cover the pure costing->shock translation, the
schema converters, bridge wiring with both heavy deps stubbed, and the
compare command; the real pipeline test is slow-marked and skips cleanly
without HUGGING_FACE_TOKEN / obr_macro.

Closes #9
Closes #10

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@vercel

vercel Bot commented Jul 17, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
macromod Ready Ready Preview, Comment Jul 17, 2026 10:17am

Request Review

@vahid-ahmadi
vahid-ahmadi merged commit aab1401 into main Jul 17, 2026
3 checks passed
@vahid-ahmadi
vahid-ahmadi deleted the score-schema-obr-bridge branch July 17, 2026 10:46
vahid-ahmadi added a commit that referenced this pull request Jul 17, 2026
- Replace the stale 'OBR arm pending' remote test (score_reform model=obr
  no longer errors — the bridge landed in #22) with the corp-tax refusal
  contract: a corporation_tax reform must be refused with a pointer to
  the direct obr_shock TCPRO lever.
- Add a fast post-deploy remote test for score_reform(model='microsim'):
  sign of the costing, ScoreResult coherence, distributional block.
- Add a slow end-to-end remote test of the full bridge pipeline (microsim
  costing -> HHDI path -> OBR second-round effects) with sign and
  magnitude gates; runs in the scheduled full validation.
- Bump validate-deployment timeout 20->35 min: the slow leg gains the
  bridge test and already timed out once at 20.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
vahid-ahmadi added a commit that referenced this pull request Aug 12, 2026
…e the two levers that do not work (#176)

* Refresh OBR figures that had been stale since the ONS anchor, and name the two levers that do not work

A fresh review of the emulator recomputed every published number and
probed each fiscal instrument. Two things needed changing here.

**The free-running scorecard had been stale since PR #22 upstream.** The
`OSHH` ONS anchor landed and these pages went on publishing the pre-anchor
figures, which *understated* the model's accuracy on every line: real GDP
4.48% not 5.75%, consumption 7.49% not 9.56%, household income 6.27% not
14.15%, real household income 6.03% not 13.86%, business investment 15.73%
not 16.12%, company profits 63.29% not 79.80%, current account 3.60% of
GDP not 4.17%. Six of the eleven computed variables land within band, not
four. Regenerated `fig_free_running_data.csv` from the model and
recomputed the MAPEs from the CSV rather than transcribing them, so the
chart and the prose now come from the same place.

**Two of the four fiscal levers do not work, and nothing said so.**
`CGIPS` moves business investment by exactly 0.0 in all twelve quarters
and leaves a wrong-signed residual GDP effect, against the OBR's published
1.0 — the channel is absent, not weak. `TCPRO` never converges: for a
sustained +5pp rise the investment response runs £97m at q3, £2,876m at
q12 and £43,387m at q25, compounding 1.21–1.27x every quarter for 25
quarters, because the stabiliser anchors the level rather than the
deviation. Truncating the shock from twelve quarters to eight barely moves
the q12 answer, so the published number is carried by accumulated drift
rather than by the tax rate.

Both now carry a caveat in the `obr_shock` payload — a docstring does not
reach an MCP caller reading JSON — and on the lever list at /obr/code. The
site already said the government-consumption multiplier is ~1 by
construction against the OBR's published 0.6, and the review confirmed it
is exactly 1.0000 on impact at both £0.1bn and £10bn and still 0.9995 at
quarter 12, with consumption moving £0.64m on a £1,250m shock. That leaves
the household-income lever as the one instrument here with behaviour in it.

Upstream: PolicyEngine/obr-macroeconomic-model#27, which also fixes three
ways to get a confidently wrong answer out of this model — a silent
all-NaN shock reading as "no effect", a corporation-tax rate above 1
returning a wrong-signed finite answer, and a transmission audit that had
published +53.99% GDP for a 1pp corporation-tax rise.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Re-pin the OBR emulator to the merged review

PolicyEngine/obr-macroeconomic-model#27: the multiplier documentation,
the TCPRO domain guard, the transmission-audit fix and the corrected
WB/WP/WV/RDELTA seeds.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Common ScoreResult schema across all scoring adapters OBR reform contract: microsim static costing path in, second-round effects out

1 participant