Impute capital gains amounts from the published joint distribution - #560
Conversation
|
Heads-up on a planned collision: #612 (carrying UK build datasets as populace Frames) is in motion — the foundations PR #617 is up, and the carrier-swap PR will stack on #610 and re-type Both orders are pre-agreed fine: if this merges first, the swap absorbs the conversion as one more stage (same pattern as the two production stages); if the swap merges first, I'm happy to push the Frame conversion onto this branch myself so it doesn't sit in your queue. No hold needed either way. |
|
The swap landed first, so this branch paid the rebase as predicted: 1470a33 merges main, and the adaptation commit carries the stage on the Frame carrier — reads through frame.table, build period from frame metadata, rebuild via uk_national_frame with kind and mass log unchanged (the person-only-replacement idiom from the migrated stages). The adaptation exposed that no test ran the stage factory's own transform (postponed annotations hid a retired type in its signature), so there's now an end-to-end test on the pinned artifact. All 46 CGT tests green; full suite running. |
|
An adversarial review pass (cross-model, pre-merge) blocked this with four criticals, all confirmed against the published table. Fixed on the branch; the review trail: C1 — rounded means outside their band crash the stage. Confirmed: four cells of the 2023-24 table imply a mean below their band floor through rounding — £463m over a rounded 1,000 people reads as £463,000 for the £500k–£1m band (also (£2m–£5m, £50k), (£1m–£2m, £100k), (£250k–£500k, £150k)). The solvers refused those means, so the stage crashed for any population reaching four of the six income bands — while every synthetic test mean sat feasibly in band, and the one real-artifact test used the single income band with no infeasible cell. Implied means are now repaired to just inside the violated boundary (2% of band width), keeping the signal that the cell's mass sits near that edge. C2 — the 500-person suppression convention overstates mass. Replaced with reconciliation: suppressed cells split the residual between the published income-column total (newly captured from the All row, on #558) and the column's unsuppressed cells. No invented count survives; allocation reconciles to published totals by construction. C3 — bottom-band draws at or below the AEA. The £0–£10k band straddles the £6,000 AEA, so draws could land people allocated as liable taxpayers below the liability threshold. Its support is now floored at the AEA where that fits inside the band. C4 — the masking. The real 2023-24 surface is now embedded in the tests as literals (public published statistics), so CI exercises the exact rounding pathology with no artifact present — including a test pinning the repaired £500k-band mean — and the pinned-artifact end-to-end test spans all six income bands instead of the one clean one. Cleared by the same review: deterministic RNG/ranking, and the Frame weight-kind/mass/linkage invariants. |
Rebuilt as one commit on the renamed tree (microcosm.* namespace landed on main mid-flight, after the Frame carrier swap had already done the same), carrying the full reviewed history of the stage: the original rank-preserving amounts-only redraw from HMRC table 3; the Frame carrier adaptation; and two adversarial review rounds — repair of infeasible rounded means (four 2023-24 cells imply a mean outside their own band), suppressed-count cells taking the count their published gains imply at the band mean with every income column rescaled onto its published total, the bottom band's support starting one pound past the annual exempt amount, and the real published surface embedded as always-run test literals. The stage owns amounts, not incidence and not weights. Taxable income is an arithmetic proxy over persisted total_income components less a tapered Personal Allowance read from the policyengine-uk parameter tree through the uk extra's deferred import. Draws are seeded on the build period. Whether the band facts become calibration targets stays behind the reviewed fence adjudicated on #552. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
98eb31d to
38be34d
Compare
The previous round claimed the end-to-end incomes covered all six income bands; they did not — £45,000 maps to £32,430 after the Personal Allowance, below the £37,700 bound, so the band went unexercised through the imputation path in both fixtures. £55,000 reaches it, and the test now asserts the proxy visits every published band, so the fixture cannot silently regress the way it silently overclaimed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Part two of #552, stacked on #558 (the source surface). Review #558 first; this PR's own diff is the last commit.
What the stage does
Redraws existing gainers'
capital_gainsfrom HMRC table 3 — the published joint distribution of size of gain by taxable income — so the population can hold the gains above £2m that carry most of the tax. The measured gap this closes: the two top bands hold 54.6% of UK CGT and effectively none of the modelled tax, because the incumbent percentile source stops near £1m.The stage owns amounts only. Incidence comes from the candidate, household weights pass through untouched (no mass change to record), and whether the published band facts also become calibration targets remains the fence adjudication recorded on the issue — nothing here assumes an answer.
Design
total_incomeconcept (ITA 2007 s.23) and subtracts a tapered allowance, with the allowance amounts read from the policyengine-uk parameter tree at a stated instant — theukextra's deferred-import pattern, per its own comment that the base package never imports policyengine-uk at import time.state_pension_reportedstands in forsocial_security_income; reliefs are not deducted. Both divergences are documented where they can move a person one band.summarize_uk_cgt_imputationcompares achieved band totals with the published surface. It is reporting, not a gate: where the population holds less gainer mass than HMRC's taxpayers the achieved totals sit below published by construction, and holding levels to the surface is the calibration question.What it deliberately does not do
No
family_coverageentry, nocgt_source_stages.json, no band-levelUK_CGT_TARGET_SPECS. Promoting the stage torequired_at_buildand declaring targets follow the fence adjudication on #552, on the individuals-only basis (the two existing facts include trusts).Tests
17 new tests: proxy arithmetic including the taper, mean-matching for both draw families (with the removable singularity at the uniform midpoint covered), determinism, rank preservation, the sub-AEA remainder, suppressed-cell fallback, artifact verification before read, and the parameter-tree read (runs when the
ukextra is installed;importorskipotherwise — it is absent from the secrets-free PR gate by design). Full populace-build suite green locally with the extra installed.🤖 Generated with Claude Code