Conversation
Adds tests/test_soi_sanity_2022.py — five weighted 2022 totals on the
TMD data, each compared against an externally published Statistics of
Income (SOI) figure with 1% relative tolerance:
filers (PUF) vs SOI count gap +0.5%
wages e00200 (PUF) vs SOI employment_income gap -0.3%
AGI c00100 (PUF) vs SOI adjusted_gross_income gap +0.1%
iitax (all rec) vs SOI tottax gap +0.35%
e26270 (all rec) vs SOI partnership_and_s_corp_ gap +0.1%
income minus _losses
The iitax comparator is SOI tottax (= income tax after credits + NIIT
+ Form 4970 tax) rather than income_tax_after_credits, because both
TaxCalc iitax and SOI tottax include NIIT and AMT in scope. Comparing
against income_tax_after_credits would show a 2.3% gap that is purely
a NIIT-inclusion mismatch, not a TMD problem.
For e26270 (Schedule E line 17, partnership and S-corp net income),
the SOI target is the difference of the two SOI variables since SOI
reports income and losses separately. No PUF-only mask is needed for
iitax or e26270 because CPS records contribute zero to both.
Removes from this PR:
- tests/test_variable_totals.py + tests/taxdata_variable_totals.yaml
Superseded entirely. The new SOI sanity test replaces its external-
benchmark role with a 1% tolerance against current SOI 2022 figures
rather than the old 45%-or-$30B tolerance against pre-TY2015
taxdata totals.
- test_misc.py::test_income_tax (was @pytest.mark.skip)
Subsumed by the iitax-vs-tottax check in the new file.
- test_misc.py::test_partnership_s_corp_income (was running)
Folded into the new SOI test as the e26270 check, with a tightened
1% tolerance and a sourced SOI target replacing the previous
unattributed $975 B / 10% tolerance.
Kept in test_misc.py: test_no_negative_weights and test_population.
Both have distinct concepts and external sources from the SOI checks
(min weight invariant; Census population) and stay as standalone
tests.
Test plan:
- make format: clean.
- make lint: exit 0.
- pytest tests/test_soi_sanity_2022.py: 1 passed in ~13s.
- make test: 59 passed, 2 skipped (unchanged set of skip markers
on test_imputed_variable_distribution and test_tax_revenue;
test_income_tax skip is gone because the test itself is gone).
Closes part of the work tracked in #501; remaining items in that
umbrella plan will be retired or moved to focused issues separately.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This was referenced Apr 24, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This is the SOI sanity-check piece described in umbrella #501 (PR 2 in that plan), plus the cleanup deletions that depended on it. It introduces a focused test that anchors five 2022 weighted totals to externally published Statistics of Income (SOI) figures, and retires three pre-existing tests whose roles it covers.
The five checks each pass with comfortable headroom against current TMD data (gaps of 0.1–0.5% against a 1% tolerance).
What's added
tests/test_soi_sanity_2022.py— one test function, five assertions, all with 1% relative tolerance:tmd/storage/input/soi.csv)countemployment_incomeadjusted_gross_incometottaxpartnership_and_s_corp_incomeminuspartnership_and_s_corp_lossesA note on the income-tax comparator
The comparator for
iitaxis SOItottax, notincome_tax_after_credits, because both TaxCalciitax(=c09200 - refund) and SOItottaxinclude NIIT in scope (and AMT, and other additional taxes except SE and Additional Medicare). Comparing againstincome_tax_after_creditswould show a 2.3% gap that is purely a NIIT-inclusion mismatch, not anything to do with TMD.A note on PUF-only filtering
Three checks restrict to the PUF subsample (
data_source == 1) because the SOI "All filers, full population" target is a filer-population concept. CPS-only records in TMD represent the non-filer population. For theiitaxande26270checks the mask is unnecessary because CPS records contribute zero to both; the simpler unmask form is used there.What's removed
tests/test_variable_totals.py+tests/taxdata_variable_totals.yaml— superseded entirely. The old test compared against pre-TY2015 taxdata PUF totals with a 45%-or-$30B tolerance — too loose to catch real regressions. The new SOI sanity test replaces that role with current SOI 2022 figures and a 1% tolerance.test_misc.py::test_income_tax(was@pytest.mark.skip) — subsumed by the iitax-vs-tottax check.test_misc.py::test_partnership_s_corp_income(was running) — folded into the new SOI test as the e26270 check, with a tightened 1% tolerance and a sourced SOI target replacing the previous unattributed $975 B / 10% tolerance.What stays in
test_misc.pytest_no_negative_weights— narrow invariant check.test_population— Census-anchored population check (distinct external source from SOI; tighter 0.1% tolerance is appropriate).Test plan
make format— clean.make lint— exit 0.pytest tests/test_soi_sanity_2022.py -v— 1 passed in ~13s.make test— 59 passed, 2 skipped (the 2 skipped are the existing markers ontest_imputed_variable_distributionandtest_tax_revenue, both unchanged by this PR; the previous third skip ontest_income_taxis gone because that test was retired).Related
iitax/payrolltaxwith a CBO or Treasury comparator #502 — sister design-discussion issue on future-year revenue comparator (open, independent, awaiting input).