feat(neutralize): P2-3 point-in-time SW industry covariate (replace current-tag) - #5
Merged
Conversation
…t-tag broadcast) P2-3 upgrades the industry neutralization covariate from the CURRENT `stock_basic.industry` tag (broadcast to every date — a mild membership look-ahead) to a POINT-IN-TIME, as-of-trade-date SW-L1 industry. The framework's layering is unchanged; the neutralize math is untouched (it already drops NaN-industry rows). - data/feed/tushare_covariates.py: `pit_sw_l1_intervals(symbols)` reads tushare `index_member_all` for each symbol's SW-L1 membership intervals (in_date/out_date; out_date None = active). - data/clean/pit_industry.py (new): `asof_industry(index, intervals)` picks, per (trade_date, symbol), the industry whose `[in_date, out_date)` covers that date (new industry wins on a reclassification date — PIT-safe; pre-start membership carries forward; no covering interval / unknown symbol -> NaN). - data/clean/covariates.py: `enrich_pit_industry(panel, industry_series)` sets a per-date `industry` column from the as-of Series. - qt/pipeline.py: `_maybe_enrich_covariates` now builds the PIT industry (real path) instead of the current tag; logs coverage. A name with no SW history stays NaN — a disclosed coverage gap the neutralizer drops — NEVER a silent current-tag fallback. `_collect_downgrades` neutralization text rewritten (PIT, UNI-010). - qt/phase2_baseline.py + qt/reports.py: report discloses PIT industry coverage (fraction of (date, symbol) rows with an as-of industry). - Docs: BIAS_AUDIT.md (regenerated), RUNBOOK.md, TEST_REPORT.md, CLAUDE.md. Tests (217 passed, network-free): test_pit_industry.py (8) — as-of switch / carry-forward / missing->NaN / overlap / enrich / pipeline wiring; +1 each to test_tushare_covariates (index_member_all parse), test_real_path_config (PIT downgrade text, not "current"), test_bias_audit_report, test_phase2_baseline (PIT coverage rendered). The current-tag `industry()` feed method remains (tested) but is no longer wired into neutralization.
…t L1) Follow-up to the P2-3 PIT industry change, after a real L1/L2 comparison. - config: add `processing.neutralize.industry_level` (L1/L2/L3, default L1); phase2_real_baseline.yaml sets it explicitly. `pit_sw_l1_intervals` is generalized to `pit_sw_intervals(symbols, level)` (reads l1/l2/l3_name from index_member_all); pipeline passes the configured level; report / downgrades / BIAS_AUDIT disclose the actual level (no hardcoded SW-L1). Why L1 default (corrected rationale): a real comparison showed the SSE50 annual return is old stock_basic.industry −17.6%, SW-L1 −10.2%, SW-L2 −9.3% — L1 ≈ L2. So the −17.6→−10 change is the tushare→SW TAXONOMY switch, which is INHERENT to going point-in-time (only SW carries in/out-date history; the old tag cannot be PIT-aligned), NOT an industry-granularity choice. The earlier "L2 ≈ old granularity isolates the PIT fix" rationale was disproven by the L2 run. L1 (31 broad sectors) is the standard granularity for industry neutralization and the DOF-safest on small cross-sections, so it is the default; L2/L3 remain available. Tests (223 passed): industry_level config (default L1, accepts L1/L2/L3, rejects invalid, phase2 config = L1), level-selective feed parse (L1/L2/L3), pipeline passes the configured level, report discloses the level. Docs (BIAS_AUDIT regenerated, RUNBOOK, TEST_REPORT, CLAUDE, AGENTS) carry the corrected taxonomy- vs-granularity finding.
… default L1' Comment/docstring-only (no behavior): feed module docstring (default L2 -> L1), pit_industry module docstring (SW-L1 / old pit_sw_l1_intervals -> SW level-configurable / pit_sw_intervals), test comments (default L2 -> L1), and phase2 coverage comments (PIT SW-L1 -> PIT SW industry coverage). The runtime already passes the configured industry_level (default L1).
…d level (not SW-L1)
StackOverFlow11
added a commit
that referenced
this pull request
Jun 9, 2026
docs: progress — PR #5 (P2-3 PIT industry) merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
P2-3 upgrades the industry neutralization covariate from the CURRENT
stock_basic.industrytag (broadcast to every date — a membership look-ahead) to a point-in-time, as-of-trade-date SW industry, with the SW level configurable (processing.neutralize.industry_level, L1/L2/L3, default L1). No new factor, no parameter tuning; the framework layering and theneutralizemath are unchanged (it already drops NaN-industry rows).What changed
data/feed/tushare_covariates.py—pit_sw_intervals(symbols, level)reads tushareindex_member_allfor each symbol's SW membership intervals at the chosen level (l1/l2/l3_name,in_date/out_date;out_date None= active). Theindustry()current-tag accessor is retained but no longer wired into neutralization.data/clean/pit_industry.py(new) —asof_industry(index, intervals)picks, per(trade_date, symbol), the industry whose[in_date, out_date)covers that date (new industry wins on a reclassification date — PIT-safe; pre-start membership carries forward; no covering interval / unknown symbol →NaN).data/clean/covariates.py—enrich_pit_industry(panel, industry_series)sets a per-dateindustrycolumn.qt/config.py—processing.neutralize.industry_level(L1/L2/L3, default L1).qt/pipeline.py—_maybe_enrich_covariatesbuilds the PIT industry at the configured level; logs coverage. A name with no SW history staysNaN— a disclosed coverage gap the neutralizer drops — never a silent current-tag fallback._collect_downgradesrewritten (PIT, configurable level, UNI-010).qt/phase2_baseline.py+qt/reports.py— the phase2 report discloses the actual SW level + PIT industry coverage.BIAS_AUDIT.md(regenerated),RUNBOOK.md,TEST_REPORT.md,CLAUDE.md,AGENTS.md.Real-data finding (this is the important part)
Going PIT is not behaviour-neutral, and a real L1-vs-L2 comparison disproved the initial "L2 isolates granularity" hypothesis. On the SSE50 baseline (2023-07..2024-06, momentum_20):
stock_basic.industry(current tag)(Raw-factor IC is unchanged at 0.0083 — neutralization affects the portfolio via the processed scores, not the raw IC.)
L1 ≈ L2, so the −17.6 → −10 jump is the tushare → SW taxonomy switch, which is inherent to going point-in-time: only the SW classification carries the
in_date/out_datehistory needed for an as-of alignment; the oldstock_basic.industrytag cannot be PIT-aligned at all. It is not a granularity choice.Default = SW-L1 (31 broad sectors): the standard granularity for industry neutralization and the DOF-safest on small cross-sections (~50 names/cross-section on SSE50; L2's ~130 sub-industries approach saturation). L2/L3 remain available via config.
Real baseline (SSE50, manual, not in CI)
Default SW-L1:
run-phase2-baselineOK — 68 symbols, 11 settled rebalances, PIT SW-L1 coverage 98.53% (67/68; 1 symbol has no SW history → NaN, disclosed), annual return −10.2%. The report regenerates with the actual level + coverage line.Quality gates
pytest -p no:cacheproviderruff check .validate-config× (example / example_tushare / phase2_real_baseline)run-phase0(demo)Hygiene: no tushare token, conflict markers, or
artifacts//*.parquettracked.Tests (network-free)
test_pit_industry.py(12): as-of switch / carry-forward / missing→NaN / overlap,enrich_pit_industry, pipeline wiring (per-date industry, configured level passed, no current-tag fallback), andindustry_levelconfig (default L1, accepts L1/L2/L3, rejects invalid, phase2 config = L1).+level-selective feed parse and report-level disclosure in the covariates / real-path / bias-audit / phase2 tests.