feat: Phase 0 MVP + Phase 1 bias-boundary (PIT · ann_date · tradability · neutralization) - #1
Conversation
- data/clean/adjust.py: front_adjust (qfq) anchored per-symbol to window-latest; store stays raw (+adj_factor), adjust in memory -> batch==incremental safe (returns are anchor-invariant for return-based factors). - TushareFeed: wire rate_limit (calls/min) + retry with backoff (SEC-004). - pipeline: front-adjust panel after read, before factor/backtest (demo no-op). - BIAS_AUDIT: 复权 section now 'implemented' with the dividend evidence. - 105 tests passed (+12).
- universe/filters.py: shared apply_tradable_filters (missing_close always + suspended/ST/at-limit when toggled and flags present); static + index reuse it. - data/feed/tushare_flags.py: TushareFlagsFeed for suspend_d/namechange/stk_limit. - data/clean/tradability.py: enrich panel with suspended/is_st/at_up_limit/ at_down_limit bool flags. - data/feed/secret.py: shared token reader (never echoed). - pipeline: enrich flags only for tushare + when a flag filter is enabled (demo no-op). - BIAS_AUDIT: tradability now implemented; honestly records that tushare full-day suspensions have no bar (already caught by missing_close) and the limit-direction nuance. 135 tests passed.
- data/clean/pit_financials.py: asof_financials joins financial fields by disclosure date (ann_date <= trade_date, backward merge_asof) — never end_date, so a report is invisible until announced. The core anti-lookahead red-line. - data/feed/tushare_fina.py: TushareFinancialFeed (fina_indicator, ann_date+end_date). - factors/compute/financial.py: FinancialFactor(roe|netprofit_yoy) surfacing the as-of column (no temporal logic; the alignment owns no-lookahead). - pipeline: _build_factor dispatch (momentum vs financial); financial factors require tushare (demo source raises, no fabricated financials). - BIAS_AUDIT: ann_date section now implemented with the 平安银行 Q1 evidence. - 150 tests passed.
- factors/process/neutralize.py: per-date OLS residual of factor on [log(market_cap), one-hot(industry)]; degenerate/missing cross-sections -> NaN (no fabrication). - ProcessingPipeline: neutralize step (order drop->winsorize->neutralize->zscore); enabled-but-missing-covariates raises a readable error. - data/feed/tushare_covariates.py + data/clean/covariates.py: industry (stock_basic, current tag) + market_cap (daily_basic.total_mv) enrichment. - pipeline: covariate enrichment + processor wiring; neutralize needs tushare. - BIAS_AUDIT: neutralization section incl. the stock_basic current-industry PIT downgrade. Validated on 12 names/4 industries: corr(momentum,log_mcap) -0.617 -> -0.000. 161 tests passed.
- _collect_downgrades now leads with an explicit DATA PATH line (DEMO/offline vs REAL tushare) and reports membership / ann_date-financial / neutralization status per run, so a demo result can never be mistaken for a real PIT/financial validation and vice versa. - config/example_tushare.yaml: documents the real path (tushare + PIT index + tradability filters + neutralization); validate-config accepts it. - 164 tests passed.
- RUNBOOK: real-data (tushare) path section + example_tushare.yaml; P1 implemented/deferred status. - TEST_REPORT: 164 passed (exact per-file collect counts, P0=93 + P1=71) + real-data validation summary.
1. financials: fetch ~16 months before start (_FINANCIAL_LOOKBACK_DAYS=500) so the prior already-disclosed report carries forward onto early trade dates (no NaN gap), still ann_date-gated. Locked by carry-forward + lookback tests. 2. price-limit flags use RAW close vs raw stk_limit: tradability enrichment now runs BEFORE front-adjust, so qfq close is used only for factors/returns. Regression test with adj_factor != 1. 3. neutralize_by_date returns NaN for saturated / no-residual-DOF cross-sections (names <= 1 + #industries) instead of fabricated ~0 residuals. 4. BIAS_AUDIT / RUNBOOK / TEST_REPORT updated. 168 tests passed.
tushare fina_indicator filters start_date/end_date by the report period (end_date), not by ann_date. The previous docstring claimed ann_date filtering / 'reports announced in the window', which is misleading: the point-in-time ann_date<=trade_date alignment is done downstream in asof_financials, not by this feed. Docstring only; no behavior change.
验收门 / Acceptance gates — P1 bias-boundaryAll gates re-run on env
Hygiene: no conflict markers; no tushare token in diff/repo; no Last change in this pass (docs only, no behavior)
Known P2 downgrades (honestly deferred, not bugs)
Status: ready for review. No merge / no force-push / no branch deletion performed — awaiting human confirmation. |
|
Update: |
Summary
A reproducible A-share cross-sectional multi-factor framework, taken from a runnable
Phase 0 MVP to a Phase 1 "does-not-fool-itself" research framework: every major
cross-sectional bias boundary is implemented on the real (tushare) data path, validated
against real data, and disclosed. The offline demo path stays deterministic and network-free.
7 layers:
data -> universe -> factors -> alpha -> portfolio -> runtime(backtest|live) -> analytics.One command runs it end-to-end:
python -m qt.cli run-phase0 --config config/example.yaml.Phase 0 — runnable MVP
DemoFeed ->
normalize_panel-> PanelStore(parquet) -> StaticUniverse ->momentum_20-> z-score (by date) -> EqualWeightAlpha -> TopN equal-weight -> monthly-rebalance backtest
(cost + turnover) -> IC / performance -> markdown report. Live
Executionport staysminimal; backtest-only hooks live in a
BacktestExecutionsub-ABC ("backtest == live").Phase 1 — bias boundary (the point of this PR)
Each is a correctness red-line, gated behind the real tushare path (on demo they raise a
readable error instead of fabricating), and disclosed in the run's DATA PATH / DOWNGRADES
section +
BIAS_AUDIT.md:data/clean/adjust.py— store stays raw +adj_factor, adjust in memory -> batch==incremental safe (returns anchor-invariant)universe/index_universe.py+data/feed/index_feed.py—index_weightas-of (latest snapshot <= date, survivorship-safe), 90-day paging, 370-day pre-start lookbackuniverse/filters.py+data/clean/tradability.py— suspend_d / namechange(ST) / stk_limit; limit flags compare raw close (enriched before front-adjust)data/clean/pit_financials.py+FinancialFactor(roe/netprofit_yoy)— figure visible only after disclosure date (neverend_date); ~16-month lookback so the prior report carries forwardfactors/process/neutralize.py— per-date OLS residual on[log(market_cap), one-hot(industry)]; saturated / no-DOF cross-sections -> NaN (no fabricated 0 residuals)_collect_downgradesleads with DATA PATH = DEMO vs REAL; a demo run can never be read as a real PIT/financial validationReal-data validation (against tushare; not in CI — the suite stays network-free)
momentum_20shifts up to 6.77pp.members(2024-06-15)resolves to the 2024-06-03 snapshot; a dropped name stays a member for its era. (Fixed a real trap:index_weightcaps ~6000 rows/call -> paged.)roestays the prior annual (10.24) until 04-19, switches to Q1 (3.12) on 04-22; anend_datejoin would have leaked it ~3 weeks early.Remaining P2 (disclosed, not hidden)
stock_basic.industry, not point-in-time -> mild industry-neutralization look-ahead.min_listing_daysconfigured but not enforced (no-op).Test plan
168 tests (P0 = 93, P1 = 75) incl. no-lookahead, ann_date-as-of, raw-close limit,
neutralize-DOF, path-disclosure regressions. Runtime env: conda
quant_mf(py3.12).No test hits the network or reads the token. Details in
RUNBOOK.md,BIAS_AUDIT.md,TEST_REPORT.md,artifacts/reports/phase1_summary.md.