diff --git a/docs/factors/d6_deferred_register.md b/docs/factors/d6_deferred_register.md new file mode 100644 index 0000000..1d237b1 --- /dev/null +++ b/docs/factors/d6_deferred_register.md @@ -0,0 +1,190 @@ +# D6 deferred register + +Things D6a **found and deliberately did not do**, with the reason and the step +that owns them. The point of writing them down is that a deferral nobody +recorded is indistinguishable from a defect nobody noticed. + +D6a's value is that it is a **path switch provable to be a no-op**: the factor +values stop coming from `factor.compute(panel)` in the runner and start coming +from `factors.service`, and the reconciliation shows the values did not move. +Every item below would have changed something else at the same time, which is +exactly what would have destroyed that property — if a number had moved there +would no longer be a single candidate explanation. + +--- + +## D6a-0 — INVARIANT (not a deferral): every daily factor is per-symbol + +**Status:** now guarded by +`tests/test_daily_factor_universe_independence.py`. **Owner:** anyone adding a +daily factor. + +The store key is `(factor_id, params, code, view)` and has **no universe +dimension** — deliberately, and it cannot have one: a PIT universe over a date +range is a time-varying *set*, not a value a key dimension could hold (design +revision A2). That is sound only while a stored value is universe-independent. + +On the minute plane this failed once already and was measured: `intraday_amp_cut` +ran its cross-sectional z-score during materialization, so its stored value *was* +a function of the loaded universe while the key said otherwise — fill a store +with 12 names, read it back asking for 24, and 24 wrong cells were served with +zero recompute, zero error, zero disclosure. **D4c** fixed it by storing the +per-symbol intermediate and moving the combine to read-assembly. + +**The daily plane never inherited that fix, because until D6a no daily factor +value was ever stored.** D6a is the step that puts them in the shared store, so +the property that keeps the daily plane sound becomes load-bearing here. It held +before by accident of what happened to be written; from here it holds on purpose. + +The guard is **behavioural**, not a source scan: it computes every registered +daily factor over a 5-name universe and a 3-name subset and requires the shared +symbols to be bit-identical. A scan for `groupby(level="date")` / `.rank(` is +both too strict (`groupby(level="symbol").rank()` is a per-symbol time-series +rank) and too loose — mutation evidence: a cross-sectional demean written as +`x.unstack().sub(x.unstack().mean(axis=1), axis=0).stack()` turns the census red +while containing **zero** of those tokens. + +**A red here is STOP-AND-REPORT.** The fix is D4c's — store the per-symbol +intermediate, combine at read-assembly — **not** an exemption list, which would +rebuild exactly the silent-wrong-value failure D4c closed. + +Note this is a different question from [D6a-1](#d6a-1--provenance-dimension-on-the-store-key) +below: that one is about telling two *real vintages* apart, this one is about +whether a value is a function of *who else was loaded*. + +--- + +## D6a-1 — provenance dimension on the store key + +**Status:** deferred, structural follow-up. **Owner:** not yet assigned. + +The store key is `(factor_id, params, code, view)` plus a data fingerprint over +the PIT/schema modules. Nothing in it says *which data* produced the value. + +D6a closed the half of this that is a **correctness** problem — a synthetic +source's values are not a function of `(symbol, date)` at all (see +`qt/factor_source.py`), so they are refused a durable store outright. + +What a provenance dimension would *additionally* buy is telling two **real** +vintages apart: a cache backfill or a tail refresh can change the bytes a +factor was computed from while the `data.clean` module hashes stay put, and the +stored value is then reused as-is. That is a **pre-existing D3 property** +(`factors/service.py::_record_fill_footprint` already records it in its closing +note), not something D6a introduced, and it applies to every consumer of the +store — the evaluation plane included. + +**Why not now:** it is a change to the D3 store's identity model, it invalidates +every stored artifact when it lands, and it is orthogonal to routing the daily +runners through the service. + +--- + +## D6a-2 — the enrichment dispatch still uses `isinstance` + +**Status:** deferred. **Owner:** D6d (or its own PR). + +`qt/pipeline.py::_maybe_enrich_financials` and `::_maybe_enrich_value` decide +which columns to fetch by testing `isinstance(f, FinancialFactor)` / +`isinstance(f, ValueFactor)`. That is the pattern red line #5 forbids +("依赖显式声明，编排通用消费"), and the declaration it should be reading +already exists and is already correct: + +* `FinancialFactor.spec.requires == (PanelField(field, source=FINA_INDICATOR),)` +* `ValueFactor.spec.requires == (PanelField(pe|pb, source=DAILY_BASIC),)` + +and `factors.registry.requirements(names, params)` — whose own docstring calls +itself "the D4 materializer's one-stop shopping list" — has **zero production +callers**. + +**Note this is a pre-existing violation, not one D6a introduced.** Converting the +dispatch needs a source-endpoint → enricher routing (the `requires` name the +*source* field `pe`, while the enricher writes the *derived* column `value_ep`), +so it is a real change with its own behaviour surface, not a rename. + +--- + +## D6a-3 — three runners do not pass the shared cache to three enrichments + +**Status:** deferred, behaviour-preserving-on-purpose. **Owner:** D6b. + +`qt/phase2_baseline.py`, `qt/oos_stability.py` and `qt/subset_validation.py` +call + +``` +_maybe_enrich_financials(cfg, panel, symbols, factors, logger) +_maybe_enrich_value(cfg, panel, symbols, factors, logger) +_maybe_enrich_covariates(cfg, panel, symbols, logger) +``` + +with five positional arguments, so the trailing `cache=None` default applies and +those three endpoints (`fina_indicator`, `daily_basic`, `index_member_all`) are +fetched **live on every run**, bypassing the P4 read-through cache. +`qt.pipeline.run_phase0` passes the cache to all four. + +**Why not now:** passing it *is* a behaviour change (gap-fetch counts and wall +time move, and a cached read can serve a different vintage than a live one), and +D6a's whole claim is that nothing but the factor-sourcing path changed. Fixing it +inside a path switch would mean a moved number with two candidate causes. + +D6b touches these three runners anyway. + +--- + +## D6a-4 — `config/phase2_real_baseline.yaml` has no cache block + +**Status:** noted. **Owner:** D6b or ops. + +The un-cached phase2 config cannot run cache-only; `config/phase2_real_baseline_cached.yaml` +is its cached twin (same universe, same window) and is what D6a's real-run +reconciliation used. + +--- + +## D6a-5 — the phase2 baseline's published headline numbers no longer reproduce + +**Status:** observed, **not** caused by the refactor. **Owner:** whoever next +cites those numbers. + +D6a captured a phase2 run on **unmodified `main`** before switching anything +(that capture is the reconciliation's reference). Its headline metrics do not +match the ones recorded in the progress archive for the same config: + +| metric | archived | measured on `main`, 2026-07-31 | +|---|---|---| +| `ic_mean` | 0.0083 | 0.008275 | +| `annual_return` | −10.19% | −9.41% | +| `max_drawdown` | −16.52% | −16.33% | +| `volatility` | 16.59% | 16.64% | +| `sharpe` | −0.5703 | −0.5168 | +| `avg_turnover` | 1.0818 | 1.0727 | + +The factor IC is unchanged, so the market panel and the factor are not what +moved; the selection is, which points at the neutralization covariates +(PIT SW industry via `index_member_all`, market cap via `daily_basic`) — both +fetched **live** on this path (see D6a-3) and both subject to upstream revision. + +**Consequence for anyone reading a reconciliation:** the archived numbers are not +a valid baseline for a code change made today. Compare a capture you took +yourself, on the tree you are changing, immediately before you change it. + +### The premise D6a's own reconciliation rests on, stated + +The same live-covariate mechanism puts a condition on D6a's before/after +comparison, and it should be written down rather than left implicit: + +> **The reconciliation is valid only if the covariates were not revised upstream +> between the two captures.** + +They were taken 18 minutes apart (legacy 11:23, served 11:41), and because those +three endpoints are fetched live on this path (D6a-3), an upstream revision +landing in that window would have moved the served run for a reason that has +nothing to do with the change under test — in either direction, and equally able +to *mask* a real difference as to invent one. + +Nothing of the sort happened here: the factor panel matched cell-for-cell and +every headline metric matched at full precision, which is itself strong evidence +that the covariates were stable across the window (a revision would have had to +leave all of them exactly unchanged). But that is evidence collected *after* the +fact, not a control designed in. A future reconciliation on this path that shows +a small movement must rule this out **before** attributing it to the code — +re-capture the legacy side and check legacy-vs-legacy first. diff --git a/qt/factor_eval_providers.py b/qt/factor_eval_providers.py index 19ef47b..7e054ff 100644 --- a/qt/factor_eval_providers.py +++ b/qt/factor_eval_providers.py @@ -11,8 +11,9 @@ ``qt.factor_hotpath_smoke`` (single source; the smoke and the probes import it from here now). Zero live calls: ``IntradayParquetStore.read_range`` has no fetch closure, so a missing month is an empty read, never a warm. -* :class:`DailyEvalPanelProvider` — serves the pipeline's loaded daily panel - through the ``DailyPanelProvider`` protocol. +* :class:`DailyEvalPanelProvider` — RE-EXPORTED from ``qt.factor_source`` + (D6a moved it there so the daily runners share the one definition; this + module imports ``qt.pipeline``, so ``qt.pipeline`` cannot import from it). * :func:`build_eval_service` / :class:`EvalServiceBundle` — the one wiring of cache -> universe -> panel -> enrichments -> store + sources, in the SAME call order the legacy eval runners used (``qt/eval_jump_amount_corr.py``). @@ -34,9 +35,9 @@ empty_intraday_bars, normalize_intraday_bars, ) -from data.clean.schema import DATE_LEVEL, SYMBOL_LEVEL from factors.materialize import MaterializeSources from factors.store import FactorValueStore +from qt.factor_source import DailyEvalPanelProvider as _DailyEvalPanelProvider from qt.pipeline import ( _build_cache, _build_universe, @@ -97,38 +98,12 @@ def minute_bars(self, symbols, start, end): return pd.concat(parts).sort_index(kind="mergesort") -class DailyEvalPanelProvider: - """``DailyPanelProvider`` over the pipeline's loaded evaluation panel. - - CLOSE-VIEW, NOT LAGGED — and that is exactly right: the materializer's own - daily path applies ``factors.view_lag.daily_decision_lag`` for the decision - view (the prev-day shift with the field-level ``open`` exception, R18), so - the provider must hand it values dated at their natural close date. A - pre-lagged panel would be shifted TWICE. ``qt.pipeline._load_panel``'s - product (raw bars enriched with tradability flags, front-adjusted in - memory) is precisely such an un-lagged close-view panel, so this provider - is a thin window/symbol slicer over it. - - WINDOW SEMANTICS: the panel covers the configured ``[data.start, data.end]`` - window (the same window the legacy runners loaded). A materializer load - request reaching before the panel's first date is served what exists — - the trailing-trading-day trim then treats the panel's left edge like the - data start (honest under-warm NaN), which reproduces the legacy runners' - warmup geometry rather than silently inventing deeper history. - """ - - def __init__(self, panel: pd.DataFrame) -> None: - self._panel = panel - - def daily_panel(self, symbols, start, end): - if self._panel.empty: - return self._panel - dates = self._panel.index.get_level_values(DATE_LEVEL) - mask = (dates >= pd.Timestamp(start)) & (dates <= pd.Timestamp(end)) - out = self._panel[mask] - keep = [str(s) for s in symbols] - syms = out.index.get_level_values(SYMBOL_LEVEL) - return out[syms.isin(keep)] +#: RE-EXPORT, not a copy (D6a): the daily close-view provider now lives in +#: ``qt.factor_source``, because the daily runners need it too and this module +#: imports ``qt.pipeline`` (so ``qt.pipeline`` cannot import back from here). +#: Its "close-view, NOT lagged" contract is load-bearing on both planes and is +#: stated once, there. ``tests/test_factor_source.py`` pins the identity. +DailyEvalPanelProvider = _DailyEvalPanelProvider @dataclass(frozen=True) diff --git a/qt/factor_source.py b/qt/factor_source.py new file mode 100644 index 0000000..125ebe0 --- /dev/null +++ b/qt/factor_source.py @@ -0,0 +1,307 @@ +"""The pipeline's factor-VALUE source: one wiring of the factor service (D6a). + +Every daily runner used to get its factor values from ``factor.compute(panel)`` +directly. That was the second factor-sourcing path the refactor exists to kill +(design decision 3), so the values now come from ``factors.service`` — the store +read-through onto the ONE materializer engine. This module is the qt side of +that wiring for the daily close-view plane: the store policy, the daily panel +provider, and the served-panel assembly. + +VIEW. The daily runners decide at the close of ``d`` and hold from ``d+1``, so +their pairing is ``close`` x ``close_to_close`` — the other of the two legal +pairings (``data.availability_policy.require_legal_pairing`` enforces it at the +service boundary). Under the close view the materializer applies NO availability +lag, which is why routing these runners through the service is a change of PATH +and not of VALUES. + +STORE POLICY (the reason it exists is the reason it is enforced structurally). +A stored factor value is addressed by ``(factor_id, params, code, view)`` plus a +data fingerprint that covers the PIT/schema modules — nothing in that identity +says WHICH DATA produced the value. For a real market source that is sound and +deliberate (design §3.4): the same symbol-date bar is the same fact whoever asks +for it. For a SYNTHETIC source it is false, and demonstrably so: +``data.feed.demo_feed`` builds each price path from ``arange(n)`` counted off +``data.start``, so the demo close for one (date, symbol) CHANGES when the +configured window start changes, while the store key cannot see ``data.start`` +at all. + + A value whose identity the key cannot capture is, by definition, not + storable. + +So the answer here is not a wider key, it is REFUSAL: a synthetic source gets an +ephemeral per-run store and :func:`factor_store_root` raises rather than hand one +a persistent root. This is not a convention the callers must remember — there is +no code path from a synthetic config to a durable artifact. + +What refusing costs: a demo run recomputes its factors every time. That is +exactly what it did before the service existed, so nothing regresses. + +What a provenance dimension on the key WOULD additionally buy — telling two REAL +vintages apart when a cache backfill changes the bytes while the ``data.clean`` +module hashes do not — is a pre-existing D3 property, not something this step +introduces, and is registered as a structural follow-up rather than done here. + +Layering: this module imports the factor layer and the config leaf only; it must +NOT import ``qt.pipeline`` (which imports this), and it carries no feed and no +token. +""" + +from __future__ import annotations + +import logging +import typing +from collections.abc import Iterable, Mapping, Sequence +from contextlib import contextmanager +from dataclasses import dataclass +from pathlib import Path +from tempfile import TemporaryDirectory + +import pandas as pd + +from data.availability_policy import ReturnBasis, View +from data.clean.schema import DATE_LEVEL, SYMBOL_LEVEL +from factors import service as factor_service +from factors.base import Factor +from factors.materialize import MaterializeSources +from factors.store import FactorValueStore + +#: Sources whose bars are FABRICATED by the process that reads them. Their +#: values are not a function of (symbol, date) alone, so they are not storable +#: under a key that names neither the source nor the window (see module docstring). +SYNTHETIC_SOURCES: frozenset[str] = frozenset({"demo"}) + +#: Sources whose bars are observed market facts: the same (symbol, date) bar is +#: the same fact for every caller, which is what makes a shared store sound. +PERSISTABLE_SOURCES: frozenset[str] = frozenset({"tushare"}) + +#: Directory name of the factor value store under a run's ``output.root_dir``. +FACTOR_STORE_DIRNAME = "factor_store" + + +def known_sources() -> frozenset[str]: + """Every source this module has classified (synthetic + persistable).""" + return SYNTHETIC_SOURCES | PERSISTABLE_SOURCES + + +def config_declared_sources() -> frozenset[str]: + """The sources ``qt.config`` actually accepts (the drift-guard reference). + + Read off the ``DataCfg.source`` Literal rather than restated, so the pair of + sets above can be checked against the config surface instead of trusted. + """ + from qt.config import DataCfg + + return frozenset(typing.get_args(DataCfg.model_fields["source"].annotation)) + + +def is_synthetic_source(source: str) -> bool: + """True iff ``source`` fabricates its bars; readable error if unclassified. + + An unknown source is refused rather than assumed real: guessing would mean + silently granting a durable store to data nobody has reasoned about, which + is precisely the failure this module exists to make impossible (red line #9). + """ + name = str(source) + if name in SYNTHETIC_SOURCES: + return True + if name in PERSISTABLE_SOURCES: + return False + raise ValueError( + f"data.source {name!r} has no factor-store storability classification. " + f"Add it to qt.factor_source.SYNTHETIC_SOURCES (fabricated bars -> " + f"ephemeral store) or PERSISTABLE_SOURCES (observed market facts -> " + f"shared store); known: {sorted(known_sources())}." + ) + + +def factor_store_root(cfg) -> str: + """The PERSISTENT factor-store root for ``cfg`` (raises for a synthetic source). + + Derived from ``output.root_dir`` rather than configured separately: a run + that redirects its outputs (every test does) must take its factor store with + it, and one fewer knob is one fewer thing to point at the wrong tree. + + Asking this of a synthetic-source config is a readable error, not a fallback + — see the module docstring for why such values are not storable at all. + """ + if is_synthetic_source(cfg.data.source): + raise ValueError( + f"data.source={cfg.data.source!r} fabricates its bars, so its factor " + f"values have no persistent store root: the demo close for a (date, " + f"symbol) is built from a counter that starts at data.start, so the " + f"same key would address different values for different windows. Use " + f"open_factor_value_store(cfg), which gives a synthetic run an " + f"ephemeral store." + ) + return str(Path(cfg.output.root_dir) / FACTOR_STORE_DIRNAME) + + +@contextmanager +def open_factor_value_store(cfg, logger: logging.Logger | None = None): + """Yield the factor value store for ``cfg``: shared for real, ephemeral for demo. + + The synthetic branch's store lives in a temporary directory that is removed + on exit, so a demo run leaves no artifact any later run could be served. The + real branch uses :func:`factor_store_root`. Either way the ONE materializer + engine behind the service is the same — what differs is only whether its + output outlives the run. + """ + synthetic = is_synthetic_source(cfg.data.source) + if synthetic: + with TemporaryDirectory(prefix="qt-factor-store-ephemeral-") as tmp: + if logger is not None: + logger.info( + "factor store: EPHEMERAL (data.source=%s fabricates its bars; " + "its values are not addressable by the store key, so they are " + "never persisted). Factors are recomputed every run.", + cfg.data.source, + ) + yield FactorValueStore(tmp) + return + root = factor_store_root(cfg) + if logger is not None: + logger.info("factor store: shared read-through at %s", root) + yield FactorValueStore(root) + + +class DailyEvalPanelProvider: + """``DailyPanelProvider`` over an already-loaded daily panel. + + CLOSE-VIEW, NOT LAGGED — and that is exactly right: the materializer's own + daily path applies ``factors.view_lag.daily_decision_lag`` for the decision + view (the prev-day shift with the field-level ``open`` exception, R18), so + the provider must hand it values dated at their natural close date. A + pre-lagged panel would be shifted TWICE. ``qt.pipeline._load_panel``'s + product (raw bars enriched with tradability flags, front-adjusted in + memory) is precisely such an un-lagged close-view panel, so this provider + is a thin window/symbol slicer over it. + + WINDOW SEMANTICS: the panel covers the configured ``[data.start, data.end]`` + window (the same window the legacy runners loaded). A materializer load + request reaching before the panel's first date is served what exists — + the trailing-trading-day trim then treats the panel's left edge like the + data start (honest under-warm NaN), which reproduces the legacy runners' + warmup geometry rather than silently inventing deeper history. + """ + + def __init__(self, panel: pd.DataFrame) -> None: + self._panel = panel + + def daily_panel(self, symbols, start, end): + if self._panel.empty: + return self._panel + dates = self._panel.index.get_level_values(DATE_LEVEL) + mask = (dates >= pd.Timestamp(start)) & (dates <= pd.Timestamp(end)) + out = self._panel[mask] + keep = [str(s) for s in symbols] + syms = out.index.get_level_values(SYMBOL_LEVEL) + return out[syms.isin(keep)] + + +def decision_points(panel: pd.DataFrame) -> list[factor_service.DecisionPoint]: + """One DecisionPoint per trading day present in ``panel``. + + The within-day cutoff is the ``DecisionPoint`` default; it selects minute + bars and is inert on this close-view daily plane, where no factor reads a + minute endpoint. (``qt.factor_eval_runner._decisions`` is the same reduction + on the evaluation plane; converging them means touching the eval path, which + this step deliberately leaves frozen — D6c.) + """ + dates = pd.DatetimeIndex( + sorted({pd.Timestamp(d).normalize() for d in panel.index.get_level_values(DATE_LEVEL)}) + ) + return [factor_service.DecisionPoint(d) for d in dates] + + +@dataclass(frozen=True) +class ServedFactorValues: + """One service read, reduced to the panel's own grid (+ what that dropped).""" + + frame: pd.DataFrame + served_rows: int + footprint_rows_dropped: int + + +def factor_values( + factors: Sequence[Factor], + panel: pd.DataFrame, + symbols: Iterable[str], + *, + store: FactorValueStore, + params_by_id: Mapping[str, Mapping[str, object]] | None = None, +) -> ServedFactorValues: + """Raw values for ``factors`` over ``panel``'s grid, from the factor service. + + Close view x close-to-close basis (the legal pairing for a runner that + decides at the close of ``d`` and holds from ``d+1``). + + WHY THE RESULT IS REINDEXED TO ``panel.index``. The service answers a + (date, symbol) question and therefore returns a row for every cell it was + ASKED about, carrying an explicit NaN where the factor has no value (D4c's + fill footprint, which is what lets a warm store tell computed-empty from + never-computed). An index universe's symbol list is the union of historical + constituents, so that grid is strictly larger than the loaded panel. + + HOW MUCH larger, and where that number comes from: 2.2% for CSI300 and 4.0% + for CSI500. Those are SCOUTING-PHASE measurements, not products of this + change and not reproducible from anything it emits — they were read off two + local (gitignored) daily panels, ``artifacts/data/i5e_csi300_daily.parquet`` + (569,736 rows over 1,210 dates x 481 symbols) and + ``artifacts/data/d1_panel_freeze_daily.parquet`` (1,158,912 over 1,210 x + 996), as ``dates * symbols / rows - 1``. They size the effect; they are not + evidence about this code. The phase2 panel this step actually reconciles on + is exactly dense (68 x 241 = 16,388), so its footprint count is zero — the + non-zero case is covered by ``tests/test_factor_source.py`` instead. + + Every one of those extra rows is all-NaN, so no VALUE changes either way — but a consumer + that counts rows (``per_factor[...]["coverage"]`` is ``notna().mean()``) is + looking at a different denominator, and a published diagnostic must not move + because the values arrived by a different route (red line #8). The count of + rows this drops is returned so the reduction is disclosed rather than + invisible. + + A cell present in the panel but ABSENT from the service's answer is a loud + error: it would mean the request and the panel disagreed about the grid, and + a silently short cross-section is the sample-coverage bias the I5a red line + forbids. + """ + factor_ids = [f.name for f in factors] + served = factor_service.panel( + factor_ids, + symbols, + decision_points(panel), + store=store, + sources=MaterializeSources(daily=DailyEvalPanelProvider(panel)), + view=View.CLOSE, + basis=ReturnBasis.CLOSE_TO_CLOSE, + params_by_id=dict(params_by_id or {}), + ) + unserved = panel.index.difference(served.index) + if len(unserved): + raise ValueError( + f"the factor service returned no row for {len(unserved)} (date, symbol) " + f"cell(s) the market panel carries, e.g. {list(unserved[:3])}. The " + f"request grid and the panel must agree; serving the panel's factor " + f"values short would drop names from a cross-section silently." + ) + return ServedFactorValues( + frame=served.reindex(panel.index), + served_rows=int(len(served)), + footprint_rows_dropped=int(len(served) - len(panel.index)), + ) + + +__all__ = [ + "FACTOR_STORE_DIRNAME", + "PERSISTABLE_SOURCES", + "SYNTHETIC_SOURCES", + "DailyEvalPanelProvider", + "ServedFactorValues", + "config_declared_sources", + "decision_points", + "factor_store_root", + "factor_values", + "is_synthetic_source", + "known_sources", + "open_factor_value_store", +] diff --git a/qt/phase2_baseline.py b/qt/phase2_baseline.py index ca0f040..056dfa3 100644 --- a/qt/phase2_baseline.py +++ b/qt/phase2_baseline.py @@ -35,6 +35,7 @@ from factors.compute.financial import FinancialFactor from portfolio.construct import TopNEqualWeight from qt.config import RootConfig, load_config +from qt.factor_source import open_factor_value_store from qt.pipeline import ( _FrameScores, _alpha_disclosure, @@ -45,7 +46,7 @@ _build_scores, _build_universe, _collect_downgrades, - _compute_factor_panel, + _serve_factor_panel, _factor_analytics, _standard_analytics, _load_panel, @@ -372,7 +373,8 @@ def run_phase2_baseline(config_path: str) -> Phase2Result: panel = _maybe_enrich_listing(cfg, panel, symbols, logger, cache) # P4-1/P4-2: one cache-stats line after every cached endpoint has run. _log_run_cache_stats(cache, logger) - factor_panel = _compute_factor_panel(cfg, panel, factors, logger) + with open_factor_value_store(cfg, logger) as store: + factor_panel = _serve_factor_panel(cfg, panel, factors, symbols, logger, store=store) processed = _process_factors(cfg, factor_panel, panel) alpha = _build_alpha(cfg) score_panel = _build_scores( diff --git a/qt/pipeline.py b/qt/pipeline.py index 98e1c53..9114233 100644 --- a/qt/pipeline.py +++ b/qt/pipeline.py @@ -60,6 +60,7 @@ from factors.process.orchestrate import covariates_from_panel, process_factor_panel from portfolio.construct import TopNEqualWeight from qt.config import RootConfig, load_config +from qt.factor_source import factor_values, open_factor_value_store from qt.reports import write_phase0_summary from runtime.backtest.driver import BacktestDriver from runtime.backtest.sim_execution import SimExecution @@ -440,7 +441,8 @@ def run_phase0(config_path: str) -> Phase0Result: # P4-1/P4-2: one concise cache-stats line after every cached endpoint has run # (market bars + universe/tradability). A warm historical rerun shows all 0s. _log_run_cache_stats(cache, logger) - factor_panel = _compute_factor_panel(cfg, panel, factors, logger) + with open_factor_value_store(cfg, logger) as store: + factor_panel = _serve_factor_panel(cfg, panel, factors, symbols, logger, store=store) processed = _process_factors(cfg, factor_panel, panel) alpha = _build_alpha(cfg) @@ -929,18 +931,93 @@ def _maybe_enrich_listing( return panel -def _compute_factor_panel( +def _factor_params_by_id(cfg: RootConfig, factors: list) -> dict[str, dict]: + """Config params keyed by the BUILT factor's id (what the store key hashes). + + Keyed by the instance name rather than the config name: the registry already + refuses a config name that disagrees with its params (D1's three-point + check), and keying by the object the service is actually asked for leaves no + second spelling to drift. ``strict`` zip guards against this list and + :func:`_build_factors`' enabled list falling out of step. + """ + enabled = [f for f in cfg.factors if f.enabled] + return { + factor.name: dict(spec.params) + for spec, factor in zip(enabled, factors, strict=True) + } + + +def _serve_factor_panel( cfg: RootConfig, panel: pd.DataFrame, factors: list, + symbols: list[str], logger: logging.Logger, + *, + store, ) -> pd.DataFrame: - """Compute every factor as its own column and persist the multi-column panel. + """Get every factor's values from the SERVICE and persist the panel (D6a). One column per enabled factor (P3-1); a single-factor config produces the same one-column frame as before. All columns share the panel's MultiIndex(date, symbol), so downstream processing stays per-date and per-column. + + D6a (factor-refactor): the values come from ``factors.service`` — the store + read-through onto the ONE materializer engine — instead of calling + ``factor.compute(panel)`` here, which was the second factor-sourcing path + the refactor exists to retire (design decision 3). The pairing is the close + view x close-to-close basis these runners already decide on, under which the + materializer applies no availability lag: a change of PATH, not of VALUES. + ``qt.factor_source.factor_values`` owns the wiring and the reduction back to + this panel's own grid; the rows that reduction drops are logged rather than + left invisible. + + This is a SECOND entry point next to :func:`_compute_factor_panel` on + purpose, and only until D6b: the migration moves the runners one group at a + time, so for the length of that migration some runners are on the service + and some are not. Two differently-named functions make "which runners are + still on the old path" a grep instead of a reading exercise, where one + function with an optional ``store`` would have made the old path a silent + default (red line #9). + """ + served = factor_values( + factors, + panel, + symbols, + store=store, + params_by_id=_factor_params_by_id(cfg, factors), + ) + factor_panel = served.frame + _write_factor_panel(cfg, factor_panel) + logger.info( + "factors: %s served by the factor service (%d rows x %d columns; the " + "service answered %d requested (date, symbol) cell(s), %d of them " + "all-NaN footprint row(s) outside the market panel and reduced away)", + [f.name for f in factors], len(factor_panel), factor_panel.shape[1], + served.served_rows, served.footprint_rows_dropped, + ) + return factor_panel + + +def _compute_factor_panel( + cfg: RootConfig, + panel: pd.DataFrame, + factors: list, + logger: logging.Logger, +) -> pd.DataFrame: + """PRE-D6a path: compute every factor here and persist the panel. + + Still the live path for the runners D6a does not migrate + (``qt.oos_stability``, ``qt.subset_validation``, and ``qt.robustness`` + through the former). It is the second factor-sourcing path the refactor is + retiring; D6b moves those runners onto :func:`_serve_factor_panel` and D6d + removes this function. Do NOT add a new caller. + + Kept behaviour-identical rather than adapted: routing those runners through + the service changes what their factor values come from, which needs its own + reconciliation on their universes and windows — it is D6b's work, not a side + effect of migrating phase0/phase2. """ columns = [factor.compute(panel).rename(factor.name) for factor in factors] factor_panel = pd.concat(columns, axis=1) diff --git a/tests/test_daily_factor_universe_independence.py b/tests/test_daily_factor_universe_independence.py new file mode 100644 index 0000000..45d2646 --- /dev/null +++ b/tests/test_daily_factor_universe_independence.py @@ -0,0 +1,178 @@ +"""Every DAILY factor's value must not depend on which universe was loaded. + +WHY THIS IS LOAD-BEARING, AND WHY IT BECAME SO IN D6a. The factor store key is +``(factor_id, params, code, view)``; it has NO universe dimension, deliberately +(design revision A2: a PIT universe over a date range is a time-varying SET, so +"the universe" is not a value a key dimension could hold). That is only sound +while the stored value is universe-INDEPENDENT. + +On the minute plane this was learned the hard way: ``intraday_amp_cut``'s +cross-sectional z-score ran during materialization, so its stored value WAS a +function of the loaded universe while the key said otherwise — measured, 12 names +filled then 24 read served 24 wrong cells with zero recompute and zero error. +D4c fixed it by storing the per-symbol intermediate and moving the combine to +read-assembly. + +THE DAILY PLANE NEVER INHERITED THAT FIX, because until D6a no daily factor value +was ever stored. D6a is the step that puts them in the shared store, so the +invariant that keeps the daily plane sound — every daily factor is per-symbol — +becomes load-bearing here. It held before by accident of what was written; from +here it has to hold on purpose. + +RED MEANS STOP AND REPORT. A daily factor whose value moves when the universe +changes cannot be stored under this key. The fix is D4c's (store the per-symbol +intermediate, combine at read-assembly) — NOT adding the factor to an exemption +list, which would restore exactly the silent-wrong-value failure D4c closed. + +WHY BEHAVIOURAL AND NOT A SOURCE SCAN. Scanning ``factors/compute`` for +``groupby(level="date")`` / ``.rank(`` is both too strict and too loose: +``groupby(level="symbol").rank()`` is a per-symbol time-series rank and would be +a false positive, while a cross-sectional standardization written as +``x.unstack().sub(x.unstack().mean(axis=1), axis=0).stack()`` contains neither +token and would sail through. Computing the factor over two universes and +comparing the shared symbols tests the property itself. +""" + +from __future__ import annotations + +import numpy as np +import pandas as pd +import pytest + +from data.clean.schema import CORE_COLUMNS, normalize_panel +from factors import registry as factor_registry +from factors.materialize import is_minute_factor + +_SYMBOLS = ["000001.SZ", "000002.SZ", "000003.SZ", "000004.SZ", "000005.SZ"] +_SUBSET = ["000001.SZ", "000003.SZ", "000005.SZ"] +_N_DAYS = 60 + + +def _registered_factors() -> list: + """One instance per registry entry — DERIVED, never a hand-kept list. + + Reads the default registry's dispatch tables directly rather than through a + public accessor: adding one would be a production change, and this step is a + path switch that has to stay provably behaviour-neutral. The point is that a + newly registered factor lands in this census without anyone remembering to + add it — including the case this test exists for, a new DAILY factor. + + An entry's builder is called with the entry key: the exact-name builders + (financial fields, value ratios) use the name, and the prefix builders ignore + it and take their window from params defaults, so one call shape serves both. + """ + registry = factor_registry.DEFAULT_REGISTRY + entries = list(registry._exact.values()) + list(registry._prefixes) + return [entry.builder(entry.key, {}) for entry in entries] + + +def _daily_factors() -> list: + return [f for f in _registered_factors() if not is_minute_factor(f)] + + +def _panel(symbols: list[str], extra_fields: tuple[str, ...]) -> pd.DataFrame: + """A deterministic panel whose columns differ ACROSS symbols. + + Detection power depends on that: a cross-sectional operator only changes a + value when the peers it standardizes against change, so every symbol gets its + own level and slope. + + A symbol's data is keyed off its identity (its position in ``_SYMBOLS``), NOT + off its position in ``symbols``. The first draft used ``enumerate(symbols)`` + and every one of the ten daily factors "failed": dropping two names shifted + the survivors' own prices, so the two panels held different DATA and the + comparison was measuring the fixture. The census caught it instantly, which + is the one piece of evidence that it has power at all — but it is also the + reminder that a probe reporting "everything is broken" is usually broken + itself. + """ + dates = pd.bdate_range("2024-01-01", periods=_N_DAYS) + rows = [] + for symbol in symbols: + i = _SYMBOLS.index(symbol) + t = np.arange(_N_DAYS, dtype=float) + close = 50.0 + 10.0 * i + (1.0 + 0.5 * i) * t + np.sin(t / 3.0) * (1.0 + i) + frame = pd.DataFrame( + { + "date": dates, + "symbol": symbol, + "open": close * 0.99, + "high": close * 1.02, + "low": close * 0.97, + "close": close, + "volume": 1_000.0 + 100.0 * i + t, + "amount": (1_000.0 + 100.0 * i + t) * close, + "adj_factor": 1.0, + } + ) + for k, field in enumerate(extra_fields): + if field in frame.columns: + continue + frame[field] = 0.5 + 0.1 * i + 0.01 * k + t / 1000.0 + rows.append(frame) + return normalize_panel(pd.concat(rows, ignore_index=True)) + + +def _extra_fields(factor) -> tuple[str, ...]: + """Panel columns the factor reads beyond the core OHLCV set.""" + return tuple(f for f in (factor.spec.input_fields or ()) if f not in CORE_COLUMNS) + + +def test_the_census_is_not_empty_and_covers_the_known_daily_families(): + """A census that enumerates nothing passes vacuously; this refuses to. + + Not an exhaustive expected list (that would be the hand-kept register this + test exists to avoid) — just enough anchors that an enumeration returning + nothing, or losing whole families, cannot read as success. + """ + names = {f.name for f in _daily_factors()} + assert len(names) >= 8, f"suspiciously small daily-factor census: {sorted(names)}" + for anchor in ("momentum_20", "volatility_20", "value_ep", "roe"): + assert anchor in names, f"{anchor} missing from the census: {sorted(names)}" + + +def test_no_minute_factor_leaked_into_the_daily_census(): + """The split is what makes the daily claim meaningful; pin it.""" + assert not [f for f in _daily_factors() if f.spec.is_intraday] + + +@pytest.mark.parametrize("factor", _daily_factors(), ids=lambda f: f.name) +def test_a_daily_factor_value_does_not_depend_on_the_loaded_universe(factor): + """Drop 2 of 5 symbols; the surviving 3 must keep bit-identical values.""" + extra = _extra_fields(factor) + full = factor.compute(_panel(_SYMBOLS, extra)) + subset = factor.compute(_panel(_SUBSET, extra)) + + shared = subset.index + missing = shared.difference(full.index) + assert not len(missing), f"{factor.name}: subset produced rows the full run did not" + + a = full.reindex(shared) + b = subset + nan_mismatch = int((a.isna() != b.isna()).sum()) + both = a.notna() & b.notna() + worst = float((a[both] - b[both]).abs().max()) if both.any() else 0.0 + + assert nan_mismatch == 0 and worst == 0.0, ( + f"{factor.name} is CROSS-SECTIONAL on the daily plane: dropping two " + f"symbols moved {nan_mismatch} NaN cell(s) and up to {worst:.3e} in value " + f"for the symbols that stayed.\n" + f"STOP AND REPORT — do not add an exemption. Its value is a function of " + f"the loaded universe, but the factor store key " + f"(factor_id, params, code, view) does not name a universe, so a value " + f"filled under one universe would be served to another (the D5b defect, " + f"measured on the minute plane before D4c fixed it). The fix is D4c's: " + f"store the per-symbol intermediate and run the cross-sectional combine " + f"at read-assembly." + ) + + +def test_at_least_one_finite_value_is_actually_compared(request): + """Guards the parametrized test above from passing on all-NaN output.""" + for factor in _daily_factors(): + extra = _extra_fields(factor) + values = factor.compute(_panel(_SUBSET, extra)) + assert values.notna().any(), ( + f"{factor.name} produced no finite value on the census fixture, so the " + f"universe-independence comparison for it is vacuous." + ) diff --git a/tests/test_factor_source.py b/tests/test_factor_source.py new file mode 100644 index 0000000..ec78e64 --- /dev/null +++ b/tests/test_factor_source.py @@ -0,0 +1,352 @@ +"""D6a: the factor-value source wiring (store policy + served-panel assembly). + +Two properties carry this step: + +* a SYNTHETIC data source can never reach a durable factor-store artifact — + structurally, not by convention (``qt/factor_source.py`` explains why such + values are not storable at all); and +* routing a daily runner through the service changes the PATH and not the + VALUES: what the service returns is reduced back to the panel's own grid, and + the rows that reduction drops are counted rather than hidden. +""" + +from __future__ import annotations + +from pathlib import Path + +import pandas as pd +import pytest +import yaml + +from factors.compute.momentum import MomentumFactor +from factors.store import FactorValueStore +from qt import factor_source +from qt.config import load_config +from qt.factor_source import ( + DailyEvalPanelProvider, + config_declared_sources, + factor_store_root, + factor_values, + is_synthetic_source, + known_sources, + open_factor_value_store, +) +from tests.fixtures.panel_factory import make_demo_panel + + +def _cfg(tmp_path: Path, example_config_path: str, *, source: str = "demo"): + """The example config with ``source`` and every output dir under tmp_path.""" + raw = yaml.safe_load(Path(example_config_path).read_text(encoding="utf-8")) + out = tmp_path / "artifacts" + raw["data"]["source"] = source + raw["output"] = { + "root_dir": str(out), + "data_dir": str(out / "data"), + "factor_dir": str(out / "factors"), + "report_dir": str(out / "reports"), + "log_dir": str(out / "logs"), + "overwrite": True, + } + path = tmp_path / f"cfg_{source}.yaml" + path.write_text(yaml.safe_dump(raw), encoding="utf-8") + return load_config(str(path)) + + +# --------------------------------------------------------------------------- # +# storability classification +# --------------------------------------------------------------------------- # +def test_every_config_source_has_a_storability_classification(): + """A new ``data.source`` must declare whether its values are storable. + + The drift guard, not a restatement: the reference side is read off the + ``DataCfg.source`` Literal, so adding a source without classifying it fails + here instead of silently inheriting a durable store. + """ + assert known_sources() == config_declared_sources() + + +def test_the_two_classes_are_disjoint(): + assert not (factor_source.SYNTHETIC_SOURCES & factor_source.PERSISTABLE_SOURCES) + + +def test_demo_is_synthetic_and_tushare_is_not(): + assert is_synthetic_source("demo") is True + assert is_synthetic_source("tushare") is False + + +def test_an_unclassified_source_is_refused_not_assumed_real(): + with pytest.raises(ValueError, match="storability classification"): + is_synthetic_source("some_new_vendor") + + +# --------------------------------------------------------------------------- # +# store root +# --------------------------------------------------------------------------- # +def test_real_store_root_follows_the_run_output_root(tmp_path, example_config_path): + cfg = _cfg(tmp_path, example_config_path, source="tushare") + assert factor_store_root(cfg) == str(tmp_path / "artifacts" / "factor_store") + + +def test_a_synthetic_source_has_no_persistent_store_root(tmp_path, example_config_path): + """Not a fallback: asking is the error, and the message says why. + + The demo close for a (date, symbol) is built from a counter that starts at + ``data.start``, so two windows produce different values under one key. + """ + cfg = _cfg(tmp_path, example_config_path, source="demo") + with pytest.raises(ValueError, match="data.start"): + factor_store_root(cfg) + + +def test_a_synthetic_run_gets_an_ephemeral_store_that_leaves_nothing( + tmp_path, example_config_path +): + cfg = _cfg(tmp_path, example_config_path, source="demo") + with open_factor_value_store(cfg) as store: + seen = _store_root_of(store) + assert seen.exists() + # not under the run's own output tree, so no later run can be served it + assert tmp_path not in seen.parents + assert not seen.exists() + assert not (tmp_path / "artifacts" / "factor_store").exists() + + +def test_a_real_run_gets_the_shared_store_at_the_derived_root( + tmp_path, example_config_path +): + cfg = _cfg(tmp_path, example_config_path, source="tushare") + with open_factor_value_store(cfg) as store: + assert _store_root_of(store) == Path(factor_store_root(cfg)) + + +def _store_root_of(store: FactorValueStore) -> Path: + """The store's on-disk root, via its public ``values_root``.""" + return store.values_root.parent + + +# --------------------------------------------------------------------------- # +# served panel: reindexed to the panel grid, footprint rows counted +# --------------------------------------------------------------------------- # +def _served(panel, symbols, tmp_path): + return factor_values( + [MomentumFactor(window=5)], + panel, + symbols, + store=FactorValueStore(tmp_path / "store"), + params_by_id={"momentum_5": {"window": 5}}, + ) + + +def test_a_dense_panel_drops_no_footprint_rows(tmp_path): + panel = make_demo_panel() + symbols = sorted(set(panel.index.get_level_values("symbol"))) + out = _served(panel, symbols, tmp_path) + + assert out.footprint_rows_dropped == 0 + assert out.frame.index.equals(panel.index) + + +def test_a_sparse_panel_reindexes_back_and_counts_what_that_dropped(tmp_path): + """The index-universe geometry: the service answers over the grid it is asked + about, the panel is smaller, and the reduction is reported.""" + panel = make_demo_panel() + symbols = sorted(set(panel.index.get_level_values("symbol"))) + dates = pd.DatetimeIndex(sorted(set(panel.index.get_level_values("date")))) + late = symbols[-1] + d = panel.index.get_level_values("date") + s = panel.index.get_level_values("symbol") + sparse = panel[~((s == late) & (d < dates[10]))] + + out = _served(sparse, symbols, tmp_path) + factor = MomentumFactor(window=5) + + assert out.footprint_rows_dropped == 10 + assert out.served_rows == len(dates) * len(symbols) + assert out.frame.index.equals(sparse.index) + # ... and the SURVIVING VALUES are right, not just the surviving labels. + # Asserting only the index would accept a reduction that lands the correct + # index carrying the wrong rows -- and that defect is invisible on a dense + # panel, where the reduction is a no-op, so no other test in this file can + # see it (mutation: reducing positionally instead of by label keeps the + # index assertion and the dense value test green and turns this red). + pd.testing.assert_frame_equal( + out.frame, + factor.compute(sparse).rename(factor.name).to_frame(), + check_exact=True, + ) + + +def test_the_served_values_do_not_depend_on_which_store_they_were_filled_into( + tmp_path, +): + """Two cold stores, one request: the same values. + + NAMED FOR WHAT IT MEASURES. It was called + ``test_the_reindex_preserves_every_value_the_service_served``, which it never + tested: both sides came from the same already-reindexed ``factor_values`` + call, so the comparison reduced to ``frame == frame`` for that property and + could only ever have caught a difference between the two STORES. + + That misnaming did real damage before it was caught: it was read (by its + author, then by a reviewer taking the author's word) as evidence that + reindex value-preservation was pinned, and on that basis nobody added the + assertion that actually pins it. It is now in + ``test_a_sparse_panel_reindexes_back_and_counts_what_that_dropped`` -- which + is where it belongs, next to the reduction it constrains. + + What is left here is worth keeping on its own: a factor value must be a + function of (factor, panel, universe) alone, so filling store A and filling + store B must agree. That is the assumption the read-through rests on. + """ + panel = make_demo_panel() + symbols = sorted(set(panel.index.get_level_values("symbol"))) + dates = pd.DatetimeIndex(sorted(set(panel.index.get_level_values("date")))) + d = panel.index.get_level_values("date") + s = panel.index.get_level_values("symbol") + sparse = panel[~((s == symbols[-1]) & (d < dates[10]))] + + into_a = _served(sparse, symbols, tmp_path).frame + into_b = factor_values( + [MomentumFactor(window=5)], + sparse, + symbols, + store=FactorValueStore(tmp_path / "store2"), + params_by_id={"momentum_5": {"window": 5}}, + ).frame + + pd.testing.assert_frame_equal(into_a, into_b, check_exact=True) + + +def test_the_service_values_equal_a_direct_compute_on_the_same_panel(tmp_path): + """The step's whole claim, on the close view: same values, different path.""" + panel = make_demo_panel() + symbols = sorted(set(panel.index.get_level_values("symbol"))) + factor = MomentumFactor(window=5) + + legacy = factor.compute(panel).rename(factor.name).to_frame() + served = _served(panel, symbols, tmp_path).frame + + pd.testing.assert_frame_equal(served, legacy, check_exact=True) + + +def test_a_panel_cell_the_service_cannot_serve_is_loud(tmp_path): + """A short cross-section is the sample bias the I5a red line forbids.""" + panel = make_demo_panel() + symbols = sorted(set(panel.index.get_level_values("symbol"))) + + with pytest.raises(ValueError, match="returned no row for"): + _served(panel, symbols[:-1], tmp_path) # one panel symbol not requested + + +# --------------------------------------------------------------------------- # +# end to end: a demo run persists no factor value, and says so +# --------------------------------------------------------------------------- # +def test_a_demo_run_leaves_no_factor_store_behind(tmp_path, example_config_path): + """The B1 guard at the runner level, not just at the helper. + + ``config/example.yaml``'s demo universe is spelled with REAL A-share tickers + over real 2024 dates, so a persisted demo value would be addressed by a key + a later real run over the same cells would hit. + """ + from qt.pipeline import run_phase0 + + cfg_path = _write_cfg_file(tmp_path, example_config_path, source="demo") + result = run_phase0(str(cfg_path)) + + assert not (tmp_path / "artifacts" / "factor_store").exists() + assert result.factor_path.exists() # the per-run panel artifact still lands + + +def test_a_demo_run_discloses_that_its_store_is_ephemeral(tmp_path, example_config_path): + from qt.pipeline import run_phase0 + + cfg_path = _write_cfg_file(tmp_path, example_config_path, source="demo") + result = run_phase0(str(cfg_path)) + + log = result.log_path.read_text(encoding="utf-8") + assert "factor store: EPHEMERAL" in log + + +def test_the_run_log_discloses_the_footprint_reduction(tmp_path, example_config_path): + """Restoring the panel's grid is allowed to be silent about nothing.""" + from qt.pipeline import run_phase0 + + cfg_path = _write_cfg_file(tmp_path, example_config_path, source="demo") + result = run_phase0(str(cfg_path)) + + log = result.log_path.read_text(encoding="utf-8") + assert "all-NaN footprint row(s) outside the market panel" in log + + +def test_the_persisted_factor_panel_keeps_the_market_panel_grid( + tmp_path, example_config_path +): + from qt.pipeline import run_phase0 + + cfg_path = _write_cfg_file(tmp_path, example_config_path, source="demo") + result = run_phase0(str(cfg_path)) + + stored = pd.read_parquet(result.factor_path) + market = pd.read_parquet(result.data_path) + + # The GRID, not the row count. `len(stored) == len(market)` was what this + # asserted, and it passes under a reduction that keeps the length and moves + # every label -- measured: shifting each date by one day left it green while + # three other tests went red. The row count is the weakest observable that + # this test's name could possibly have meant. + grid = ["date", "symbol"] + pd.testing.assert_index_equal( + stored.set_index(grid).index.sort_values(), + market.set_index(grid).index.sort_values(), + ) + + +def _write_cfg_file(tmp_path: Path, example_config_path: str, *, source: str) -> Path: + raw = yaml.safe_load(Path(example_config_path).read_text(encoding="utf-8")) + out = tmp_path / "artifacts" + raw["data"]["source"] = source + raw["data"]["end"] = "2024-06-30" # keep the run small + raw["output"] = { + "root_dir": str(out), + "data_dir": str(out / "data"), + "factor_dir": str(out / "factors"), + "report_dir": str(out / "reports"), + "log_dir": str(out / "logs"), + "overwrite": True, + } + path = tmp_path / "run_cfg.yaml" + path.write_text(yaml.safe_dump(raw), encoding="utf-8") + return path + + +# --------------------------------------------------------------------------- # +# mid-migration: the runners D6a does not move must still be able to call +# --------------------------------------------------------------------------- # +# WHICH files still call the pre-D6a entry point, and whether their calls still +# bind, is guarded by ``tests/test_legacy_factor_panel_callers.py`` -- a census +# DERIVED from the repo. It replaced a hand-kept pair of filenames here, which a +# third caller in an unnamed file walked straight past. What stays here is the +# only thing that actually EXERCISES the legacy path. + + +def test_the_legacy_entry_point_still_computes(tmp_path, example_config_path): + """Real coverage for the path those runners are still on.""" + from qt.pipeline import _compute_factor_panel + + cfg = _cfg(tmp_path, example_config_path, source="demo") + panel = make_demo_panel() + factor = MomentumFactor(window=5) + logger = __import__("logging").getLogger("test.legacy_entry") + + out = _compute_factor_panel(cfg, panel, [factor], logger) + + pd.testing.assert_series_equal(out[factor.name], factor.compute(panel).rename(factor.name)) + + +# --------------------------------------------------------------------------- # +# author-once: the eval wiring re-exports this provider, it does not copy it +# --------------------------------------------------------------------------- # +def test_the_eval_provider_is_this_provider(): + from qt import factor_eval_providers + + assert factor_eval_providers.DailyEvalPanelProvider is DailyEvalPanelProvider diff --git a/tests/test_legacy_factor_panel_callers.py b/tests/test_legacy_factor_panel_callers.py new file mode 100644 index 0000000..687f173 --- /dev/null +++ b/tests/test_legacy_factor_panel_callers.py @@ -0,0 +1,225 @@ +"""Who still calls the PRE-D6a factor path, derived from the repo — not listed. + +D6a moved phase0 and phase2_baseline onto ``_serve_factor_panel`` (the factor +service) and left ``qt.pipeline._compute_factor_panel`` in place, unchanged, for +the runners D6b migrates. Two things then need guarding, and NOTHING in the test +suite executes those runners: they need real tushare data, so a break in them is +invisible to every other test here. That is not hypothetical — the first draft of +D6a changed the shared signature under them and the whole suite stayed green. + + 1. every call to the legacy entry point still matches its live signature; and + 2. the set of files that call it is EXACTLY the set we think it is. + +(2) is the one that matters, and it is why this census is DERIVED rather than +listed. A hand-kept list of "runners still on the old path" only ever confirms +what its author already knew: add a third caller in a file the list does not +name, change the signature, and dutifully update everything the list DOES name, +and a listed guard passes in silence. Measured on the previous, hand-kept +version of this guard: exactly that scenario left all 21 tests green. + +SCOPE OF THE DERIVATION — stated because a guard that does not say what it +cannot see is worth less than no guard: + +* FILES: ``git ls-files`` UNION ``git ls-files --others --exclude-standard``, + i.e. tracked files plus untracked-but-not-ignored ones. Tracked-only would be + blind to a caller added but not yet staged; a bare directory walk would drag in + gitignored artifacts (and, in a worktree, follow the ``artifacts`` symlink into + another checkout entirely). +* CALL FORMS: a direct call (``_compute_factor_panel(...)``), an attribute call + on any object (``pipeline._compute_factor_panel(...)``), and a call through a + local import alias (``from qt.pipeline import _compute_factor_panel as X``; + ``X(...)``). ``getattr(obj, "_compute_factor_panel")`` cannot be bound, so it + is REFUSED by name rather than skipped. +* WHAT IT STILL CANNOT SEE: a call assembled from a dynamically built name + (``getattr(obj, "_compute_" + "factor_panel")``). No test here claims + otherwise. + +⚠️ THIS FILE IS SUPPOSED TO GO RED IN D6b, AND THEN BE DELETED. D6b moves +``oos_stability`` / ``subset_validation`` onto the service and D6d deletes +``_compute_factor_panel`` entirely; at that point ``EXPECTED_CALLERS`` no longer +matches and the symbol no longer exists. That is the guard reporting the +migration finished, not a stale test. Delete it with the function; do not "fix" +it by emptying the expected set while the function still has callers. +""" + +from __future__ import annotations + +import ast +import inspect +import subprocess +from pathlib import Path + +import pytest + +from qt import pipeline + +#: The legacy entry point this census tracks. +SYMBOL = "_compute_factor_panel" + +#: The module that DEFINES it — excluded from the caller census because a +#: definition is not a call. Derived below by finding the ``FunctionDef``, so a +#: move to another module does not silently drop the exclusion. +DEFINING_MODULE = Path(inspect.getsourcefile(pipeline)).name + +#: Every file that calls it today, repo-relative. +#: +#: ``qt/oos_stability.py`` and ``qt/subset_validation.py`` are the two runners +#: D6b migrates. ``tests/test_factor_source.py`` calls it too, deliberately: it +#: is the only execution coverage the legacy path has, since the two runners +#: cannot be run without real data. +EXPECTED_CALLERS: frozenset[str] = frozenset( + { + "qt/oos_stability.py", + "qt/subset_validation.py", + "tests/test_factor_source.py", + } +) + +#: A census that scans nothing passes vacuously. The repo has ~300 python files; +#: this floor only has to be high enough that an empty or broken file listing +#: cannot read as "no callers found". +_MIN_FILES_SCANNED = 100 + + +def _repo_root() -> Path: + return Path(__file__).resolve().parents[1] + + +def _python_files() -> list[Path]: + root = _repo_root() + seen: dict[str, Path] = {} + for args in ( + ["git", "ls-files", "--", "*.py"], + ["git", "ls-files", "--others", "--exclude-standard", "--", "*.py"], + ): + out = subprocess.run( + args, cwd=root, capture_output=True, text=True, check=True + ).stdout + for line in out.splitlines(): + rel = line.strip() + if rel: + seen.setdefault(rel, root / rel) + return [path for _rel, path in sorted(seen.items())] + + +def _local_aliases(tree: ast.AST) -> set[str]: + """Local names bound to ``SYMBOL`` by an import in this module.""" + names = {SYMBOL} + for node in ast.walk(tree): + if isinstance(node, ast.ImportFrom): + for alias in node.names: + if alias.name == SYMBOL and alias.asname: + names.add(alias.asname) + return names + + +def _calls_in(path: Path) -> tuple[list[ast.Call], list[str]]: + """(bindable call nodes, unbindable references) for one file.""" + try: + tree = ast.parse(path.read_text(encoding="utf-8")) + except (SyntaxError, UnicodeDecodeError): # not our business to compile the repo + return [], [] + aliases = _local_aliases(tree) + calls: list[ast.Call] = [] + unbindable: list[str] = [] + for node in ast.walk(tree): + if not isinstance(node, ast.Call): + continue + func = node.func + if isinstance(func, ast.Name) and func.id in aliases: + calls.append(node) + elif isinstance(func, ast.Attribute) and func.attr == SYMBOL: + calls.append(node) + elif ( + isinstance(func, ast.Name) + and func.id == "getattr" + and any( + isinstance(a, ast.Constant) and a.value == SYMBOL for a in node.args + ) + ): + unbindable.append(f"{path.name}:{node.lineno} getattr(...)") + return calls, unbindable + + +def _census() -> tuple[dict[str, list[ast.Call]], list[str], int]: + root = _repo_root() + callers: dict[str, list[ast.Call]] = {} + unbindable: list[str] = [] + files = _python_files() + for path in files: + if path.name == DEFINING_MODULE: + continue + calls, bad = _calls_in(path) + unbindable.extend(bad) + if calls: + callers[path.relative_to(root).as_posix()] = calls + return callers, unbindable, len(files) + + +def test_the_file_census_actually_scanned_the_repo(): + """Distinguishes 'no callers' from 'the file listing came back empty'.""" + files = _python_files() + assert len(files) >= _MIN_FILES_SCANNED, ( + f"the python-file derivation returned only {len(files)} file(s); every " + f"census below would pass vacuously. Check the git listing, not the " + f"callers." + ) + + +def test_the_set_of_legacy_callers_is_exactly_what_we_think_it_is(): + """The load-bearing one: a NEW caller anywhere in the repo turns this red. + + Red here is STOP-AND-REPORT, and which report depends on which way it moved: + + * an ADDED file — someone put a new caller on the pre-D6a path. Route it + through ``_serve_factor_panel`` instead, or say why it must not be. + * a REMOVED file — expected exactly once, when D6b migrates the two runners. + Then this whole module goes away with ``_compute_factor_panel`` (D6d); do + not keep it alive against an empty set. + """ + callers, _unbindable, _n = _census() + found = frozenset(callers) + assert found, ( + f"no caller of {SYMBOL} found anywhere in the repo. Either the function " + f"is already dead (then delete it and this module together) or the " + f"derivation broke — check " + f"test_the_file_census_actually_scanned_the_repo first." + ) + assert found == EXPECTED_CALLERS, ( + f"the set of files calling {SYMBOL} changed.\n" + f" added: {sorted(found - EXPECTED_CALLERS)}\n" + f" removed: {sorted(EXPECTED_CALLERS - found)}\n" + f"STOP AND REPORT — see this test's docstring for which of the two " + f"situations you are in." + ) + + +def test_every_legacy_call_matches_the_live_signature(): + """Parsed from source, bound against the function object as it exists now.""" + callers, _unbindable, _n = _census() + signature = inspect.signature(getattr(pipeline, SYMBOL)) + for rel, calls in sorted(callers.items()): + for call in calls: + try: + signature.bind( + *[""] * len(call.args), + **{kw.arg: "" for kw in call.keywords if kw.arg}, + ) + except TypeError as exc: # noqa: PERF203 - one message per bad call site + pytest.fail( + f"{rel}:{call.lineno} calls {SYMBOL} with a shape its current " + f"signature {signature} does not accept ({exc}). Nothing in " + f"the suite executes that call, so this guard is the only " + f"thing between the change and a runtime TypeError." + ) + + +def test_no_caller_reaches_the_symbol_in_a_form_this_guard_cannot_bind(): + """``getattr`` by name is refused, not silently skipped.""" + _callers, unbindable, _n = _census() + assert not unbindable, ( + f"{SYMBOL} is referenced through getattr at {unbindable}, a form this " + f"census can find but cannot bind against the signature. Make it a " + f"direct call, or extend the guard deliberately — do not leave a " + f"reference it can only half-see." + )