Skip to content

refactor(cache): split TushareCache endpoint specs and parsers (D2) - #43

Merged
StackOverFlow11 merged 1 commit into
mainfrom
feat/data-layer-d2-tushare-cache-modularization
Jun 19, 2026
Merged

refactor(cache): split TushareCache endpoint specs and parsers (D2)#43
StackOverFlow11 merged 1 commit into
mainfrom
feat/data-layer-d2-tushare-cache-modularization

Conversation

@StackOverFlow11

Copy link
Copy Markdown
Owner

Summary

Data Layer D2 — behavior-preserving internal refactor of
data/cache/tushare_cache.py. The 814-line facade is split so endpoint metadata
and raw parsers are reviewable in small files, while TushareCache stays the
public facade and every cache behavior is unchanged. This is not a cache
semantics change, not a performance/concurrency change, and not a data-quality
layer.

Changed files (6)

  • new data/cache/tushare_specs.py (89 lines) — endpoint identifiers,
    ALL_ENDPOINTS, _GLOBAL_KEY, canonical-raw column sets + natural keys,
    FINA_FIELDS, _INDEX_WINDOW_DAYS, raw→canonical rename maps. Pure constants.
  • new data/cache/tushare_parsers.py (216 lines) — the 10 raw endpoint
    parsers (_parse_daily/_adj/_suspend/_stk_limit/_index_weight/_namechange/ _stock_basic/_daily_basic/_fina/_index_member), moved verbatim.
  • new data/cache/tushare_planning.py (22 lines) — the two leaf helpers
    _fields_hash / _compact.
  • mod data/cache/tushare_cache.py814 → 596 lines; keeps the public
    TushareCache facade + read-through engine + FetchOne/FetchSnapshot;
    imports the moved pieces and re-exports the public names.
  • mod docs/data/data_layer_contracts.md — factually records D2 implemented
    (D3–D6 still not).
  • new tests/test_tushare_cache_modularization.py (5 tests) — narrow
    re-export-identity + module-boundary regression guard.

What moved vs what stayed in TushareCache

  • Moved out: endpoint constants/specs, raw parsers, and the two leaf planning
    helpers.
  • Stayed (unchanged): all public methods + signatures (daily_bars,
    adj_factor, suspend_d, stk_limit, daily_basic, fina_indicator,
    index_member_all, index_weight, namechange, stock_basic, stats,
    update_summary); dense per-symbol gap planning; index_weight 90-day paging;
    snapshot/dimension staleness; refresh_recent_days / recent_tail_overrides /
    not_ready_days / force_refresh; fetch_counts / written_counts /
    not_ready_counts; ok/empty/not_ready/failed coverage semantics; one-row-per-
    natural-key upsert.

Backward-compatible import guarantees

from data.cache.tushare_cache import TushareCache and the constant imports
(DAILY_BASIC, FINA_INDICATOR, INDEX_MEMBER_ALL, INDEX_WEIGHT, NAMECHANGE,
STK_LIMIT, STOCK_BASIC, SUSPEND_D, FINA_FIELDS, ALL_ENDPOINTS) all keep
working — the facade re-exports them (verified identity in the new test; exercised
by data/feed/tushare_fina.py and the existing cache tests).

Verification

  • pytest tests/test_tushare_cache_market.py tests/test_tushare_cache_universe.py tests/test_tushare_cache_p4_3.py tests/test_data_updater.py tests/test_cache_config.py → all pass.
  • pytest -p no:cacheprovider584 passed (579 prior + 5 new D2 tests).
  • ruff check .All checks passed!
  • validate-config over all 17 config/*.yaml → all OK.
  • run-phase0 --config config/example.yamlic_mean=0.9600, annual_return=0.8408 (no behavior drift).
  • git diff --check main...HEAD → clean; merge-marker scan → 0.
  • Secret scan over changed files → real .config.json token value 0 hits; no FAKE token in changed files.

Behavior-preserving internal refactor — no cache semantics change. Please
review before merge.

Behavior-preserving internal refactor of data/cache/tushare_cache.py — no cache
semantics change. The 814-line facade is split so endpoint metadata and raw
parsers are reviewable in small files:

- data/cache/tushare_specs.py (new): endpoint identifiers, ALL_ENDPOINTS,
  _GLOBAL_KEY, canonical-raw column sets + natural keys, FINA_FIELDS,
  _INDEX_WINDOW_DAYS, raw->canonical rename maps.
- data/cache/tushare_parsers.py (new): the 10 raw endpoint parsers
  (_parse_daily/_adj/_suspend/_stk_limit/_index_weight/_namechange/_stock_basic/
  _daily_basic/_fina/_index_member), moved verbatim.
- data/cache/tushare_planning.py (new): the two leaf helpers _fields_hash /
  _compact.
- data/cache/tushare_cache.py: keeps the public TushareCache facade + the
  read-through engine (gap planning, 90-day index_weight paging, snapshot
  staleness, refresh_recent_days / recent_tail_overrides / not_ready_days /
  force_refresh, ok/empty/not_ready/failed coverage, one-row-per-natural-key
  upsert, stats / update_summary) and FetchOne/FetchSnapshot. 814 -> 596 lines.

Backward-compatible imports preserved: the facade re-exports the endpoint ids +
FINA_FIELDS + ALL_ENDPOINTS, so `from data.cache.tushare_cache import DAILY_BASIC,
FINA_FIELDS, INDEX_WEIGHT, ...` (used by data/feed/tushare_fina.py and the cache
tests) keep working unchanged.

Public method signatures, internal planning, and every coverage semantic are
unchanged; the existing cache tests are the behavior lock. Adds a narrow
modularization regression test (re-export identity + parser module boundary).

Non-goals (D3+): no schema registry, no data-quality validator, no concurrency,
no CoverageLedger storage change, no PanelStore append/partition, no intraday
cache refactor, no rate-limit/retry change, no factor/alpha/portfolio/runtime/
report change.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant