Skip to content

docs: record data-layer D5 concurrency progress#52

Merged
StackOverFlow11 merged 1 commit into
mainfrom
docs/progress-data-layer-d5
Jun 19, 2026
Merged

docs: record data-layer D5 concurrency progress#52
StackOverFlow11 merged 1 commit into
mainfrom
docs/progress-data-layer-d5

Conversation

@StackOverFlow11

Copy link
Copy Markdown
Owner

Summary

Docs-only progress sync recording Data Layer D5 (opt-in bounded concurrency + one shared global rate limiter) as implemented. No code change.

PR #51 merge status

MERGED into main (mergeCommit d4b665d). D5 code accepted by Codex; this PR only updates documentation to match.

Changed files (3, docs only)

  • docs/data/data_layer_contracts.md — D5 recorded as implemented (opt-in bounded concurrency for updater/cache-warm fetches guarded by one shared GlobalRateLimiter; default max_workers=1 keeps serial; dense per-symbol daily endpoints can use workers while snapshot / index_weight paging / intraday stay serial; cache=raw endpoint SoT / PanelStore per-run artifact boundary unchanged); section-4 title gains / D5; the unimplemented block moves to D6+ (endpoint schema registry still deferred; D6 = PanelStore append/partition / optional materialized derived panel store, only if factor research needs reusable derived panels). Only bounded global-rate-limited concurrency was completed — no other "D5+" idea is claimed.
  • CLAUDE.md — PR feat(data): D5 opt-in bounded concurrency with a global rate limiter #51 added to the merged-PR list; D5 progress bullet; quality gate 656 → 677 passed (+ D5 scheduler=9 + concurrency=12); phase0 anchor 0.9600 / 0.8408; roadmap D5 done, endpoint schema registry deferred, D6 optional derived panel store.
  • AGENTS.md — made byte-identical to CLAUDE.md.

D5 scope (as recorded)

GlobalRateLimiter (thread-safe ticket limiter, injectable monotonic/sleep, no secret); request_with_retry(scheduler=) acquires a global slot per attempt; data_update.concurrency.max_workers default 1 (<1 rejected); run_data_update builds one scheduler only when max_workers>1, shared by all feeds; TushareCache(max_workers>1) parallelizes only the dense per-symbol fetch, then upserts/records on the main thread in deterministic plan order (successes durable, failures uncovered+retryable). Default serial; existing configs validate unchanged; no factor/alpha/portfolio/runtime/backtest math change.

Endpoint schema registry / D6 — NOT implemented

Confirmed. This is a publish + docs-sync task only; no endpoint schema registry, no D6 PanelStore append/partition / materialized derived panel store, no data-quality / research change, no further concurrency expansion.

Verification (docs PR)

  • git diff --name-status main...HEAD → only AGENTS.md, CLAUDE.md, docs/data/data_layer_contracts.md (docs only).
  • cmp -s AGENTS.md CLAUDE.mdidentical.
  • git diff --check main...HEAD → clean; merge-marker scan → NO_MERGE_MARKERS.
  • stale-text scan: no 656 passed; no "D5 deferred/unimplemented" residual; 677 passed recorded; endpoint schema registry / D6 not claimed implemented.
  • docs-only ⇒ full pytest not required (the D5 suite already 677 passed on feat(data): D5 opt-in bounded concurrency with a global rate limiter #51).

PR #38 untouched

Confirmed: #38 OPEN base=feat/i5c-mmp-minute-factor — not merged, closed, retargeted, or edited.

- data_layer_contracts.md: D5 recorded as implemented (opt-in bounded concurrency
  for updater/cache-warm fetches guarded by one shared global rate limiter;
  default max_workers=1 keeps serial; dense per-symbol daily endpoints can use
  workers while snapshot / index_weight paging / intraday stay serial; cache/store
  boundary unchanged); section-4 title gains D5; the unimplemented list moves to
  D6+ (endpoint schema registry still deferred; D6 PanelStore append/partition /
  optional materialized derived panel store only if factor research needs it).
- CLAUDE.md / AGENTS.md (kept byte-identical): PR #51 added to the merged-PR list;
  D5 progress bullet (global rate limiter, max_workers default 1, serial-default
  compatibility, deterministic main-thread writes); quality gate 656 -> 677 passed
  (+ D5 scheduler=9 + concurrency=12); phase0 anchor 0.9600/0.8408; roadmap D5
  done, endpoint schema registry deferred, D6 optional derived panel store.

Docs only — no qt/ data/ tests/ config/ change.
@StackOverFlow11
StackOverFlow11 merged commit 78b0ad9 into main Jun 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant