Remove fabricated strength-scan diagnostics and enforce data integrity

[charlesmartin14]

Motivation

• Eliminate synthetic/placeholder immediate WW-PGD outputs that fabricated alpha/D/fit counts and silently substituted tiny fixture datasets, which produced unverifiable scientific artifacts.
• Ensure immediate WW-PGD diagnostics are produced only from real WeightWatcher measurements and that production strength-scan execution fails transparently when prepared scientific data is missing.
• Add provenance, schema fields, audit tooling, and stricter resume/manifest checks so scientific analyses only accept measured, verifiable data.

Description

• Removed the synthetic immediate spectral row generator and local-fixture fallback and wired immediate before/after WeightWatcher measurement into projection events by running weightwatcher_details immediately before projection and (when immediate_projection_spectral is true) once after the full projection; measured layer records are matched by longname/name and written to wwpgd_projection_spectral.csv with measured fields only. (src/wwgpt/train.py, src/wwgpt/ww.py)
• Deleted the silent prepare_local_text fallback in production run_strength_scan and replaced it with a provenance-rich RuntimeError that includes requested data root, level, token multiplier, expected directory, original exception type and message. (src/wwgpt/strength_scan.py)
• Added resume compatibility checks so a completed arm is reused only when a comprehensive set of provenance fields match (seed, strength, level, token multiplier, model/train configs, projection schedule, target alpha, init/data/tokenizer hashes, realized tokens, schema version, WW-PGD commit, and immediate_projection_spectral); incompatibilities are recorded append-only. (src/wwgpt/strength_scan.py)
• Introduced measured-file schema and provenance fields on immediate spectral files: immediate_spectral_source = "weightwatcher_measured", weightwatcher_version, weightwatcher_configuration, and measurement_valid_for_science = true; analysis rejects legacy files lacking these fields as legacy_fabricated_or_unverified and emits prominent warnings. (src/wwgpt/strength_scan_analysis.py, src/wwgpt/train.py)
• Added wwgpt audit-strength-scan --scan-root PATH which writes analysis/strength_scan_integrity_audit.csv and analysis/strength_scan_integrity_summary.json and distinguishes Cases A/B/C (fabricated immediate alpha, fixture/toy budget, fully measured). (src/wwgpt/strength_scan_analysis.py, src/wwgpt/cli.py)
• Notebook updates: notebooks/07_strength_scan_overview.ipynb and notebooks/08_strength_scan_weightwatcher.ipynb now refuse missing real scan roots by default and gate test-fixture usage behind WWGPT_ALLOW_TEST_FIXTURE=1, and immediate-alpha analysis refuses legacy/unverified files.
• Added documentation docs/STRENGTH_SCAN_DATA_INTEGRITY.md explaining the prior issue and audit procedure.
• Tests and test harness updated to assert production behavior (no silent fixture fallback, fabricated constants removed, immediate WW calls before/after projections, and audit classifications). (tests modified/added)

Testing

• Installation: ran python -m pip install -e . successfully.
• Full automated test suite: ran pytest -q and all automated tests passed (42 passed, 3 warnings).
• Notebooks: executed the relevant notebooks in CI-style tests; notebook execution tests passed after notebook gating for fixtures.
• Audit verification: exercised the new wwgpt audit-strength-scan against three synthetic scan directories representing Cases A/B/C and confirmed the audit classifications:
  • Case A: immediate alpha file flagged as legacy_fabricated_or_unverified and immediate-alpha analysis invalid.
  • Case B: fixture/toy dataset detected and scan marked invalid for scientific claims.
  • Case C: immediate spectral files measured and scan eligible for scientific analysis (subject to fit-quality checks).
• Static checks: compiled package (python -m compileall src/wwgpt) and scanned repository for removed fabrication markers (no remaining hardcoded alpha_before = 2.5, D_before = 0.02, num_evals_before = 10, prepare_local_text fallback in production).

All fabricated immediate-alpha generation and silent dataset fallbacks have been removed from production flows, immediate WeightWatcher before/after diagnostics are genuinely measured when enabled, and legacy fabricated immediate-alpha files are excluded from scientific alpha analysis unless re-measured.

---

Codex Task