Skip to content

Docs: corpus README says 5-case smoke per PR (CI runs full 10); cassette.py docstring path is stale #75

Description

@JumpTechCode

Summary

Two stale doc strings that no longer match the code.

Evidence

  • sentinel/evals/corpus/README.md:38-39 — claims "the deterministic 5-case smoke subset CI runs on every PR." CI actually runs the full 10-case corpus per PR (.github/workflows/ci.yml:108-109, make evals with no --smoke); the corpus has 10 cases.
  • sentinel/evals/cassette.py:11-12 — docstring says cassettes live under cassettes/<prompt_version>/<model_id>/<key>.json, but the actual layout is flat (cassettes/<key>.json); prompt_version/model_id are folded into the key hash (compute_cassette_key, cassette.py:52-70; path at :153).

Fix

Update both doc strings to match reality (full 10-case per-PR run; flat cassette layout).

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationImprovements or additions to documentation

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions