[Automation Reliability Gap] Persist scheduler execution receipts and resumable checkpoints

[seonghobae]

Buyer/operator outcome

Make Clearfolio's autonomous development loop diagnosable and resumable when an hourly invocation ends at a platform/tool boundary. Operators must be able to distinguish scheduling, run admission, queue construction, repository execution, clean budget continuation, and failure instead of receiving only a generic task-error sentence with no durable execution identity.

Fresh incident evidence — 2026-08-10

The external hourly control plane has repeatedly returned only:

There was a problem with your scheduled task.

The connected repository state proves that Clearfolio work can continue and that multiple PR/issue lanes exist, but the available external task interface does not expose an exception type, failing phase, action receipt, checkpoint, or stack trace for those generic failures. Therefore the internal cause must not be invented.

Repository-local Draft PR #271 defines a future GitHub Actions OpenCode development path, but it is not protected-main behavior and is dependency-gated behind #270 plus the central .github control plane. It cannot be treated as evidence that the currently failing external scheduler invocation executed successfully.

Current RCA classification

• Observed symptom: generic external scheduled-task failure after an execution-oriented prompt.
• Proven missing control: no durable, user-visible phase/action/checkpoint receipt links the scheduled invocation to exact Clearfolio repository evidence.
• Supported hypotheses requiring telemetry: practical invocation/tool-budget exhaustion; connector/provider failure; prompt/control-plane transport failure; repository action failure; or another platform boundary.
• Rejected behavior: choosing one hidden cause without evidence, treating scheduler activation as repository execution, or treating the generic error as repository completion.

Required control contract

Keep the external scheduler a thin control plane and repository documents as detailed authority, while introducing a privacy-safe execution receipt model equivalent to:

• automation_run
• automation_checkpoint
• queue_snapshot
• deferred_lane
• action_receipt
• failure_envelope
• continuation_handoff

Each run must record, where the platform permits:

1. immutable run identity and scheduler prompt/control version or digest;
2. scheduled/admitted/started/completed/failed/budget-continuation phases as separate states;
3. protected-main SHA plus exact PR/issue identities observed at queue construction, explicitly historical after the run;
4. last completed atomic action and its exact repository evidence identity;
5. current atomic action, if any, and whether it committed, failed, or was safely abandoned;
6. controlled failure category and failing phase without secrets, raw document content, tokens, tenant/subject identifiers, provider-controlled exception text, or stack traces exposed to users;
7. deferred lanes keyed by exact PR/head/live-base/run/review identity;
8. clean budget_continuation before a hard boundary when available signals show that a new multi-step mutation cannot be verified safely;
9. next-run handoff that always starts with fresh GitHub state and treats the checkpoint as historical evidence, never current authority;
10. no temporary/self-modifying/competing writer workflow and no knowingly broken non-test-only branch head left behind.

Prompt and repository authority

• The scheduler prompt remains a concise execution policy, not a duplicated PRD/TRD/architecture corpus.
• AGENTS.md, canonical product/architecture/security/operability documents, current code and live GitHub policy remain detailed authority.
• Prompt edits and documentation updates have zero completion credit while safe product work exists.
• A generic scheduled-task error is local control-plane debt and must hand off to another safe repository lane in the same invocation when tools remain available.
• Repository-local OpenCode automation uses NVIDIA_NIM_API_KEY only through GitHub Secrets and never COPILOT_GITHUB_TOKEN as a development-model credential.

Test-first acceptance

Add deterministic contract tests before implementation/documentation changes for at least:

• activation without execution is not success;
• run-start without queue construction is distinguishable;
• generic connector/provider failure records a controlled failing phase and last safe checkpoint;
• hard-budget simulation produces budget_continuation, not a generic failure and not repository completion;
• failure after a test-only RED head either reaches GREEN in the same run or leaves an explicit safe test-only continuation state;
• partial contents/ref mutation uses exact-parent/blob/ref identity and cannot claim completion after CAS movement;
• duplicate/overlapping invocation is rejected or branch-locally deferred by writer lease;
• next run refetches all live identities rather than trusting checkpoint SHAs;
• receipts contain no raw secrets, tokens, document content, tenant/subject values or uncontrolled exception messages;
• issue/PR/documentation traceability distinguishes the external scheduler from Draft feat(automation): prevent status-only early termination #271's repository-local future automation.

Documentation updates

Reconcile the canonical documentation line with:

• ADR for scheduler execution receipts, controlled failure envelopes and resumable continuation;
• docs/OPERABILITY.md incident/continuation procedure;
• docs/DATA_MODEL.md conceptual entities and persistence ownership (do not invent a database if the external platform owns storage);
• docs/UML.md schedule → admission → fresh queue → atomic action → receipt → continuation/failure flow;
• docs/TRACEABILITY.md and docs/DOCUMENTATION_ASSESSMENT.md maturity;
• machine-checkable documentation contracts;
• CHANGELOG.md.

Scope / ownership

This is the Clearfolio leaf contract for autonomous-run reliability. The external scheduler platform and central .github control plane remain separate authorities; the Clearfolio writer must not mutate other repositories or claim unsupported platform internals. Repository-local PR #271 may adopt the same semantic contract after its stack dependencies stabilize, but it must not be churned or merged ahead of #270/central prerequisites merely to conceal the external scheduler incident.

[seonghobae]

Additional incident evidence — repository execution can succeed

A later external invocation progressed beyond scheduling/admission into real Clearfolio repository execution, which narrows the earlier generic-error hypotheses without inventing a hidden platform exception.

Observable repository evidence from this invocation:

• protected main was freshly resolved as 83ec6f7fe2b04bdcd28bf98ec350e41e55730a18;
• PR feat(conversion): establish provider-neutral Office adapter contract #306 exact green head 914d1680e83a9935dd0375430bb1f9c0b79ac4ff was revalidated;
• a new test-only head 466a5ede72d3df04140f3732df727953788562d9 was published for the ODF manifest minimum-content contract;
• exact-head CI 31382929438 reached Maven and failed exactly the intended new regression: 565 tests, 1 failure, 0 errors, 0 skips, adapterRejectsManifestWithoutAnyFileEntry, while SAST and Security Scan were successful;
• a narrow GREEN source correction was then published as a5c86719b8adef1c2957a29bc6de6042c59cdd9a; its exact-head checks are now the authoritative verification boundary.

This proves that the generic scheduled-task failures are not evidence of a permanent Clearfolio write-authorization failure or a permanently unavailable GitHub execution path. They remain intermittent/phase-unknown control-plane failures until the external runtime exposes durable phase/action/failure receipts. Issue acceptance remains unchanged: successful repository mutations still do not reveal why predecessor invocations emitted only the generic task-error sentence, and no hidden cause should be inferred.

[seonghobae]

Fresh scheduler-control evidence — 2026-08-10

A direct read of the connected Clearfolio recurring task before this repository action showed the task enabled, while its exposed last_run_time still pointed to 2026-08-10T10:23:10Z. This invocation subsequently reached live GitHub reads and a real Clearfolio branch mutation, so activation/admission and repository execution are empirically distinguishable states.

The available task surface still does not expose a durable run identifier, phase transition log, queue snapshot, action receipt, failure envelope, or last-safe-checkpoint record for the currently executing invocation. Therefore no hidden scheduler failure code/root cause is inferred from the stale last_run_time field. The falsifiable control gap remains the same: after completion/failure, an operator must be able to bind the scheduled invocation to exact repository action evidence without treating activation, an old completion timestamp, or a generic task error as execution proof.

Concrete repository evidence from this invocation: protected main remained 55d7ae8647208e301f282350f076eeddaba61d11; PR #340 was revalidated on exact head c5d2c40809cf6ede5d411dcd1a9f38e993a48e0c, then advanced by commit 1eeeb5fbfc09b0feb3bda257d08ad10c9fd92e23 adding a deterministic dependency-contract regression. These SHAs are historical evidence for this run only and must be refetched next invocation.

This observation strengthens the acceptance requirement that scheduler activation/admission, GitHub action execution, and completion/failure/budget-continuation receipts remain separate evidence authorities.