feat(web-check): 404 in an unreachable repo is web_unverifiable, not web_not_found

[txemi]

What

On a 404, web-check couldn't distinguish two very different cases and reported both as
web_not_found (which fails a gate):

• the file genuinely moved in a repo we can see, vs.
• the repo/ref itself is unreachable — a private repo we can't read (GitHub returns 404 to hide
  its existence) or a deleted branch.

Now, on a 404, web-check probes the repo root at the same ref (cached per owner/repo/ref). If that
is also unreachable, the finding is web_unverifiable (a warning, never a failure) — a broken
link in a repo we can't even see is not ours to assert. A genuine move in a reachable repo stays
web_not_found.

Why

It makes a web gate usable in a real repo that references client/private repos it can't access
(and links to deleted feature branches): those become web_unverifiable instead of failing the build,
while genuine breakage in repos we can read still fails. Measured on one such tree: 30 → 1
web_not_found (29 were inaccessible client repos).

Coverage

Two new tests (unreachable-repo → unverifiable; reachable-repo-moved-file → not_found) and the existing
test_online_404_is_web_not_found_exits_4 updated to make the repo root reachable. Full suite: 148 passing.

[Copilot]

Pull request overview

This PR refines darnlink web-check --online’s handling of GitHub 404s by distinguishing between a genuinely missing/moved file in an accessible repo versus an unreachable repo/ref (e.g., no access or missing ref), so unreachable destinations are reported as web_unverifiable rather than failing the gate as web_not_found.

Changes:

• On destination 404s, probe the repo root at the same (owner, repo, ref) (with per-repo/ref caching) to decide whether to emit web_not_found vs web_unverifiable.
• Extend/update tests to cover both “404 in unreachable repo/ref” and “404 in reachable repo” scenarios, and adjust the existing 404 test to keep the repo root reachable.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
src/darnlink/weblinks.py	Adds repo-root reachability probing (cached per owner/repo/ref) and threads the result into 404 classification.
tests/test_weblinks.py	Updates the existing 404 test and adds two new tests to validate unreachable-vs-reachable 404 classification.

Comments suppressed due to low confidence (1)

src/darnlink/weblinks.py:221

• The comment and surrounding PR description mention "deleted branch", but parse_github_url() only captures ref as a single path segment ((?P<ref>[^/]+)), so common branch names like feature/foo won’t be parsed and therefore won’t reach this 404/root-probe logic (they’ll be reported as web_unverifiable as an unrecognised URL shape). Consider clarifying this limitation in the comment (or expanding parsing) so the behavior matches expectations for deleted feature branches.

            # A 404 is ambiguous: the file moved in an ACCESSIBLE repo, or the repo/ref itself is not
            # reachable (a private repo we can't read — GitHub 404s to hide it — or a deleted branch).
            # Probe the repo root at the same ref to tell them apart: if THAT is unreachable, we cannot
            # verify (so `web_unverifiable`, a warning), rather than assert the file moved (`web_not_found`).

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[txemi]

Valid — reworded in 373ddf9: 'no access, or the ref no longer exists' (ref can be a branch, tag or SHA; the root probe detects an unreachable/missing ref generally).

[txemi]

Fixed in 9a6c911 — reworded the test comment to 'a ref that no longer exists' too.

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (2)

src/darnlink/weblinks.py:150

• The new 404 handling text still says “deleted branch”, but gu.ref can be a branch, tag, or SHA; the root probe is really detecting an unreachable/missing ref in general. Rewording avoids a misleading, overly specific explanation in both the comment and the user-facing detail string.

            # The repo root at this ref is unreachable too: a private repo we can't read (GitHub 404s to
            # hide it) or a deleted branch. We can't tell "moved" from "no access", so we do NOT fail the
            # gate on it — a broken link in a repo we can't even see is not ours to assert.
            return WebFinding("web_unverifiable", f, link.href,
                              "destination repo/ref not reachable (no access, or a deleted branch) — cannot verify")

src/darnlink/weblinks.py:221

• This comment explains the ambiguous 404 case as possibly coming from a “deleted branch”, but the ref may also be a tag or SHA; the probe is distinguishing “repo/ref reachable” vs “repo/ref unreachable” more generally. Updating the wording prevents incorrect guidance for future maintainers.

            # A 404 is ambiguous: the file moved in an ACCESSIBLE repo, or the repo/ref itself is not
            # reachable (a private repo we can't read — GitHub 404s to hide it — or a deleted branch).
            # Probe the repo root at the same ref to tell them apart: if THAT is unreachable, we cannot
            # verify (so `web_unverifiable`, a warning), rather than assert the file moved (`web_not_found`).

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.