Skip to content

fix(vendor): count skipped packages, not advisory warnings - #166

Open
Mikola Lysenko (mikolalysenko) wants to merge 1 commit into
mainfrom
fix/vendor-honest-event-counts
Open

fix(vendor): count skipped packages, not advisory warnings#166
Mikola Lysenko (mikolalysenko) wants to merge 1 commit into
mainfrom
fix/vendor-honest-event-counts

Conversation

@mikolalysenko

@mikolalysenko Mikola Lysenko (mikolalysenko) commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator

Problem

The vendor command's skipped count is untrustworthy because per-package advisory warnings are recorded as Skipped events that bump summary.skipped.

record_warning (crates/socket-patch-cli/src/commands/vendor.rs:242) routed every backend VendorWarning through Envelope::record as a Skipped event. But a vendor warning is an advisory ABOUT how a package was vendored — a successful vendor_prebuilt_downloaded service fetch (crates/socket-patch-core/src/vendor/cargo.rs:164, gem.rs:707, composer_lock.rs:566, ...), an artifact rebuild, a content-mismatch overwrite — not a package that was skipped. The package's genuine Applied/Skipped/Failed outcome is recorded separately, so the advisory double-counted.

Observed against real production data (2026-08-12):

  1. A SUCCESSFUL service download surfaced as action:"skipped" (vendor_prebuilt_downloaded), so a single service vendor reported applied:1 skipped:1.
  2. summary.skipped counted warning EVENTS, not skipped PACKAGES — "2 skipped" on a 1-package project, "1 skipped" on a refresh that skipped nothing.

Fix

crates/socket-patch-cli/src/commands/vendor.rs:242 (record_warning): push the advisory Skipped event directly onto env.events instead of through Envelope::record. The advisory stays visible to JSON consumers (unchanged Skipped action + errorCode/reason; Skipped never flips run status, so no status signal is lost) but no longer bumps summary.skipped. That counter now reflects only packages that were genuinely skipped — those recorded via Envelope::record (already_vendored, package_not_installed, vendor_unsupported_ecosystem, benign refusals). A successful service vendor is now counted applied:1 skipped:0.

Test

crates/socket-patch-cli/src/commands/vendor.rs new hermetic module warning_counting_tests:

  • advisory_warning_does_not_bump_skipped_summary — an applied package + a vendor_prebuilt_downloaded advisory yields applied:1 skipped:0, with the advisory still present in events[].
  • multiple_advisories_do_not_accumulate_skips — two advisories on one package keep skipped:0 (reproduces the "2 skipped" report).
  • genuine_package_skip_still_counts — an already_vendored skip still bumps summary.skipped (the fix narrows the counter, it does not zero it out).

RED before / GREEN after (verified by reverting the one-line change: the first two assertions fail with skipped:1). Also re-ran the hermetic vendor suites unchanged: --lib vendor (19), in_process_vendor (27, incl. mismatched_baseline_vendors_with_warning_event which asserts a warning is still a skipped event), scan_vendor_e2e (18), e2e_vex_vendor (8), json_envelope (23), full --lib (350). Build clean.

Scope

Kills the count/label sweep findings: vendor-skip-count-inflation (P2), vendor-envelope-service-download-as-skip (P3), vendor-event-applied-plus-skipped-info (P3), service-success-recorded-as-skipped (P3), vendor-unverifiable-skip-noise (P2).

Deferred (not in this focused PR):

  • vendor-noop-redownload (P2) — the write side. The covered backends already short-circuit an idempotent re-vendor without rewriting when the installed copy is present (e.g. cargo's wiring_in_sync + copy_matches_after_hashes hot path at crates/socket-patch-core/src/vendor/cargo.rs:396; the docker_e2e_vendor_* suites assert byte-stable re-vendor). This PR already makes the no-op REPORT honestly (Vendored 0; 1 skipped for already_vendored, no advisory inflation). The residual re-fetch on a not-installed / fresh-clone re-vendor routes through find_packages_for_purls at crates/socket-patch-cli/src/commands/vendor.rs:699 — the qualified-purl "not installed" lookup owned by a separate PR — so the write-avoidance for that path is intentionally left to it and not touched here.

🤖 Generated with Claude Code


Note

Low Risk
CLI reporting and JSON envelope counting only; no change to vendoring behavior, lockfile wiring, or run exit semantics for successful vendors.

Overview
Fixes vendor reporting where summary.skipped and human “N skipped” lines counted advisory warnings (e.g. successful vendor_prebuilt_downloaded service downloads) as skipped packages, so one applied package could show applied:1 skipped:1 or “2 skipped” on a single-package run.

record_warning still emits a Skipped-shaped event with stable errorCode/reason for JSON, but pushes it onto events directly instead of Envelope::record, so summary.skipped only reflects genuine package skips (already_vendored, package_not_installed, etc.). Run status is unchanged because advisory Skipped events never drove failure status.

Adds warning_counting_tests to lock in applied + advisory → skipped:0, multiple advisories on one package, and real skips still incrementing the counter.

Reviewed by Cursor Bugbot for commit 52a950f. Configure here.

`record_warning` routed every backend advisory through `Envelope::record`
as a `Skipped` event, so each one bumped `summary.skipped`. A successful
`vendor_prebuilt_downloaded` service fetch then reported `applied:1
skipped:1`, a 1-package project could print "2 skipped", and a refresh
that skipped nothing still counted "1 skipped".

Push the advisory event directly onto `events` instead: it stays visible
to JSON consumers (same `Skipped` action + code/detail, which never flips
run status) but no longer inflates `summary.skipped`, which now counts
only packages that were genuinely skipped (recorded via `Envelope::record`
— `already_vendored`, `package_not_installed`, benign refusals, ...).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant