Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
95 changes: 95 additions & 0 deletions .cargo/mutants.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
# Can the repository laws go red?
#
# Every other mechanism here asks whether a law PASSES. This one asks the
# opposite question, and it is the question this repository's own history says
# is the dangerous one. Three defects found in these laws were laws that passed
# while what they claimed was false: an obligation that
# lost its `red:` row and still qualified, a green route seated by any test that
# happened to be in the file, and a manifest table made invisible to the
# topology reader by a trailing comment. None of the three was a failing check.
# Each was a check whose numerator and denominator moved together, so the
# printed PASS stayed exactly as true-looking as it had been.
#
# A mutation run is the only mechanism in this repository that attacks that
# directly. It damages one decision in a source, rebuilds, and runs the tests: a
# mutant the tests CATCH is a decision some test is actually about, and a mutant
# that SURVIVES is a decision nothing executed would have noticed being wrong.
# Survivors are the finding. They are not repaired by the run and they are not
# repaired by whoever reads it in the same breath — a surviving mutant is
# evidence debt, named at its own home, and the repair is a test or a type there.
#
# WHY `xtask` FIRST, and the scope below says so in one line. xtask is the plane
# whose checks most need proof they can go red, because a repository law is the
# one kind of claim here with no second reader: a test that stops testing is
# caught by the law that counts it, and a law that stops enforcing is caught by
# nothing. It is also the crate all three defects above were found in. The
# machine, the services, and the judge are out of scope for now and their
# opening condition is this run being cheap enough to widen.
#
# WHY THIS IS NOT A GATE, and never becomes one. A mutation run rebuilds and
# retests once per mutant, so its wall-clock cost is the tests' cost multiplied
# by the number of decisions in the sources. That does not belong between a
# commit and a merge: a contributor waiting on it would learn to route around
# it, and a timeout in it would read as a verdict about the change. It runs on a
# schedule in `.github/workflows/mutation.yml`, where a survivor means "go look"
# rather than "this change is bad" — the same split the dependency workflow
# already makes between its graph job and its advisory job.
#
# WHAT THIS DOES NOT CLAIM. A mutant that is CAUGHT says some test noticed one
# damaged decision; it says nothing about whether that test is about the right
# thing. A mutant reported UNVIABLE says the damaged source did not compile, and
# on this tree that is most of them — a strongly typed reader refuses most
# damage before any test is reached. Unviable is not coverage and it is not a
# gap: it is a question the run never got to ask, and reading a low survivor
# count off a high unviable count would be reading coverage out of silence.
#
# WHERE THIS FILE IS COUNTED: in the roster `xtask/src/main.rs` holds.
# `alarm-artifacts-are-present-and-distinct` reads this file and refuses when it
# is deleted or emptied, which is the opening condition `.config/nextest.toml`
# wrote once for both surfaces, now discharged. Its own report still has a
# denominator of a different kind — the run derives it, every mutant examined,
# split into caught, survived, timed out and did-not-build, and the job prints
# it — and that one is published by a run rather than joined by a law, so what
# refuses when this file goes is the roster entry and not the report.

# The scope, and the whole of it. Mutants are generated from these sources
# alone; the tests that judge them are the mutated package's own, which for
# every file here is xtask's.
#
# THE ONE THING THIS LINE CANNOT SAY is which PACKAGES a run examines — the
# configuration has no key for it, so the selection is an argument. MEASURED:
# `cargo mutants` with no selection takes the root package alone, this glob
# matches nothing inside it, and the run finds no mutant and exits 0. It does say
# so, on a WARN line — and a warning that fails nothing is what a job reads as
# success, which is the whole difference between a diagnostic and a refusal.
# `cargo mutants --workspace` is therefore the invocation, on a working machine
# exactly as in the hosted job, and the job COUNTS what it examined rather than
# trusting that it examined something.
examine_globs = ["xtask/**/*.rs"]

# CHOSEN: cap the lints, and this is the setting that decides what the run
# MEASURES. `.cargo/config.toml` makes a surviving warning fatal, so without
# this a mutant that merely trips `unused_variables` or `clippy::let_and_return`
# fails to build and is reported UNVIABLE — a verdict that reads as "not a
# coverage gap" when the tool never got far enough to ask. Capping the lints
# asks the one question a mutation run is for: does a TEST catch it. What the
# lint wall catches is a different seat with its own stage in the entry bar, and
# it is not weakened by anything here — this flag reaches the throwaway build in
# a scratch directory and no build of this tree.
cap_lints = true

# CHOSEN: write the report inside `target`, which is already ignored. The
# default puts `mutants.out` beside `Cargo.toml`, and the qualification road
# ends by refusing a checkout that does not match what is committed — so the
# default would make a working machine's mutation run fail the next `cargo xtask
# qualify` for a reason that has nothing to do with the tree. A tool that writes
# into the checkout is a tool that has to be remembered; one that writes into
# `target` does not.
output = "target"

# NOT SET, ON THE RECORD: `test_tool`. nextest would run these suites faster,
# and `.config/nextest.toml` configures it two directories away. It is left at
# cargo deliberately: a mutation result must not depend on whether the second
# harness works, because the two surfaces landed together and a common cause
# would take both readings out at once. It is adopted the day the second harness
# has a history of its own.
Loading
Loading