Test Suite Curation

<objective> Test A and Test B cover the exact same lines, so a tired engineer deletes B — and three weeks later a production defect slips through because B was the only test that asserted the rounding was correct. Coverage equality is not redundancy. This skill audits an entire regression suite as a corpus (the redundancy analysis no human does by hand), prunes it on evidence rather than vibes, and treats every deletion as a destructive change that requires a quarantine grace period, human sign-off, and a record you can defend to an auditor. </objective>

Quick Route

Discovery Questions

First, check .agents/qa-project-context.md in the project root and skip anything it already answers. Then clarify:

• Which language/runner? pytest+coverage.py, Jest/Vitest+Istanbul, Go, JUnit — the per-test context mechanism differs per stack (and so does the mutation tool).
• Is there CI test-result history, and how far back? No JUnit/Datadog/Trunk history means you cannot mine never-failing or flaky signals — you only have coverage and clustering.
• What is the suite size and current wall-clock time? This sizes the tiering target (e.g. smoke under 5 min) and whether per-test coverage is feasible in one run or must be sharded.
• What is the business risk map / critical paths? Tiering and the "keep" disposition both depend on it. If absent, run risk-based-testing first.
• Who signs off on deletions, and is there a CODEOWNERS file? Deletion is destructive; you need a named approver before this skill removes anything.
• What is the acceptable observation window? How long the team will run quarantined tests as skipped before permanent removal (default: 2 sprints / 2 releases).

Core Principles

1. Coverage equality is not redundancy. Two tests hitting the same lines can assert completely different things — different oracles, inputs, edge cases. Line/branch coverage tells you what code ran, never what was checked. The only evidence that one test subsumes another is that the survivor catches the same faults, which you prove with mutation testing, not a coverage diff.

1. The agent's edge is whole-corpus analysis, not deletion authority. An agent can fingerprint 4,000 tests, cluster near-duplicates, and cross-reference CI history in minutes — work no human does by hand. That is the entire value. But the agent proposes; a human approves. Never let the corpus-scale analysis become corpus-scale auto-deletion.

1. Deletion is destructive and must be reversible in practice. "It's in git history" is not a recovery plan. Quarantine first (skip/xfail, or move to a deprecated suite), observe for a defined window, watch for escaped defects, then delete with sign-off. The grace period is the safety net, not the commit log.

1. Every disposition is differentiated. Redundant, obsolete, and low-value are three different states with three different actions. Collapsing them into one "delete" bucket is how you lose real coverage. A test that never failed is not the same as a test that cannot fail.

1. Evidence over intuition, recorded per test. Each removal carries its own row: category, the test that supersedes it, the coverage delta, who approved, and how to restore. If you cannot fill the row, you cannot delete the test.

1. Coverage Fingerprinting (per-test)

The wrong answer is a single combined --cov report that tells you per-file percentages. That cannot tell you which test covered which line, so it cannot tell you which tests overlap. You need per-test (dynamic) contexts.

pytest / coverage.py — record which test hit each line with dynamic contexts, and turn on branch coverage:

pytest --cov=src --cov-branch --cov-context=test

--cov-context=test makes coverage.py call switch_context() around each test, tagging every measured line with the test that ran it. Add --cov-branch so a test that takes the if and one that takes the else are not treated as covering "the same line." The result is written into the .coverage SQLite database, in the context and line_bits/arc tables.

Then read the contexts table out of the `.coverage` SQLite DB to build a per-test fingerprint: for each test, the exact set of (file, line) and (file, branch-arc) pairs it covered. From those sets you compute:

• Uniquely covered lines — lines/branches that only one test covers. Lose that test and you lose that coverage outright. These tests are pulling their weight; protect them.
• Subsumption — test A's covered set is a superset of test B's. A candidate for redundancy (but see §2 — it is not proof).

See references/coverage-fingerprinting.md for the SQL to pull contexts from .coverage, the Python that builds per-test line/branch sets and computes uniquely-covered and subsumption relations, and the JS/Vitest+Istanbul --coverage equivalent (coverage-final.json with per-test reporters).

Do not rank tests by line count per test file, and do not delete tests merely for having a low overall coverage percentage — a one-line test can be the only thing guarding a critical branch.

2. The Coverage-Equality Trap (the load-bearing rule)

This is the single most important rule in the skill. When per-test data shows Test A and Test B cover exactly the same lines, the naive conclusion is "redundant, delete one." That is wrong, and here is the gotcha:

Coverage equality does not prove the tests have the same assertions, the same inputs, or the same oracle. Two tests can cover identical lines while one asserts the HTTP status and the other asserts the response body, or while they pass different edge-case inputs. One covers same lines but asserts different values; the other covers same lines but checks different state. Coverage measures execution, not verification.

To find out whether B is actually redundant — whether A truly subsumes B's fault-detection — run mutation testing:

• mutmut (3.6.0+) or cosmic-ray for Python, StrykerJS (9.x) for JS/TS, PIT for Java/JVM, cargo-mutants for Rust.
• Mutation testing injects faults (mutants) into the covered code. A test "kills" a mutant if it fails on the mutated code. If A kills every mutant that B kills, A genuinely subsumes B's fault detection and B is a defensible delete candidate. If B kills a mutant A misses, B catches a defect class A does not — keep B, even though coverage was identical.

Decision: identical coverage → flag as a candidate → confirm with mutation testing → only then propose deletion. When assertions differ and mutation results differ, retain both, do not delete. See references/mutation-confirmation.md for the mutmut/Stryker config that scopes mutation runs to the suspect lines and the kill-set comparison.

3. Near-Duplicate Clustering

Goal: surface copy-pasted tests without flagging every test in the same file. Grouping tests by filename is not clustering — it tells you nothing about similarity. Two defensible signals, combined: