petrkindlmann/qa-skillsMIT5 files

Bug Reproduction

>-

Specification
Skill ID
petrkindlmann/qa-skills/bug-reproduction
Publisher
petrkindlmann
Repository
qa-skills
Installs
0
Files
5
License
MIT
Synced
Sep 23, 2026
How to use it

Open any RiverX project, open the Skills panel in the chat, and search for this identifier. The files are fetched from the source repository at install time.

petrkindlmann/qa-skills/bug-reproductionInstalls these files
  • SKILL.md
  • references/bisect.md
  • references/determinism.md
  • references/intake.md
  • references/ticket-writeback.md

What this skill tells the agent

<objective> A bug you cannot reproduce is a bug you cannot fix or prove fixed. This skill takes a thin, hand-wavy report ("order total is wrong sometimes") and drives it to a VERIFIED minimal reproduction, a deterministic failing test written BEFORE the fix, a git bisect that names the introducing commit, and a structured evidence block in the ticket. The discipline it enforces: reproduce before theorizing, minimize one cut at a time, freeze time/seed/network so the repro fails identically every run, watch the test go red first, and confirm the fix flips it green — and that reverting the fix turns it red again. </objective>

Quick Route

You have…Go to
A thin report and no idea how to trigger itStep 1: Extract the implicit repro
A messy 14-step repro to clean upStep 2: The reproduce-minimize-isolate-capture loop
"Worked last month, broken now"Step 3: Bisect to the introducing commit
A repro that passes/fails inconsistentlyStep 4: Make the repro deterministic
A clean repro, no fix yetStep 5: Write the failing regression test (red) first
"The dev says it's fixed, test is green"Step 6: Verify the fix actually fixes it
"It won't reproduce for me but does for the user"Step 7: Flaky vs environment vs not-reproducible
Repro + test doneStep 8: Write the evidence back into the ticket

Discovery Questions

First, check .agents/qa-project-context.md in the project root — it carries the tech stack, test runner, known-flaky areas, and environment matrix. Pass over any question it already answers. If it is missing, suggest creating one with the qa-project-context skill.

  • What is the report, verbatim? The thinner it is, the more you must extract before touching code — a one-line report sets the whole intake agenda (Step 1).
  • Does it reproduce at all yet, and how reliably? "Every time" vs "sometimes" decides whether you go straight to minimizing or into determinism work first.
  • Did it ever work? A known-good past release unlocks git bisect to the introducing commit; no known-good point means you debug forward instead.
  • What is the test runner and stack? Vitest/Jest vs Playwright changes the determinism API (vi.setSystemTime vs page.clock) and where the regression test lands.
  • What are the non-determinism sources? Time-of-day rules, randomness, third-party APIs, locale — each must be pinned for the repro to be trustworthy.

Core Principles

  1. Reproduce before you theorize. The strongest wrong instinct is to read a symptom and jump to a root cause ("sounds like a float rounding bug, let me patch the total calc"). Don't. Extract the repro, make it fail on demand, and only then form a hypothesis. A fix without a reproduction is a guess you can't falsify.
  1. A repro is a deterministic artifact, not a story. "It happens around midnight with a random code" is a story. Freeze the clock, seed the RNG, and stub the network so the same inputs produce the same failure on every run and every machine. If it isn't deterministic, you can't bisect it, test it, or prove it fixed.
  1. Minimize one variable at a time, and re-confirm after every cut. Shrinking the repro is a search, not a rewrite. Remove one step, data field, or dependency, then re-run and confirm it still reproduces. Removing several at once tells you nothing about which one mattered.
  1. The regression test is written RED, before the fix. Assert the real expected value, watch it fail first (proving it catches this bug), then watch the fix flip it green. A test added after the fix, or one disabled / marked pending, proves nothing.
  1. Green isn't done — revert-to-verify is. A passing test can pass for the wrong reason. Temporarily remove the fix and confirm the test goes red again. Only a test that fails without the fix actually guards against the bug.

Step 1: Extract the implicit repro

A thin report ("Checkout is broken, order total is wrong sometimes") names a symptom, not a path. Before any code, extract or ask for every reproducibility dimension. Never invent the repro steps from imagination, and never theorize a root cause yet — both come after the bug reproduces.

For a "wrong total" / data-correctness bug, the load-bearing dimensions a thin report most often omits are:

  • Exact steps to reproduce — the click-by-click path, not "checkout is broken."
  • Build / version / commit (git SHA) — they may be on a build where it's already fixed.
  • Environment — browser + version, OS, device.
  • Input data — cart contents, quantities, the account/user, coupon, the exact fixture. A total is a pure function of its inputs; without them you are guessing.
  • Expected vs actual — the number they expected and the number they saw.
  • Frequency — every time, or intermittent? "Sometimes" points at non-determinism.
  • Locale / timezone / currency — rounding, tax, and formatting are locale-specific; a total "wrong" in de-DE may be correct in en-US.
  • Timestamp of occurrence, plus any logs/screenshots/network trace.

Write these into a single repro spec before touching code. If a row is blank, that is your next question to the reporter — not a license to start writing the fix or theorizing.

See references/intake.md for the full extraction checklist, why each load-bearing dimension matters for "wrong total," and the repro-spec template.


Step 2: The reproduce→minimize→isolate→capture loop

Given a confirmed-but-messy repro (e.g. 14 manual UI steps across 3 pages), do not hand the 14-step version to the developer and do not rewrite it wholesale. Run this loop:

  1. REPRODUCE / confirm. First establish a baseline: run the full repro and confirm it actually fails. You can only minimize something that currently reproduces.
  2. MINIMIZE. Remove one step, field, or dependency. Re-run. If it still reproduces, keep the cut; if it no longer fails, that element was load-bearing — restore it. Repeat, one variable at a time, until every remaining piece is necessary. This is the core ordering rule: never minimize before confirming it reproduces, and never omit verifying it still fails after each cut.
  3. ISOLATE. Narrow the failure to the smallest layer that still shows it — drop from a 3-page UI flow to a single page, then to a unit/API call against the offending function if the bug lives below the UI.
  4. CAPTURE. Record the now-minimal repro as evidence: the smallest steps or the single command, plus logs/trace/screenshot. This is what the developer and the regression test consume.

The output is the smallest sequence that still reproduces — not the original walkthrough.


Step 3: Bisect to the introducing commit

The bug is on HEAD but a past release was clean, and you have a command that exits non-zero when the bug is present. Use git bisect run to binary-search history automatically — do not manually check out each commit, and do not use git revert to hunt for it.

git bisect start
git bisect bad HEAD          # current commit has the bug   (alias: git bisect new)
git bisect good v2.4.0       # last clean release            (alias: git bisect old)
git bisect run npm test -- checkout-total.spec.ts   # ONE targeted test, never the full suite
# bisect prints "<sha> is the first bad commit"
git bisect reset             # ALWAYS clean up — restores the original HEAD

Use this skill in a project

Open any RiverX project, search the Skills panel for petrkindlmann/qa-skills/bug-reproduction, and install it in one click.

Use this skill