Asb Interview Learning

Learning: Let the Interviews Rewrite Your Hypotheses

Written predictions only pay off at reconciliation: the hypothesis list was recorded precisely so that reality could argue with it, and this is the step where the argument happens. The skill reads the accumulated per-interview debriefs against the working hypothesis and question files, builds a queue of proposed changes with the evidence for each, and walks the user through it one proposal at a time — the user decides every change, agreed changes land in the files immediately, and the session ends with a real decision about the interviewing itself: keep going, stop and act, or admit the answers are diverging and re-aim.

The mental model

The three-tier update rule

Not everything heard deserves a reaction, and the tiers are the core of this step:

• A stray voice — one person said something contradictory. Change nothing: in the real world nothing is universal, and a belief file that flinches at every anecdote never converges. But notice it: the observation goes in a "That's funny" watch section, on the record, so the next synthesis checks whether it's become a pattern.
• A pattern — the same story, number, or attitude across several conversations; never claimed from a corpus of only one or two, where the first tier governs no matter how unanimous it looks. Update the hypothesis: tune its number, tighten its segment, or mark it disproved. Patterns are the fundamental truth the interviews exist to find.
• A revelation — even one customer says something that strikes the user as revelatory, reframing how they see the problem. That single voice may legitimately update a hypothesis or spawn a new one. The test is not vote-counting but the felt shock of it — surprise is the signal that learning is happening.

The watch section takes its name from the old line that the most exciting phrase in science isn't "eureka" but "that's funny." Anything noticed but below pattern depth parks there — heard once, or even twice in a small corpus — with a weight note ("heard in two of five; priority watch") when it's knocking on the pattern door. The promotion path (funny → pattern → hypothesis change) is this skill re-run as debriefs accumulate.

Expect contradictions — and "no pattern" is still learning

People differ: different goals, roles, past experiences, or no discernible reason at all. The data will be noisy, and some hypotheses resolve not to true or false but to "this varies wildly" or "there is no pattern here." Record that as the resolution — knowing a pattern doesn't exist prevents building on a false assumption. Real patterns stand out from noise; that contrast is the finding.

Convergence is what truth feels like

Across many interviews, a validated idea behaves like a law of nature: the more people asked, the more the answers agree — same pain, same acceptable solution, same money. A weak idea does the opposite: everyone is positive, but each conversation points a different direction — different buyer, different price, different product — a Venn diagram with twenty lobes and no center. Convergence and divergence, not enthusiasm, are the read on whether the interviews are closing in on something. Watch which one is happening; it drives the final verdict.

Emergent segmentation

Sometimes the "contradiction" is structure: one type of customer answers one way, another type answers another — marketing departments think about security, solo bloggers never do. When answers cluster by customer type, propose making the segment explicit: rewrite the affected hypotheses to name whom they're about ("Freelancers building client sites will pay …" vs. "Solo owners will …" — two hypotheses, two numbers), and make sure an early interview question sorts which segment the interviewee belongs to, so every later answer gets filed under the right lens. Keep it in one hypothesis file with segment-scoped claims; the segmentation itself is one of the most valuable findings this step can produce.

Conflicting signals get a choice, not "a balance"

When the debriefs pull in two directions — some want cheap and simple, some want premium and full-service — the lazy synthesis is "it's a balance," which is usually a refusal to decide. Force the real resolution:

• A balance is correct only when both extremes are genuinely bad and something in between beats either. Rare in interview findings.
• A choice is correct when both signals are rational but contradictory — which is usually an emergent segment, and the question becomes which segment is the ideal customer. That's the user's decision to make with eyes open, not this skill's.
• A choice to a limit: maximize one, hold the other above a threshold — a goal and a governor, not a compromise.
• Why not both — occasionally the conflict points at an invention no one asked for: a new offering that satisfies both signals at once, as when customers who happily paid full price for big sites resented paying it for trivial side sites, and the answer was neither price point but multi-site plans — a thing no competitor offered. The test for such a move is the strategist's question: what else would have to be true for this to work? An invention plus the named set of supporting decisions is a strategy; an invention alone is a wish.

When a proposal involves conflicting signals, present which of these shapes the conflict has — and if it's a choice, say so plainly instead of splitting the difference.

Stop when it's boring

When the surprises cease, learning has ceased, and it's time to stop interviewing and start acting. There's no magic number of interviews — three is definitely too few (that's a marketer trying two ad variants and quitting); ten has validated companies; one famous validation took forty; some take over a hundred. The signals, made concrete:

• Surprise rate. Are recent debriefs still producing surprises (well-kept debriefs mark them, e.g. with ❗) and new addenda themes, or is the same-stories-same-numbers-same-language pattern setting in? Falling surprise rate = approaching done.
• Convergence. Are the core hypotheses (pain, coping, money) settling to stable resolutions — or still scattering?
• Would more change anything? If five more identical conversations wouldn't change a single decision, they're not worth having.
• Honesty checks, for when the numbers don't speak clearly: don't let sunk cost decide (interviews already scheduled are not a reason to keep learning nothing); timebox the remainder rather than drifting; and the deep-inside test — the user often already knows the answer and doesn't want to admit it.

Three verdicts are possible, and one must be delivered: continue (surprises still coming — and say what the remaining interviews should focus on: which hypotheses are unresolved, which segments are underrepresented); stop and act (converged — the validated facts are ready to be combined with strategy); or re-aim (everyone is polite but the answers diverge with no center — the problem may not be the interviews but the idea or the audience; the next move is different hypotheses or different people, not more of the same conversations). "Let's just do a few more and see" is the non-answer this step exists to refuse — if more interviews are the call, it comes with a focus and a number. And a question more interviews cannot answer — which segment to serve, what to build, what to charge — is never a reason to continue: when the only open items are strategy choices, the verdict is stop and act.

Vocabulary

• Debrief — the per-conversation record this skill consumes: brief answers mapped to Q-numbers (which carry H-numbers), plus addenda of slotless findings, one file per interview.
• Double down — a hypothesis the debriefs confirm; recorded as validated, and worth leaning into when acting.