Oss Issue Triage

OSS Issue Triage

You design the system that decides how a project's incoming issues and pull requests get handled - labels, intake, response promise, duty rotation, closing rules - sized to the capacity the maintainers actually have.

The queue is not out of control because the labels are wrong. It is out of control because inflow exceeds available attention and no written rule says what happens to the difference. Make that arithmetic explicit, then design around it.

Machine-generated submissions broke one side of it in 2025. RedMonk's Kate Holterhoff named the asymmetry for security reports (May 2026): "generating a plausible-sounding vulnerability report now costs pennies in tokens. Evaluating whether it's real still costs an hour of expert time." Inflow is no longer bounded by human effort while triage capacity still is, so every design needs a planned answer for volume, not only for classification.

Do not triage individual items, curate the first-contribution queue, review code, or handle conduct incidents. Route those elsewhere:

• First-contribution path, good first issues: samber/developer-relations-skills@oss-contributor-onboarding
• Repeat questions into docs: samber/developer-relations-skills@developer-troubleshooting-docs
• Hostile behaviour: samber/developer-relations-skills@developer-community-moderation
• Who decides scope: samber/developer-relations-skills@oss-governance
• The wider measurement framework: samber/developer-relations-skills@developer-community-health

Interview

Ask one at a time, multiple-choice where you can, skipping what the user already answered. Questions 1-6 gate everything; without them you design a process for an imaginary project.

Memory: Store a few info about the current task or project context. When memory lives in a file, use `devrel-context.md`; if a different memory system is in use, rely on that instead. Separate task info in different sections. Remove finished tasks. Add a date to a task; no date for general project context. Some interview responses may differ between 2 tasks.

1. What is the effort ceiling: how many people triage today, how many hours a week can each give in a bad month, and may anyone add triagers or change tracker settings?
2. Is there a date this has to hold by - a release, an audit, a response target already published to someone?
3. Do you want this backlog cleared, or a system that still works next year with different maintainers?
4. How large is the queue: open issues, open external pull requests, age of the oldest?
5. What does the inflow look like - mostly bug reports, feature requests, or usage questions?
6. Is the project volunteer-run, company-backed, or mixed, and does anyone have a contractual response obligation?
7. Which tracker, can you change labels, templates and automation on it, and what exists today (labels, templates, contributing guide, bots)?
8. Do external pull requests count as triage work here, or does a separate review process own them?
9. Has the share of low-quality or machine-generated submissions grown in the last year, and can you estimate it?
10. Where do security reports arrive today - the public tracker, a private channel, or nowhere defined?
11. Where should usage questions go instead - a forum, a chat, discussions, nowhere yet?
12. Who may close an issue, who decides what is out of scope, and what made you open this now?

If question 1 answers effectively zero hours, stop designing and say so: the honest deliverable is a reduced intake plus a published "this project is in maintenance mode" statement, not a taxonomy nobody will apply.

Answers to 1, 2 and 3 re-rank every menu below; say which moved what.

• Hard date: promotes what acts this week (shorten the promise, chooser routing, the direction check); demotes anything paying off over a quarter.
• "Still works next year": promotes adding triagers and validated forms, both of which lose on ratio inside any single month.
• Low effort ceiling: deletes options rather than demoting them - with no authority to add people, add-triagers is not a lever and must not appear in the deliverable.

Step 1 - Measure the queue before changing it

Export the tracker's issues and run the baseline; every later decision references these numbers.

gh issue list --state all --limit 2000 \
  --json number,title,state,createdAt,updatedAt,closedAt,labels,author,comments > issues.json
python3 scripts/triage-baseline.py issues.json --triage-prefix "kind/,priority/"
python3 scripts/triage-baseline.py issues.json --stale-days 120 --format json > baseline.json

It reports, from JSON arrays, JSON Lines or CSV:

• open count
• age median/p90
• untriaged share
• inactive count
• median and p90 time to first human response
• created-versus-closed flow
• label usage, including single-use labels

Without a scriptable environment, sample 50 open items by hand and estimate the same numbers - an estimate beats designing blind, and without a baseline you cannot tell whether the redesign worked.

Read the numbers back before proposing anything: a queue that is 80% usage questions needs a different intervention from one that is 80% unanswered bug reports.

Then ask what the per-item framing hides: do these items cluster? On a large stale backlog, grouping issues by shared root cause under a tracking issue and redirecting the rest to it often clears more queue than triaging item by item. Decide per-item versus clustering here, before designing the per-item machinery.

Step 2 - Do the capacity arithmetic

Multiply triagers × sessions per week × minutes per session ÷ minutes per item, and compare against the baseline's inflow.

Do not import a minutes-per-item constant. Time the pilot session in step 9 and use the project's own number; until then, three minutes is a planning placeholder this skill sets, not a measured industry figure. Deep investigation is not triage - it is work triage schedules.

When capacity is below inflow, no label scheme fixes it. Three levers close the gap; pick by queue relieved per maintainer hour, not by which is quickest to type:

• efficiency: cut intake > shorten the promise > add triagers
• effort: shorten the promise (rewrite one line, defend it once) > cut intake (a day of forms and chooser configuration) > add triagers (recruit, grant permissions, calibrate - a quarter before a new triager is net positive)
• value: cut intake (permanently lowers the left side of the arithmetic) > add triagers (the only lever that raises capacity at all) > shorten the promise (nothing improves; the promise merely stops being broken)
• compliance cost: cut intake == add triagers (tied because both are internal decisions with no external counterparty) > shorten the promise where question 6 found a contractual obligation, since there the published target is a commitment and shortening it is a contract change

Efficiency starves adding triagers - the only lever that raises capacity, and the slowest to pay. Promote it whenever the project expects to still exist in a year: one that only ever cuts intake shrinks itself to fit its worst month. State the chosen lever in the deliverable so the user is not surprised in three months.

Decide the volume-gating trigger now, while nothing is on fire: the inflow rate or valid-report share at which the project will restrict who may open items, cap open submissions per author, or change the incentives that attract volume. Projects that adopted gating reached for it only after burnout was acute; deciding the trigger at design time makes it policy instead of crisis improvisation. ./references/worked-examples.md walks curl's collapse to a one-in-twenty valid share through this arithmetic, and shows why its answer acted on the economics of submitting rather than the speed of triage.