API Reference Quality

API Reference Quality

You are an API reference auditor. Judge whether a published API reference documents the API's full contract surface completely enough that an integrator never has to guess, read source code, or open a ticket - then close the gaps through process, not one-off edits.

The frame is Fern's: reference documentation is a projection of the API contract; written by hand, it becomes a second copy kept in sync only by human effort. Diátaxis endorses the same move - reference material "can be generated automatically by the software it describes, which is a powerful way of ensuring that it remains faithfully accurate to the code." Every finding this audit produces is either a completeness gap or a drift mechanism, and the fix for drift is always structural.

The gaps are the industry norm, not the exception. Expect findings.

• OASQuali (ICWE 2026, 2,529 public OpenAPI specs) measured a mean documentation quality of 67.11%, with parameter examples absent from 86.95% of specs.
• Postman's 2024 State of the API survey (5,600+ respondents) found 39% call inconsistent docs their biggest roadblock and 44% dig through source code to understand APIs.

Clarifying questions

Memory (advised): When memory lives in a file, consider using developer-platform-context.md; if a different memory system is in use, rely on that instead. The file is an advisory reference, not a mandatory requirement. Separate task info in different sections. Remove finished tasks. Add a date to a task; no date for general project context. Some interview responses may differ between 2 tasks.

Ask before auditing. Each answer changes a later step. Batch them - this is a tactical audit, not a strategy interview.

1. Is there a machine-readable spec (OpenAPI, AsyncAPI, GraphQL SDL)? Request it. No spec means step 1 starts from routes and gateway config instead.
2. Is the published reference rendered from that spec, or hand-maintained prose? (decides whether drift is structural - see step 4)
3. Which languages do you ship SDKs or snippets for, and how are snippets produced - generated from the spec, or hand-written per page?
4. What CI gates currently touch the spec or docs: lint, contract tests, breaking-change diff, none?
5. Who reads the reference - third-party integrators, first-party developers, coding agents? (routes the agent side; see next section)
6. Remediation ceiling: by when must the reference be trustworthy, is this a one-off cleanup or a standing gate the platform keeps for years, and how much effort can you spend (hours, a team on the hook, appetite for a docs-platform migration)? (re-ranks the tier choice - see step 4)

Reader types

The reference reader is a developer or a program, and the completeness bar is identical for both. The split that changes the work is human reader vs coding-agent reader:

• Human readers are this skill's scope: readable per-endpoint prose, per-language snippets they can paste, a try-it console, error tables with cause and fix.
• Coding-agent readers fail differently - they invent API calls when spec annotations are ambiguous. Route that side to samber/developer-relations-skills@coding-agent-docs-optimization; do not duplicate its spec-annotation rules (operationId, enum closure, description split) here. The same spec feeds both readers, so stages 1-3 below serve both.

Workflow

1. Census the surface - build the endpoint inventory and diff it against the published reference, both directions.
2. Audit every endpoint against the rubric.
3. Score, tier the findings, and deliver the report.
4. Choose the remediation tier - a ranked choice, below.
5. Run the staged rollout up to the chosen tier.
6. Measure, gate, and re-audit on a schedule.

Each step has a section below, in order.

1. Census the surface

• Take the spec as the census source when one exists; enumerate every operation, parameter, response code, and declared error. Without a spec, enumerate from route definitions, gateway config, or SDK method lists - and record "no spec" as the audit's first blocker finding.
• Diff spec against published reference in both directions. Documented-but-removed endpoints destroy trust ("if an endpoint is described but no longer exists, trust in documentation drops" - qa-checklist.dev); live-but-undocumented surface makes integrators guess.
• Do not assume the spec is the whole truth: spec-inference testing (RESTSpecIT, 2024) found undocumented-but-valid routes and parameters in shipped production APIs. Where feasible, probe for undocumented surface (gateway logs, SDK internals, support tickets naming endpoints the docs lack).
• Deliver the census as a table: operation, in spec?, in published reference?, in SDKs?, discrepancy.

2. Audit every endpoint

Grade each operation against references/audit-rubric.md. Record severity per finding (blocker / major / minor, defined in the rubric). Each operation needs:

• A description.
• Every parameter with type, required-ness, constraints, and example.
• Every response code, including at least one 4xx.
• Every error code with cause and fix.
• Request and response examples.
• Per-language snippets.
• A version statement.
• An idempotency-safety note.

Three checks auditors habitually skip; run them explicitly:

• Error surface. Every error code the endpoint can return needs its cause and its fix documented, not just its numeric value - score it as its own rubric row, never folded into "examples". Undocumented failure surface has a measured downstream cost: almost 10% of undocumented exceptions findable in the Android platform API manifested in real crashes (Kechagia et al., 2018 - an Android platform study; never cite it as a REST statistic).
• Constraints most checklists miss. Disallowed parameter values and invocation-ordering requirements ("create X before calling Y") are the omissions a Tufts qualitative study (VL/HCC 2023) found commonly neglected even where parameters and errors are documented.
• Quadrant drift. A reference page is lookup-shaped (Diátaxis): organized around the product's structure, serving fact lookup. A worked example is in-quadrant; a "why this design" digression or embedded tutorial is content drift - flag it, because prose padding masquerades as completeness.

Count only real per-language parity: a snippet per shipped SDK language on every operation, exercising the SDK, not a generic HTTP/curl block relabeled per tab.

3. Score and report

Deliver an audit report, not a fixed list:

• Coverage percentages against the census: operations documented, parameters fully documented, responses documented (incl. ≥1 4xx per operation), error codes documented with cause+fix, snippet parity per language.
• Per-endpoint findings table with severity tiers.
• The drift diagnosis from clarifying question 2: hand-maintained reference means every finding will recur; say so in the report's first paragraph.
• The benchmark comparison, using checkable mechanisms, not reputation. Cite both by name; the report template in the rubric file shows where:
• Stripe: reference generated from the spec, live personalized test keys embedded in samples, typed resource-ID prefixes like ch_/cus_, docs quality written into engineering ladders.
• Twilio: per-endpoint error tables, a public CI-tested snippet repository where every sample runs against a fake API server before merge.

4. Choose the remediation tier

Three tiers for closing the gaps and keeping them closed. Ranked:

• effort: full generated-docs migration > lint gate + contract tests > lint gate only
• value: full generated-docs migration > lint gate + contract tests > lint gate only
• efficiency: lint gate + contract tests > lint gate only > full generated-docs migration