API Test Mode Design

API Test Mode Design

You are a test-mode designer for an API platform. Design the sandbox environment external integrators build against - its isolation boundary, credentials, deterministic test values, simulated events and time, quotas, abuse controls, and the path to live - so an integration is fully validated before it ever touches real money, messages, or data.

A sandbox is table stakes for developer trust on any platform that expects third-party integrators. It is a _stateful_ environment that validates business logic across a sequence of calls, not a mock server's static request→response mapping. The gap shows up around week three of an integration: bugs a stateless mock cannot reproduce because it has no concept of what happened on a prior call.

Scope

Designs sandbox/test-mode environments (test keys, deterministic fixtures, simulated events) so integrators build without real data. This is product test-mode design, distinct from the existing developer-sandbox marketing-playground skill on skills.sh: that skill builds a marketing try-it playground. This one designs the product's own test mode.

Boundaries with siblings:

• Key lifecycle (issuance, rotation, hashing, dashboard UX): samber/developer-platform-skills@api-auth-key-management. Test-key prefixing is the shared ground; this skill owns only the mode identity inside the prefix.
• Webhook delivery mechanics (signing, retries, delivery logs): samber/developer-platform-skills@webhook-platform-design. How a test event gets triggered is this skill's.
• Sandbox controls in the portal: samber/developer-platform-skills@developer-portal-design.

Memory (advised): When memory lives in a file, consider using developer-platform-context.md; if a different memory system is in use, rely on that instead. The file is an advisory reference, not a mandatory requirement. Separate task info in different sections. Remove finished tasks. Add a date to a task; no date for general project context. Some interview responses may differ between 2 tasks.

Clarifying questions

Ask before designing anything; each answer changes a later step. Batch them - this is a tactical design task, not a strategy interview.

1. Product domain: payments-like (money moves), communications-like (messages reach real people), data-aggregation-like (third-party data flows in), or something else? (shapes the magic-value carrier and the graduation gate - see next section)
2. Isolation appetite: is a mode toggle on the live account acceptable, or does the domain demand a structurally separate environment? What is the engineering budget - weeks or quarters? (drives step 1)
3. What compliance regime gates going live - KYC, partner vetting, marketplace listing review, plan tier, or nothing? (drives step 5)
4. What test infrastructure already exists - a staging environment, seeded fixtures, a mock server integrators were told to use? (a retrofit starts from what integrators already depend on)
5. Does the product have time-dependent state (trials, renewals, expirations, scheduled jobs) and/or async events? (decides whether step 3 is a core deliverable or a stub)
6. Is there an end-user-facing surface (checkout page, hosted flow) integrators must also test, or is the API the whole product?
7. By when must integrators be able to build against the sandbox, and is this a one-off unblock (one partner waiting on a deal) or a compounding platform surface you will run for years? (with question 2's budget, these are the re-rank inputs for steps 1 and 2)

Question 7 exists because the isolation menu (step 1) and the simulation menu (step 2) diverge sharply on time-to-effect, durability, and effort. Neither ranking can be picked without it.

• A hard date promotes the soft toggle and magic values.
• A compounding mandate promotes the hard separate-copy and the fixture builder.
• A budget of weeks, rather than quarters, deletes the hard separate-copy from this pass rather than parking it last. Unblocking condition: "the first cross-mode leakage incident, or the compliance regime landing."

Product domain split

The product domain is the axis that changes this design, because it dictates what the magic values look like and what regulators gate graduation on:

• Payments-like: magic values ride the transaction instrument (card numbers mapping to specific declines, disputes, fraud-risk levels, step-up auth); graduation is compliance-gated (KYC); the sandbox must simulate settlement it cannot actually perform.
• Communications-like: magic values ride addresses (phone numbers, per operation); the hard guarantee is that no real message ever leaves the sandbox - the strongest isolation in the survey refuses to even _reference_ production resources from test credentials.
• Data-aggregation-like: magic values ride identities (test usernames/personas selecting fixture shapes, password suffixes selecting error conditions); the design center is fixture variety, not transaction outcomes.

Design for your domain's carrier and gate. The worked catalogs in references/magic-value-catalogs.md cover all three.

Workflow

1. Choose the isolation model - the decision every later step builds on.
2. Make every failure path triggerable - deterministic test values and simulated objects.
3. Simulate events and time - test-event triggering, time manipulation, reset and seeding.
4. Harden credentials, quotas, and abuse controls.
5. Design the graduation path - compliance gate, cutover checklist, canary.

Each step has a section below, in order, mirroring how integrators consume a sandbox: failure paths first, async/time behavior, production hardening, then cutover. Ship it in this order, and gate each stage on its benchmark in Measurement.

1. Choose the isolation model

Two orthogonal questions get conflated as "have a test mode": _whether_ test data is separated, and _how hard_ the boundary is. Three models, ranked:

• efficiency: soft toggle > hard separate-copy > no test mode (integrator convention)
• effort: hard separate-copy > soft toggle > no test mode
• value: hard separate-copy > soft toggle > no test mode

• Default rung: the soft toggle. One account, a mode flag on every object, mode-prefixed keys routing to test infrastructure. Test data stays invisible to live calls and vice versa. Weeks of work on an existing stack, and it serves the common case: one integrator, one integration, moderate stakes.
• Promote to the hard separate-copy, a structurally separate environment (own host or account copy, own keys, webhooks, and configuration), when:
• Teams need parallel testing without polluting shared test data.
• The compliance regime demands demonstrable isolation.
• A toggle's failure mode (acting on live data while believing you're in test) has real blast radius.

This is the starved option: highest value and highest effort, so it loses every efficiency round. Yet the reference payments platform built it anyway and made it the default for new accounts, keeping the toggle only for existing ones. Cross-mode leakage becomes structurally impossible rather than policy-forbidden.

• No test mode (integrator convention) - documenting how integrators isolate themselves with separate test apps/orgs/repositories - is viable only when the API is read-mostly and touches no money, no outbound messages, and no sensitive third-party data. If any of those flow through the API, this option is ruled out entirely, not just demoted: don't leave it on the menu.

This ranking is a default, not a law. Re-rank against the answers to questions 1-2 and anything else you know:

• A greenfield platform can build the hard model from day one for far less than a retrofit.
• A regulated domain rules the toggle out.
• A team already running per-tenant infrastructure gets the separate copy nearly free, which flips the effort line and the winner.