Design Testing Strategy
Use before writing any type of tests. Distills 14 industry sources into deterministic decision gates, schemas, and worked test examples.
- Skill ID
- neolabhq/context-engineering-kit/design-testing-strategy
- Publisher
- neolabhq
- Repository
- context-engineering-kit
- Installs
- 0
- Files
- 1
- Synced
- Sep 23, 2026
Open any RiverX project, open the Skills panel in the chat, and search for this identifier. The files are fetched from the source repository at install time.
neolabhq/context-engineering-kit/design-testing-strategyInstalls these files- SKILL.md
What this skill tells the agent
Design Testing Strategy
A reference manual for designing a fit-for-purpose, fit-for-criticality testing strategy.
This skill is decision-oriented, not philosophical: every gate is deterministic (ON when X / OFF when Y), every schema is enforced (field ordering matters), every example is worked end-to-end.
How To Use This Skill
- Read Decision Gates in order (Gate 0 -> Gate 6). Each gate is independent — you may finish with any subset of test types ON.
- Apply Strategic Skip Heuristics to remove ON gates that would yield low ROI for this artifact.
- For each ON gate, fill the Test Matrix Schema (
selected_typesentry) — the field order is load-bearing. - List rejected types in
rejected_typesand deliberate skips indeliberately_skipped. - Produce a Test Cases to Cover markdown bullet list using ISTQB techniques from Case Design Techniques.
- Cross-check against the matching Worked Example (A pure function / B HTTP+DB endpoint / C UI component).
Decision Gates
Apply gates in numeric order. Each gate produces an independent boolean (applies: true|false). Gates do NOT veto each other — a single artifact may have unit + integration + contract + property-based all ON.
| # | Type | ON when | OFF when | Source |
|---|---|---|---|---|
| 0 | Skip All | Criticality is NONE (docs-only, comments, formatting, generated code, config without logic, throwaway prototypes) | Anything with branching, computed output, side effects, or user-visible behavior | Pragmatic Programmer — "Test ruthlessly and effectively" implies effective skipping when ROI is zero |
| 1 | Unit | Code contains any logic: branches, loops, conditionals, computation, transformation, parsing, validation, formatting | Pure declarative wiring (DI registration, route table) with no behavior | Test Pyramid (Vocke) base layer + Beck TDD Red-Green-Refactor unit |
| 2 | Integration | Boundary crossing: HTTP call, DB query, external SDK, message queue, filesystem I/O, OR collaboration with >=2 distinct collaborators where unit doubles distort behavior | Pure function with no I/O and 0-1 stable collaborators | Testing Trophy (Dodds) — integration is the highest-ROI layer; Google "Follow the User" |
| 3 | Component or E2E | UI surface AND criticality >= MEDIUM-HIGH AND user-facing critical path (signup, checkout, auth, payment, primary CTA) | Internal admin-only screens, dev tooling, or non-critical UI | Test Pyramid top + ISO/IEC/IEEE 29119 risk ranking + Google e2e principles |
| 4 | Contract | Public API consumed by >=1 distinct clients (mobile + web, multiple internal services, external partners) AND independent deploy cadence | API where consumer and provider deploy together | Pact / CDC + Pactflow CDC explainer |
| 5 | Smoke | Deployable surface (web app, API, service) AND a deploy/CI pipeline exists where post-deploy validation is meaningful | Library, internal helper, or no deploy pipeline | Google "What Makes a Good End-to-End Test" — smoke = minimal e2e for deploy gate |
| 6 | Property-Based | Input domain is large or unbounded (numeric ranges, strings, lists, parsers, serializers, encoders, math) AND invariants are stable (round-trip, idempotency, monotonicity, commutativity) AND criticality >= MEDIUM-HIGH | Small finite input domain, unstable invariants, or LOW criticality | Hypothesis / QuickCheck |
Gate Application Algorithm
for gate in [Gate 0, Gate 1, ..., Gate 6]:
if gate.ON_condition_met(artifact):
result[gate.type] = applies: true
else:
result[gate.type] = applies: false
if Gate 0 is true:
short-circuit: emit empty selected_types, document criticality=NONE, stopCriticality Scale (used by Gates 3 and 6):
| Level | Definition |
|---|---|
NONE | Docs, formatting, generated code, throwaway code, configs without logic |
LOW | Internal dev tooling, admin-only screens, logging formatters |
MEDIUM | Standard CRUD, internal APIs with a single team consumer, non-critical UI, helpers and utilities |
MEDIUM-HIGH | User-facing UI on critical paths, public APIs with multiple consumers, business workflows |
HIGH | Money movement, auth/authz decisions, security-critical validation, data integrity, regulated domains |
Test Type Reference
| Type | Use when | Do NOT use when | Frameworks | Typical dependencies | Google Size |
|---|---|---|---|---|---|
| unit | Pure logic, single function/method/class, deterministic inputs | Code is just I/O orchestration with no logic | vitest, jest, pytest, go test, JUnit, xUnit, RSpec | None (or in-memory fakes) | Small |
| integration | Boundary crossing (DB, HTTP, queue, FS); multiple collaborators where mocking distorts behavior | Pure function with no boundary | vitest, jest, pytest, go test, JUnit + Testcontainers, supertest, TestRestTemplate | Real Postgres/Redis/Kafka via Testcontainers, in-process HTTP server, real FS in tmpdir | Medium (single machine, localhost OK) |
| component | UI rendering + interaction within a single component, no full app context | Backend-only logic; multi-page user flow | React Testing Library, Vue Test Utils, Angular TestBed, Storybook interaction tests | jsdom or happy-dom, mocked network at fetch/axios level | Small to Medium |
| e2e | Full user path through running app: real browser, real backend, real DB | Internal helper, single component, non-critical UI | Playwright, Cypress, Selenium | Real running app + Testcontainers-backed DB or seeded staging | Large (multi-process, possibly multi-machine) |
| smoke | Post-deploy go/no-go: hit / health, key endpoints respond, login works | Detailed correctness; smoke is shallow by design | Playwright (1-3 critical paths), HTTP probe scripts, k6 minimal scenarios | Real deployed environment | Large |
| contract | Public API consumed by 2+ distinct clients with independent deploy cadence | Single-consumer internal API; provider and consumer deploy together | Pact, Spring Cloud Contract, OpenAPI schema validators | Pact broker or contract files in repo | Medium |
| property-based | Large/unbounded input domain with stable invariants (parser, serializer, encoder, math) | Small finite input space; unstable invariants | Hypothesis (Python), fast-check (TS), QuickCheck (Haskell), jqwik (Java), proptest (Rust) | Same as unit | Small |
Google Test Size Mapping
Google Test Sizes (Bland) and SWE at Google Ch.11 classify tests by resources (size), independent of scope (paths covered):
| Size | Process model | Network | Filesystem | Time budget | Notes |
More from neolabhq
Use this skill in a project
Open any RiverX project, search the Skills panel for neolabhq/context-engineering-kit/design-testing-strategy, and install it in one click.
