AI System Testing

<objective> AI features fail differently from deterministic software. The same input produces different outputs, correctness is subjective, and failure modes include hallucination, prompt injection, and silent quality decay. A chatbot that confidently cites a fabricated URL passes every toBeDefined() check. This skill covers how to test AI features rigorously despite nondeterminism: versioned prompts, eval suites scored against golden datasets, statistical and property-based assertions, tool-call validation, grounding checks, and red-team safety scans. </objective>

Quick Route

Discovery Questions

Check .agents/qa-project-context.md first. If it exists, use it as context and skip questions already answered there.

AI features under test:

• What AI features exist? (Chat, summarization, classification, code gen, recommendations, search) — determinism expectations differ per type.
• Which provider/model? (Anthropic, OpenAI, Google, open-source) — drives the eval harness and red-team backend.
• Are prompts hardcoded, template-based, or dynamically constructed? — only versioned prompts are regression-testable.
• Is RAG involved, and what is the knowledge source? — RAG needs grounding tests, not just output checks.

Determinism requirements:

• Which outputs must be deterministic (classification, extraction) vs. creative (chat)? — decides exact-match vs. property vs. statistical assertions.
• What temperature runs in production? — test there, not at 0 "to make tests pass."
• Can the output be constrained to a JSON schema? — schema-constrained extraction is testable with a plain validator, no LLM judge.

Quality requirements:

• How is quality defined? (Accuracy, relevance, completeness, safety, tone) — these become eval metrics with weights and thresholds.
• Is there a golden dataset of inputs and acceptable outputs? — the anchor for every regression check.
• Who evaluates quality today? (Humans, metrics, nobody) — if an LLM judges, it must be calibrated against humans.

Safety requirements:

• Does the AI process user-generated input? — prompt-injection surface.
• Are there content-policy or PII constraints, or a regulated domain? — drives the red-team probe set.

Core Principles

1. Nondeterminism is inherent, not a bug. LLMs are stochastic; the same prompt yields different outputs across runs. Assert on properties and boundaries, never exact strings. expect(output).toBe("The answer is 42") breaks the next run when the model says "42 is the answer."

1. Test properties, not exact outputs. A good assertion asks: does the response contain the required information, stay in the length range, exclude prohibited content, match the expected format? When you can constrain output to a JSON schema, do that and validate with a plain schema validator — it converts a statistical check into a deterministic one.

1. Evals are the test suite for AI. An eval defines inputs, runs them through the system, and scores outputs against quality criteria. Invest in evals the way you invest in test infrastructure: version them, run them in CI, gate merges on them.

1. Safety testing is non-negotiable. AI can produce harmful content, leak the system prompt, echo PII, or be steered by adversarial input. Safety tests are the security tests of AI features — run them on every prompt change, and red-team before launch.

1. A judge you have not calibrated proves nothing. LLM-as-judge scales evaluation, but a judge that disagrees with humans just launders a wrong answer. Measure judge-vs-human agreement on a labeled held-out set and set a minimum bar before trusting it (see Response Quality Evaluation).

Tooling

Pick the layer that fits the job. Don't reach for hand-rolled TS unless none of these match.

Public benchmarks (HELM, LMSYS Chatbot Arena, Inspect AI's catalog) are for model selection, not app regression — they don't know your domain. Use them when picking a base model; use the tools above for everything after.

For runnable entry points — DeepEval tool-call validation, the Promptfoo cross-provider YAML suite, and the Garak red-team command — see references/tooling-evals.md.

Prompt Regression Testing

Version prompts like code

Prompts drive your application's behavior; version, review, and test them like code. A versioned prompt object carries its version, template, typed parameters, and a changelog. See references/prompt-regression.md for the SUMMARIZE_PROMPT object.

Baseline response quality

Establish a quality baseline per prompt and detect regressions when the prompt, model, or parameters change. Each eval case pairs an input with criteria (maxLength, mustContain, mustNotContain, sentenceCount, formatCheck); the test asserts every applicable criterion. See references/prompt-regression.md for the baseline eval suite.

A/B test prompts