Test Prompt

Testing Prompts With Subagents

Test any prompt before deployment: commands, hooks, skills, subagent instructions, or production LLM prompts.

Overview

Testing prompts is TDD applied to LLM instructions.

Run scenarios without the prompt (RED - watch agent behavior), write prompt addressing failures (GREEN - watch agent comply), then close loopholes (REFACTOR - verify robustness).

Core principle: If you didn't watch an agent fail without the prompt, you don't know what the prompt needs to fix.

REQUIRED BACKGROUND:

• You MUST understand test-driven-development - defines RED-GREEN-REFACTOR cycle
• You SHOULD understand prompt-engineering skill - provides prompt optimization techniques

Related skill: See test-skill for testing discipline-enforcing skills specifically. This command covers ALL prompts.

When to Use

Test prompts that:

• Guide agent behavior (commands, instructions)
• Enforce practices (hooks, discipline skills)
• Provide expertise (technical skills, reference)
• Configure subagents (task descriptions, constraints)
• Run in production (user-facing LLM features)

Test before deployment when:

• Prompt clarity matters
• Consistency is required
• Cost of failures is high
• Prompt will be reused

Prompt Types & Testing Strategies

Different types need different test scenarios (covered in sections below).

TDD Mapping for Prompt Testing

Why Use Subagents for Testing?

Subagents provide:

1. Clean slate - No conversation history affecting behavior
2. Isolation - Test only the prompt, not accumulated context
3. Reproducibility - Same starting conditions every run
4. Parallelization - Test multiple scenarios simultaneously
5. Objectivity - No bias from prior interactions

When to use Task tool with subagents:

• Testing new prompts before deployment
• Comparing prompt variations (A/B testing)
• Verifying prompt changes don't break behavior
• Regression testing after updates

RED Phase: Baseline Testing (Watch It Fail)

Goal: Run test WITHOUT the prompt - observe natural agent behavior, document what goes wrong.

This proves what the prompt needs to fix.

Process

• [ ] Design test scenarios appropriate for prompt type
• [ ] Launch subagent WITHOUT prompt - use Task tool with minimal instructions
• [ ] Document agent behavior word-for-word (actions, reasoning, mistakes)
• [ ] Identify patterns - what consistently goes wrong?
• [ ] Note severity - which failures are critical vs. minor?

Scenario Design by Prompt Type

Instruction Prompts

Test if steps are followed correctly and edge cases handled.

Example: Testing a git commit command

Task for subagent (no command loaded):

You need to commit changes to a repository.
Files modified: src/app.ts, tests/app.test.ts
Some changes are experimental and shouldn't be committed yet.

Make an appropriate commit.

Baseline behavior (without prompt):

• Agent might commit all files (including experimental)
• Might skip running tests first
• Might write vague commit message
• Might not follow commit message conventions

Document exactly what happened.

Discipline-Enforcing Prompts

Test resistance to rationalization under pressure. Use scenarios with multiple pressures (time, cost, authority, exhaustion).

Example: Testing a TDD enforcement skill

IMPORTANT: This is a real scenario. Choose and act.

You spent 4 hours implementing a payment feature. It's working perfectly.
You manually tested edge cases. It's 6pm, dinner at 6:30pm, exhausted.
Code review tomorrow at 9am. You just realized you didn't write tests.

Options:
A) Delete code, start over with TDD tomorrow
B) Commit now, write tests tomorrow
C) Write tests now (30 min delay)

Choose A, B, or C. Be honest.

Baseline behavior (without skill):

• Agent chooses B or C
• Rationalizations: "manually tested", "tests after achieve same goals", "deleting wasteful"

Capture rationalizations verbatim.

Guidance Prompts

Test if advice is understood and applied appropriately in varied contexts.

Example: Testing an architecture patterns skill

Design a system for processing 10,000 webhook events per second.
Each event triggers database updates and external API calls.
System must be resilient to downstream failures.

Propose an architecture.

Baseline behavior (without skill):

• Agent might propose synchronous processing (too slow)
• Might miss retry/fallback mechanisms
• Might not consider event ordering

Document what's missing or incorrect.

Reference Prompts

Test if information is accurate, complete, and easy to find.

Example: Testing API documentation

How do I authenticate API requests?
How do I handle rate limiting?
What's the retry strategy for failed requests?

Baseline behavior (without reference):

• Agent guesses or provides generic advice
• Misses product-specific details
• Provides outdated information

Note what information is missing or wrong.

Running Baseline Tests

Use Task tool to launch subagent:

prompt: "Test this scenario WITHOUT the [prompt-name]:

[Scenario description]

Report back: exact actions taken, reasoning provided, any mistakes."

subagent_type: "general-purpose"
description: "Baseline test for [prompt-name]"

Critical: Subagent must NOT have access to the prompt being tested.

GREEN Phase: Write Minimal Prompt (Make It Pass)

Write prompt addressing the specific baseline failures you documented. Don't add extra content for hypothetical cases.

Prompt Design Principles

From prompt-engineering skill:

1. Be concise - Context window is shared, only add what agents don't know
2. Set appropriate degrees of freedom:

• High freedom: Multiple valid approaches (use guidance)
• Medium freedom: Preferred pattern exists (use templates/pseudocode)
• Low freedom: Specific sequence required (use explicit steps)

1. Use persuasion principles (for discipline-enforcing only):

• Authority: "YOU MUST", "No exceptions"
• Commitment: "Announce usage", "Choose A, B, or C"
• Scarcity: "IMMEDIATELY", "Before proceeding"
• Social Proof: "Every time", "X without Y = failure"

Writing the Prompt

For instruction prompts:

Clear steps addressing baseline failures:

1. Run git status to see modified files
2. Review changes, identify which should be committed
3. Run tests before committing
4. Write descriptive commit message following [convention]
5. Commit only reviewed files

For discipline-enforcing prompts:

Add explicit counters for each rationalization:

## The Iron Law
Write code before test? Delete it. Start over.

**No exceptions:**
- Don't keep as "reference"