Visual Testing

<objective> Catch visual regressions that functional tests miss. A button that works perfectly but renders at 2px height passes toBeVisible() and click() — visual testing compares screenshots against approved baselines and flags pixel-level differences. This skill covers Playwright's built-in toHaveScreenshot, dedicated tools (Chromatic, Percy, Argos CI), and the baseline-management workflows around them. </objective>

Quick Route

Discovery Questions

Check .agents/qa-project-context.md first — if it exists, use it and skip anything answered there.

Tool selection

• Playwright built-in or dedicated tool? toHaveScreenshot is free, stores baselines in-repo (git/LFS cost is on you). Dedicated tools (Chromatic, Percy, Argos) store artifacts off-repo, add review workflows, browser farms, historical tracking, and — since late 2025 — AI diff triage. Choose on team size, review needs, and whether you want artifacts in your repo or theirs.
• Storybook in the project? If yes, Chromatic is the natural fit (every story becomes a visual test) — but you can also drive stories with Playwright without paying (see recipes.md). If no Storybook, Playwright built-in or Percy.
• CI platform and artifact budget? Visual tests generate large screenshot/diff artifacts. Built-in baselines live in git (repo bloat, LFS); hosted tools keep retention off-repo at a subscription cost. Confirm CI has the storage and time before scaling up.

Scope

• Full-page or component screenshots? Full-page catches layout issues but is sensitive to unrelated changes. Component-level screenshots are more stable and focused.
• Which pages/components are visually critical? Not everything needs it. Focus on user-facing pages, marketing pages, design-system components, and complex layouts.
• Which viewports? Desktop, tablet, mobile — define the viewport matrix upfront from analytics, not every possible width.

Dynamic content

• What changes between runs? Dates, timestamps, user-generated content, analytics IDs, randomized content, ads, avatars — all must be masked or frozen.
• Animations or transitions? They cause false positives if not disabled or finished before capture.
• External resources? Fonts, CDN images, third-party widgets can vary between runs.

Core Principles

1. Visual tests catch what functional tests miss

Functional tests assert behavior ("clicking Submit shows a success message"). Visual tests assert appearance ("the success message is green, correctly positioned, and does not overlap the form"). Both are needed. Visual tests complement functional tests; they do not replace them.

2. Baseline management is the hard part

Taking screenshots is easy. Managing baselines — updating them when design changes intentionally, reviewing diffs, coordinating approvals across a team — is the real challenge. Invest in the review workflow early.

3. Dynamic content causes false positives

Any content that changes between runs (timestamps, avatars, ads, random IDs) produces pixel differences that are not real regressions. Aggressively mask or freeze it. A suite with a 10% false-positive rate gets ignored within a month. Hosted tools (Percy's Visual Review Agent, Chromatic AI triage) now auto-filter a large share of these — but masking and freezing remain your first line of defense regardless of tool.

4. Threshold tuning is iterative

The right diff threshold depends on the component, rendering engine, and what counts as "visually different." Start strict (zero tolerance), observe false positives, loosen per-component. Document why each threshold was chosen.

5. Screenshots are artifacts, not test results

The screenshot file is the evidence. Store it, version it, make it accessible for review. A test that says "visual diff detected" without showing the diff is useless — always upload expected/actual/diff images as CI artifacts.

Playwright Visual Comparisons

Playwright's built-in toHaveScreenshot and toMatchSnapshot provide visual regression testing without an external service.

Basic screenshot comparison

import { test, expect } from '@playwright/test';

test('dashboard matches baseline', async ({ page }) => {
  await page.goto('/dashboard');
  // Wait for data to load before capturing — never waitForTimeout
  await expect(page.getByRole('heading', { name: 'Dashboard' })).toBeVisible();
  await expect(page.getByTestId('chart-container')).toBeVisible();

  await expect(page).toHaveScreenshot('dashboard.png', { animations: 'disabled' });
});

First run creates the baseline. Subsequent runs compare and fail if pixels differ beyond the threshold.

Configuration options

await expect(page).toHaveScreenshot('dashboard.png', {
  maxDiffPixels: 100,          // Allow up to 100 pixels to differ
  // OR
  maxDiffPixelRatio: 0.01,     // Allow up to 1% of pixels to differ
  threshold: 0.2,              // Per-pixel color difference tolerance (0-1, YIQ space)
  animations: 'disabled',      // Freeze CSS animations and transitions
  caret: 'hide',               // Hide blinking cursor
  stylePath: './screenshot.css', // Inject a stylesheet at capture to hide dynamic chrome
  timeout: 15000,              // Wait up to 15s for a stable screenshot
});

The default comparator is pixelmatch (YIQ color space). The relevant knobs are comparator: 'pixelmatch' plus threshold/maxDiffPixel* — there is no stable perceptual-color mode key, so do not set mode: (perceptual SSIM is an internal/experimental underscore API, not stable config).

When to use which threshold:

playwright.config.ts visual settings

import { defineConfig } from '@playwright/test';

export default defineConfig({
  expect: {
    toHaveScreenshot: {
      comparator: 'pixelmatch',    // Default comparator (YIQ color space)
      maxDiffPixelRatio: 0.005,    // Global default: 0.5% tolerance
      animations: 'disabled',
      caret: 'hide',
    },
    toMatchSnapshot: {
      maxDiffPixelRatio: 0.005,
    },
  },
  projects: [
    {
      name: 'visual-desktop',
      use: { viewport: { width: 1280, height: 720 }, colorScheme: 'light' },
      testMatch: /.*visual.*\.spec\.ts/,
    },
    {
      name: 'visual-mobile',
      use: { viewport: { width: 375, height: 667 }, colorScheme: 'light', isMobile: true },
      testMatch: /.*visual.*\.spec\.ts/,
    },
  ],
});

Masking & freezing dynamic content

Two complementary tactics. Mask blanks specific elements; stylePath / freezing kills time- and animation-driven noise.

test('profile page visual test', async ({ page }) => {
  await page.goto('/profile');
  await expect(page.getByRole('heading', { name: 'Profile' })).toBeVisible();

  await expect(page).toHaveScreenshot('profile.png', {
    mask: [
      page.getByTestId('user-avatar'),       // User-specific image
      page.getByTestId('last-login-time'),    // Timestamp
      page.getByTestId('activity-feed'),      // Dynamic content
    ],