Cross Browser Testing

<objective> Chrome-only testing gives false confidence: a layout that works in Chromium can break in WebKit, a clipboard call that succeeds in Chrome silently no-ops in Firefox, and a partitioned-cookie flow can pass everywhere except the one engine your users are on. This skill produces an analytics-driven browser matrix, a Playwright (or cloud-platform) config that runs it, and a committed log of known browser divergences — each verified by a test that asserts the user outcome, not the CSS. </objective>

Quick Route

Discovery Questions

Check .agents/qa-project-context.md first — if it exists, use it and skip anything already answered there. Then:

1. Target browsers from analytics: What do actual users use? Pull browser/OS data from your analytics tool. Testing browsers nobody uses is waste; missing a browser 15% of users rely on is a bug.
2. Desktop and mobile? Mobile Safari on iOS and Chrome on Android render differently than their desktop counterparts. Treat them as separate matrix entries.
3. Cloud platform: BrowserStack, Sauce Labs, LambdaTest, or local engines only? Cloud platforms provide real branded browsers and OSes; Playwright's bundled engines cover Chromium, Firefox, and WebKit (not Chrome/Safari themselves).
4. Progressive enhancement or pixel-perfect? Progressive enhancement accepts graceful degradation. Pixel-perfect demands identical rendering. The answer determines pass/fail criteria.
5. Existing Playwright config? If the project already uses Playwright, cross-browser testing is a configuration change, not a new tool.

Core Principles

1. Analytics-driven matrix. Test what your users actually use. A browser at 0.3% traffic does not need the same investment as one at 40%. Check analytics quarterly — browser share shifts.

1. Progressive enhancement over pixel-perfect. Identical rendering across all browsers is neither achievable nor necessary. Define what "works" means: core functionality operates, content is accessible, layout is usable. Visual differences in shadows, gradients, or animation timing are acceptable.

1. Safari and Firefox surface the most cross-browser bugs. Chrome-only testing catches Chrome bugs. Safari's WebKit engine and Firefox's Gecko engine have the most behavioral differences from Chromium. Prioritize them.

1. Test functionality, not rendering-engine internals. A cross-browser test should verify that the user can complete a task, not that a CSS property renders identically. Visual comparison tools handle pixel-level differences.

1. Engines are not brands. Playwright's WebKit is not Safari and its Chromium is not Chrome — they share an engine, not the shipped product (codecs, fonts, enterprise policy, update cadence all differ). Report "WebKit coverage," not "Safari coverage," unless you ran real Safari on a cloud grid.

1. One test, multiple browsers. Write tests once. Run them across browser configurations. Never duplicate test logic for different browsers.

Browser Matrix Design

Analytics-Based Methodology

Step 1: Export browser/OS data from analytics (last 90 days)
Step 2: Rank by session share
Step 3: Group into tiers
Step 4: Assign test coverage per tier
Step 5: Review quarterly

Tier System

Example Matrix (derived from analytics)

## Browser Matrix — Q1 2026 (next-review: 2026-04-01)

| Browser | Version | Platform | Traffic % | Tier | Notes |
|---------|---------|----------|-----------|------|-------|
| Chrome | Latest | Windows | 34% | P0 | |
| Chrome | Latest | macOS | 12% | P0 | |
| Safari | Latest | macOS | 11% | P0 | WebKit-specific issues |
| Chrome | Latest | Android | 15% | P0 | Mobile viewport |
| Safari | Latest | iOS | 14% | P0 | Mobile Safari quirks |
| Firefox | Latest | Windows | 5% | P1 | Gecko rendering |
| Edge | Latest | Windows | 4% | P1 | Chromium-based but different UA/policy |
| Samsung Internet | Latest | Android | 3% | P1 | Chromium fork, lagging engine |
| Firefox | Latest | macOS | 1.5% | P2 | |
| Chrome | N-1 | Windows | 1.2% | P2 | Previous major version |

Version Coverage Strategy

• Latest: Always test current stable release.
• Latest - 1: Test previous major version only for P0 browsers where analytics show >1% on older versions.
• Extended Support Release (ESR): Test Firefox ESR only if enterprise users are a significant segment.
• Do not test: Beta/Canary/Nightly releases unless you are a browser vendor or building browser-facing tools.

Playwright Browser Configuration

Playwright ships three browser engines — Chromium, Firefox, WebKit — so no cloud platform is needed for basic engine-level coverage. This is engine coverage, not brand coverage: bundled WebKit ≠ Safari and bundled Chromium ≠ Chrome (see Core Principle 5). Define one project per matrix entry, map mobile devices via devices[...], and drive locally installed branded browsers with the channel option.

See references/playwright-and-cloud-config.md for the full playwright.config.ts project list, branded-channel snippets, and --project run commands.

When to use channels: When you need real branded behavior that differs from the bundled engine — installed Chrome (channel: 'chrome') or Edge (channel: 'msedge') for extension support, enterprise policy, or codecs. WebKit and Firefox have no channel option; they are always Playwright's bundled engines. Note the edge project in the config and the msedge channel snippet are illustrative alternatives, not two projects to merge — a config needs one edge project, not both.

`page.screencast()` (Playwright 1.59+, current in 1.60) captures annotated video of a cross-browser run — useful when a matrix failure needs human review across engines. For agent-driven re-runs and stepping through a failure, use --ui (UI mode) or --debug (Inspector); PWDEBUG=1 and --headed are the other real entry points. There is no --debug=cli flag.

Cloud Platform Setup

Cloud platforms (BrowserStack, Sauce Labs) provide real branded-browser/OS instances Playwright connects to over a CDP/Playwright WebSocket endpoint. Pass credentials and capabilities via environment variables, and keep the platform's playwrightVersion aligned with the Playwright version in package.json (currently 1.60.x — a client/server mismatch causes socket errors).

BrowserStack now recommends the npx browserstack-node-sdk runner plus a client.playwrightVersion capability (in addition to browserstack.playwrightVersion) to keep the client and grid sockets in lock-step. The raw wsEndpoint/CDP config below still works for direct connections; use the SDK path for new setups.

See references/playwright-and-cloud-config.md for the BrowserStack config (with the client.playwrightVersion cap), the Sauce Labs config, and the GitHub Actions parallel matrix that fans out across cloud browsers.

Common Cross-Browser Issues