Flows Design Review

Flows Design Review

This is step 3 of the Flows app certification flow:

flows-app-brief  →  build  →  flows-code-review  →  flows-design-review (this skill)  →  flows-external-app-submit

This is the manual design quality assessment described in docs.cognite.com/cdf/flows/guides/quality-guidelines. Target overall average: 3.8 or higher to be launch-ready.

Operating rules

• Automate first, ask second. For every question Q1–Q10, run the probes listed below to gather hard evidence from the repo and propose a draft score (1–5) with rationale before asking the user. The user's job is to confirm or override the proposed score, not to grade from scratch. This dramatically reduces the manual burden.
• The task walkthrough (Step 2) is the one part that cannot be skipped — automation cannot tell whether a user "gets lost" navigating a screen. Capture it manually and use it to override the auto-derived scores where lived experience disagrees.
• Use AskQuestion for every score so answers are structured. For each question present three options: (a) accept the draft score, (b) override with a specific score, (c) override + add a note.
• Pre-fill user, tasks, and persona context from App-Brief.md frontmatter when present.

Step 0 — Pre-scan before prompting

Always pre-scan before asking the user anything. Read these sources silently and surface what you found as evidence — never as scores, never auto-saved:

Show the user the pre-scan results in your opening message before any scoring. They are starting points, not verdicts. The manual task walkthrough (Step 2) and user-assigned scores remain authoritative.

Step 0b — Choose feedback round

Look at reviews/design-review/. If it doesn't exist, this is round 1. Otherwise increment to the next missing feedback-round-<N>/ directory.

Step 1 — Confirm user and tasks

Per the docs, "the quality assessment is only as useful as the clarity of the user and tasks it's based on."

If App-Brief.md exists, parse userRole, oneSentenceStory, and successCriteria from its frontmatter and propose them as the primary user and tasks. Ask the user to confirm or extend.

Capture, via AskQuestion:

• Primary user — specific role and context (e.g. "Maintenance engineers on offshore platforms").
• 2–3 critical tasks — the workflows this user needs to complete (e.g. "Check pump vibration alerts", "Schedule maintenance work").
• Context — experience level, time constraints, device, success criteria.

Step 2 — Walk each task end-to-end (manual)

Instruct the user to:

1. Open the app as that user in a clean browser session with representative test data.
2. Complete each task from beginning to end without shortcuts.
3. Note pain points: where they get stuck, confused, or make errors.

For each task, prompt the user to paste back: what happened, where they got stuck, and any screenshots / notes. Capture these as taskWalkthroughs[] for the report.

Do NOT proceed to scoring until the user confirms they walked every task. If they refuse, write a stub report that records "task walkthrough skipped" and exits — do not score.

Step 3 — Score the 10 questions (probe → propose → confirm)

For every question Q1–Q10, follow the same loop:

1. Run the listed probes. They are concrete shell / grep / lint / build commands that produce hard evidence from the repo.
2. Propose a draft score (1–5) based on the probe results and the rubric. Show your work: which probe results led to which score.
3. Cross-check against the user's task-walkthrough notes from Step 2 (especially for navigation, clickability, error prevention).
4. Ask the user via `AskQuestion` with three options: (a) accept the proposed score `N`, (b) override with a specific score, (c) override + add a note.
5. Capture the final score, a one-line rationale, and an improvement note.

Heuristics for translating probe results into a draft score

These thresholds are starting points — adjust based on the specific evidence and the rubric language. The user always has the final say.

Per-question automated probes

Each question's probe list is the first thing the agent should run before asking the user anything about that question. Always state which probes were run and what they returned.

The 10 questions and rubric

Q1 — Aura design system consistency. Are you using Aura tokens, layouts, components and patterns correctly?

Probes (automatable):

• grep -c '@cognite/aura' package.json — confirm Aura is a dependency
• grep -rlE "from '@cognite/aura'" --include='*.ts' --include='*.tsx' src | wc -l — count files importing Aura
• grep -rlE '#[0-9a-fA-F]{3,8}' --include='*.css' --include='*.tsx' --include='*.ts' src — files with hard-coded hex colors
• grep -rlE '\b(rgb|rgba|hsl|hsla)\(' --include='*.tsx' --include='*.css' src — files with raw rgb/hsl values
• npx eslint . --ext .ts,.tsx --rule '{"aura/no-overriding-styles":"error"}' --no-eslintrc --quiet 2>&1 | tail -5 or read the existing lint output for aura/no-overriding-styles warning counts

Translate to draft score: 0 hard-coded colors + 0 aura/no-overriding-styles warnings → 5. Few warnings (1–5) → 4. Many warnings (>15) or no Aura imports → 2–3.

• 5 Excellent: All Aura tokens applied correctly, no hard-coded values. Proper responsive sizing and page layouts. Aura components used without style overrides. Best practices followed.
• 4 Good: Mostly Aura tokens and components with 1–2 minor exceptions. Layout spacing mostly consistent. Minimal style overrides.
• 3 Average: Mix of Aura and custom elements. Some proper spacing, some random values. Overriding styles in multiple places.
• 2 Below average: Frequently custom colors, typography, or spacing instead of Aura tokens. Heavy customization that breaks patterns.
• 1 Poor: Not using Aura at all. Custom colors, fonts, spacing throughout.

Q2 — Navigation, layout and hierarchy. Can users tell where they are and navigate easily?

Probes (partially automatable — relies on Step 2 walkthrough):

• grep -rcE '<Route\b' --include='*.tsx' src — count routes (informs navigation surface)
• grep -rlE 'Breadcrumb' --include='*.tsx' src — files using breadcrumb components (location cues)
• grep -rlE 'NavLink|Link to=|useLocation' --include='*.tsx' src — navigation primitives in use
• grep -rlE '<Topbar|<Sidebar|<Header' --include='*.tsx' src — top-level chrome
• Look at the route tree (src/routes/) and ask: does each non-trivial page show its own title and a way back?

Translate to draft score: Default to the walkthrough finding since navigation feel is hard to measure statically. Use probes to flag risks (e.g. routes without breadcrumbs).