[Feature] shadcn-audit: deterministic browser collection (collect)

.claude/skills/shadcn-audit/SKILL.md

@@ -42,7 +42,12 @@ an error as an empty result. Talk to the user in their language;
 - `gh` authenticated (`gh auth status`) with read access to
   `ruby-ui/ruby_ui`; posting the comment needs write access.
 - Node and pnpm (the shadcn sandbox and the docs assets).
-- The chrome-devtools MCP server, for every browser step.
+- Playwright for `collect`: `cd audit && pnpm install` in every checkout or
+  worktree (it installs `audit/node_modules`), then `pnpm exec playwright
+  install chromium` (the browser download is shared by every checkout on the
+  machine).
+- The chrome-devtools MCP server, for the browser steps `collect` doesn't
+  cover (the kind-specific behavior sequences, animation, exploration).
 - Optional: Herdr. When `HERDR_ENV=1` the servers run in Herdr panes;
   otherwise as background shell processes (step 2.6).
 
@@ -168,24 +173,62 @@ For each folder in `components`:
   list is ⏸ "map gap".
 - `divergences <folder>` → registered differences (`dimension`, `rubyui`,
   `shadcn`, `reason`). A matching difference is ✅ basis `intentional`.
+  Entries with a structured `match` are applied by `collect` on its own;
+  prose-only entries you apply by hand.
+- `collect <folder> --run <run> --preview <preview path>` (add
+  `--shadcn-url`/`--rubyui-url` with the `url_template`s when step 2.6 picked
+  other ports). It drives both fixtures in headless Chromium at 1280×800,
+  light and dark, through the states the `kind` has (closed, open, focus,
+  after-esc; `form_control`: closed, focus; `static`: closed), and grades the
+  objective lines: computed styles per part (Look ❌ / ✅ grouped), the
+  accessibility snapshot plus `aria-expanded`/`aria-controls`/`aria-haspopup`/
+  `aria-describedby` per part (ARIA ⚠️ / ✅ grouped), structured divergences
+  (✅ intentional), differences explained by a theme token or an environment
+  diff (⏸), unreachable states and selector count mismatches (⏸). It writes
+  `<run>/collect.json` and `<run>/screens/`, with a pixel diff per state as
+  evidence only, and only for states reached on both sides. Append its `lines` and `screenshots` to the component and
+  copy its `environment_diffs` into the report. Running it twice on the same
+  SHAs gives the same `collect.json`.
+  - `{"error": "… has no selectors …"}` → the component has no `selectors`
+    in `component_map.yml`: tell the user to add them (RubyUI part → CSS
+    selector in the rendered fixture; `null` = shadcn renders no element for
+    that part). Until they exist, collect that component by hand with the
+    chrome-devtools MCP: for each mapped part, the shadcn classes from
+    `[data-slot=<slot>]` against `effective`, the RubyUI element found from
+    its Ruby source at `<preview path>/gem/lib/ruby_ui/<folder>/` (tag,
+    Stimulus target, class string), look-alikes settled with
+    `getComputedStyle`; then the whole set for its `kind` in
+    `references/interactions.md`, including the Setup, the states and the
+    screenshots.
+  - `{"error": "playwright not found …"}` / `"chromium failed to launch"` →
+    install as in Prerequisites and re-run.
 
 ### 4. Look
 
-Light and dark. For each mapped part: shadcn classes from the DOM
-(`[data-slot=<slot>]`) checked against `effective`; RubyUI classes from the
-preview DOM via `evaluate_script`. RubyUI parts have no `data-slot`: find the
-element from the part's Ruby source at `<preview path>/gem/lib/ruby_ui/<folder>/`
-(tag, Stimulus target, class string). List the tokens missing and extra on each
-side; settle look-alikes with computed styles. Variant props (size, side) are
+`collect` already graded Look from computed styles. Review its lines instead
+of re-collecting: explain each ❌ with the class tokens behind it (shadcn
+`effective` from `reference` vs the RubyUI class string) in the `note`, and
+settle the ⏸ it left (count mismatches, theme/environment explanations,
+reference `warnings`) when the DOM lets you. Variant props (size, side) are
 compared against `variants`.
 
+**Overrides.** Change a `collect` grade only with a verified reason that the
+rules in `references/grading.md` allow, written in that line's `note` (e.g.
+`text-align: left` vs `start` in LTR renders the same → ✅ with the note). A ✅
+with a basis still needs its usual route: a registered divergence, a native
+replacement or the spec. Never drop a `collect` line.
+
 Fixture-originated wrappers aren't findings (`references/grading.md`): Sheet
 body (shadcn `div.px-4` vs `SheetMiddle`), Dialog body (bare `<p>` vs
 `DialogMiddle`).
 
 ### 5. HTML, ARIA, Behavior, Animation
 
-Run the set for the component's `kind` from `references/interactions.md` on
+`collect` already covered the setup, the environment probe, the states
+closed/open/focus/after-esc, the ARIA snapshot and attributes per part, and
+the screenshots. Run the rest of the `kind`'s set from
+`references/interactions.md` (focus trap order, Esc and focus return, outside
+click, scroll lock, arrow keys, typeahead, Enter/Space, animation) on
 both URLs (`url_template`s; `{folder}` = folder, `{light|dark}` = theme), in
 the same viewport on both sides (aim for 1280×800; record the actual size from
 the environment probe), with chrome-devtools MCP. Open by clicking `[data-audit-trigger]`.

.claude/skills/shadcn-audit/references/grading.md

@@ -21,7 +21,7 @@ Behavior or Animation. When unsure, use ⏸ and say what's missing.
 
 | `basis` | Shows as | Rule |
 | --- | --- | --- |
-| `intentional` | ✅ … — intentional | The difference matches an entry from `divergences <folder>` (same `dimension`). `source` = `audit/divergences.yml`, `note` = its `reason`. |
+| `intentional` | ✅ … — intentional | The difference matches an entry from `divergences <folder>` (same `dimension`). `source` = `audit/divergences.yml`, `note` = its `reason`. Entries with a structured `match` are applied by `collect` automatically; prose-only entries by hand. |
 | `native` | ✅ … — native | RubyUI swaps shadcn's JS for a native feature **and keeps** what shadcn delivers: focus in and back, Esc, dismiss, inertness of the page behind. Labelling is markup, graded on the ARIA line, not here. Tag differences go in `note`. No registration needed. |
 | `spec` | ✅ … — follows spec | shadcn breaks a spec (stage 2) and RubyUI follows it. `source` = the spec URL#anchor. No registration needed. |
 
@@ -52,6 +52,16 @@ still graded on its own and is still a finding.
    `native-html.md` does → ❌ "candidate for <feature>", citing MDN. Native
    already used and nothing lost → ✅ basis `native`.
 
+## Lines from `collect`
+
+`collect` grades Look (computed styles: ❌ / ✅) and ARIA (snapshot + attributes:
+⚠️ / ✅) per part, by the same rules as below. Its ⏸ lines say why (state not
+reached, selector count mismatch, theme token, environment diff). Keep every
+`collect` line. Change a grade only with a verified reason that the rules
+below allow (e.g. values that render the same), written in its `note`; a ✅
+with a basis still needs its usual route (registered divergence, native, spec).
+Otherwise keep the grade.
+
 ## Per-dimension notes
 
 - **Look**: compare per part, light and dark. Class tokens that differ but

.claude/skills/shadcn-audit/references/interactions.md

@@ -6,6 +6,13 @@ from the `.tsx` or the Ruby source. Run the same set on both URLs:
 - shadcn: `http://localhost:5179/?c=<folder>&theme=<light|dark>` (sandbox `url_template`)
 - RubyUI: `http://localhost:3179/__audit/<folder>?theme=<light|dark>` (preview `url_template`)
 
+`collect` (SKILL.md step 3) already runs the setup below, the environment
+probe, the states closed/open/focus/after-esc, the computed-style and ARIA
+probes per part, and the screenshots, deterministically. Use this file for
+what it doesn't cover: the per-`kind` sequences (focus trap order, Esc and
+focus return, outside click, scroll lock, arrows, typeahead, Enter/Space),
+animations, and any state `collect` reported as not reached.
+
 ## Setup (once per page)
 
 1. `new_page` (one per side), then `resize_page` both to the **same viewport**,

audit/.gitignore

@@ -1 +1,2 @@
 /.bundle/
+/node_modules/

audit/Gemfile

@@ -3,3 +3,5 @@ source "https://rubygems.org"
 gem "minitest"
 gem "rake"
 gem "standard"
+gem "playwright-ruby-client", "1.64.0"
+gem "chunky_png", "1.4.0"

audit/Gemfile.lock

@@ -2,17 +2,29 @@ GEM
   remote: https://rubygems.org/
   specs:
     ast (2.4.3)
+    base64 (0.3.0)
+    chunky_png (1.4.0)
+    concurrent-ruby (1.3.8)
     drb (2.2.3)
     json (3.1.0)
     language_server-protocol (3.17.0.6)
     lint_roller (1.1.0)
+    logger (1.7.0)
+    mime-types (3.7.0)
+      logger
+      mime-types-data (~> 3.2025, >= 3.2025.0507)
+    mime-types-data (3.2026.1027)
     minitest (6.0.6)
       drb (~> 2.0)
       prism (~> 1.5)
     parallel (2.3.0)
     parser (3.3.13.0)
       ast (~> 2.4.1)
       racc
+    playwright-ruby-client (1.64.0)
+      base64
+      concurrent-ruby (>= 1.1.6)
+      mime-types (>= 3.0)
     prism (1.9.0)
     racc (1.8.1)
     rainbow (3.1.1)
@@ -58,7 +70,9 @@ PLATFORMS
   ruby
 
 DEPENDENCIES
+  chunky_png (= 1.4.0)
   minitest
+  playwright-ruby-client (= 1.64.0)
   rake
   standard
 

audit/README.md

@@ -10,11 +10,13 @@ mise exec -- bundle exec exe/shadcn-audit workspace [--sha SHA]
 
 `workspace` keeps a sparse clone of `shadcn-ui/ui` in `.shadcn-audit/ui` (git-ignored) and checks out `main`, or the given SHA.
 
+`collect <folder> --run <dir> [--preview <path>] [--shadcn-url T] [--rubyui-url T]` drives both fixtures in headless Chromium (playwright-ruby-client) with the sandbox and preview servers running, and writes `<dir>/collect.json`: computed styles, accessibility snapshots, screenshots and pixel diffs per state, plus the objectively graded Look/ARIA lines. It needs `pnpm install && pnpm exec playwright install chromium` here once. Part selectors come from `selectors` in `component_map.yml`; a `match` key in `divergences.yml` lets `collect` apply that divergence on its own.
+
 Tests use `ShadcnAudit::FakeShell` (`test/test_helper.rb`): stub canned stdout by argv or matcher, then assert on recorded calls.
 
 ## Prerequisites
 
-Ruby 3.4 (via mise, as below, or an equivalent version manager; without mise, drop `mise exec --`), `gh` authenticated (`gh auth status`), Node and pnpm, and, for the browser steps, the chrome-devtools MCP. The audit runs its two dev servers in Herdr panes when `HERDR_ENV=1`, otherwise as background shell processes; either way it stops them at the end. `preview` boot commands use `mise exec --` only when `mise` is on PATH.
+Ruby 3.4 (via mise, as below, or an equivalent version manager; without mise, drop `mise exec --`), `gh` authenticated (`gh auth status`), Node and pnpm, Playwright's Chromium for `collect` (above), and, for the remaining browser steps, the chrome-devtools MCP. The audit runs its two dev servers in Herdr panes when `HERDR_ENV=1`, otherwise as background shell processes; either way it stops them at the end. `preview` boot commands use `mise exec --` only when `mise` is on PATH.
 
 A component added to `component_map.yml` also needs fixtures in `fixtures/<folder>/` (`content.yml`, `shadcn.tsx`, `rubyui.rb`).
 

audit/component_map.yml

@@ -2,6 +2,8 @@
 # parts: RubyUI class => shadcn data-slot (null = no shadcn part).
 # props: RubyUI class => {shadcn prop => RubyUI kwarg (null = none)}; shadcn props only.
 # rubyui_only: RubyUI class => [kwargs with no shadcn counterpart]. className is never listed (all parts take class via attrs).
+# selectors: RubyUI class => CSS selector of that part in the rendered fixture (fixed, so collect is deterministic);
+#   null = not compared (shadcn renders no element for it, e.g. a Base UI root).
 # fixture_shadcn: other shadcn components the fixture renders (added to the sandbox, not audited here).
 dialog:
   shadcn: dialog
@@ -16,6 +18,14 @@ dialog:
     DialogTitle: dialog-title
     DialogDescription: dialog-description
     DialogMiddle: null
+  selectors:
+    Dialog: null
+    DialogTrigger: '[data-ruby-ui--dialog-target="trigger"] > button'
+    DialogContent: 'dialog[data-ruby-ui--dialog-target="dialog"]'
+    DialogHeader: 'dialog[data-ruby-ui--dialog-target="dialog"] > div:first-child'
+    DialogFooter: 'dialog[data-ruby-ui--dialog-target="dialog"] > div:last-of-type'
+    DialogTitle: '[data-ruby-ui--dialog-target="title"]'
+    DialogDescription: '[data-ruby-ui--dialog-target="description"]'
   props:
     Dialog: {open: open}
     DialogContent: {showCloseButton: show_close_button}
@@ -37,6 +47,16 @@ alert_dialog:
     AlertDialogDescription: alert-dialog-description
     AlertDialogAction: alert-dialog-action
     AlertDialogCancel: alert-dialog-cancel
+  selectors:
+    AlertDialog: null
+    AlertDialogTrigger: '[data-controller="ruby-ui--alert-dialog"] > [data-action] > button'
+    AlertDialogContent: '[role="alertdialog"]'
+    AlertDialogHeader: '[role="alertdialog"] > div:first-child'
+    AlertDialogFooter: '[role="alertdialog"] > div:last-child'
+    AlertDialogTitle: '[role="alertdialog"] > div:first-child > h2'
+    AlertDialogDescription: '[role="alertdialog"] > div:first-child > p'
+    AlertDialogAction: '[role="alertdialog"] > div:last-child > button:last-child'
+    AlertDialogCancel: '[role="alertdialog"] > div:last-child > button:first-child'
   props:
     AlertDialogContent: {size: null}
     AlertDialogCancel: {variant: variant, size: size}
@@ -54,6 +74,14 @@ sheet:
     SheetTitle: sheet-title
     SheetDescription: sheet-description
     SheetMiddle: null
+  selectors:
+    Sheet: null
+    SheetTrigger: '[data-controller="ruby-ui--sheet"] > [data-action] > button'
+    SheetContent: '[data-ruby-ui--sheet-content-target="panel"]'
+    SheetHeader: '[data-ruby-ui--sheet-content-target="panel"] > div:first-child'
+    SheetFooter: '[data-ruby-ui--sheet-content-target="panel"] > div:last-of-type'
+    SheetTitle: '[data-ruby-ui--sheet-content-target="panel"] > div:first-child > h3'
+    SheetDescription: '[data-ruby-ui--sheet-content-target="panel"] > div:first-child > p'
   props:
     SheetContent: {side: side, showCloseButton: null}
 
@@ -71,6 +99,15 @@ command:
     CommandDialog: dialog
     CommandDialogTrigger: null
     CommandDialogContent: dialog-content
+  selectors:
+    Command: '[data-ruby-ui--command-target="panel"] > div'
+    CommandInput: 'input[data-ruby-ui--command-target="input"]'
+    CommandList: '[data-ruby-ui--command-target="panel"] > div > div:has([data-ruby-ui--command-target="group"])'
+    CommandEmpty: '[data-ruby-ui--command-target="empty"]'
+    CommandGroup: '[data-ruby-ui--command-target="group"]'
+    CommandItem: '[data-ruby-ui--command-target="item"]'
+    CommandDialog: null
+    CommandDialogContent: '[data-ruby-ui--command-target="panel"]'
   props:
     CommandInput: {placeholder: placeholder}
     CommandGroup: {heading: title}
@@ -97,6 +134,17 @@ form:
     FormFieldContent: field-content
     FormFieldTitle: field-label
     FormFieldSeparator: field-separator
+  selectors:
+    FormField: '[data-slot="field"]'
+    FormFieldLabel: '[data-slot="field-label"]'
+    FormFieldHint: '[data-slot="field-description"]'
+    FormFieldError: '[data-slot="field-error"]'
+    FormFieldSet: '[data-slot="field-set"]'
+    FormFieldLegend: '[data-slot="field-legend"]'
+    FormFieldGroup: '[data-slot="field-group"]'
+    FormFieldContent: '[data-slot="field-content"]'
+    FormFieldTitle: null
+    FormFieldSeparator: '[data-slot="field-separator"]'
   props:
     FormField: {orientation: orientation}
     FormFieldLegend: {variant: variant}

audit/divergences.yml

@@ -1,5 +1,7 @@
 # Intentional RubyUI-vs-shadcn differences. A divergence listed here makes the
 # matching finding a `pass` with basis `intentional`.
+# Optional `match: {part, property (Look) | attribute (ARIA, or `snapshot`), rubyui, shadcn}` lets `collect` apply it
+# automatically; entries without it are applied by the agent.
 - component: dialog
   dimension: HTML
   rubyui: "<dialog> + showModal()"