Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 33 additions & 2 deletions docs/output.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,7 @@ per-shot output when piped, `--no-progress` the summary on a terminal.
| `findings: N repeated on 2+ routes` | After the routes: the same `dom` entry, text line, `fonts` face or element-less band delta (`band -24px`) on two or more routes, once per line with `×N routes` and where (`/a@1440,375`, then `+N routes`), most widespread first, accepted ones aside. Text lines seen on exactly the same routes and widths share one line, `text -"Users" -"Billing" -"Docs" (+9 more)` (a nav folded into a menu button), and N counts lines as printed. Top 20; `summary.findings` in results.json has every finding, one per text line. One cause behind many rows: fix it once. A selector names an element per page, so the same path may be a different element on another route: check one crop. |
| `computed: N findings` | With `--computed`: each `key prop left→right` once, with the routes and widths it was seen at; `dom identical`: the markup matches apart from asset hashes and generated class names, so look at CSS or script. See [below](#computed-styles---computed). |
| `rules: N changed groups` | With `--computed`: style rules only one side has, by stylesheet and `@media`, and whether any shot had an element they could style. `matched nothing`: the run proved nothing about those rules. See [below](#rule-coverage). |
| `prefixes: N prefixed declarations only one side has` | With `--computed`: `-webkit-`, `-moz-`, `-ms-` and `-o-` declarations in one side's CSS source only, which Chrome's parsed rules can't show. See [below](#vendor-prefixes). |
| `aria: identical on N of M route(s)` | With `--aria`: routes whose accessibility tree matches at every width, then each differing route's lines, `+` left only, `-` right only. See [below](#aria-snapshots---aria). |
| `crops …` | Up to 3 PNGs per comparison, worst first, relative to `results.json`. Left is on top or on the left, and the change is boxed in red. |
| `ERROR …` | Unmeasured, which is neither a pass nor a fail. Exits 3. |
Expand Down Expand Up @@ -317,6 +318,34 @@ rules: 3 changed groups, 1 never matched, 2 matched on some shots
rules then show as one side's. renderdiff's own CSS (hidden selectors, stilled animations,
`extraCss`) isn't read.

### Vendor prefixes

Chrome parses `-webkit-user-select: none` and `user-select: none` into the same rule, so a lint
autofix that drops a prefix Safari still needs is no pixel change and no rule change. So
`--computed` also reads each side's CSS as written and counts its prefixed properties:

```
prefixes: 3 prefixed declarations only one side has
only left has: -webkit-background-clip: text; -webkit-text-size-adjust: 100%; -webkit-user-select: none /a@1440,375
```

- The source is every stylesheet the page loaded, as served (cross-origin and `@import`ed ones
too), and the text of every `<style>`, open shadow roots included. Frames aren't read.
- What counts is how many declarations of each prefixed property a side has; which selector,
rule or file holds them, and their values, don't. So a dropped prefix is one fewer
`-webkit-user-select` on that side, while minifying (`0.3s` → `.3s`), reordering, moving a
declaration between files or a new asset hash is no change. The value is shown as context:
that side's declarations of the property, ones the other side lacks as written first.
- Prefixed selectors (`::-webkit-scrollbar`), media features and values (`-webkit-box`) aren't
read.
- Up to 10 declarations per side shown; results.json has them all. `prefixes: no prefixed
declaration differs between the sides`: the sources hold the same ones.
- Informational, like `rules:`: it never changes the exit code.
- CSS with no source text isn't seen: rules a script adds with `insertRule`, constructed
(adopted) stylesheets, and `style` attributes.
- A third-party sheet that changes between loads (an ad or chat widget) shows as prefixes
either side has; block it with `blockUrlPatterns` in the config.

## ARIA snapshots (`--aria`)

With `--aria`, after the shot (and `--computed`) renderdiff takes Playwright's ARIA snapshot
Expand Down Expand Up @@ -531,7 +560,8 @@ file, not each line), scores rounded down to four decimals (so a shot that diffe
"fonts": { "added": [], "removed": ["Inter 600"] }, // web font faces one side loaded; absent when equal
"masked": ["rotating quote (data-renderdiff-mask)"], // absent unless a side marked elements data-renderdiff-mask
"computed": { "dom": 1, "changes": [], "truncated": 0, "unmatched": { "left": [], "right": ["body > nav > a[href=\"/pricing\"]"] },
"rules": [{ "side": "left", "sheet": "docs.css", "conditions": ["@media (width >= 60em)"], "rules": 5, "elements": 67 }] }, // --computed only; null: snapshot failed, "computedError" says why; rules: one-sided rule groups, elements they could style here
"rules": [{ "side": "left", "sheet": "docs.css", "conditions": ["@media (width >= 60em)"], "rules": 5, "elements": 67 }],
"prefixes": { "left": ["-webkit-user-select: none"], "right": [] } }, // --computed only; null: snapshot failed, "computedError" says why; rules: one-sided rule groups, elements they could style here; prefixes: { left, right }, prefixed declarations only that side's CSS source has
"aria": { "added": ["- text: Buy now"], "removed": ["- button \"Buy now\""], "truncated": 0 }, // --aria only; both empty: identical; null: snapshot failed, "ariaError" says why
"images": { "left": "about.1440.left.png", "right": "about.1440.right.png", "diff": "about.1440.diff.png" }, // packed: .webp, right may name left's file, no diff when diffPixels is 0
"crops": [{
Expand All @@ -552,7 +582,8 @@ file, not each line), scores rounded down to four decimals (so a shot that diffe
"stale": [], // accepted entries the run measured but didn't see
"findings": [], // { finding, selector?, at: [{ url, widths }] }, seen on 2+ routes
"computed": [], // --computed: { key, prop, values: [left, right], at: [{ url, widths }] }
"rules": [] // --computed: { side, sheet, conditions, rules, matched: [{ url, widths }], unmatched: [{ url, widths }] }
"rules": [], // --computed: { side, sheet, conditions, rules, matched: [{ url, widths }], unmatched: [{ url, widths }] }
"prefixes": [] // --computed: { side, declaration, at: [{ url, widths }] }
}
}
```
1 change: 1 addition & 0 deletions skills/renderdiff/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ the § named at each step. Setting up a project (`/renderdiff init`): [init.md](
| Change | Run |
| --- | --- |
| CSS only, markup untouched (refactor, dead-CSS removal, token swap) | pixels + `--computed`. First diff the two builds (§ Reference and routes): if only asset hashes differ, or every changed rule means the same, no shots are needed; otherwise shoot the routes the changed rules reach. Expect `dom identical` on every route: then each `computed:` finding is a real style change, even on a clean route. A changed rule that matched nothing (`rules:`) means the run proved nothing about it: add a route or hook that reaches it. `dom differs`: often a wrapper's class changed, re-keying its subtree (§ `--computed`). Hover menus and open menus aren't shot without a `before` hook that ends waiting for the open state, and dead-CSS removal breaks exactly those. |
| Lint autofix, Autoprefixer or browserslist change | pixels + `--computed`; read `prefixes:` and decide whether each dropped prefix's browsers still need it. |
| Markup changes (framework, template or component migration) | pixels, plus `--aria`: `aria: identical` means a screen reader gets the same page. `--computed` findings would only be hints. |
| Dependency or build upgrade | pixels, plus `--computed` if the markup should be identical. |
| One component | pixels with `--selector <sel>` or `'/route @sel'`. |
Expand Down
10 changes: 7 additions & 3 deletions skills/renderdiff/reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -228,6 +228,9 @@ computed: 2 findings; dom identical on 2 of 3 route(s), differs /pricing (1), so
that reaches them. Rules compare as Chrome parses them: `rgb()` notation is no change,
`min-width:` → `width >=` is. Details and limits: docs/output.md § Rule coverage in the
renderdiff repo.
- `prefixes: N prefixed declarations only one side has`: `-webkit-`/`-moz-`/`-ms-`/`-o-`
properties one side's CSS source declares more often. Details and limits: docs/output.md
§ Vendor prefixes in the renderdiff repo.

## `--aria`

Expand Down Expand Up @@ -449,15 +452,16 @@ commas: `JSON.parse` the whole file, not each line. Paths are relative to the ou
(`kind`, each side's box with `h` 0 at an insertion point, `layout`, `scale`, `clipped`);
`accepted`; `rightReplayedFrom`; with `--computed`, `computed` (`dom`, 0 = identical;
`changes` `{ key, prop, values }`, first 500; `truncated`; `unmatched`; `rules`
`{ side, sheet, conditions, rules, elements }`) or null with
`{ side, sheet, conditions, rules, elements }`; `prefixes` `{ left, right }`) or null with
`computedError`; with `--aria`, `aria` (`added`, `removed`, first 200 each, both empty =
identical; `truncated`) or null with `ariaError`.
- **`summary`:** `compared`, `errors`, `mean` and `worst` (accepted shots aside),
`belowThreshold`, `differing` (unaccepted shots with any differing pixel; without
`--threshold`, any exits 1), `accepted`, `stale`, `findings` (each `dom` entry, text line
or element-less band on 2+ routes once: `finding`, `selector?`, `at: [{ url, widths }]`),
and with `--computed` `computed` (each change once with `at: [{ url, widths }]`) and `rules`
(each rule group once with `matched` and `unmatched` `[{ url, widths }]`).
and with `--computed` `computed` (each change once with `at: [{ url, widths }]`), `rules`
(each rule group once with `matched` and `unmatched` `[{ url, widths }]`) and `prefixes`
(each declaration once: `side`, `declaration`, `at`).
- **`meta`:** origins, widths, threshold, `clock`, block patterns, hidden selectors, mask
reasons, `browser` (`chromium`, or `chrome` after a fallback), storage state file names,
`config` hash, `rightReplayedFrom`, `freezeVideo`, `lottieFrame` (only when set), `platform`, `rerun` (`{ rerun, reused }` or
Expand Down
27 changes: 26 additions & 1 deletion src/capture.ts
Original file line number Diff line number Diff line change
Expand Up @@ -153,6 +153,8 @@ const MAX_PAGE_AREA = 1440 * 40_000;
const network = new WeakMap<Page, { open: Set<Request>; last: number; failed: Set<string> }>();
/** Each page's last main-frame document response; redirect hops come first, so it is the final one. */
const doc = new WeakMap<Page, Response>();
/** With `--computed`: the bodies of this load's main-frame stylesheets, as served. */
const sources = new WeakMap<Page, Promise<string>[]>();

const html = (r: Response) => {
const type = r.headers()["content-type"];
Expand Down Expand Up @@ -201,9 +203,17 @@ export async function prepare(page: Page, cfg: ResolvedConfig, now: number): Pro
// Aborted: cancelled by the page or by the next navigation, not a load that failed.
if (r.failure()?.errorText !== "net::ERR_ABORTED") sheet(r);
});
if (cfg.computed) sources.set(page, []);
page.on("response", (r) => {
if (r.status() >= 400) sheet(r.request());
if (r.request().isNavigationRequest() && r.frame() === page.mainFrame()) doc.set(page, r);
if (r.request().isNavigationRequest() && r.frame() === page.mainFrame()) {
doc.set(page, r);
// A new document, from the load, a hook or a redirect; a pushState has no response.
sources.get(page)?.splice(0);
}
// Routing is always on, so the HTTP cache is off and no sheet is a bodiless 304; read now, before navigating drops it.
if (r.ok() && r.request().resourceType() === "stylesheet" && r.frame() === page.mainFrame())
sources.get(page)?.push(r.text().catch(() => ""));
});
// A request the old document left open never finishes or fails, so it can't hold the next load.
page.on("framenavigated", (f) => {
Expand Down Expand Up @@ -262,6 +272,21 @@ export async function prepare(page: Page, cfg: ResolvedConfig, now: number): Pro
});
}

/** `--computed`: this document's CSS as written, which Chromium's CSSOM isn't: stylesheet bodies and `<style>` text. */
export async function sheetSources(page: Page): Promise<string[]> {
const inline = await page.evaluate(() => {
const out: string[] = [];
const walk = (root: Document | ShadowRoot) =>
root.querySelectorAll("*").forEach((e) => {
if (e.localName === "style") out.push(e.textContent ?? "");
if (e.shadowRoot) walk(e.shadowRoot);
});
walk(document);
return out;
});
return [...(await Promise.all(sources.get(page) ?? [])), ...inline];
}

/** Not addStyleTag: it fails on any CSP violation logged meanwhile, and a <style> is subject to style-src. */
function adoptCss(css: string) {
const sheet = new CSSStyleSheet();
Expand Down
115 changes: 114 additions & 1 deletion src/computed.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
import type { Page } from "playwright";
import { textDiff } from "./compare.ts";
import { name } from "./config.ts";
import { andMore } from "./format.ts";
import { MAX_STATES, SNAP_PROPS } from "./locate.ts";
import type { ComparedResult, RouteResult } from "./report.ts";

Expand Down Expand Up @@ -33,6 +34,16 @@ export interface ComputedDiff {
unchecked?: number;
/** Style rules only that side has, grouped by stylesheet and enclosing at-rules; absent when not read. */
rules?: RuleGroup[];
/** Vendor-prefixed declarations in the CSS source only that side has, each once; absent when not read. */
prefixes?: { left: string[]; right: string[] };
}

/** One prefixed declaration only one side has, with every route and width it was seen at. */
export interface PrefixFinding {
side: "left" | "right";
/** As written, whitespace normalised: `-webkit-user-select: none`. */
declaration: string;
at: Finding["at"];
}

export interface RuleGroup {
Expand Down Expand Up @@ -447,6 +458,67 @@ function oneSided(a: string[], b: string[]): number[] {
});
}

/** The `-webkit-`/`-moz-`/`-ms-`/`-o-` declarations in CSS source, whitespace normalised. */
export function prefixed(css: string): string[] {
// A value is quoted strings, parenthesised groups (a data URI's `;`) and other text, each one way only.
const decl =
/(?<![\w-])(-(?:webkit|moz|ms|o)-[a-z-]+)\s*:((?:"[^"]*"|'[^']*'|\([^)]*\)|[^;{}"'(])+)(?=[;}])/gi;
return [...uncomment(css).matchAll(decl)].map(
([, prop, value]) =>
`${prop!.toLowerCase()}: ${value!
.trim()
.replace(/\s+/g, " ")
.replace(/\s*!\s*important$/i, " !important")}`,
);
}

/** `css` without its comments; a `/*` inside a string or parentheses (`url(/img/*.png)`) isn't one. */
function uncomment(css: string): string {
const out: string[] = [];
let [from, quote, depth] = [0, "", 0];
for (let i = 0; i < css.length; i++) {
const c = css[i];
if (quote) {
if (c === "\\") i++;
else if (c === quote) quote = "";
} else if (c === '"' || c === "'") quote = c;
else if (c === "(") depth++;
else if (c === ")") depth = Math.max(0, depth - 1);
else if (!depth && c === "/" && css[i + 1] === "*") {
out.push(css.slice(from, i));
const end = css.indexOf("*/", i + 2);
i = end < 0 ? css.length : end + 1;
from = i + 1;
}
}
out.push(css.slice(from));
return out.join("");
}

/**
* Per side, the prefixed properties it has more declarations of, as that many of its
* declarations, ones the other side lacks verbatim first. Counting names, not values, keeps a
* minifier's `0.3s`→`.3s` from reading as a pair.
*/
export function prefixDiff(left: string[], right: string[]): { left: string[]; right: string[] } {
const [l, r] = [left, right].map((texts) => texts.flatMap(prefixed));
const prop = (d: string) => d.slice(0, d.indexOf(":"));
const only = (a: string[], b: string[]) => {
const extra = new Map<string, number>();
for (const d of a) extra.set(prop(d), (extra.get(prop(d)) ?? 0) + 1);
for (const d of b) extra.set(prop(d), (extra.get(prop(d)) ?? 0) - 1);
const picked: string[] = [];
for (const i of new Set([...oneSided(a, b), ...a.keys()])) {
const n = extra.get(prop(a[i]!))!;
if (n <= 0) continue;
extra.set(prop(a[i]!), n - 1);
picked.push(a[i]!);
}
return [...new Set(picked)].sort();
};
return { left: only(l!, r!), right: only(r!, l!) };
}

/** `--computed`'s rule coverage: the style rules only one side has, and what each side's could style. */
export async function ruleCoverage(
pages: [Page, Page],
Expand Down Expand Up @@ -531,6 +603,46 @@ export function groupRules(results: RouteResult[]): RuleFinding[] {
return [...out.values()].sort((a, b) => rank(a) - rank(b));
}

/** Each one-sided prefixed declaration once, most widespread first. */
export function groupPrefixes(results: RouteResult[]): PrefixFinding[] {
const out = new Map<string, PrefixFinding>();
for (const r of computedOf(results)) {
for (const side of ["left", "right"] as const) {
for (const declaration of r.computed!.prefixes?.[side] ?? []) {
const id = JSON.stringify([side, declaration]);
const f = out.get(id) ?? { side, declaration, at: [] };
out.set(id, f);
addAt(f.at, r);
}
}
}
const shots = (f: PrefixFinding) => f.at.reduce((n, a) => n + a.widths.length, 0);
return [...out.values()].sort(
(a, b) => shots(b) - shots(a) || a.declaration.localeCompare(b.declaration),
);
}

/** The `prefixes:` block: per side, the prefixed declarations only it has, and where. */
function fmtPrefixes(rows: ComparedResult[]): string[] {
if (!rows.some((r) => r.computed?.prefixes)) return [];
const all = groupPrefixes(rows);
if (!all.length) return ["prefixes: no prefixed declaration differs between the sides"];
const out = [
`prefixes: ${all.length} prefixed declaration${all.length === 1 ? "" : "s"} only one side has`,
];
for (const side of ["left", "right"] as const) {
const fs = all.filter((f) => f.side === side);
if (!fs.length) continue;
const at: Finding["at"] = [];
for (const r of rows) if (r.computed?.prefixes?.[side].length) addAt(at, r);
const shown = fs.slice(0, PREFIXES_SHOWN).map((f) => f.declaration);
out.push(
` only ${side} has: ${shown.join("; ")}${andMore(fs.length, PREFIXES_SHOWN)} ${fmtAt(at)}`,
);
}
return out;
}

const fmtGroup = (f: RuleFinding) =>
`${f.side === "left" ? "+" : "-"}${[f.sheet, ...f.conditions].join(" ")} (${f.rules} rule${f.rules === 1 ? "" : "s"})`;

Expand Down Expand Up @@ -559,6 +671,7 @@ function fmtRules(rows: ComparedResult[]): string[] {

/** Findings shown in the console summary; results.json has them all. */
export const SHOWN = 20;
const PREFIXES_SHOWN = 10;
const AT_ROUTES = 3;
/** Unmatched routes, and keys per route, in the console summary. */
const UNMATCHED_SHOWN = 5;
Expand Down Expand Up @@ -621,7 +734,7 @@ export function fmtComputed(results: RouteResult[], routes: string[]): string[]
out.push(...unmatched.slice(0, UNMATCHED_SHOWN));
if (unmatched.length > UNMATCHED_SHOWN)
out.push(` (+${unmatched.length - UNMATCHED_SHOWN} more routes with unmatched elements)`);
return [...out, ...fmtRules(rows)];
return [...out, ...fmtRules(rows), ...fmtPrefixes(rows)];
}

// ponytail: raw indented lines, so a wrapper role added or removed re-indents everything below it; key lines by role path if that gets noisy
Expand Down
Loading
Loading