Skip to content

chore(skills): add PR screenshot skill with capture scripts - #5906

Open
GVodyanov wants to merge 2 commits into
mainfrom
chore/pr-screenshots-skill
Open

GVodyanov wants to merge 2 commits into
mainfrom
chore/pr-screenshots-skill

Conversation

@GVodyanov

@GVodyanov GVodyanov commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Adds an agent skill in .skills/nextcloud-pr-screenshots/ for taking screenshots of UI changes and putting them on a PR. It decides which shots a change needs (viewports, themes, loading/error states, long strings, before/after), takes them, combines them into one labeled image and posts it in a collapsible block in the PR description or a comment.

Until now, the skill was only instructions, so every run rebuilt the same tooling: login, test data, waiting for the page, image layout. This version ships scripts for that:

  • preflight.mjs checks the instance, URLs, Node version, build and browser, and explains how to fix problems
  • seed.mjs creates a test user with neutral data (fixtures for contacts, files and calendar), using only the Nextcloud APIs, so it works on any instance and not only nextcloud-docker-dev
  • run.mjs runs a short per-PR shot list: one login, scenes in parallel, light/dark without reloading, 2x close-ups, held or failing requests for loading and error states, and --tag before|after for before/after pairs
  • compose.mjs combines the shots into one image, and post.mjs adds or replaces it in the PR description or a comment

Nothing in it is specific to Contacts; it works for any app and for the server repo. I tested it on #5889 (8 shots in about 40 s, from preflight to finished image) and on the Files app using only the example template. .skills is excluded from the app package through .nextcloudignore.

🤖 AI (if applicable)

  • The content of this PR was partly or fully generated using AI

Assisted by ClaudeCode:claude-opus-5-5

Adds a nextcloud-pr-screenshots agent skill with scripts that check the
environment, seed a test user with neutral data, capture the shots a PR
needs in parallel, combine them into one labeled image and post it to
the PR description or a comment. The scripts are not tied to this app
and work for any Nextcloud app or the server repo.

The folder is excluded from the app package via .nextcloudignore.

Assisted-by: ClaudeCode:claude-opus-5-5
Signed-off-by: Grigory V <scratchx@gmx.com>
@nextcloud-command nextcloud-command added the AI assisted This PR contains AI-assisted commits label Nov 16, 2026
@ChristophWurst

Copy link
Copy Markdown
Member
image

import { CACHE_DIR, importPlaywright, loadEnv, parseArgs, saveEnv } from './lib/env.mjs'
import { adminAuth, ocs } from './lib/http.mjs'

const PLAYWRIGHT_VERSION = '1.56.1'

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should either use a more up-to-date version or simply use the latest one when running the skill? 🤔 Don't know how this is typically being handled in skills

Copy link
Copy Markdown
Member

Claude Code review — 10 findings on commit 8144e86

🔴 Normal — .skills/nextcloud-pr-screenshots/scripts/lib/nc.mjs:222

Shot filenames leave out the before/after tag, so the documented --tag after run writes over the --tag before PNGs in the same out dir, and the composite shows the same image on both sides.

🔴 Normal — .skills/nextcloud-pr-screenshots/scripts/seed.mjs:37

If NC_SHOTS_USER or --user names an existing real account (for example admin, for the 'admin vs. user' contexts SKILL.md suggests), seed.mjs resets that account's password to a random value, and the fixtures delete all of its files, contacts and calendar events. [also at: .skills/nextcloud-pr-screenshots/scripts/seed.mjs:41 - When user creation fails for any reason, seed.mjs resets the password of whatever existing account has that id (from --user or NC_SHOTS_USER). It also changes that account's display name, language, locale and theme. A real account can be taken over without any warning. The scripts are meant to run on 'any instance', not only docker-dev.]

🔴 Normal — .skills/nextcloud-pr-screenshots/scripts/post.mjs:39-71

The gh() helper always calls .trim() on what execFileSync returns. Calls made with { stdio: 'inherit' } get null back, so null.trim() throws a TypeError right after the PR edit has already succeeded. Fix: don't trim when stdio is inherited (e.g. ?.trim()). Same pattern at 2 sites (post.mjs:71, post.mjs:88).

🔴 Normal — .skills/nextcloud-pr-screenshots/scripts/post.mjs:39

Every gh call runs with cwd set to the image's directory. The documented output location (~/.cache/nc-pr-shots///out) is outside any git repo, so without --repo gh cannot work out which repository is meant and post.mjs fails.

🔴 Normal — .skills/nextcloud-pr-screenshots/scripts/build-swap.mjs:33-42

detectOutputDirs only keeps git-ignored build dirs. In repos that commit their build output (e.g. the server's tracked dist/), it saves dirs: [], and use base swaps nothing but still prints 'Serving the base build'.

🔴 Normal — .skills/nextcloud-pr-screenshots/scripts/run.mjs:151-157

When run.mjs merges into the manifest, it only replaces entries whose scene name ran in this run (with the same tag). Entries for scenes that were renamed or removed from shots.mjs stay in manifest.json for good. Their old PNGs are never deleted, so compose.mjs puts those stale shots back into every later composite. Since a scene's name defaults to its section, renaming a section is enough to trigger this.

🔴 Normal — .skills/nextcloud-pr-screenshots/scripts/lib/nc.mjs:225-238

Cropped shots measure the target's boundingBox without scrolling it into view, then clamp the clip to the viewport. A target below the fold, for example an item further down a scrolled list, gets a clip with negative height and the screenshot throws.

🟡 Nit — .skills/nextcloud-pr-screenshots/scripts/run.mjs:68-75

applyUserSettings ignores the OCS responses when it enables a server theme or sets the language. If either call fails, the scenes still run and their shots are saved and labeled as highcontrast/dyslexia/de even though the default theme or English is showing.

🟡 Nit — .skills/nextcloud-pr-screenshots/scripts/run.mjs:95

Per-scene failure isolation in runJob can be escaped. createNc() is awaited before the try block (line 95), and the catch block assumes e.message is a string (line 123). Either way runJob rejects, pool's Promise.all rejects, and the top-level await aborts the whole run. The manifest is never written and the failure report is never printed, even though SKILL.md:26 says 'failing scenes are reported with a screenshot, the rest still run'. Fix: runJob must never reject. Move createNc into the try with a null-safe finally/close, and format the error with String(e?.message ?? e). That covers both escape paths listed. Same pattern at 2 sites (run.mjs:95, run.mjs:123).

🟡 Nit — .skills/nextcloud-pr-screenshots/scripts/lib/nc.mjs:49-52

The 'Login failed, run seed.mjs' error is unreachable. After a failed login Nextcloud stays on (or redirects back to) a URL whose path ends in /login, so waitForURL waits 60 s and throws a generic TimeoutError instead.


Generated by Claude Code

…ipts

- Before/after shots no longer overwrite each other, and a run replaces
  all shots of its tag, so renamed or removed scenes don't linger
- seed.mjs only uses an account it created and never resets passwords
- post.mjs resolves the repo itself and no longer crashes after posting;
  comments are always new so reviewers are notified
- Failing scenes and failing theme/language changes are reported
  without aborting the run
- Shots scroll their target into view before cropping
- Login problems fail fast with a clear message
- Install the latest Playwright and its browser instead of pinning
- Drop build-swap.mjs, the browser host mapping and --only
- Make navigation and settings selectors independent of the UI language
  and keep library-dependent selectors in one place
- Refresh Playwright monthly instead of keeping the first install forever
- Require Node 22 for the scripts
- Disable the first-run wizard through its API
- Point coding agents to the skill in AGENTS.md

Assisted-by: ClaudeCode:claude-opus-5-5
Signed-off-by: Grigory V <scratchx@gmx.com>
@GVodyanov
GVodyanov force-pushed the chore/pr-screenshots-skill branch from fe868f5 to 4d6a7cb Compare November 17, 2026 11:23
@GVodyanov

Copy link
Copy Markdown
Contributor Author

@coderabbitai are all those scripts actually useful? can this be cleaned up?

@coderabbitai

coderabbitai Bot commented Nov 17, 2026

Copy link
Copy Markdown

Tip

For best results, initiate chat on the files or code changes.

Analysis chain

Script executed:

#!/bin/bash
set -u
printf '%s\n' '=== Skill files and line counts ==='
fd -H -t f . .skills/nextcloud-pr-screenshots | sort | xargs -r wc -l
printf '%s\n' '=== Script outlines ==='
ast-grep outline .skills/nextcloud-pr-screenshots/scripts --items all 2>&1
printf '%s\n' '=== Skill instructions ==='
for f in $(fd -H '^SKILL\.md$' .skills/nextcloud-pr-screenshots); do
  echo "=== $f ==="
  cat -n "$f"
done
printf '%s\n' '=== Script and workflow references ==='
rg -n -C2 'preflight\.mjs|seed\.mjs|run\.mjs|compose\.mjs|post\.mjs|build-swap\.mjs|shots\.mjs' .skills/nextcloud-pr-screenshots
printf '%s\n' '=== Changed files relative to base ==='
git diff --stat origin/master...HEAD 2>/dev/null || git diff --stat origin/main...HEAD 2>/dev/null || true

Length of output: 22549


Script executed:

#!/bin/bash
set -u
for f in .skills/nextcloud-pr-screenshots/scripts/{preflight,seed,run,compose,post}.mjs .skills/nextcloud-pr-screenshots/scripts/templates/shots.example.mjs; do
  printf '\n=== %s ===\n' "$f"
  cat -n "$f"
done
printf '\n=== Skill references and potential tests across repository ===\n'
rg -n -H -i -C1 'nextcloud-pr-screenshots|nc-pr-shots|scripts/fixtures|manifest\.json' -g '!package-lock.json' -g '!yarn.lock' . || true
printf '\n=== PR changes to instructions ===\n'
git diff origin/master...HEAD -- AGENTS.md .nextcloudignore 2>/dev/null || git diff origin/main...HEAD -- AGENTS.md .nextcloudignore 2>/dev/null || true

Length of output: 24390


Script executed:

#!/bin/bash
set -u
for f in .skills/nextcloud-pr-screenshots/scripts/lib/{env,http,nc,data}.mjs .skills/nextcloud-pr-screenshots/scripts/fixtures/{contacts,files,calendar}.mjs .skills/nextcloud-pr-screenshots/reference/troubleshooting.md; do
  printf '\n=== %s ===\n' "$f"
  cat -n "$f"
done

Length of output: 24824


@GVodyanov Yes. The scripts serve distinct purposes, but the skill can be smaller. I would keep capture, composition, and safe test-data setup as the core. The five entry points are not inherently a problem; their scope matters more than their count.

I suggest:

  • Keep the Contacts fixture. Make Files and Calendar fixtures optional if cross-app support is not required for this PR.
  • Keep preflight checks focused on prerequisites. Pin Playwright instead of automatically installing the latest version.
  • Make posting optional; manual posting can cover cases where gh attachment support or permissions are unavailable.
  • Add tests for seeding safety, before/after filenames, manifest replacement, and composition. This PR adds no tests for the scripts.

Check seeding safety before use on an instance with real data. In seed.mjs, Line 24, the existing-account check verifies cached credentials, not that the tool created the account. Fixtures can replace that account’s data, and --delete can delete it. Track and verify account ownership, and fail closed when ownership is uncertain.

The earlier review covered an older revision: build-swap.mjs is gone, and tagged screenshot filenames are now distinct. I inspected the current code but did not run it.

You are interacting with an AI system.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

AI assisted This PR contains AI-assisted commits

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants