Repository navigation
feat: accept or reject screenshot diffs from a browser page #313
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
e202360
cad1db7
0f9b141
d98318e
14f7a2f
e56aea4
552eb0a
6584b68
73023a4
f797a96
32d8872
d19dbf8
6f770ed
c9a3aaf
15ad84b
5d38a7e
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Large diffs are not rendered by default.
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,34 @@ | ||
| --- | ||
| name: 'Accept SnapDiff screenshots' | ||
| description: >- | ||
| Commit screenshot baselines accepted in a PR comment or a new issue | ||
| (`/snap-diff accept run=<run id> [pr=<n>] <file>@<hash> ...`) from that run's report artifact. | ||
| Run it from an issue_comment + issues workflow gated to OWNER/MEMBER/COLLABORATOR. | ||
| inputs: | ||
| token: | ||
| description: 'Token with contents: write, pull-requests: write, issues: write, actions: read (actions: write to dispatch ci-workflow); reads the author''s permission' | ||
| default: ${{ github.token }} | ||
| screenshots-path: | ||
| description: 'Every accepted file must be committed under this directory (relative to the repository root)' | ||
| default: 'doc/screenshots' | ||
| ci-workflow: | ||
| description: >- | ||
| Optional workflow file name (with workflow_dispatch) to run on the PR branch after the commit. | ||
| A push made with GITHUB_TOKEN does not start workflows; a workflow_dispatch does. | ||
| default: '' | ||
| runs: | ||
| using: 'composite' | ||
| steps: | ||
| - name: Accept screenshots | ||
| shell: bash | ||
| env: | ||
| GH_TOKEN: ${{ inputs.token }} | ||
| CI_WORKFLOW: ${{ inputs.ci-workflow }} | ||
| SCREENSHOTS_PATH: ${{ inputs.screenshots-path }} | ||
| EVENT_NAME: ${{ github.event_name }} | ||
| ISSUE_NUMBER: ${{ github.event.issue.number }} | ||
| # issue_comment: the comment on the PR; issues: the new issue itself. | ||
| BODY: ${{ github.event_name == 'issues' && github.event.issue.body || github.event.comment.body }} | ||
| AUTHOR: ${{ github.event_name == 'issues' && github.event.issue.user.login || github.event.comment.user.login }} | ||
| AUTHOR_ASSOCIATION: ${{ github.event_name == 'issues' && github.event.issue.author_association || github.event.comment.author_association }} | ||
| run: ruby "$GITHUB_ACTION_PATH/accept_screenshots.rb" |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,46 @@ | ||
| --- | ||
| name: Accept screenshots | ||
|
|
||
| # The report's "Commit decisions" button opens a new issue whose body is the | ||
| # review command (or a maintainer pastes it as a PR comment): | ||
| # /snap-diff accept run=<run id> [pr=<n>] <file>@<hash> ... | ||
| # The action commits those files from that run's report artifact to the PR branch. | ||
|
|
||
| on: | ||
| issue_comment: | ||
| types: [ created ] | ||
| issues: | ||
| types: [ opened ] | ||
|
|
||
| jobs: | ||
| accept: | ||
| if: >- | ||
| (github.event_name == 'issue_comment' && github.event.issue.pull_request && | ||
| startsWith(github.event.comment.body, '/snap-diff ') && | ||
| contains(fromJSON('["OWNER","MEMBER","COLLABORATOR"]'), github.event.comment.author_association)) || | ||
| (github.event_name == 'issues' && | ||
| startsWith(github.event.issue.body, '/snap-diff ') && | ||
| contains(fromJSON('["OWNER","MEMBER","COLLABORATOR"]'), github.event.issue.author_association)) | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 10 | ||
| permissions: | ||
| contents: write | ||
| pull-requests: write | ||
| issues: write | ||
| actions: write | ||
| # Per PR for comments, per issue for issues (the PR is not known yet); | ||
| # two issues for one PR may run at once, and the push lease lets only one land. | ||
| concurrency: | ||
| group: snap-diff-accept-${{ github.event.issue.number }} | ||
| cancel-in-progress: false | ||
| steps: | ||
| # The default branch (the ref of both events): this repository's trusted | ||
| # code, needed for the local action. The PR's code is never checked out here. | ||
| - uses: actions/checkout@v7 | ||
| with: | ||
| persist-credentials: false | ||
|
|
||
| - uses: ./.github/actions/accept-screenshots | ||
| with: | ||
| screenshots-path: test/fixtures/app/doc/screenshots | ||
| ci-workflow: test.yml |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -20,6 +20,7 @@ | |
| *.diff.png | ||
| *.base.png | ||
| snap_diff_report.html | ||
| snap_diff_manifest.json | ||
|
|
||
| *.attempt_*.webp | ||
| *.diff.webp | ||
|
|
||
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
|
|
@@ -283,4 +283,81 @@ Two things this minimal recipe does not do: | |||||
| 2. Enter the branch name (e.g. your PR branch) | ||||||
| 3. The workflow records new and changed baselines, commits, and pushes | ||||||
|
|
||||||
| ## Accept screenshots from the pull request | ||||||
|
|
||||||
| When a CI run fails on screenshot diffs, a reviewer can accept the new pixels without checking out the | ||||||
| branch. In the HTML report (the inline link in the PR comment), each changed or new screenshot has | ||||||
| Accept and Reject buttons (keys `Y` and `N`). "Commit decisions" opens a new GitHub issue already | ||||||
| filled in with a command like `/snap-diff accept run=123 pr=45 linux/cuprite/index.png@0123456789abcdef`. | ||||||
| Click Submit. A workflow commits those exact files from that run's report artifact to the PR branch | ||||||
| with the repository's own `GITHUB_TOKEN`, posts the result on the PR, closes the issue and re-runs CI. | ||||||
| The buttons appear only in reports built on GitHub Actions. | ||||||
|
|
||||||
| If the link would be too long for GitHub, the button is disabled. Use "Copy review command" and paste | ||||||
| the command as a comment on the pull request instead; it does the same thing. | ||||||
|
|
||||||
| To install it, copy this workflow. Set `screenshots-path` to your `save_path`. `ci-workflow` is | ||||||
| optional: it names your test workflow, which needs an `on: workflow_dispatch` trigger. Without it the | ||||||
| reply asks you to re-run CI by hand, because a push made with `GITHUB_TOKEN` does not start workflows. | ||||||
|
|
||||||
| ```yaml | ||||||
| # .github/workflows/snap-diff-accept.yml | ||||||
| name: Accept screenshots | ||||||
|
|
||||||
| on: | ||||||
| issue_comment: | ||||||
| types: [created] | ||||||
| issues: | ||||||
| types: [opened] | ||||||
|
|
||||||
| jobs: | ||||||
| accept: | ||||||
| if: >- | ||||||
| (github.event_name == 'issue_comment' && github.event.issue.pull_request && | ||||||
| startsWith(github.event.comment.body, '/snap-diff ') && | ||||||
| contains(fromJSON('["OWNER","MEMBER","COLLABORATOR"]'), github.event.comment.author_association)) || | ||||||
| (github.event_name == 'issues' && | ||||||
| startsWith(github.event.issue.body, '/snap-diff ') && | ||||||
| contains(fromJSON('["OWNER","MEMBER","COLLABORATOR"]'), github.event.issue.author_association)) | ||||||
| runs-on: ubuntu-latest | ||||||
| timeout-minutes: 10 | ||||||
| permissions: | ||||||
| contents: write | ||||||
| pull-requests: write | ||||||
| issues: write | ||||||
| actions: write # read is enough if you leave out ci-workflow | ||||||
| concurrency: | ||||||
| group: snap-diff-accept-${{ github.event.issue.number }} | ||||||
| cancel-in-progress: false | ||||||
| steps: | ||||||
| - uses: snap-diff/snap_diff-capybara/.github/actions/accept-screenshots@master | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔒 Security & Privacy | 🟠 Major | ⚡ Quick win 🔎 Supported by static analysis🏁 Script executed: sed -n '282,355p' docs/ci-integration.md
sed -n '1,38p' .github/actions/accept-screenshots/action.ymlRepository: snap-diff/snap_diff-capybara Length of output: 5234 🏁 Script executed: set -u
printf '%s\n' '--- pinning and acceptance references ---'
rg -n -i -F --glob '!vendor/**' --glob '!node_modules/**' -e 'immutable' -e 'commit SHA' -e 'pin' -e 'accept-screenshots' -e 'contents: write' -e 'issue_comment' docs .github README.md 2>/dev/null || test "$?" -eq 1
printf '%s\n' '--- reviewed head object ---'
if command -v gh >/dev/null 2>&1; then
gh api repos/snap-diff/snap_diff-capybara/commits/15ad84b70c1397f901a829c13e43f808309e0767 --jq '.sha'
else
printf '%s\n' 'gh unavailable'
fi
printf '%s\n' '--- local reviewed head object ---'
git cat-file -t 15ad84b70c1397f901a829c13e43f808309e0767 2>&1 || trueRepository: snap-diff/snap_diff-capybara Length of output: 7663 Pin the acceptance action to the reviewed commit. When a consumer copies this workflow, 🔧 Suggested fix--- "a/docs/ci-integration.md"
+++ "b/docs/ci-integration.md"
@@ -321,7 +321,7 @@
group: snap-diff-accept-${{ github.event.issue.number }}
cancel-in-progress: false
steps:
- - uses: snap-diff/snap_diff-capybara/.github/actions/accept-screenshots@master
+ - uses: snap-diff/snap_diff-capybara/.github/actions/accept-screenshots@15ad84b70c1397f901a829c13e43f808309e0767
with:
screenshots-path: doc/screenshots
ci-workflow: test.yml📝 Committable suggestion
Suggested change
🤖 Prompt for AI Agents |
||||||
| with: | ||||||
| screenshots-path: doc/screenshots | ||||||
| ci-workflow: test.yml | ||||||
| ``` | ||||||
|
|
||||||
| `issue_comment` and `issues` workflows run only from the default branch, so merge this file first. | ||||||
| Until then the issue or comment does nothing. | ||||||
|
|
||||||
| Reviewers need write access to the repository. The workflow checks the permission API as well as the | ||||||
| `author_association`, so read-only collaborators are refused, and their issue is closed with the reason. | ||||||
|
|
||||||
| Limits: | ||||||
|
|
||||||
| - Pull requests from the same repository only. A fork PR is refused. | ||||||
| - Git LFS paths are refused. Accept those locally. | ||||||
| - Screenshot paths may use only `A-Z a-z 0-9 . _ - /` (no spaces) and must sit under `screenshots-path`. | ||||||
| - The report artifact must still exist. `upload-screenshots` keeps it `retention-days` days (default 2). | ||||||
| - Only the PR's current head can be accepted. After a new push, review the new run's report. | ||||||
| - Only the last capture of a file can be accepted. Earlier captures of it show a note instead of the buttons. | ||||||
|
|
||||||
| Security: the workflow never checks out or runs the PR's code. It downloads the report artifact, takes | ||||||
| the paths from the report's manifest, and commits a file only if its SHA-256 matches the hash in the | ||||||
| reviewer's command and its bytes are a PNG or WebP. It pushes only to the PR branch, and only while | ||||||
| that branch is still at the commit the run tested. The design does rely on trust you already extend: | ||||||
| whoever opens a same-repository PR can already push to the repository, and their CI run produces the | ||||||
| artifact. Beyond that, the permission check and your branch protection limit the damage. Do not enable | ||||||
| this for forks, and do not give the bot a ruleset bypass: with branch protection in place, a push the | ||||||
| rules reject fails instead of landing. | ||||||
|
|
||||||
| [← Back to README](../README.md) | ||||||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Copying
save_pathintoscreenshots-pathdoes not always work. The action needs a path from the git root with no leading./.If an app lives in
apps/web,save_path: doc/screenshotsproduces manifest paths underapps/web/doc/screenshots. Following this instruction then makes every acceptance fail. A leading./, as used in this repository's test setup, is also refused.Document the git-root-relative path requirement and show a subdirectory example.
Prompt To Fix With AI
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!