feat: SNAP_DIFF_ACCEPT_SCREENSHOTS selects record: :all per run, on CI too

CHANGELOG.md

@@ -20,6 +20,11 @@ kept as history. This entry is the one to read if you are coming from **1.15.1**
 
 ### Added
 
+- **`SNAP_DIFF_ACCEPT_SCREENSHOTS=1` re-records changed baselines for one run.** The variable
+  selects `record = :all` for the run that sees it, outranking `record =` and `fail_if_new`; a
+  `record:` on the screenshot still outranks it. `1` and `true` turn it on; any other value is the
+  same as not setting it. Unset, the mode reads as before (`:none` under CI, `:once` off it).
+
 - **AI triage (optional, offline-first, advisory by default).** `SnapDiff::Reporters::AISimple`
   classifies every failed comparison as `flaky` / `intentional` / `real_bug`,
   logs one line per diff, quotes the verdict in the failure message, and
@@ -341,6 +346,10 @@ One removal 2.0 cannot warn about, written down instead: the legacy `LOADED_DRIV
   rollback)
 
 ### Changed
+- **`SNAP_DIFF_ACCEPT_SCREENSHOTS=1` lets `record: :all` run under CI.** A job that re-records
+  baselines sets the variable and nothing else: no `CI: ""`, no `continue-on-error`, and a failing
+  unrelated test still fails the job. A `record = :all` that comes from code is still refused under
+  CI when the variable is not set. The update-baselines recipe in `docs/ci-integration.md` uses it.
 - **`record: :all` now compares before re-recording and the report shows what changed.**
   A screenshot that differs from its committed baseline is accepted as the new baseline
   (it never fails the run) and leaves the usual `.base.png` and diff/heatmap files,

README.md

@@ -199,30 +199,24 @@ diff next to the code that caused it.
 
 ### Accepting many at once
 
-One failure at a time is fine for one screenshot. After a redesign that changed forty, set the
-**record mode** to `:all` and re-record them in a single run:
-
-```ruby
-# test_helper.rb
-# Set the mode only when asked: an explicit mode, even :once, outranks the
-# :none-under-CI default and would stop CI from failing on a missing baseline.
-SnapDiff.config.record = :all unless ENV["ACCEPT_SCREENSHOTS"].to_s.empty?
-```
+One failure at a time is fine for one screenshot. After a redesign that changed forty, re-record
+them in a single run:
 
 ```bash
-ACCEPT_SCREENSHOTS=1 bin/rails test:system
+SNAP_DIFF_ACCEPT_SCREENSHOTS=1 bin/rails test:system
 # [snap_diff] 212 verified, 40 changed (40 re-recorded), 0 new (not verified).
 
 git status                     # forty modified baselines, and nothing else
 git add doc/screenshots/
 git commit -m "chore: re-record baselines after the checkout redesign"
 ```
 
-`:all` compares every capture to its committed baseline like `:once` does, but a screenshot that
-differs is accepted as the new baseline instead of failing the test — the report lists what changed,
-and `git status` is the review surface — look at the images before you commit. It **refuses to run under CI**,
-where there would be nobody to look. The other modes are `:once` (the default) and `:none`
-(strict — a missing baseline always fails); see
+The variable puts the run in the `:all` **record mode**: every screenshot is still compared to its
+committed baseline, but one that differs becomes the new baseline instead of failing the test. The
+report lists what changed and `git status` is the review surface — look at the images before you
+commit. To do the same on CI, set the variable on a dedicated job; see
+[CI integration](docs/ci-integration.md#update-baselines-in-ci). The other modes are `:once` (the
+default off CI) and `:none` (the default under CI; a missing baseline always fails); see
 [Record modes](docs/configuration.md#record-modes--accepting-changes).
 
 ## Web UI for Reviewing Screenshot Changes

docs/UPGRADING.md

@@ -339,12 +339,9 @@ process, from the point you set it:
 **Nothing changes if you set none of them.** With no `record` line, `record` reads back as `:none`
 under CI and `:once` off it — exactly what `fail_if_new` already did, sniff and all. The
 missing-baseline default is deliberately unchanged; `:none` is how you opt into strictness
-explicitly. **Precedence:** an explicitly set mode outranks `fail_if_new`, and `fail_if_new`
-decides only when no mode was set — the same rule `fail_if_new` itself has over the `CI` sniff.
-
-`record = :all` **refuses to run under CI**: it accepts every difference by design, so left in a
-committed config file it would be a build that never fails. See
-[Record modes](configuration.md#record-modes--accepting-changes).
+explicitly. An explicitly set mode outranks `fail_if_new`, and `SNAP_DIFF_ACCEPT_SCREENSHOTS=1`
+outranks both for the run that sets it; the full chain is under
+[Precedence](configuration.md#precedence).
 
 #### Silent by design
 

docs/ci-integration.md

@@ -243,9 +243,8 @@ jobs:
 
       - name: Record baselines
         run: bin/rails test:system
-        continue-on-error: true    # the run fails by design; it rewrites the baselines
         env:
-          CI: ""                   # see below — without this, a NEW screenshot is never written
+          SNAP_DIFF_ACCEPT_SCREENSHOTS: "1"    # record: :all for this job only
 
       - name: Commit updated baselines
         run: |
@@ -258,23 +257,26 @@ jobs:
 
 </details>
 
-> **Set the record mode, or clear `CI`, to let this job record a *new* baseline.** With
-> nothing set, a missing baseline fails whenever `ENV["CI"]` is set and non-empty — so on a
-> stock GitHub Actions runner a screenshot with no committed baseline raises `No existing
-> screenshot found for …`. `SnapDiff.config.record = :once`
-> (or clearing `CI` for this one step, or the older `SnapDiff.config.fail_if_new = false`)
-> records new baselines instead. *Changed* baselines are rewritten either way; only new ones
-> need this. See [Record modes](configuration.md#record-modes--accepting-changes).
->
-> The screenshot itself **is** written before the raise ([`screenshot_matcher.rb`](https://github.com/snap-diff/snap_diff-capybara/blob/master/lib/snap_diff/screenshot_matcher.rb)
-> — `capture_screenshot` precedes `fail_if_new_screenshot`), so the `git add` the message names
-> is a command you can actually run. But the raise still fails the test it happened in, and a red
-> job usually never reaches the commit step — which is why a recording job sets the mode rather
-> than relying on the files being there.
->
-> **`record = :all` is not the mode for this job** — it refuses to run under CI, because it
-> would accept every *changed* rendering unreviewed and never fail. `:once` records what is new and
-> keeps comparing everything that already has a baseline.
+`SNAP_DIFF_ACCEPT_SCREENSHOTS` puts the run in the `:all` record mode: every screenshot is
+compared to its committed baseline, one that differs becomes the new baseline, one with no
+baseline yet is recorded, and the run exits green with a summary of what it re-recorded
+(`3 changed (3 re-recorded)`). A failing test that has nothing to do with screenshots still fails
+the job. See [Record modes](configuration.md#record-modes--accepting-changes) for the mode itself.
+
+Set the variable on this job only. The gem cannot tell a variable set on one job from one exported
+to every job, and on a verification job it would accept every change. Do not put
+`record = :all` in `test_helper.rb` for the same reason; under CI the gem refuses a `:all` that
+comes from code.
+
+Record in the same environment that verifies (OS, browser, fonts). A baseline recorded elsewhere
+is often worthless, which is why this job runs on CI at all.
+
+Two things this minimal recipe does not do:
+
+- It keeps the committed PNGs, not the HTML report or the diff images. Add an
+  [upload step](#2-reusable-composite-action-recommended) without an `if: failure()` guard if you want them.
+- A push made with the default `GITHUB_TOKEN` does not start the ordinary verification
+  workflow. Review the images in the pushed commit, then run the normal check against it.
 
 **How it works:**
 1. Go to Actions → "Update Screenshot Baselines" → "Run workflow"

docs/configuration.md

@@ -68,20 +68,18 @@ SnapDiff.config.record = :once   # what a local run already does; see Precedence
 ### `:all` — the bulk-accept verb
 
 After an intentional redesign that changed forty screenshots, re-record them in one run rather
-than accepting them one failure at a time:
-
-```ruby
-# test_helper.rb. There is no CLI flag — the gem has no runner to hang one on —
-# so gate it on an environment variable of your own if you want one. Set the
-# mode only when asked: an explicit mode, even :once, outranks the
-# :none-under-CI default and would stop CI from failing on a missing baseline.
-SnapDiff.config.record = :all unless ENV["ACCEPT_SCREENSHOTS"].to_s.empty?
-```
+than accepting them one failure at a time. Set `SNAP_DIFF_ACCEPT_SCREENSHOTS` for that run:
 
 ```bash
-ACCEPT_SCREENSHOTS=1 bin/rails test:system     # with the line above in test_helper.rb
+SNAP_DIFF_ACCEPT_SCREENSHOTS=1 bin/rails test:system
 ```
 
+The variable selects `:all` for the run that sees it. `1` and `true` (any case) turn it on; any
+other value, including `0`, `false` and empty, is the same as not setting it. It wins over
+`record =` and `fail_if_new` (see Precedence below), so a suite that pins `record = :none` accepts
+with the same command. Do not write `record = :all` in `test_helper.rb` instead: a mode written
+there applies to every run, including the ones that are supposed to fail.
+
 Every screenshot is compared to its committed baseline, as under `:once`. One that differs is kept
 as the new baseline instead of failing the test, and leaves the usual `.base.png` and diff files
 (ignored by the recommended `.gitignore`); one that matches changes nothing. So `git status` lists
@@ -95,14 +93,14 @@ At the end of the run the gem names what it accepted:
 [snap_diff] record: :all compared 3 screenshots, found them different and accepted them as the new baseline: checkout/cart, checkout/payment, checkout/review. `git add` the file(s) to keep the change, `git restore -- <path>` to discard it. The `.base.<fmt>` and diff files left beside each one are covered by the recommended .gitignore.
 ```
 
-> **`:all` refuses to run under CI** (when `ENV['CI']` is set to a non-empty value). It accepts
-> every difference by design, so a mode left in a committed config file would buy you a build that
-> never fails, with the "recorded" screenshots discarded when the runner
-> is torn down. Re-record locally, where you can look at the result.
+> **Under CI (`ENV['CI']` non-empty), `:all` runs only when `SNAP_DIFF_ACCEPT_SCREENSHOTS` is
+> set.** When the variable is not set, a `:all` written in code (a config line or a per-screenshot
+> option) raises there, because a committed `:all` would make every build pass. When it is set, the
+> whole run records, whatever the code says. To re-record baselines on CI, set
+> the variable on a dedicated job: see [Update baselines in CI](ci-integration.md#update-baselines-in-ci).
 >
-> A CI job that needs to record screenshots with **no baseline yet** does not need `:all`:
-> `record = :once` records those and still compares everything that has a baseline. See
-> [CI integration](ci-integration.md).
+> A CI job that only needs to record screenshots with **no baseline yet** does not need `:all`:
+> `record = :once` records those and still compares everything that has a baseline.
 
 ### Per screenshot
 
@@ -112,9 +110,16 @@ assert_matches_screenshot "flaky_widget", record: :none   # this one must alread
 
 ### Precedence
 
-**An explicitly set mode outranks `fail_if_new`; `fail_if_new` decides only when no mode was set.**
-That is the same rule `fail_if_new` itself has over the `CI` sniff — explicit outranks implicit, all
-the way down. Per screenshot outranks the config; the config outranks `fail_if_new`.
+First match wins:
+
+1. `record:` on the screenshot
+2. `SNAP_DIFF_ACCEPT_SCREENSHOTS=1`, which means `:all`
+3. `SnapDiff.config.record =`
+4. `fail_if_new`, when set
+5. the default: `:none` under CI, `:once` off it
+
+A setting written for one screenshot beats the run's variable, the run's variable beats the
+suite's configuration, and an explicitly set mode beats the legacy `fail_if_new` boolean.
 
 With **nothing** set, `record` reads back as `:none` under CI and `:once` off it — which is exactly
 what `fail_if_new` has always done, so a suite that never mentions `record` behaves as it always
@@ -126,9 +131,11 @@ laptop is often worthless on another OS. `:none` makes strictness an explicit ch
 |-----------|----------------|------------------|
 | nothing, off CI | `:once` | recorded |
 | nothing, under CI | `:none` | fails |
+| nothing, `SNAP_DIFF_ACCEPT_SCREENSHOTS=1` | `:all` | recorded; a changed baseline is re-recorded |
 | `record = :once` | `:once` | recorded, on CI too |
 | `record = :none` | `:none` | fails, off CI too |
 | `record = :once`, `fail_if_new = true` | `:once` | recorded — the mode wins |
+| `record = :none`, `SNAP_DIFF_ACCEPT_SCREENSHOTS=1` | `:all` | recorded — the variable wins |
 | `record = nil`, `fail_if_new = true` | `:none` | fails — nil hands it back |
 
 A misspelt mode raises `ArgumentError` at the point you set it, rather than reading back as

lib/snap_diff/config.rb

@@ -107,6 +107,11 @@ def self.ci?
       !ENV["CI"].to_s.empty?
     end
 
+    # The per-run spelling of <tt>record = :all</tt>; see {#record}.
+    def self.accept_screenshots?
+      Removal.truthy_env?("SNAP_DIFF_ACCEPT_SCREENSHOTS")
+    end
+
     # shift_distance_limit, driver and the three older new-screenshot
     # booleans are excluded from the generated writers and hand written
     # below (they announce their 2.1 removal, or validate); fail_if_new and
@@ -200,13 +205,17 @@ def fail_on_difference=(value)
     # +:all+::  +:once+ that accepts instead of fails. Every baseline is
     #           compared; a screenshot that differs is kept as the new
     #           baseline and never raises -- the bulk-accept verb, for the
-    #           redesign that changed forty screenshots at once. Refused
-    #           under CI (see {SnapDiff::ScreenshotMatcher}).
+    #           redesign that changed forty screenshots at once. Written in
+    #           code, it is refused under CI; the variable below is how a
+    #           CI job records (see {SnapDiff::ScreenshotMatcher}).
     #
-    # PRECEDENCE: an explicitly set mode outranks +fail_if_new+; nil means
-    # nobody said, and only then does +fail_if_new+ answer -- so a setup
-    # with no +record+ line behaves EXACTLY as it did before this setting
-    # existed, CI sniff and all. +:none+ is the mode spelling of
+    # PRECEDENCE: the per-screenshot +record:+ option outranks everything.
+    # Then <tt>SNAP_DIFF_ACCEPT_SCREENSHOTS=1</tt> selects +:all+ for this one
+    # run: the bulk-accept verb is a per-run decision, so the run's own
+    # intent beats a mode written into test_helper.rb, which lasts forever.
+    # Then an explicit +record =+, then +fail_if_new+ -- so a setup with no
+    # variable behaves EXACTLY as it did before this setting existed, CI
+    # sniff and all. +:none+ is the mode spelling of
     # <tt>fail_if_new = true</tt> and +:once+ of <tt>= false</tt>, which is
     # why the fallback can express today's behaviour without a special case
     # anywhere downstream: the matcher branches on the mode alone.
@@ -217,6 +226,8 @@ def fail_on_difference=(value)
     # worthless across OS. +:none+ makes strictness an explicit choice
     # instead; that is the whole point of having the mode.
     def record
+      return :all if Config.accept_screenshots?
+
       @record || (fail_if_new ? :none : :once)
     end
 

lib/snap_diff/removal.rb

@@ -84,6 +84,16 @@ def self.unknown_option(key)
         "ArgumentError for it. Check the spelling against the option list in docs/configuration.md."
     end
 
+    # Is the environment variable +name+ switched on? Truthy is "1" or "true"
+    # (any case); anything else, empty included, is off. The one rule for the
+    # gem's own on/off switches, so a value that stops failures is never
+    # enabled by "0" or "false" (+CI+ keeps its non-empty rule: it is not
+    # ours, and "false" is how some runners spell it). Lives here because this
+    # file loads under every entry point, Config included.
+    def self.truthy_env?(name)
+      %w[1 true].include?(ENV[name].to_s.downcase)
+    end
+
     MUTEX = Mutex.new
     @seen = {}
     @suppressed = false
@@ -147,13 +157,7 @@ class << self
     #   either via the {silence_deprecations} accessor or the
     #   SNAP_DIFF_SILENCE_DEPRECATIONS env var (truthy = "1"/"true").
     def silence_deprecations?
-      !!silence_deprecations || truthy_env?(ENV["SNAP_DIFF_SILENCE_DEPRECATIONS"])
-    end
-
-    private
-
-    def truthy_env?(value)
-      %w[1 true].include?(value.to_s.downcase)
+      !!silence_deprecations || Removal.truthy_env?("SNAP_DIFF_SILENCE_DEPRECATIONS")
     end
   end
 end

lib/snap_diff/screenshot_matcher.rb

@@ -74,23 +74,27 @@ def resolve_record_mode(per_screenshot)
     end
 
     # `:all` accepts every differing rendering as the new baseline -- that is
-    # the feature. On CI there is nobody to review the result and the
-    # re-recorded files go away with the box, so a mode left in a committed
-    # config file buys a build that never fails, forever. That is precisely
-    # how Percy goes green on a job that lost its token; refuse it rather
-    # than shipping our own version.
+    # the feature. A mode written in code (a config line, a per-screenshot
+    # option) applies to every run, so left on CI it buys a build that never
+    # fails, forever. That is precisely how Percy goes green on a job that
+    # lost its token; refuse it rather than shipping our own version.
     #
-    # A CI job that must record NEW baselines does not need `:all` at all:
-    # `record: :once` records them and still compares everything that has a
-    # baseline (docs/ci-integration.md).
+    # The per-run variable is the operator asking for this run to record, so
+    # a deliberate recording job (docs/ci-integration.md) gets through. With
+    # it set, the whole run is `:all` already; where the mode was written no
+    # longer matters, so the guard looks only at the variable.
+    #
+    # A CI job that must record only NEW baselines does not need `:all`:
+    # `record: :once` records them and still fails on everything that differs.
     def refuse_bulk_record_under_ci!
       return unless SnapDiff::Config.ci?
+      return if SnapDiff::Config.accept_screenshots?
 
       raise SnapDiff::ExpectationNotMet.new(<<~ERROR.chomp, caller)
-        `record: :all` re-records every screenshot that differs from its baseline without failing, so it refuses to run under CI (ENV["CI"] is set).
-        No difference could fail the build and the re-recorded screenshots would be discarded with the runner.
-        Re-record locally, review the result, and commit the screenshots.
-        To let a CI job record screenshots that have no baseline yet: SnapDiff.config.record = :once
+        `record: :all` re-records every screenshot that differs from its baseline without failing, and it was set in code, so it refuses to run under CI (ENV["CI"] is set).
+        Left in committed config, no difference could fail this build, ever.
+        To record on CI for one run: SNAP_DIFF_ACCEPT_SCREENSHOTS=1
+        To record only the screenshots that have no baseline yet: SnapDiff.config.record = :once
       ERROR
     end
 

test/fixtures/summary_line_case.rb

@@ -12,6 +12,7 @@
 #   SNAP_IMAGES -- directory holding the fixture PNGs a.png / b.png
 #   SNAP_CASES  -- comma-separated subset of verified,changed,new (may be empty)
 #   SNAP_RECORD -- optional record mode for the whole run (once, none, all)
+#   SNAP_FAIL_ONE -- when set, adds one unrelated test that fails
 require "minitest/autorun"
 require "snap_diff/integrations/minitest"
 # The HTML report is a FEATURE and stays opt-in; the summary line is core
@@ -69,4 +70,10 @@ class SummaryLineCase < Minitest::Test
       end
     end
   end
+
+  if ENV["SNAP_FAIL_ONE"]
+    def test_unrelated
+      assert false, "unrelated failure"
+    end
+  end
 end