Skip to content

feat(browser): add reset: :soft to keep the HTTP cache between tests - #1

Open
maxwell wants to merge 4 commits into
mainfrom
soft-reset
Open

maxwell wants to merge 4 commits into
mainfrom
soft-reset

Conversation

@maxwell

@maxwell maxwell commented Nov 15, 2026 •

Copy link
Copy Markdown
Member

Internal review, do not merge. This branch is offered upstream to rubycdp/cuprite once approved, using the description below. BackerKit's Gemfile already points at the soft-reset branch, so merging here gains nothing and would make the fork's main drift from upstream.

🤖 AI-generated. This note is internal: rubycdp/cuprite's AGENTS.md forbids AI attribution, so the description below leaves it out.


Refs rubycdp#128

This is the feat/soft-reset branch from this repo brought up to date with main, following the plan in rubycdp#128 (comment): reset: :soft is opt-in and the full reset stays the default.

The first commit is that branch's commit, with what it needed to run on current main folded in:

  • Ferrum::Contexts#each yields the id and the context as two values, so contexts.each.map(&:first) raised NoMethodError: undefined method 'first' for an instance of String on every soft reset. The fallback only rescues Ferrum::Error, so it propagated. The ids are now collected with map { |id, _| id } before the other contexts are disposed.
  • /cuprite/cacheable is back in the test app. a35c5c1 removed it, and "keeps the HTTP cache" visits it.
  • The IndexedDB example waits for indexedDB.open to succeed before listing databases. open is asynchronous, so the assertion before the reset failed on every run.
  • The examples moved from spec/features/soft_reset_spec.rb to describe "#reset!" in spec/driver_spec.rb, under context "with reset: :soft", to match the layout from test: restructure specs by public method rubycdp/cuprite#357.
  • The changelog entry is under Unreleased.

The second commit adds four examples for state that lives in the old window, so they pass because the soft reset closes it and opens a new one:

  • "drops cookies set by a request still in flight when the reset starts". Session#reset! resets the driver before it waits for pending server requests, so a response can set a cookie after the cookies are cleared. Closing the old window cancels the request. With the old window left open (no close_pages), the example fails: expected: 0, got: 10.
  • "stops scripts the previous page left running", "forgets a modal the previous test never asserted on", and "restores the window size and drops device and media emulation". All three fail when the soft reset reuses the current window instead of opening a new one.

The third commit fixes a leak a review found: the soft reset collected origins from the windows still open and their traffic, and close_window removes the target, so an origin a test opened in a window and closed before the reset kept its storage into the next test. close_window now remembers the window's origins and the next soft reset clears them. "clears storage of an origin whose window closed before the reset" fails without it: expected: nil, got: "left behind".

I first wrote a soft reset that reused the window, and the in-flight request case is what broke it: a session cookie from a request still running when the test ended survived into the next test. That's why I think the new window is the right design and worth pinning in specs.

bundle exec rake passes, and so does CUPRITE_RESET=soft bundle exec rake, which runs Capybara's shared specs with the soft reset. RuboCop is clean.

Benchmarks

Cuprite's own suite, bundle exec rake on one machine with nothing else running:

Default reset reset: :soft
Run 1 13:23 (1,825 examples) 12:00 (1,822 examples, before three of this PR's specs existed) (−10%)
Run 2 14:05 (1,825 examples) 12:21 (1,825 examples) (−12%)
Run 3, with the third commit 13:23 (1,826 examples) 12:18 (1,826 examples) (−8%)

The test app serves almost nothing cacheable, so this mostly measures skipping the browser context: disposing it and creating a new one on every reset.

A Rails app's system specs on CI (Semaphore, 18 jobs × 2 processes, about 31,200 examples), comparing two runs that differ only in the reset mode:

Default reset reset: :soft
Summed spec time 7,085s 6,473s (−9%)
Pipeline wall time 7:39 7:27

The wall time barely moves because it is mostly setup and the slowest job. Locally, a 113-file sample of the same app ran in 217s with reset: :soft and in 259s with my earlier tab-reusing version.

route and others added 2 commits November 15, 2026 07:45
The default reset disposes the browser context, so every test downloads all assets again. With reset: :soft
Cuprite keeps the context and opens a fresh blank window, closes the others and other contexts, clears cookies
and clears storage for every origin found in the network traffic. Any failure falls back to the full reset.
CUPRITE_RESET=soft runs the suite in soft mode.

Refs rubycdp#128

Co-authored-by: Maxwell Salzberg <maxwell@backerkit.com>
Capybara's Session#reset! resets the driver before it waits for pending server requests, so a response can set a
cookie after the soft reset clears them. Page timers, an unused modal response and window size, device and media
emulation also live in the old window.

Each example passes because the soft reset closes the old window and opens a new one. The in-flight request example
fails when the old window is left open, and the other three fail when the current window is reused.
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-15T17:06:23.749030Z c8f74ba PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c8f74ba4b6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread lib/capybara/cuprite/browser.rb Resolved
The soft reset collected origins from the windows still open and their network traffic. close_window removes the
target, so an origin a test opened in a window and closed before the reset kept its storage into the next test.
close_window now remembers the window's origins, and the next soft reset clears them.
Target.closeTarget returns before Chrome has torn the page down. A busy page kept running for a few milliseconds and
could set a cookie, and a response arriving in that window could store one, after the soft reset had cleared them.
The soft reset now waits until the closed targets leave Target.getTargets, up to the browser timeout, before it
clears cookies. A timeout falls back to the full reset like any other soft reset error.

"clears cookies only after the old windows are gone" failed in 4 of 8 runs under full CPU load without the wait and
passed 8 of 8 with it.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants