Repository navigation
feat(playwright): seed each test's random numbers; run each test once - #127
Merged
Merged
Conversation
keploy runs each test once in a run and charges a test's calls to its one window. A Playwright retry (from --retries, the config or test.describe.configure) or a --repeat-each repeat began the test a second time. Such a run now fails before its begin, saying why and how to turn it off, and keploy is told (POST /agent/refusal), so its verdict on the run names the test. The message names the test, not the attempt, so the agent keeps one refusal per test however often Playwright runs it again. keploy's reason for skipping a test was only in the skip annotation, which Playwright's list, line and dot reporters do not print. The test's output now shows it as well. A begin, an end or a refusal answered with a plain 404 (keploy running without KEPLOY_PROVEN=1) said only that the agent answered 404. It now names the switch, as the claim's 404 already did. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
Signed-off-by: slayerjain <shubhamkjain@outlook.com>
A page that draws random numbers can make other calls each run, and no recording answers a call it never saw: stripe.js flips a 50/50 A/B coin from crypto.randomUUID() at each page load, and its two sides load files that never overlap, so a test on such a page fails about half its replays. An ID or idempotency key the browser makes differs too. Under keploy, each test's begin now sends `random: "seeded"` and a seed drawn afresh (128 bits as hex); keploy answers the seed to draw from: in a record the begin's own, which it keeps with the test's recording, in a replay the recording's. The test's context gets an init script, before its first page, that puts seeded streams behind Math.random, crypto.getRandomValues and crypto.randomUUID in every frame before any script of the page runs. Each function has its own stream, and each document its own, named by its page (in the order the test opened its pages; a popup by the window its opener opened), its frame (by its element's path) and how many documents of that path the tab loaded before: a reload draws new numbers, the same ones each run. The end sends what the test did (`seeded` or `own`). A document the seeding cannot place (a popup opened with noopener or at another origin, a frame that cannot count its documents, a top document with no session storage) draws the browser's own numbers, never another page's, and the test names it once per address. A begin answered with no seed (an older recording or an older keploy) draws the browser's own numbers and says so. On by default. KEPLOY_SEED_RANDOM=off turns it off for the run, and the option keploySeedRandom: false for a project, a file or a test; the begin then sends `random: "own"` and no seed. Without keploy nothing changes. Web Workers and Service Workers keep the browser's own numbers (README limits). No row checks a retry's random numbers: a retry fails before its begin, since keploy runs each test once in a run. The rows run in Chromium, Firefox and WebKit, so CI installs all three. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
Firefox can report a call the new document makes as it loads (a script or an image its parser asks for) before it reports the frame's commit. Settle dropped every call of the frame at the commit, that one included, so a call still in flight at the test's end was not waited for, and the context's close cut it. Each call now gets a number in the order the browser reports it, and a frame's navigation answer keeps the number of the last call reported before it. The commit drops only that frame's calls numbered up to it: the new document can make no call before its answer. A redirect's answer is not the document's; the final answer's number replaces it. A limit: a call the old document starts after its replacement's answer (an unload handler's) is waited for too, until the test's timeout if the browser never ends it. It fails toward waiting, never toward a cut call. Rows: a unit row feeds settle the order a Firefox run reported, which fails before this change; a browser row, in all three engines, ends a test with its new document's image in flight. The 300-frames row's nested case now expects what every engine counts: the srcdoc frame inside each sandboxed frame counts no frame beside that sandboxed frame. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
A suite that makes its test data in the Node worker that runs the test (a lead named with Math.random()) made other data on each replay, while a mocked dependency answered with the recording's, so the test's own assertion failed. Only the test browser was seeded. Under keploy, the test process's Math.random now draws from the seed keploy answered the test's begin with: the one a record keeps with the test's recording and a replay answers back. It has a stream of its own (sfc32 over the SHA-256 of the seed and the stream's name, pinned by a row to a reference made outside the package), so it does not move the browser's. It is seated from the begin to the end: the test's beforeEach and afterEach hooks and its fixtures draw from it too. At the end the worker's own Math.random is back, unless the test put its own stub there, which it keeps. Outside every test (beforeAll, afterAll, a file as it loads) the worker draws its own, as does its crypto. What a worker does once, in whichever test first needs it, is kept out of that test's stream, so a test draws the same numbers whichever tests its worker ran before it: replayed alone, after keploy set an earlier test aside, or in a new worker after a test failed. - Playwright parses a file's source map the first time a test's stack reaches that file, and source-map's sort draws Math.random. While a test is seeded, Error.prepareStackTrace holds a wrapper of the function last set there, and a draw made inside it is the worker's own. At the test's end the property is a plain one again, holding the function last set, unless the test defined it itself. - A worker fixture registered with this package's test.extend or test.use, or those of a test type extend makes from it, draws the worker's own Math.random from its start until it calls use(), or fails before. A test fixture runs as written, and Playwright still names a fixture at the spec's line that called extend or use. A worker fixture registered through @playwright/test's own test or a test type mergeTests returned, and a library that draws on its first use inside a test, are not covered: the README says so. The seeding line is now said at the first seeded begin, not the first seeded context, so a worker whose tests open no page says it too. The line for a begin keploy answered with no seed names the test process. With seeding off (KEPLOY_SEED_RANDOM=off, or keploySeedRandom: false), the worker says once what that costs. The README states that the test process's clock is keploy's run clock, not a seeded one, and that one stream serves all of a test's draws in the test process. Rows: a test's draws in its beforeEach, body and afterEach are its recording's at replay, and another test's, an opted-out test's and code outside tests are not; no seed answered, or seeding off, each saying so; tests in one worker whose helper file calls test.step or reads an Error's stack, replayed alone, with the first set aside, and after a record whose first test failed; a worker fixture set up in the first test that asks for it, through extend and through use, replayed alone and with the first set aside, named at the spec's line; the worker's own Math.random back after a test that throws, one that times out, and one that defined Error.prepareStackTrace itself. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
A document's kind is top, frame or other; whyOf returns for top and other before its last test, so `s.kind !== 'frame'` was never true there. The reason a document gives is unchanged. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
No row reached a frame of its parent's origin inside a frame that cannot count its loads, so the fixture's count of the frames beside the uncounted one could have counted the inner frame's instead and every row passed. many.spec.ts's new test: a cross-site frame fills its origin's session storage in the tab, then the page puts in a second frame of that origin, which cannot count its load, holding a frame of its own origin. Both lines must name one other frame (the first, beside the uncounted one under the page), not zero (the inner frame is alone in its parent), in every engine, at record and at replay. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
A document's Math.random stream is now pinned to a reference made outside the package (sfc32 over a 128-bit hash of the seed, page, path and load, 15 draws dropped, 53 bits a draw), as the test process's stream already was: a recording keeps the seed, not the numbers, so a change to the stream would make every recording's data anew. The rows before it compared the browser's draws only with the same code's. The row for junk in a tab's load count now also checks the junk is replaced by a count (and adds a bare number), so the tab's next document at the path draws as its second load. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
No row gave keploy's context fixture a context that already held a page, so dropping the check that counts such a page left every row passing, while a test that drove only that page would end with no word on what its browser drew. opened.spec.ts merges a suite fixture that opens a page under keploy's and drives that page alone: its end must say seeded, and own with KEPLOY_SEED_RANDOM=off. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
A sandboxed frame's line is said when its parent's parser ends, when the parent is removed, or at the test's end. No row told those apart from a line said only once the context closed: dropping the flush at the test's end, or letting the parent's removal skip the line, left every row passing. The rows' reporter now also logs each write to a test's stdout among its steps, in order. The unparsed and unloaded rows check that the line comes before the test's wait for its calls in flight. A new test, removed, puts u0 and u1 in a frame whose parser never ends and removes that frame: its line must name no other frame and come before the test's next step. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
With no Error.prepareStackTrace before the seat, the seat's accessor is enumerable, as an assignment would make the property. No row read that, so making it non-enumerable left every row passing, and an app that sets the formatter and lists Error's keys would miss it during a test. The prepareStackTrace row now checks that the formatter a test sets while seated is among Error's own keys. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
Two rules no row read: - A test that takes the browser context and opens no page drew nothing in its browser, so its end says neither seeded nor own. Counting a page for every context left every row passing. opened.spec.ts gains a test that takes the context and opens no page; its end must carry no random, seeding on or off. - A document that drew before its page was named (the test's own context init script draws first) draws the browser's own numbers and is named once as drawing them; it keeps drawing its own. Working out the page again at each draw left every row passing, while the document's later draws turned seeded. The ownctxinit test now also draws after the page's scripts, and a replay must draw anew there. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
placePages promises a page's first document is named, and marked as the page's first, before newPage returns. The rows before used a page whose evaluate ran at once, so a newPage that did not wait for that script passed them. The new row holds the first document's script until it says, and checks newPage has not come; then that a first document that cannot be named (its page closed) fails newPage instead of handing the test a page whose numbers are not its own. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
The build moves to TypeScript 7.0, and a lint step runs oxlint with its type-aware rules (tsgolint, TypeScript 7's checker) on src in CI, on every Node leg. The rows run on Playwright 1.65.0; peerDependencies stay at @playwright/test >=1.40.0, and @types/node is Node 20's, the oldest Node the package supports. TypeScript 7 no longer loads @types packages unless tsconfig names them (types: ["node"]). Its declaration of test named playwright, the package that @playwright/test re-exports, which pnpm does not let a project see. The type of test is now written out from @playwright/test, and with it TypeScript 7's output, JavaScript and declarations, is the same as 5.9's, byte for byte. A new row fails when a declaration names a package that is not one of the peerDependencies. Lint: two copies of a collection made only to iterate it are dropped. Two lines carry a reason to keep a flagged form: a regex that replaces control characters, and an agent's version printed as it sent it. .oxlintrc.json turns off three rules that Playwright's own idioms trip: an empty fixture pattern, void on a fixture asked for its order, and methods kept to call later with their receiver. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
Two promises of the seeding had no row that failed without them. newPage returns only once the init script that names the page's later documents is in place: the test's next goto must not race it, and a page that refuses the script (it was closed) fails newPage instead of handing the test a page whose later documents draw the browser's own numbers. The new row holds addInitScript until it says and checks newPage has not come, then rejects it and checks newPage fails. A document's page is decided once, when first asked. A page's first document that a test's context.addInitScript made draw before keploy named it draws the browser's own numbers; its key, which its frames and the popups it opens are placed by, stays none after the name and the mark. The new row checks that key, and that a first document named before it drew keeps the same key each time it is asked. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
The seeding puts its own Math.random, getRandomValues and randomUUID in place with the flags the original had. Nothing checked that: a seeded page whose code lists Math's keys (Object.keys, a for-in, a spread) would see random there if the seeding made it enumerable. The first row reads each function's flags in a document with no seeding and in a seeded one, checks the seeded document has its own functions with the same flags, and that Object.keys(Math) is empty. The second row covers the two other cases the seeding handles: a Math.random that a script before it made read-only stays read-only, and a method that Crypto.prototype only inherits gets an own one, writable and not enumerable. Both draw the same seeded numbers as a plain seeded document. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
@faker-js/faker's instances (faker, the locales' fakerXX, simpleFaker) seed their own generators from Math.random as the package loads, before any test, so seeding Math.random never reached them: every name, email and ID a suite made with faker differed at replay while a mocked dependency answered with the recording's. Each test's seat now also puts each instance's draws (faker 10's fakerCore.randomizer, faker 8 and 9's _randomizer) on a stream of the test's seed named by its export, so a replay makes the data its recording made and one instance's draws move neither another's nor Math.random's. The instances are those of the module the test file's own import loads, resolved from the test file: faker 10 is one ES module for CommonJS and ES module test files alike (require(esm)), and faker 9's ES module test files get its ES module copy. They are found, and the module loaded, before the seat, so faker's own draws as it loads are the worker's. While a worker fixture is set up, and once a test seeds an instance itself, that instance draws its own; at the test's end each instance's own generator is back, unmoved. A faker that is installed where the tests import it but cannot be loaded is named once per worker, with the reason and the next step. The seeded line and the README say so; faker 10.6.0 is a dev dependency for the rows. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
A faker instance the suite makes itself as a test file loads (new Faker(...), new SimpleFaker(...)) seeds its own generator from the worker's Math.random then, before any test, so it made other names, emails and IDs at replay than in its recording, and nothing said how to fix it. seedFaker(instance, name), exported from @keploy/playwright, seeds such an instance as the prebuilt ones are seeded: from each test's begin to its end, on a stream of the test's seed named by the name, and returns the instance. One given as a file loads, in beforeAll or while a worker fixture is set up is seeded in every test after; one given in a test until that test's end. An instance already seated (a prebuilt one) keeps its stream. It throws, naming the next step, for a value that is no faker 8, 9 or 10 instance or an empty name. The worker's seeded line now says: own Faker instances draw their own, seed them with seedFaker(instance, name). The README shows the call. Rows: the unit rows of seedFaker's streams, lifetimes and refusals, and in a CommonJS and an ES module test file an instance given at load makes the recording's data at replay, one made in a test does too, and one not given makes its own. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
A test's pages were seeded, but a worker they started drew the browser's own random numbers, unnamed, so its draws could differ at replay. The page's Worker and SharedWorker now add a __keploy_worker mark to the script URL's query: a Web Worker's names its document, its script's path and how many workers the document started at that path; a Shared Worker's names its script's path and name only, so the test's pages still share it. A route of the test's context takes the mark off before the request leaves the browser and answers with the worker's seeding, then the app's own script as its server answered it (importScripts for a classic worker, an import for a module worker). In the worker, location shows no mark and scripts beside it load as before. Each start of a Shared Worker is the next instance. A worker keploy cannot mark (a blob: or data: URL, or a URL refused with the mark) and, in Firefox, whose route does not reach it, a Shared Worker, start as the app asked and are named once as the test runs, with the next step. Rows: the mark, the page's wrappers, the worker's seeding and the route, and in Chromium, Firefox and WebKit a classic and a module worker and a Shared Worker drawing their recording's numbers at replay. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
Chromium and WebKit let a context route reach none of what a Shared Worker loads after its script, so the importScripts or import of the app's own script that a Web Worker's answer makes reached the app's server with keploy's mark in its URL, and a module Shared Worker imported the app's script where its seeding should be. Chromium also reports a Shared Worker's script request but never its end (Playwright does not follow Shared Workers), so settle waited on it until the test timed out. A Shared Worker's script is now answered whole: a classic one with its seeding on the script's first line (after a hashbang, under the script's own 'use strict'), a module one with its seeding imported as a data: module ahead of its own imports. A module one whose policy refuses data: scripts gets the app's own script and the run says so. Each request the route fulfills counts as ended for settle. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
The browser row for a page's workers now also starts module Shared Workers in every engine: one plain, which draws from the test's seed in Chromium and WebKit (its own in Firefox, named), and one served with a policy that refuses data: scripts, started twice by two names, which draws its own and is named once. The row also checks that the app's server is never asked for a URL carrying keploy's mark, at record and at replay, and that it was asked for each worker's script and what it loads. cspAllowsData's row adds directive names in upper and mixed case, and strictPrologue's adds the next line's -, !=, instanceof (the expression goes on), --x and an identifier that starts with "in" (it does not), and a line or block comment that ends the directive's line. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
keploy's route answers a marked worker script with the worker's seeding, which then loads the app's own script from the URL the app asked for. The app's script so runs at its own URL: its import.meta.url, the URLs its imports resolve against and a stack trace through it show no mark, as recorded and as replayed. Before, a module worker's import.meta.url and a stack trace carried the mark, and a module Shared Worker whose policy refuses data: scripts ran unseeded at the marked URL, its location showing the mark. - A classic worker (Web or Shared) runs its seeding, then importScripts of the app's script; a module Web Worker imports its seeding, then the app's script; a module Shared Worker imports its seeding as a data: module, then the app's script. - Where a module Shared Worker's policy refuses data: scripts, the answer runs the seeding itself, then imports the app's script, and gives the connect events that came before that script ran to its handlers after, in order. That worker is now seeded, and its line is gone. - keploy's answer keeps the status and headers the app's server gives the script, so the server is asked for it twice: by keploy, and by the worker. When keploy cannot fetch it, the worker gets its seeding and loads the app's script itself. A route answered with a redirect to the unmarked URL is not an option: Chromium never follows a fulfilled 307 for a worker's script, and Playwright refuses to fulfill with a redirect in WebKit. The browser row checks import.meta.url and stacks, a second connection to a held Shared Worker, and that module Shared Worker seeded at replay. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
workerRoutes asked cspAllowsData only for a module Shared Worker, though workerAnswer reads dataOk for no other kind: the guard changed nothing. The module Shared Worker row now also checks that its answer stops holding connect events once the app's script has run. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
A classic Web or Shared Worker ran the app's script through importScripts called from keploy's answer, so in Chromium and Firefox a stack trace taken at the top level of the app's script ended in a frame at the marked URL. keploy's answer now queues the import in a microtask (queueMicrotask with importScripts bound to the URL): it runs as the answer ends, before the worker takes any message or connection, with none of keploy's code on the stack, and what the app's script throws is reported as the worker's own uncaught error, as before. A module Shared Worker whose policy refuses data: gives each connect event it held to the app's handlers in a microtask of its own, for the same reason. Rows: the browser row checks the stack at the top level of every worker kind and in a held connection's handler, and that a message the page posts as it starts a Web Worker reaches the app's handler; the unit rows check that the answer queues one microtask per import or held event and runs nothing of the app's before it. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
Firefox keeps, in a stack, the frames of the code that queued what is running. So in a seeded classic Web Worker, an Error made at the top level of the app's script still showed the frame of keploy's answer that queued its import, at the worker's URL with keploy's mark. A worker's seeding now wraps the stack getter of Error.prototype and DOMException.prototype where the engine reads a stack through it (Firefox): a stack read in the worker leaves out each frame at a URL whose query has keploy's mark, and keeps the getter's setter and flags. Chromium and WebKit keep each error's stack on the error itself and show no such frame; their prototypes are left as they are. Rows: a unit row reads an Error's and a DOMException's stack through the prototype's getter and checks which frames are left out, that the setter, the flags and a value that is no string are kept, and that a stack that is no getter stays as it was; the browser row already checks the Firefox stack at the top level of the app's script. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
Two rows for the page's Worker wrapper: - A document that a second run of the seeding took over (WebKit runs it again in a document that takes over its window's first about:blank) starts a Web Worker: its script URL carries one mark, the same as at a document with no takeover, and nothing is named. The first run passes the worker on as the app asked. - A worker URL the browser cannot read throws the browser's own error, and nothing is named. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
Chromium reports no end for a Shared Worker's script request, since Playwright does not follow Shared Workers. A keploy test whose browser was not seeded had no route to answer that request, so settle waited on it until the test's timeout failed the test. A seeded Chromium test was fine, because keploy's route answered the script whole. Every keploy test's context, seeded or not, now runs a small init script. It wraps the page's SharedWorker: the worker starts as the app asked, with the same arguments, and then the test learns the script's URL (resolved against the document's base, no fragment) through a binding. The script takes the binding off the window. Settle does not wait on a request of that frame at that URL that is no navigation, whether the browser reports it before or after. Playwright reports none of a Shared Worker's own calls in any engine, so the test says once for each Shared Worker script address that its script and calls are not waited for, and what to do: wait in the test for what the worker does. The README says the same. Rows: unit rows for Inflight (before and after, fragment, another frame, URL or a navigation still waited for, two scripts of one frame) and for the init script (arguments as given, URL, binding removed, a refused worker throws the browser's own error and reports nothing, new, instanceof, constructor, name, length, prototype and flags kept, no binding leaves the browser's own); a browser row in each engine for an unseeded test whose page starts a Shared Worker, which passes under a 4 s timeout with no settle step and says the line once; the seeded workers row now expects the line for each Shared Worker script. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
Settle waits for a navigation of the frame that started a Shared Worker to its script's URL when the browser reports it after the frame's report too, as it does for one reported before. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
In a seeded Web or Shared Worker, an error the browser's own call throws inside a seeded function or getter (crypto.getRandomValues of a float array or of more than 65536 bytes, crypto.randomUUID or the location's href, search or toString with a this of another kind) kept, in Chromium and WebKit, a stack made as it was thrown: with a frame of keploy's seeding, at the worker's URL with keploy's mark, or at the data: module a module Shared Worker imports its seeding as. The seeding now catches what those calls throw and leaves each frame at a URL keploy marked out of the error's own stack, through the stack's setter (V8) or as its value (WebKit), flags kept, then throws it on. A stack with no such frame, one that is no string, an error with no stack of its own (Firefox's, which its prototype's getter already filters) and a thrown value that is no object stay as they were. The data: module's media type now carries keploy's mark as a parameter, so its frames are found as the others are. A page's seeded functions let an error go on as it is: no page URL carries the mark. Rows: a unit row throws, from each seeded function and getter, errors whose stack is a setter's or a value, with frames at a query's mark, a 'p' import and a data: parameter, and checks what is left, that the flags are kept, and that the four kinds above and a page's errors are not touched; the workers browser row has each worker kind (classic and module Web Workers, classic and module Shared Workers, one served with a policy that refuses data:) throw six such errors and checks, in each engine, that no stack shows the mark or a data: frame and, in Chromium and WebKit, that each frame is the app's script's or the browser's own. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
A worker's code resolves URLs against its script's URL, which carries
keploy's mark. So fetch(''), a fragment alone or an XMLHttpRequest of ''
in a seeded Web Worker reached the context's route and got keploy's
seeding instead of the app's script, and counted as another start of the
worker's token.
The route now answers a marked URL only when the browser loads it as a
worker's script: by its sec-fetch-dest (worker or sharedworker) where the
route sees one (Firefox, WebKit), else by its type (script, Chromium).
Any other marked request goes on to the test's other routes and the
app's server with the mark taken off, and starts no instance.
Rows: a unit row with each engine's request shapes, and the workers
browser row, where each Web Worker's own fetch(''), fetch('#h') and
XMLHttpRequest of '' get the app's script in Chromium, Firefox and
WebKit.
The README says where the mark still reaches the app's server: a seeded
worker's Referer headers and, in Chromium and WebKit, a Shared Worker's
own request at its script's URL.
Signed-off-by: slayerjain <shubhamkjain@outlook.com>
keploy's answer to a Web Worker's marked script load is now the app's script it fetched through the test's routes, with the worker's seeding loaded first from a URL keploy marks and answers itself (importScripts for a classic worker, after the app's own 'use strict' when it has one; an import ahead of the app's imports for a module worker), on the script's first line, then a sourceURL of the app's URL. The worker never loads the app's script a second time, so a route that answers once, counts loads or picks the worker's script by its destination, and the app's server, see one load under any referrer policy, as with seeding off. The kept scripts and the Referer matching are gone. Stacks through the app's script name its URL with no mark in each engine; the worker's location and an ErrorEvent's filename hide the mark. A module Web Worker's import.meta.url shows it. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
Rows in each engine, seeded and with seeding off: a document whose referrer policy is no-referrer, origin or strict-origin, set by a meta tag or by a header on the HTML alone, starts a classic and a module Web Worker whose script a route sends on once with the header the server needs (times: 1), and two whose loads a route counts. Each runs seeded at the app's URL, the counting route sees one load per worker, and the app's server is asked for each Web Worker's script once, as with seeding off. A row for the error event: an error a seeded Web Worker throws, classic and module, names the app's script and the line it was thrown on, in the worker and in the page, as with seeding off. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
…irst The README's Workers paragraph said a Web Worker's answer was its seeding, which loaded the app's script again, and how keploy told that second load apart. Say what the code does: a Web Worker's answer is the app's script, with its seeding loaded ahead of it on its first line and a sourceURL of the app's URL; the test's routes and the app's server see the script loaded once under any referrer policy; an error event's filename shows no mark; a module Web Worker's import.meta.url is its marked URL; line numbers are kept and the first line's columns shift. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
Chromium reports no end, ever, for the script load of a Web Worker that a page terminates while that script loads: no requestfinished, no requestfailed and no 'worker' event. Settle waited for it, so the test hung until its timeout failed it, seeding on or off (0.1.0 and 0.1.1). The init script every keploy test's context runs (now workersScript) wraps the page's Worker as it wraps SharedWorker: each document numbers its Web Workers at a URL (resolved, no fragment) in the order it starts them, asks the test for its document number at its first one, and its terminate() reports the document, the URL and the worker's place. In Chromium, settle keeps each frame's script requests by URL (keploy's mark aside) per document, a new one at each commit with an answer, and drops the terminated worker's load, now or when it is reported. A load whose URL keploy marked is left to keploy's route; a document that terminates a worker as it unloads names its own document. Firefox and WebKit go on loading the script and report its end, which settle waits for. keploy's route now counts a worker's script as answered once its fulfill has run, even when that fulfill throws (the browser dropped the request of a terminated worker), so a seeded test does not wait on it either. Rows: unit rows for the page's Worker (numbering, terminate reports, kept constructor and terminate) and for settle's drop (place, frame, document, mark, engine); browser rows in each engine, seeded and off, for a classic and a module Web Worker terminated while their scripts load and for two Web Workers at one URL with the second terminated (the first one's load is still waited for). The module Web Worker row now also shows that URLs built from import.meta.url with a relative path or a query carry no mark, and one with a fragment alone keeps it. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
The README's "a call does not end" case says how settle treats the script load of a Web Worker a page terminates while it loads (Chromium reports no end for it, so the page's word drops it; Firefox and WebKit report the end, which settle waits for). A new case says that a seeded module Web Worker's import.meta.url is its marked URL, that URLs built from it with a relative path or a query of their own carry no mark, and that only code reading the exact URL, or adding a fragment alone, sees it, the same at record and at replay. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
The settle row for Web Workers a page terminates while their scripts load now also checks that every such load the server got is answered before the test ends in Firefox and WebKit, seeded or with seeding off: they go on loading it. Chromium ends the load at once, and, seeded, keploy's fetch of it is cut too. A unit row checks that a commit with no answer (a route change inside the page) keeps the document, so its word of a terminated worker still drops that load, and that a load at a URL keploy marked is waited for even when its place was terminated before it was reported. webWorkerStopped no longer checks that the place is a whole number from 1: no load is ever at another place, so it dropped nothing either way. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
A page route of the test's own that takes itself off as it gets a Web Worker's script load, then continues it, fails with Playwright's error "route.continue: Route is already handled!" when seeding is off: the page has no route left, so Playwright sent the load on already. Seeded, keploy's own route on the page kept the page intercepted, so the continue went on and the test passed. keploy's routes stay on the page and on the context. When the test's unroute or unrouteAll, with any behavior, leaves a page no route of its own, keploy does what Playwright does there before the call returns: it answers each worker script load a route taken off still held, seeded, fetching the script once, and sends any other request on, through Playwright's own answers that leave the request to the route holding it. That route's continue, fulfill or abort then fails at the browser's end with Playwright's own error (dropped after 'ignoreErrors'), and its fallback goes on, as with no keploy. A load keploy or the test began to answer is waited for. The context's last route does the same for a request from no page or from a page with no route of the test's. The test's unrouteAll takes off the test's routes alone, with its behavior, and keeps keploy's, so the page or context is never left unrouted: Playwright never sends a request on with keploy's mark. With 'wait' it waits for the test's routes, as Playwright does. Where Playwright has no such way, keploy's routes are added again after it, as before. Rows: unit rows on a fake of Playwright's routing (its routes' handlers and the requests each holds, its unrouteAll path, its answers past its API, and one without them) for the routes kept, each of continue, fulfill, abort and fallback after unroute and unrouteAll with each behavior, a request other than a worker's, a load the test began to answer, a page that keeps a route of the test's, 'wait' from outside the route, and the context's last route from a page with and without a route of the test's and from no page; a browser row in each engine, seeded and with seeding off: after unroute a continue fails with Playwright's error and a fallback passes, after unrouteAll() a fulfill fails, after 'ignoreErrors' an abort passes, 'wait' from the test waits for the route's continue; the app's server is asked for each script once, never with the mark, and each passing test's worker draws its seeding's numbers. The README says what an unroute and unrouteAll do, and, under "What the run says when", that in Chromium a page's script at a terminated Web Worker's URL can shift which load settle stops waiting on. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
keploy puts the load of a classic Web Worker's seeding first in the app's script, after a 'use strict' of the app's own when its directive prologue has one, so the script stays strict. The prologue scan ended at a line terminator inside a string, but U+2028 and U+2029 may stand in a string literal: a script that starts "a b"; "use strict"; is strict, and keploy's answer made it sloppy. The scan now ends a string only at its closing quote; a raw line feed or carriage return in one is a syntax error, which the browser refuses whatever comes ahead of it. Rows: - useStrict: a directive after a string holding U+2028 or U+2029 counts; a line comment that ends the source ends the directive's statement. - a Shared Worker's seeding leaves an ErrorEvent filename holding "null" as it is. - a route that calls fulfill, abort, fallback or continue while keploy still answers its continue gets Playwright's error once that answer ended, and the load is answered once; a load the browser dropped while keploy fetched its script is still counted answered, and the route's next call waits no more. - when the test takes its context's last route off, keploy answers a load from a page it has not seen yet, as Playwright sends it on. - a page that shows no list of its routes takes them off with its own unrouteAll, and keploy's route is added again. - settle (Chromium): a fetch at a Web Worker's URL is no worker's load, so the terminated worker's load is the one dropped. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
Firefox and WebKit send a request that a page or a worker starts while its browser context closes to the network past every route, keploy's too. A Web Worker's script load that a test's last step started, or the load of the seeding keploy's answer makes when the test ends just after keploy answered the worker's script, then reached the app's server with keploy's __keploy_worker mark in its URL. keploy now takes a seeded test's context offline before it closes, whoever closes it: after keploy's end-of-test wait; as the test closes it itself; or as its browser closes (once a test's timeout cuts keploy's wait short, Playwright skips the context's own close, and the context closes with the browser as the worker ends). Such a request then fails in the browser instead. With seeding off it would reach the server, unmarked; no test step reads what it gets, and it is the same at record and at replay. A context that cannot go offline (its setOffline throws, or there is none) closes anyway, and the test prints one line naming it and saying that a request started as its context closes can reach the server with the mark. A context Playwright reuses for the next test (reuseContext, or PW_TEST_REUSE_CONTEXT, which VS Code's Show browser sets) is not closed, and Playwright's reset of it for the next test needs the network (Firefox clears its storage by loading its origin; in Chromium an offline error page cuts its navigation to about:blank), so keploy leaves it online. The run says once, at its first seeded test there, that a request its pages or workers start after a test ends can reach the server with the mark, and that running without reuse avoids it. Rows: - in every engine, seeded: a test whose last step starts a Web Worker (classic, module, from a timer) or a Shared Worker and returns, one whose page route takes itself off and then fulfills a Web Worker's script load, and one that closes its own context as a Web Worker starts, 20 runs of each with four runs at once: the server never sees the mark (before, of 32 runs of each, it did in up to 10 in Firefox and 8 in WebKit). - a test whose context's setOffline throws, or that has none, passes and says so once, seeded; with seeding off it says nothing. A test that puts back the context's own close, past keploy's, finds the context offline as Playwright closes it, seeded (keploy's call after its wait), and online with seeding off. - in every engine, in a reused context: keploy leaves it online, so three tests in it each reach the app's page online, as with seeding off, and the run says so once. - in every engine, a test whose timeout cuts keploy's wait short prints no line about going offline (the sandboxed-frame rows' exact lines): its context goes offline as its browser closes (before, it closed with the browser ahead of keploy's setOffline, which then failed in Firefox). - offlineBeforeClose: offline once, before the close, whichever comes first; a close during setOffline waits for it; before a browser closes, each seeded context it still has open goes offline, and not one that closed or that Playwright handed on for reuse. reusedContext: Playwright's own flag, and without one PW_TEST_REUSE_CONTEXT. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
This reverts a0c6060 (take a seeded test's context offline before it closes). context.setOffline(true) is answered only once each page's renderer has handled it. A page with a dialog left open, or a main thread busy as the test ends, never does in Chromium and WebKit: the test then times out in teardown, and keploy's wait in its browser.close kept the whole run from ending. It also told the app it was offline before Playwright took the failure snapshot, so a failed test's error context showed the app's offline state, in every engine. Both are worse than what it closed. The close is again Playwright's own. The README's "What the run says when" says what that leaves: a request the browser starts while Playwright closes a seeded test's context, after the test has ended, can reach the app's server with keploy's mark in its URL in Firefox and WebKit, with the rates measured on a test whose last step starts a worker and returns. No step of the test reads its answer. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
keploy's answer to a Web Worker's script decoded the app's script as UTF-8 and wrote it out again. Chromium reads a worker's script by its charset, so a script served as iso-8859-1 read differently seeded than with seeding off. The answer now holds the bytes the server sent, with keploy's ASCII statements ahead of them and after them (in UTF-16 of the script's byte order when its byte order mark names UTF-16). A byte order mark and a hashbang line stay first, and the app's lines keep their numbers. A Content-Security-Policy on the worker script's response could refuse keploy's load of the seeding (importScripts or import at a marked URL), so the worker died at keploy's first statement in every engine. When the response carries one, the seeding now goes on the script's first line instead, on one line, which no policy refuses. In a module that line runs after the modules the script imports, so the test says so once for each address and names the next step. A classic worker's answer starts with keploy's own 'use strict' when the app's directive prologue has one. Rows: the bytes of a Latin-1 script, a UTF-8 and a UTF-16 byte order mark, hashbang lines ended by CR LF, CR, LF, U+2028 and U+2029; the seeding on one line draws what it draws on its own lines; the inline answer under a policy, for classic and module workers, and its line; a directive after a string ended by U+2028 alone is strict. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
keploy's answer to a worker's script followed the server's redirect and served the final script at the worker's first URL, so a classic worker's importScripts and a module worker's imports resolved against the wrong directory, and its location named the first URL. keploy now fetches a worker's script without following a redirect (maxRedirects 0). On a 301, 302, 303, 307 or 308 with a Location, the worker runs where the redirect takes it, as with seeding off, unseeded: Playwright's routes reach only a redirect's first request, so keploy cannot put its seeding ahead of the script there. In Firefox keploy answers with the server's redirect, its Location resolved against the URL keploy asked for, so the mark never reaches the server. Chromium and WebKit follow no redirect a route answers with (WebKit refuses one; Chromium never ends a worker's load of one), so keploy sends the request on as the test's routes asked for it, and the server is asked for the script's first URL twice. A Shared Worker's request sent on is marked answered, as Chromium reports no end for it. The test prints one line per address naming the worker, why it runs unseeded and the next step. Rows: unit rows for each redirect status and kind, a 300 and a 302 with no Location, a fragment-only Location, a route's own headers on the request sent on, and a dropped request; a browser spec in each engine, seeded and with seeding off, for a Latin-1 worker script, four Content-Security-Policies on a worker's script (the module one named as seeded late), and classic, module and Shared Workers redirected to v2/, with the server's counts and no mark at the server. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
Firefox lets no route reach a Shared Worker's script, so keploy never sees the redirect of /rds/s.js there: the run names that worker with the line for a Shared Worker keploy cannot reach, not the redirect line. The served row now expects two redirect lines and that one line in Firefox, and three redirect lines and no such line in Chromium and WebKit. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
In Chromium, a seeded worker whose script the server redirects never
started: keploy sent the load on at the unmarked URL, and Playwright
sends a request whose URL a route changed to that URL again on each of
the redirect's hops, so the server was asked for the first URL until
the browser gave up and the worker failed. A test's own
route.continue({ url }) on a worker's script does the same with no
keploy; with seeding off and no such route the worker runs.
keploy cannot take its mark off without changing the URL, so in
Chromium it now stops the load (net::ERR_BLOCKED_BY_CLIENT) once its
own fetch has seen the redirect: the server is asked for the script
once, and the worker fails to load. The test prints one line per
address naming the worker, why it does not start and the next step
(serve the script with no redirect, turn seeding off for that test, or
run it in Firefox or WebKit). Firefox and WebKit keep their answers.
workerRoutes takes the browser's name in place of two flags.
A load held by a route the test takes off is sent on as Playwright
sends on a request it held (a fallback's continue), or stopped past
Playwright's API, so the route's later call gets Playwright's own error.
Rows: unit rows per engine for each redirect status and kind, through
a route of the test's that continues or falls back, a dropped load, and
the release as the test takes its route off (each bounded, so an
answer that never ends a load fails the row); the served browser row
in Chromium: each redirected worker fails seeded and is named, the
server asked for its first URL once and never for the next.
Signed-off-by: slayerjain <shubhamkjain@outlook.com>
A worker script in UTF-16 whose hashbang line ends with U+2028 or U+2029 (one code unit there) keeps that line first, with keploy's statement after it: a unit row for each byte order. The served browser row starts a second module worker under a Content-Security-Policy, and the redirected classic worker a second time, in each engine, seeded and with seeding off: the late line and the redirect line are each said once for their address, and the server's counts follow each start. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
Since a worker's answer keeps the app's script's bytes (withBytes), nothing in the package calls withPrefix, the string form of the same placement after a hashbang line. It and its two unit rows go; withBytes and its rows cover the placement. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
The row for a page route that takes itself off was flaky in WebKit:
two of its tests fail by their route's error ('Route is already
handled!') and ended at once, while the seeded worker they started
still loaded its seeding. As Playwright closed the context, WebKit
sent that load past every route, so the app's server got a URL with
keploy's mark (2 of 76 such tests in WebKit, none of 76 in Firefox).
That is the close-time case the README states, whose next step is to
wait for what the worker does before the test ends.
Each test now ends once its worker's message has come and its route's
answer has ended; the two that fail keep their route's error and throw
it then ('ignoreErrors' still drops its own), so they fail with the
same error, and the row checks every test's worker:
it runs at the app's URL and, seeded, draws its seeding's numbers. The
README and the case's measurements name a test that fails while its
worker loads, and the seeding load keploy's answer makes.
Signed-off-by: slayerjain <shubhamkjain@outlook.com>
A worker's script served with any Content-Security-Policy had keploy's
seeding put on the script's first line, so a module Web Worker under a
policy that allows a script from its own origin ('self', as helmet's
default does) ran its imports before its seeding: a random number an
import drew as it loaded was the browser's own. And a Shared Worker under
a policy that refuses a script from its own origin (default-src 'none')
did not start when seeded, as keploy's answer loaded the app's script
from its URL, which the policy refused; with seeding off it runs.
cspAllowsOwn reads the policy. keploy keeps its load at the worker's own
URL where every policy allows it ('self', 'strict-dynamic', *, the scheme
or the origin; no sandbox; no Trusted Types for a classic worker; no
upgrade of http, but in Chromium on a loopback host), and puts the
seeding on the first line only where one could refuse it. A Shared Worker
whose policy could refuse its load of the app's script is answered so
too: the app's bytes with the seeding on the first line, run at its
marked URL; a module one is named as seeded late, as a module Web Worker
is.
Rows: the policy table; the route's answers for each kind; a browser row
for each engine with module workers whose imports draw under default-src
'self' and helmet's default, 'strict-dynamic' workers, and a classic and
a module Shared Worker under default-src 'none'. Also corrects two README
sentences on redirects in Firefox and a comment on byte order marks.
Signed-off-by: slayerjain <shubhamkjain@outlook.com>
A row of the worker policy table where a host source with no wildcard meets its own host (allowed), a subdomain of it, and a longer name that ends in it (both refused). Signed-off-by: slayerjain <shubhamkjain@outlook.com>
A test whose own route answered a request with a redirect waited out its timeout at the end: settle waits for the browser to report each call's end, and none comes. Chromium never ends a Web Worker's script load a route fulfills with a 302 or 307 (its worker never starts, with no keploy too), Firefox never ends the redirect's first request, and WebKit refuses such a fulfill, whose error fails the test, and leaves the request with the route. Seeded and with seeding off alike. settle now watches each route Playwright hands the test's pages and context. Once a route's fulfill returns, its request has the whole answer the route gave and nothing of it is on the network, so settle stops waiting on it. Once a route's handler throws while the route still holds its request (no route answered it or sent it on), Playwright sends it nowhere, so settle stops waiting on that one too. A handler that returns and leaves the request held might still send it on, so settle waits on it as on any other call. A context Playwright reuses between tests has its routes watched by the current test's settle alone. On a Playwright without this route handling, settle waits as before. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
helmet's default Content-Security-Policy has upgrade-insecure-requests. On an http worker script served with it, keploy counted the policy as one that could refuse its own load of the seeding in Firefox and WebKit, so a module worker there was seeded only after its imports ran, and the test said so, where Chromium kept the seeding first on a loopback host. Measured in Firefox and WebKit on localhost and on 127.0.0.1: neither upgrades a worker's http load of its own origin under that policy, and every Web Worker (classic and module, with importScripts and with imports) ran seeded with the seed's numbers. keploy now keeps the seeding first there too. Other loopback names (a name under localhost, the rest of 127.0.0.0/8, [::1]) are not measured in those engines and still count as refusing: seeded late, and named. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
Chromium reports a Web Worker's script load finished only once the worker's top-level code returns. A worker whose top-level code runs long (a loop, or long setup as its script loads) so holds the test's end, with seeding off too, until that code returns or the test's timeout names the load. Playwright shows nothing else that says the script has arrived, so settle keeps waiting: it never stops waiting on a script that may still be arriving. The wait now says why. Settle keeps each Web Worker's script URL from Playwright's worker event (Chromium). Once the test's end has waited 3 s with only Web Workers' script loads in flight, and again as each call ends after that, the test prints one line for each such URL, naming the worker, why Chromium waits, and the next step: move the worker's long-running top-level work into its onmessage handler, or end the worker before the test ends. Firefox and WebKit report the load's end as the script arrives, and keploy's route ends a seeded Web Worker's load with its answer, so nothing is said there. Rows: a unit row for the line's timing, its once-per-URL rule, a fetch at a worker's URL and a page script beside the load, the stop, and another engine; a browser row per engine, seeded and off, with a worker whose top-level code runs on for 10 s after the body ends. The README names the case under "What the run says when ...". Signed-off-by: slayerjain <shubhamkjain@outlook.com>
Settle's route watch called each route's fulfill through Function.prototype.call. Playwright names an API call by how its caller invoked it, so a fulfill that failed read "route.call: Route is already handled!" where Playwright's own error, with no keploy, reads "route.fulfill: ...". A test's route that took itself off and then answered a worker's script load failed with that wrong text, seeded and off. Settle now calls the bound method, as keploy's own route wrappers do, and the error is Playwright's own again. The settle rows for a route that answers with a redirect asserted that settle's step names none of those requests, while the fixture's body could end before the routes' handlers had: the step then named calls that ended a moment later. The body now ends once each handler has ended (its fulfill returned, or threw as WebKit refuses a redirect), and the rows match only the first requests, not a redirect's next hop. The "sends on, then fulfill" row asserts Playwright's whole error text. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
The settle rows for a call the test's route sends on and then tries to answer asserted "route.fulfill: Route is already handled!". Playwright's own check refuses that fulfill before it makes any API call, so its error names no API, with keploy or without it: the rows now assert the first line Playwright gives, "Error: Route is already handled!". The rows for a route that takes itself off are the ones that see how a fulfill is called (their error comes back through Playwright's API). Signed-off-by: slayerjain <shubhamkjain@outlook.com>
Playwright ends a test at the first error a route's handler throws (the handler's rejection is unhandled, and the test runner stops the test), while the test's other handlers may still wait on their fulfill. Their requests are then in flight, held by a running handler, and settle's step named them for the moment until WebKit refused their answers too. The rows for a test whose routes answer with a redirect asserted that no step names them, and in WebKit they failed in about half the runs. The fixture's handlers now each wait until every route's first fulfill has returned or thrown, and only then throw what their fulfill threw, so in WebKit all three end together. In WebKit no route can reach a request whose redirect the browser refused: Playwright marks the route handled before WebKit refuses, so a continue() of it fails with Playwright's own "route.continue: Route is already handled!". The WebKit row that expected such a call sent on and waited for now asserts that error and that settle does not wait on it. WebKit holds the call until the test's browser context is deleted and may then send it to the server (Playwright sends nothing for it, with or without keploy), so the row asserts nothing of the server. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
WebKit sometimes never runs a Shared Worker whose script the server redirects: the server answers the script's URL with the redirect and the redirect's URL with the script, and then WebKit asks for nothing more, with no error event and no message. Plain Playwright with no keploy does the same (1 of 90 runs of the spec file in a busy 2-CPU container), so WebKit's served row with seeding off failed now and then on that Shared Worker's missing answer. The row keeps every assertion, and takes a missing answer from that Shared Worker only on exactly WebKit's record: the engine is WebKit, neither a message nor an error event came, and the server served /rds/s.js and /rds/v2/s.js once each and never the h.js the script imports first. The check lives in test/webkit.mjs with a row of its own (the recorded run is taken; another engine, an error event, a message, the first URL asked twice as a seeded WebKit run asks it, either URL missing or asked twice, or the import asked, are not). A run it takes prints a line in the row's output naming WebKit's drop. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
A route handler that called route.fallback() and then threw left settle waiting on its request until the test's timeout: Playwright hands such a request to no other route and sends it nowhere, so with no keploy the test fails at once with the handler's error. Settle read Playwright's private _handlingPromise to tell a held request apart, and fallback() clears it. Settle now wraps each route's continue() and abort() as it wraps fulfill(), bound so Playwright names them as the test called them. When a handler throws, a request no continue() or abort() of its route returned for is not waited for; one they sent on or stopped is waited for until the browser reports its end. Rows: fallback() then throw fails at once with the handler's error in every engine; abort() after WebKit refused a redirect fails with Playwright's own route.abort error; unit cases for a fallback, a failed continue() and an abort() before the throw, and a throw on a context two settles share (the newest ends the request). README and comments updated; the WebKit drop record names microsoft/playwright#44134. Signed-off-by: slayerjain <shubhamkjain@outlook.com>
Settle wraps each route's abort(), and keploy's worker routes wrap the
abort of a worker script's load they hand to the test's routes, but no
row checked that the error code reaches the browser: a wrapper that
dropped it left every row green.
Rows: in every engine, a fetch the test's own route aborts with
accessdenied (on its page and on its context) or with no code, and a
Web Worker's script it aborts with accessdenied, with blockedbyclient
or with no code, fail with the page's requestfailed text they have
with no keploy, seeded and with seeding off, and never reach the
server. In Chromium, keploy's stop of a seeded Web Worker's script the
server redirects fails it as a route's abort('blockedbyclient') does
with no keploy. Engines name codes apart only in part (WebKit gives
every abort one text; Chromium reports every failed Web Worker script
load as net::ERR_ABORTED), so the row checks that a dropped code would
show where they do: a fetch's text in Chromium and Firefox, and a
worker script's in Firefox. A unit row checks that the watched route's
fulfill, continue and abort hand Playwright what the test gave them,
on a page's route and on its context's.
README: in a Playwright trace, the source link of a route action
points into @keploy/playwright, as settle wraps the route's calls.
Signed-off-by: slayerjain <shubhamkjain@outlook.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
feat(playwright): seed each test's random numbers; run each test once
Why
@keploy/playwright0.1.1 breaks three rules of keploy's scope protocol v1, the wire that go-sdk'skeploytestspeaks too:--retries, the config, ortest.describe.configure({ retries })) and each repeat of--repeat-eachbegan the same test again, so keploy saw two windows for one test ID.KEPLOY_PROVEN=1. keploy speaks the wire only under that switch. The claim's 404 already said so. A begin or an end answered with a plain 404 said onlyanswered 404 Not Found.Three more gaps make a recorded test fail its replay:
crypto.randomUUID()at each page load, and its two sides load files that never overlap, so a test on such a page failed about half its replays: no recording answers a call it never saw. An ID or idempotency key the browser makes differed too.Math.randomdiffered each run. A suite that names its test data in the Node worker that runs the test (a lead named withMath.random()) made other data on each replay, while a mocked dependency answered with the recording's, so the test's own assertion failed.What
testInfo.retry > 0, ortestInfo.repeatEachIndex > 0) fails before its begin. Its body and hooks do not run, and the error says why and how to turn it off:keploy: Playwright retried <project>::<file>::<titles>, and keploy runs each test once in a run (run the suite with --retries=0, and with no retries in its config or its test.describe.configure). keploy is told withPOST /agent/refusal(codetest_run_again), so its verdict on the run names the test. The message names the test, not the attempt, so keploy's agent, which keeps a repeated refusal once, records one refusal per test even though each retry runs in a new worker. If the refusal call fails, the test still fails with the same text, plus; keploy could not be told: <the agent's answer>. The test's first run is begun and judged as before.keploy: <reason> (keploy skipped <titles>), so every terminal reporter prints the reason. The skip annotation is unchanged.keploy: this keploy agent answered 404 to POST <route>, so ...: keploy speaks @keploy/playwright's wire, scope protocol v1, only when it runs with KEPLOY_PROVEN=1 (...). A 404 that carries keploy's own error body still shows keploy's message.random: "seeded"and a seed drawn afresh (128 bits as hex). keploy answers the seed to draw from: in a record the begin's own, which it keeps in the test's mapping entry, in a replay the recording's. The test'scontextgets an init script, before its first page, that puts seeded streams behindMath.random,crypto.getRandomValuesandcrypto.randomUUIDin every frame before any script of the page runs. Each function has its own stream, and each document its own, named by its page (in the order the test opened its pages; a popup by the window its opener opened), its frame (by its element's path) and how many documents of that path the tab loaded before: a reload draws new numbers, the same ones each run. The end says what the test's pages drew from (seededorown).Math.randomdraws from the seed keploy answered the test's begin with, on a stream of its own (sfc32 over the SHA-256 of the seed and the stream's name), so it does not move the browser's. It is seated from the begin to the end: the test'sbeforeEachandafterEachhooks and its fixtures draw from it too. At the end the worker's ownMath.randomis back, unless the test put its own stub there, which it keeps. Outside every test (beforeAll,afterAll, a file as it loads) the worker draws its own, as does itscrypto.@faker-js/faker(8, 9 or 10), the instances the package makes as it loads (faker, the locales'fakerXX,simpleFaker) draw from the test's seed while it is seated, each on a stream named by its export, so the names, emails and IDs a suite makes with faker are its recording's at replay. keploy takes them from the module the test file's own import loads (faker 10 is one module for CommonJS and ES module test files alike; for faker 9 an ES module test file gets its ES module copy), and finds and loads it before the seat, so faker's own draws as it loads are the worker's. While a worker fixture is set up, and once a test seeds an instance itself (faker.seed(n)), that instance draws its own; at the end each instance's own generator is back, unmoved. A faker keploy cannot load where the tests can import it is named once per worker with the reason and the next step. A faker instance the suite makes itself as a file loads (new Faker(...)) seeds its own generator from the worker'sMath.random, so it is seeded only once given toseedFaker(instance, name), which the package exports and the seeded line names: it then draws on its own stream of each test's seed, named by the name (in every test after when given as a file loads, inbeforeAllor a worker fixture's setup; until the test's end when given in a test), and an instance already seated keeps its stream.seedFakerthrows, naming the next step, for a value that is no faker 8, 9 or 10 instance or an empty name. One made inside a test needs no call: it seeds itself from the test's seededMath.random.Error.prepareStackTrace(Playwright parses a file's source map the first time a test's stack reaches it, and source-map's sort drawsMath.random), and a worker fixture registered with this package'stest.extendortest.useuntil it callsuse().KEPLOY_SEED_RANDOM=off(or0,false) turns seeding off for the run; any value other than those andon,1,truefails every test, naming the variable. The optionkeploySeedRandom: falseturns it off for a project, a file or a test. The begin then sendsrandom: "own"and no seed.type: 'module') draws from its own streams, named by the document that started it, its script's path and how many workers that document started there before it; a Shared Worker, which the test's pages share, by its script's path, its name and how many times the test started it. The page'sWorkerandSharedWorkeradd a__keploy_workermark to the script URL's query, and a route of the test's context answers the marked request under the status and headers the app's server gives that script. A Web Worker's answer is the app's script keploy fetched, the bytes its server sent, never decoded and encoded again (a script in another charset, which Chromium reads a worker's script by, reads as with seeding off; a byte order mark and a hashbang line stay first), with the worker's seeding loaded ahead of it on its first line, from a URL keploy marks and answers itself (importScriptsfor a classic worker, after a"use strict";of keploy's own when the app's script has'use strict'in its directive prologue, which keploy leaves where it is, so the app's code stays strict; animportahead of the app's imports for a module worker, so the seeding runs before the modules the app imports), then a//# sourceURLof the app's URL unless the app names one. When the script's response carries a Content-Security-Policy that could refuse that load (one keploy cannot tell allows a script from the worker's own origin:'self','strict-dynamic',*, the origin or its scheme in its script source list, and for a module worker in its worker source list; nosandbox; no Trusted Types for a classic worker; noupgrade-insecure-requestson http, but on a loopback host: any in Chromium, which never upgrades one, andlocalhostand127.0.0.1in Firefox and WebKit, measured to leave a worker's http load of its own origin alone there), the seeding itself is on the app's first line instead, which no policy refuses; in a module worker it then runs after the modules the app's script imports, and the test says so once for each address; a Shared Worker's is the worker's seeding, which then loads the app's own script from the URL the app asked for, with no mark, or, where its script's policy could refuse that load, the app's script itself with the seeding on its first line, as a Web Worker's (a module one named as seeded late). So the app's script runs at its own URL: itslocation, thefilenameof an error event (in the worker and in the page), the URLs its imports resolve against and its stacks show no mark (in Firefox, a worker'sErrorandDOMExceptionstacks leave keploy's frames out), and neither does the stack of an error the browser's own call throws inside a seeded function or the worker'slocation(a float array togetRandomValues, athisthat is noCrypto). A module Web Worker'simport.meta.urlis its marked URL: one load cannot both run keploy's seeding first and give the module the app's URL (its imports still resolve to the app's URLs, as the mark is in the query; a Shared Worker's shows no mark, but in a module one whose seeding is on its script's first line, which runs at its marked URL). The app's line numbers are kept; keploy's seeding shifts the columns of the app's first line. The test's routes and the app's server see a Web Worker's script loaded once, as with seeding off, under any referrer policy of the script's or of the document that started it; the seeding's own load reaches neither. A worker whose script the server redirects runs at the URL the redirect gives, unseeded, in Firefox and WebKit, and does not start in Chromium (below). Chromium and WebKit route nothing a Shared Worker loads after its script, so the server is asked for a Shared Worker's script twice, with no mark, by keploy and by the worker, at record and at replay alike (once, by keploy, for one whose seeding is on its script's first line, as keploy's answer is then the script itself). Only the browser's load of a worker's script is answered with the seeding: a request the worker's own code makes at its script's URL (fetch(''), a fragment alone, which resolve against the marked URL) goes on to the server with the mark taken off. A seeded worker'sfetchandRequesttake the mark off what they send: a fetch or aRequestat the script's own URL asks for it with no mark, and a fetch whose referrer the browser would pick (no init, an init that names none, aRequestmade with no referrer and the default policy) sends the script's URL with no mark as itsReferer. JavaScript cannot set the rest, so the server still sees the mark in theRefererof a worker'sXMLHttpRequestandimportScripts, of a fetch of aRequestwith areferrerPolicyof its own and no referrer, and, in Chromium and WebKit, of a Shared Worker's load of its own script, and in the URL of a Shared Worker'sXMLHttpRequestof its own script; a worker that runs at its marked URL with no seeding (below: its script answered by the test's own route, or a Shared Worker's sent on with options of its own) sends the mark in theRefererof each of its requests and, in Chromium and WebKit, a Shared Worker's in the URL of itsfetch(''); the mark comes from the test's seed, so the server sees the same at record and replay, and the README says so. A module Shared Worker whose script's policy refusesdata:runs its seeding in keploy's answer and then imports the app's script; a connection that comes before that script has run reaches itsconnecthandler after, in order, as an event keploy dispatches (isTrustedis false). While a context has a route (keploy's, on a seeded test, or anycontext.route), every engine asks the server for each request rather than answering it from its cache, at record and at replay alike, so a replay asks the app's server for what its recording asked: on one test's four page loads (two pages on the same stylesheet, script and image, a reload, the first page again) the server got 20 calls in Chromium and WebKit (11 with no route) and 24 in Firefox (12), the same ones at record and at replay. keploy's route stays ahead of every route the test adds to its context or its pages (withrouteorrouteFromHAR, and again afterunrouteAll), so a test's own route never sees the mark: it gets a worker's script load at the URL the app asked for, and a route that continues it (a catch-allroute.continue()) or falls back has keploy answer it with the seeding, keploy asking the app's server for the script with what that route asked (the URL, method, headers or post data it gavecontinueorfallback: a header the server needs, a stand-in's URL); a Web Worker then runs the script that answer got, so a route that answers once (times: 1), counts its loads or picks the worker's script by its destination sees what it sees with seeding off. When the test'sunrouteorunrouteAlltakes the last route of its own off a page, Playwright sends each request a route it took off still held on, as the browser asked for it, so that route'scontinue,fulfillorabortthen fails with Playwright's own error (route.continue: Route is already handled!, dropped afterunrouteAll({ behavior: 'ignoreErrors' })) and itsfallbackgoes on to the next route. keploy's route stays, so keploy does the same before theunrouteorunrouteAllreturns: it answers each worker script load such a route held, seeded, asking the app's server for the script once, and sends any other request on; the route's call after that fails with the same error, seeding on or off, and the app's server never gets the mark. The same holds for the context's last route, for a request from a page with no route of the test's.unrouteAlltakes off the test's routes alone, with the behavior the test gives, and keeps keploy's: with'wait'it waits for the test's routes, as Playwright does, so a route that then continues a worker's script load has keploy answer it as above. Chromium and WebKit route nothing a Shared Worker loads after its script, so a Shared Worker whose script such a route sends on otherwise than the browser asked (headers other than the browser's, a URL other than the app's, another method, or post data; of two routes, what the last one asks) gets the answer to what that route asked, whole, and runs unseeded at its marked URL, and the test names it with the next step (no options for that script); one sent on with the browser's own headers, or none, is seeded. A worker keploy cannot mark (ablob:ordata:URL, a marked URL the browser refuses, and in Firefox a Shared Worker, whose script Firefox's route does not reach), or whose script a route of the test's own answers itself (route.fulfill: it runs that script unseeded, at its marked URL) or, for a Shared Worker, sends on with options of its own, draws the browser's own numbers, and the test names it once with the next step. In WebKit, a Web Worker that a seeded Web Worker starts (classic or module, and the ones that one starts in turn) is seeded too, named by the worker that started it, its script's path and how many workers that worker started there before it. Chromium's and Firefox's routes reach no script a worker starts, so there, and for one a seeded worker starts from ablob:ordata:URL or a URL the browser refuses with the mark, it starts as the app asked and the test names it once for each address: the worker that started it posts the address to keploy's route, which answers it, so neither a route of the test's nor the app's server sees that request. Every worker line but a Service Worker's says that each worker the named one starts draws the browser's own numbers too.SharedWorkernow starts each worker as the app asked and tells the test the script's URL, and settle does not wait on a request of that frame at that URL, nor on any request whose URL carries keploy's Shared Worker mark, whichever route answers it. Playwright reports none of a Shared Worker's own calls in any engine, so the test says once for each Shared Worker script that its script and calls are not waited for, and that the test should wait for what the worker does before it ends.Workernow numbers each document's Web Workers at a URL in the order it starts them, and itsterminate()tells the test which one it terminated; in Chromium, settle stops waiting on that worker's script load (the matching script request of that frame at that URL, while that document was the frame's: a document that terminates one as it unloads names its own). Firefox and WebKit go on loading the script and report its end, which settle waits for. On a seeded test keploy's route fetches the script, and the load counts as ended once keploy's answer has gone to the browser or the browser has dropped it.fulfillthrows, which fails the test) and leaves the request with the route, so the test waited until its timeout failed it, seeded or not (0.1.0 and 0.1.1). So did a test whose route handler callsfallback()and then throws, in every engine, where with no keploy it fails at once with that error. Settle now watches the routes Playwright hands the test's pages and context: a request whose route'sfulfillreturned has the route's whole answer and is not waited for, nor is one whose route's handler throws before acontinue()orabort()of the route returned, as Playwright then hands it to no other route and sends it nowhere. A request acontinue()sent on, or anabort()stopped, is waited for until the browser reports its end. A handler that returns leaving its route holding the request might still send it on, so that request is waited for as any other. The watch calls each route'sfulfill,continueandabortas the test called them, so Playwright's errors name them as they do with no keploy (route.fulfill: Route is already handled!for a fulfill afterunrouteAll(),route.continue: Route is already handled!for a continue after WebKit refused a redirect), and hands them what the test gave them (an abort's error code fails the call with the text it has with no keploy). In a Playwright trace, a route action's source link (Fulfill, Continue or Abort request) points into this package's watch rather than at the test's line; the reports show no route action and do not change. This reads Playwright's own route handling past its API; with a Playwright that has none, settle waits as before.importScripts, at that URL takes a place too, so settle may stop waiting on that script's load while it is in flight and, with seeding off, go on waiting on the terminated worker's load until the test timeout fails the test, naming that load. Playwright shows both as script requests of the page's frame, so settle cannot tell them apart; the README states this case under "What the run says when ...", with the next step (a URL of the worker's own).keploy: in <test>, the test's end waits for the script of the Web Worker at <url>: ..., with the next step (move the worker's long-running top-level work into itsonmessagehandler, or end the worker before the test ends). Settle knows a Web Worker's URL from Playwright'sworkerevent. Firefox and WebKit report the load's end as the script arrives, and keploy's route ends a seeded Web Worker's load with its answer, so nothing is said there.import.meta.urlshows keploy's mark. On a seeded test it is the app's URL with the__keploy_workermark in its query, the URL the one load ran at. Its path is the app's, so a URL built from it with a relative path or a query of its own, and the URLs its imports resolve to, carry no mark; only code that reads or compares the exact URL, or adds a fragment alone to it, sees the mark, the same at record and at replay. The README states this case under "What the run says when ...".Location, the worker (classic or module, a Web Worker, and in WebKit a Shared Worker too) runs at the URL the redirect gives, as with seeding off: itslocation,import.meta.urland the URLs its imports resolve against are the same, with no mark. Playwright's routes reach only a redirect's first request, so keploy cannot put its seeding ahead of the script there, and the worker draws the browser's own numbers. In Firefox keploy's answer is the server's redirect, itsLocationresolved against the URL keploy asked for. WebKit follows no redirect a route answers with (it refuses one), so keploy sends the request on as the test's routes asked for it (as Playwright sends on a request it held), and the server is asked for the script's first URL twice: once by keploy, to see the redirect, and once by the browser. The test prints one line for each address, naming the worker and saying so, with the next step (serve the script at the URL the app starts the worker at, with no redirect, where the tests run); the README states this case under "What the run says when ...". Firefox's route never reaches a Shared Worker's script, so a redirected one there runs where the redirect takes it as with seeding off, and gets the line every Shared Worker gets in Firefox.route.continue({ url })on a worker's script does the same with no keploy). keploy must change that URL to take its mark off, so in Chromium it now stops the load (net::ERR_BLOCKED_BY_CLIENT) after asking the server for the script once, and the worker fails to load (itserrorevent), at record and at replay alike. The test prints one line for each address naming the worker, why it does not start and the next step (serve the script with no redirect where the tests run, turn seeding off for that test, or run it in Firefox or WebKit); the README states this case under "What the run says when ...".__keploy_workermark in its URL; with seeding off the script load would reach the server unmarked. No test step reads what it gets. Measured over 32 runs of four such tests at once: Firefox 9 to 10 of 32 for a Web Worker started by the last step and none for a Shared Worker, WebKit 1 to 8 of 32, Chromium none, and none in any engine for a test that waits for its worker's message; for a test that fails by its route's error while its Web Worker loads, WebKit 2 of 76 and Firefox none of 76. keploy does not take the context offline to stop it: that waits on every page's renderer (a run with an alert left open never ends) and shows the app's offline state in the failure snapshot. The README states this case under "What the run says when ...".serviceWorkers: 'block'in the project'susewhen the tests do not need it); a registration Playwright blocks says nothing. A worker whose script a Service Worker may answer would show that Service Worker keploy's mark, so it starts as the app asked, unseeded, and the test names it with the Service Worker: a Web Worker of a controlled page, and a Shared Worker of a controlled page or whose URL the scope of a registration keploy knows of holds. keploy marks a Shared Worker only when it knows that no registration's scope holds its URL: each page asks the browser for its origin's registrations as it begins, and the test's pages tell each other, on aBroadcastChannelof keploy's, the scope of each registration they find and of eachregisterthey call (before the browser registers it). Eachregisterholds a Web Lock of keploy's (keploy-sw-register) from before the browser registers it until it settles, and a page asks for its registrations only once the browser has said that no page holds that lock. A Shared Worker started before the browser answers (a script that runs as the page loads may start one), in a page that began while another page was registering a Service Worker, or in a page that cannot read its Service Workers or its Web Locks, starts as the app asked and is named, withserviceWorkers: 'block'as the next step: keploy then knows at once that none can answer. Such a worker still takes its place at its path, so the next worker there draws what it drew at record.note, JSON of at most 16 KiB): how many times the test process drew each stream (Math.random, each seeded faker instance), how many times each placed document drewMath.random,crypto.getRandomValuesandcrypto.randomUUID, and how many frames each document put in at each path. The page's seeding reports a document after each task in which it drew or numbered a frame, and only once its place is known. keploy keeps the note with a record and answers it to the replay's begin, and the replay's end prints one line for each difference, naming the test, what differs and the next step, at most 8 a test. A seeded replay answered no note says so once per worker; a note this package cannot read is named. A begin answered a note that is no string of at most 16 KiB fails the test. keploy keeps the note only when it is a keploy that keeps notes: with keploy v3.7.5 or older, a seeded replay says once per worker that it has no note, and names nothing.npm run linton each Node leg: oxlint with its type-aware rules (tsgolint, TypeScript 7's checker) onsrc. The rows run on Playwright 1.65.0;peerDependenciesstay at@playwright/test >=1.40.0, and@types/nodeis Node 20's, the oldest Node the package supports. The type oftestis written out from@playwright/test: TypeScript 7 otherwise declared it fromplaywright, the package@playwright/testre-exports, which pnpm does not let a project see. With it, TypeScript 7's JavaScript and declarations are the same as 5.9's, byte for byte.given, seeds or notes; a call that does not end, among them, in Chromium, a Web Worker's script load while its top-level code runs; a page that starts a Shared Worker; a document that cannot be placed; a worker that cannot be seeded, among them a Shared Worker started before keploy knows that no Service Worker's scope holds its URL (one a page's own script starts as it loads, or one in a page begun while another page of its origin registers a Service Worker), whose line names the one remedy,serviceWorkers: 'block'; and what moves a stream at replay (a worker fixture registered through@playwright/test's owntestor a test typemergeTestsreturned, a library that draws on its first use inside a test, a varying number of draws, frames at one path in a varying order or number), which the replay's note lines name. "What keploy leaves as it is" names the app's server,crypto.subtle's keys, the run clock and a context the test makes itself, which keploy neither seeds nor waits for. Two cases no line names yet are said there plainly: in WebKit, a worker that a Shared Worker starts (WebKit's route reaches nothing a Shared Worker asks for after its script), and two frames at one path that trade places and draw alike. The package version is now 0.1.2.Without keploy (
KEPLOY_MOCK_AGENTunset oroff), nothing changes: a retried test is retried as written, and the browser draws its own random numbers. Rows check both.Tests
continue()orabort()of its route returned (refused, after afallback(), after acontinue()that failed, or before any answer) ends; one the handler holds, sent on or aborted, before or after a refused answer, is waited for; a route the page's routes hand to the context's is watched there too; a Playwright with no route handling of its own gets none; of two settles on one reused context, the newest watches, and ends a request whose handler throws there; the watched route'sfulfill,continueandaborthand Playwright's own what the test gave them (an abort's error code, a continue's options, a fulfill's answer), on a page's route and on its context's;cspAllowsOwnunder helmet's default policy: allowed onlocalhostand127.0.0.1in Firefox and WebKit, refused there on another loopback name, another host and an engine it does not know;ok;runAgainfor a first run, a retry, a later retry and a--repeat-eachrepeat, with and without a project;KEPLOY_SEED_RANDOM's values, a begin's seed and an answered seed; a document's stream and the test process's stream pinned to a reference made outside the package (a recording keeps the seed, not the numbers, so a change to a stream would make every recording's data anew); junk in a tab's load count replaced by a count; while the test process is seated, a formatter a test sets onError.prepareStackTraceis among Error's own keys, as an assignment lists it;newPageonly once its first document is named and the script that names its later documents is in place; a first document that cannot be named, or a page that refuses that script (its page closed), failsnewPage.context.addInitScriptdrawing) keeps no key after the name: the frames and popups that ask it are not placed by a name it did not draw in.Math.random,getRandomValuesandrandomUUIDkeep the original's property flags (not enumerable, soObject.keys(Math)stays empty); aMath.randoma script before the seeding made read-only stays read-only; a methodCrypto.prototypeonly inherits gets an own one, writable and not enumerable; each draws the seeded numbers.peerDependencies.unrouteandunrouteAll(Playwright's own unrouteAll path takes the test's routes alone, with the behavior given, and lets go of its HARs; without it keploy's are added again); when the test takes a page's last route of its own off while that route holds a worker script's load, keploy answers it, seeded, before theunrouteorunrouteAll(any behavior) returns, and the route'scontinue,fulfillorabortthen gets Playwright's own error (dropped after'ignoreErrors') and itsfallbackgoes on; any other request it held is sent on; a load the test had begun to answer, a page that keeps a route of the test's, and'wait'from outside the route leave the answer to the route; the context's last route releases only a request from no page or from a page with no route of the test's;registerkeeps none; a Web Worker of a controlled document, and a Shared Worker of a controlled document or whose URL the scope of a registration it made or found holds, start unmarked and are named with that Service Worker's address (the controller, else the registration with the longest scope that holds the URL: its active, waiting or installing worker, or the script the document's register named; none when keploy cannot read it); a Shared Worker started before the browser answers, or where it never does, starts unmarked and is named for that; one with no Service Workers (an insecure context), or in a context Playwright blocks, is marked at once; a register's scope is the one it names (else its script's directory) and is known before the browser registers it, and one Playwright's block answers with none is taken back unless the browser found it; the documents of a context say each registration they make or find on keploy's channel, and another script's messages there, of any other shape, change nothing; a worker keploy cannot mark is named for that; the next worker at a path keeps its place.Worker: where the route reaches its script (WebKit), each Web Worker it starts is marked with a token of its own (its URL resolved against the URL the app asked for; the same each run, another for the next worker at that path, another path, token or instance); elsewhere, and for ablob:ordata:URL or a marked URL the browser refuses, it starts as the app asked and posts its address to the worker's own URL markedt, which keploy's route answers itself with no body, hearing only an array of a known reason and an address, before any route of the test's; the lines say what the worker is, that each worker it starts is unseeded too, why, and the next step.--repeat-each=2repeat;beforeEach, body andafterEachare its recording's at replay, and another test's, an opted-out test's and code outside tests are not; no seed answered, or seeding off, each saying so; tests in one worker whose helper file callstest.stepor reads an Error's stack, replayed alone, with the first set aside, and after a record whose first test failed; a worker fixture set up in the first test that asks for it, throughextendand throughuse; faker's default and locale instances in a CommonJS and an ES module test file made again at replay, others for another test, an opted-out test and code after the tests, and a test that seeds an instance itself drawing as seeded; the suite's own instances, one given toseedFakeras a CommonJS and an ES module test file loads and one made in a test, made again at replay, and one not given made anew;seedFaker's streams (never a prebuilt instance's), its lifetimes (given as a file loads, in a test, while a worker fixture is set up, after a test's end), an instance seated once, and its refusals; the seat of each instance (its own back, a setAside, a test's own seed or replacement kept), which instances a module exports, the faker a test file's import loads (CommonJS, ES module, faker 9's two copies, absent, unloadable) and Node's choice of a package's entry; the worker's ownMath.randomback after a test that throws and one that times out;serviceWorkers: 'block'names neither, and its Web Worker draws from the test's seed, the same at replay; a Shared Worker started before the browser answers is named and starts as the app asked, one whose URL the scope of a registration another page made holds is named with that Service Worker, which sees no mark, and one started once the browser said there is none draws from the test's seed, the same at replay;times: 1, on its page, and on its context for a module worker), one that adds it only to the worker's script by its destination, and the same once under a script'sReferrer-Policyofno-referrer(classic and module) andorigin(module), and under a document's referrer policy (no-referrer,originandstrict-origin, by a meta tag or by a header on the HTML alone; a classic and a module worker each sent on once with the header, and two whose loads a route counts): the worker runs in both modes, and seeded at the app's URL on the test's seed; a route that counts a classic and a module worker's script loads sees 2 in both modes, and the app's server is asked once for each Web Worker's script in both; an error a seeded Web Worker throws, classic and module, names the app's script and the line it was thrown on, in the worker and in the page, as with seeding off; a Web Worker at one address whose script the test's route answers itself and a Shared Worker there that it continues with a header are each named; a page route that falls back with the browser's own headers then a context route that adds the header: the last one's options count (the Shared Worker runs what it asked, unseeded, and is named). Unit rows: a Web Worker's answer is the app's script with its seeding's load on its first line (after a hashbang line; after the app's'use strict', read from the script's directive prologue as the engine reads it), each line keeping its number, then asourceURLunless the app names one; the script is fetched once under each referrer policy, and nothing is kept for a later load; keploy answers the seeding's load only in each engine's shape of it (a classic worker'simportScripts, a module worker's import), and a request the worker's own code makes at that URL goes on with the mark taken off; an error event'sfilenameshows no mark; a Shared Worker continued with its own URL (fragment aside) or method is seeded, and with another query, another method, post data or what is no URL is not.unroute, one that continues the load fails the test withroute.continue: Route is already handled!and one that falls back passes; afterunrouteAll()a fulfill fails withroute.fulfill: Route is already handled!, afterunrouteAll({ behavior: 'ignoreErrors' })an abort passes;unrouteAll({ behavior: 'wait' })from the test while the route holds the load waits for the route's continue, which passes; each test ends once its worker's message has come and its route's answer has ended, and one that fails throws its route's error then; the app's server is asked for each script once, never with the mark, and each test's seeded worker draws its seeding's numbers;blob:URL draws its own and is named once; the app's server never sees keploy's mark;default-src 'none',script-src 'none',require-trusted-types-for 'script', and two module ones underdefault-src 'none') draws from the test's seed, the module ones named once as seeded late; a module one underdefault-src 'self'whose import draws as it loads draws the seed's numbers there too, and is named by no line, as are module and classic ones under'strict-dynamic'and one under helmet's default policy, on 127.0.0.1; a classic and a module Shared Worker underdefault-src 'none'start, as off, seeded in Chromium and WebKit (the module one named as seeded late); one the server redirects (classic, module and Shared, and the classic one started again) runs at the URL the redirect gives and loads what sits beside it there, unseeded and named once for each address, in Firefox and WebKit (in WebKit the server is asked for its first URL twice for each start; WebKit itself sometimes never runs the Shared Worker, with no keploy too, and the row takes that only on exactly WebKit's record, the server asked for the first URL and the redirect's once each and nothing after them, with no message and no error event, and prints a line naming it), and in Chromium does not start seeded and is named once for each address, the server asked for its first URL once for each start and never for the next; the app's server never sees keploy's mark;Cannot fulfill with redirect status, with no wait on them in either (in WebKit, Playwright's runner stops the test at the first error a route's handler throws, while other handlers may still be answering, so the fixture's handlers end together; keploy lets go of each call within 0.1 ms of its handler's end, and the row still asserts that settle names none of them); these fail before this change (the test waits out its timeout);Route is already handled!) is waited for; in WebKit, a call whose redirect the browser refused can be sent on by no route: the route'scontinue()fails with Playwright's ownroute.continue: Route is already handled!and itsabort()withroute.abort: Route is already handled!, and nothing waits on the call (WebKit holds it until the context is deleted, and may then send it to the server, as with no keploy); a call the test's route falls back on and then throws is not waited for, and the test fails at once with the handler's error (before this change it waits out the test's timeout);accessdenied(on its page and on its context) or with no code, and a Web Worker's script it aborts withaccessdenied, withblockedbyclientor with no code, fail with the page'srequestfailedtext they have with no keploy, seeded and with seeding off, and never reach the server; in Chromium, keploy's stop of a seeded Web Worker's script the server redirects fails it as a route'sabort('blockedbyclient')does with no keploy (Chromium names every failed worker-script loadnet::ERR_ABORTED, so the code itself is held by the unit rows). Engines name codes apart only in part (WebKit gives every abort one text,Blocked by Web Inspector; Firefox givesblockedbyclientthe text of no code; Chromium reports every failed Web Worker script load asnet::ERR_ABORTED), so the row checks that a dropped code would show where they do: a fetch's text in Chromium and Firefox, and a worker script's in Firefox;Math.randomand a seeded faker instance), each placed document's draws per function and its frames per path; a replay that draws alike prints nothing; one that draws otherwise names each of 7 differences once; a replay answered no note, or an empty one, says so once per worker; an unreadable note is named; a document keploy cannot place, and its frames, are left out; a document that numbered a frame before its place was known is reported under its place. Each end's note is checked, and a test with seeding off sends none. Unit rows: the note's 16 KiB cut (documents first, then streams, counted), reading a note back, the lines for each kind of difference, at most 8 and the count of the rest, and a begin answered a note that is no string of at most 16 KiB;npm teston Node 20, 22 and 24, as CI runs it (npm ci,npm run lint,npx playwright install --with-deps chromium firefox webkit,npm test): lint is clean and 273 of 273 pass on each (Node 22 underKEPLOY_PROVEN=1, Node 20 and 24 with it unset).upgrade-insecure-requests, settle's watch of the test's routes and of theircontinueandabort, and the busy Web Worker line: 9 inindex.ts, 200 inrandom.ts, 66 insettle.ts): 254 are killed by a row, and the other 21 are among those below. Three more, of the error code a route's abort carries (settle's wrapper dropping it, or giving Playwright's default, and keploy's hand-on of a worker script's abort dropping it), are each killed by the abort rows above. The 15 mutants that widen the record on which the WebKit served row takes WebKit's own dropped Shared Worker (test/webkit.mjs: each part of it dropped or loosened) are each killed by that record's row. The earlier runs held 787 (105 inindex.ts, 580 inrandom.ts, 49 insettle.ts, 17 inagent.ts, 36 innote.ts), 773 killed by a row; those of code a later commit replaced are not run again. The others change nothing a test or a user can see, as the code shows:ErrorEventfilename(that it is there, that it is a getter, that it gives a string), in the page and in a Web Worker: WebIDL makesErrorEventan interface of every window and worker,filenamea getter on its prototype, and its value a string, in every engine;WorkerandSharedWorkerwrapping: a window with no such constructor fails the wrapping inside its own guard and keeps what it has;terminateis a WebIDL operation (a function) in every engine; a terminate whose document never got its number (the report threw) fails inside its own guard, as when it tells nothing;WeakSetholds only objects and answers no for anything else;unrouteorunrouteAll: that step stops at its own check for no list before it asks whether the test has a route left, and the only other place that asks it (the context's release, for a request's page) runs only after a context that had a list, whose pages have one too;unrouteAlltakes the test's routes off through Playwright's own, which keeps keploy's, andunroutecannot name keploy's matcher, so no route of keploy's is ever taken off;WeakMapanswers nothing for no page, which is read as no route of the test's either way;Error.prepareStackTraceformatter: only functions are keys of that map, so anything else is found in neither form;openedlooking among the pages still coming:newPageadds its page to the known pages beforeopened's wait on the coming pages resumes, so that look finds no page the known pages lack;pageOf, which answers no page, so the call is kept either way;